Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device capable of suppressing an inappropriate response, an information processing method, and an information processing program.SOLUTION: An information processing device comprises: an acceptance unit which accepts information indicative of a user's request; a determination unit which determines whether or not to generate information indicative of a response to the request indicated by the information accepted by the acceptance unit; and a provision unit, when the determination unit determines to generate the information indicative of the response, provides the user with the response information generated using a generative AI as information indicative of the response to the request. The determination unit determines whether or not to generate information indicative of a response to the request indicated by the information accepted by the acceptance unit on the basis of whether or not the request indicated by the information accepted by the acceptance unit is a request on a first target set as a non-response target or a request on a second target set as a response target.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, technologies for responding to requests from users using language models such as transformer models have been developed. For example, Patent Document 1 discloses a technology for generating an answer to a question using a language model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above conventional technology does not consider how to handle inappropriate responses such as answers, and there is room for improvement.

[0005] The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of suppressing inappropriate responses.

Means for Solving the Problems

[0006] The information processing apparatus according to the present application includes a reception unit, a determination unit, and a provision unit. The reception unit receives information indicating a request from a user. The determination unit determines whether to generate information indicating a response to the request indicated by the information received by the reception unit. The provision unit includes a provision unit that provides, to the user, response information generated using a generation AI as information indicating a response to the request when it is determined by the determination unit that information indicating a response is to be generated. The determination unit determines whether to generate information indicating a response to the request indicated by the information received by the reception unit based on whether the request indicated by the information received by the reception unit is a request related to a first target set as a non-response target and a request related to a second target set as a response target.

Effect of the Invention

[0007] According to one aspect of the embodiment, there is an effect that inappropriate responses can be suppressed.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments for implementing the information processing apparatus, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. Also, the respective embodiments can be appropriately combined within a range that does not conflict with the processing contents. Further, in the following respective embodiments, the same parts are denoted by the same reference numerals, and duplicate explanations are omitted.

[0010] 〔1. An Example of Information Processing〕 First, an example of information processing according to the embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram for explaining the information processing according to the embodiment.

[0011] The information processing apparatus 1 shown in FIG. 1 is an information processing apparatus that cooperates with each terminal device 2 of the user U and provides various services to the user U online, and is realized by, for example, one or more servers or a cloud system. The terminal device 2 is, for example, a smartphone, a tablet, or a personal computer.

[0012] The services provided by the information processing apparatus 1 are, for example, various services such as a Q&A (Question and Answer) service such as Yahoo! Knowledge Bag, a content creation service, a customer support service, and a learning support service, but are not limited to such examples.

[0013] In various services, the information processing apparatus 1 generates response information, which is information indicating a response corresponding to the request of the user U, using a generative AI (Artificial Intelligence), and provides the generated response information to the user U.

[0014] The generative AI is, for example, text generative AI. The text generative AI is, for example, a large language model trained to estimate and output the next token from an input token sequence, such as a Transformer-based model, an RNN (Recurrent Neural Network)-based model, etc., or it may be a hybrid model like these. Also, the text generative AI may be a composite system combined with an identification mechanism for preventing unauthorized use, etc.

[0015] Examples of the Transformer-based model include, but are not limited to, GPT (Generative Pre-trained Transformer) (registered trademark), PaLM2 (Pathways Language Model Version 2), or LLaMA (Large Language Model Meta AI), etc. Examples of the RNN-based model include, but are not limited to, RWKV (Receptance Weighted Key Value), etc.

[0016] Note that it is desirable for the generative AI to be trained not to include personal information, etc. in its generation results. The generative AI is arranged in an external information processing device, and the information processing device 1 uses the generative AI via an API (Application Programming Interface), but the generative AI may also be arranged within the information processing device 1.

[0017] As shown in FIG. 1, the information processing device 1 receives request information, which is information indicating the request of the user U (step S1). The information processing device 1 receives request information, which is information indicating the request of the user U, for example, by receiving a usage request from the terminal device 2.

[0018] The usage request includes request information. The request information includes, for example, information indicating the question of the user U, information indicating the instruction of the user U, or information indicating the desire of the user U, etc., but is not limited to such examples.

[0019] In addition, the request information includes designation information for designating the category of the request. For example, when the question of user U is a question in a Q&A service or a customer support service, the usage request includes designation information for designating the category of the question.

[0020] Also, when the instruction of user U is an instruction for content creation in a content creation service, the usage request includes designation information for designating the category of the content to be created. Further, when the instruction of user U is an instruction for interactive learning in a learning support service, the usage request includes designation information for designating a subject as the category of learning.

[0021] The designation information is, for example, information indicating the category of the request or information corresponding to the category of the request. The information corresponding to the category of the request is, for example, information of a part or all of the URL (Uniform Resource Locator) or domain name of the category in the service provided by the information processing apparatus 1, but is not limited to such an example.

[0022] Subsequently, the information processing apparatus 1 determines whether to generate information indicating a response to the request indicated by the request information received in step S1 (step S2). In the following, it will be described assuming that the request indicated by the request information received in step S1 is a question in a Q&A service.

[0023] For example, the information processing apparatus 1 performs an explicit non-response determination process, which is a process of determining whether the request indicated by the request information received in step S1 is a request regarding a first target set as a non-response target (step S2-1). The first target is a target explicitly set as a non-response target.

[0024] A non-response target is a target that does not respond to a request indicated by request information. A plurality of first targets may be set as non-response targets, or there may be only one. Such a first target is a target explicitly indicated as a non-response target. Therefore, the first target can be said to be a criterion for determining a non-response target, and can be called an explicit non-response determination criterion.

[0025] The first target, which is an explicit exclusion determination criterion, includes, for example, a target set as a non-response target for a specified category that is a category specified by user U. A target set as a non-response target for a specified category is, for example, a target that depends on the specified category. The information processing apparatus 1 identifies the specified category based on the specified information included in the usage request.

[0026] Targets that depend on the specified category are, for example, other categories (categories other than the specified category) for which a response is not appropriate in the specified category, other categories (categories other than the specified category) where the boundary with the specified category is ambiguous and the probability of an appropriate response is below a threshold, etc., and may be referred to as non-response categories in the following. Non-response categories are preset for each specified category.

[0027] In addition, the first target set as a non-response target includes, for example, targets that do not depend on the specified category in addition to the targets set as non-response targets for the specified category. Targets that do not depend on the specified category are, for example, but not limited to, violations of the law, violations of social morals, slander, and defamation of reputation.

[0028] In addition, targets that depend on the specified category and targets that do not depend on the specified category may include targets for which a response using generative AI is not appropriate. Targets for which a response using generative AI is not appropriate are, for example, but not limited to, medical, health, finance, etc.

[0029] The information processing apparatus 1 can determine, for example, using a language model, whether a request indicated by request information is a request regarding a first target set as a non-response target. Such a language model is an example of a first language model and may be described as the first language model hereinafter. The first language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples.

[0030] The information processing apparatus 1 inputs, as input information, information including instruction information instructing output of information indicating whether a request indicated by request information included in a usage request is a request regarding a first target and information indicating the request included in the request information to the first language model.

[0031] In this case, information indicating whether it is a request regarding the first target is output from the first language model. In this way, the information processing apparatus 1 can determine, using the first language model, whether a request indicated by request information is a request regarding a first target set as a non-response target.

[0032] Input information input to an AI such as a language model is called a prompt. In the following, the input information input to the language model may be described as a prompt. A prompt is, for example, information indicating instructions, requests, etc. given to an AI such as a language model to execute a specific task for the AI such as a language model.

[0033] The instruction information is, for example, when the request indicated by the request information is a question in a specific category (designated category) in a Q&A service, information such as the string "You are an excellent filter. Please determine whether the given question meets the conditions.\n#Conditions\nReturn 'true' when it meets the following, and return 'false' when it does not.\nQuestions regarding {category1}\nQuestions regarding {category2},...". {category1} and {category2} are, for example, information indicating non-response categories set for the designated category.

[0034] The instruction information includes information indicating a first target, but the information indicating the first target may be included in the input information separately from the instruction information. Further, the instruction information may include information indicating a request included in the request information.

[0035] Also, the instruction information includes information indicating an output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request related to the first target, and information to be output when the request indicated by the request information is not a request related to the first target.

[0036] The information to be output when the request indicated by the request information is a request related to the first target is information indicating a non-response determination, for example, "true", and the information to be output when the request indicated by the request information is not a request related to the first target is information indicating a response determination, for example, "false", but is not limited to such examples.

[0037] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request related to the first target. In this case, the specific information is information indicating a non-response determination, and the fact that the specific information is not output becomes information indicating a response determination. For example, the information indicating the output format is information in the character string "when the request indicated by the request information is a request related to the first target, output 'corresponding', and if not, output nothing", but is not limited to such examples. Also, the information indicating the output format may be information indicating a sample of the output.

[0038] In the above example, although the input information input to the first language model includes the instruction information, the first language model may be a language model learned to output information indicating whether a request indicated by the request information from input information not including the instruction information is a request regarding the first target. Further, the first language model may be a language model learned to output information indicating whether a request indicated by the request information from input information not including the instruction information and information indicating the first target is a request regarding the first target. In these cases, the language model is generated, for example, for each category, but is not limited to such examples.

[0039] Instead of or in addition to the first language model, the information processing apparatus 1 can also determine, by natural language processing without using the first language model, whether a request indicated by the request information is a request regarding a first target set as a non-response target. The natural language processing without using the first language model is, for example, keyword-based natural language processing. The information processing apparatus 1 has, for example, a first target dictionary including a plurality of keywords each directly or indirectly indicating the first target, and determines whether a keyword included in the first target dictionary is included in the request information.

[0040] When a keyword included in the first target dictionary is included in the request information, the information processing apparatus 1 determines that the request indicated by the request information is a request regarding the first target set as a non-response target, and otherwise determines that the request indicated by the request information is not a request regarding the first target set as a non-response target.

[0041] Further, the keyword-based natural language processing may be natural language processing using a model other than a large language model. The model in this case is a model generated by machine learning using learning information including, for each input information, the input information and information (label information) indicating whether a request indicated by the input information is a request regarding the first target. Such a model is, for example, GBDT (Gradient Boosting Decision Tree), a neural network, etc., but is not limited to such examples.

[0042] Subsequently, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is not a request related to the first target, the information processing apparatus 1 performs an implicit non-response determination process of determining whether the request indicated by the request information is a request related to the second target set as the response target (step S2-2).

[0043] The second target is the target set as the response target. By determining whether the request is not a request related to the second target, it is determined whether the target is not a non-response target, and it can be said that the target implicitly indicates the non-response target. Therefore, it can be said that the second target that becomes the response target implicitly indicates the non-response determination criterion, and it can be called the implicit non-response determination criterion. The second target includes, for example, the designated category if there is a category designated by the user U.

[0044] The information processing apparatus 1 can determine, for example, using a language model whether the request indicated by the request information is a request related to the second target set as the response target. Such a language model is an example of the second language model and may be described as the second language model hereinafter. The second language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples. Note that the second language model may be the same language model as the first language model.

[0045] The information processing apparatus 1 inputs, as input information, information including instruction information instructing the output of information indicating whether the request indicated by the request information included in the usage request is a request related to the second target and information indicating the request indicated by the request information included in the usage request to the second language model.

[0046] In this case, information indicating whether the request is a request related to the second target is output from the second language model. In this way, the information processing apparatus 1 can determine, using the second language model, whether the request indicated by the request information is a request related to the second target set as the response target.

[0047] For example, when the request indicated by the request information is a question in a specific category (designated category) in the Q&A service, the indication information is information such as the string "You are an expert in {category}. Return 'true' when it corresponds to a question regarding {category}, and do not return 'false' when it does not correspond to a question regarding {category}." Here, {category} is the designated category.

[0048] The indication information includes information indicating the second target, but the information indicating the second target may be included in the input information separately from the indication information. Also, the indication information may include information indicating the request included in the request information.

[0049] In addition, the indication information includes information indicating the output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the second target, and information to be output when the request indicated by the request information is not a request regarding the second target.

[0050] The information to be output when the request indicated by the request information is a request regarding the second target is information indicating a response determination, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the second target is information indicating a non-response determination, for example, "false", but is not limited to such examples.

[0051] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the second target. In this case, the specific information is information indicating a response determination, and the non-output of the specific information becomes information indicating a non-response determination.

[0052] For example, the information indicating the output format is information such as the string "When the request indicated by the request information is a request regarding the second target, output 'corresponding', and otherwise, do not output anything", but is not limited to such examples. Also, the information indicating the output format may be information indicating a sample of the output.

[0053] In the above example, although the input information input to the second language model includes the instruction information, the second language model may be a language model learned to output information indicating whether a request indicated by the request information from input information not including the instruction information is a request regarding a second target.

[0054] Further, the second language model may be a language model learned to output information indicating whether a request indicated by the request information from input information not including the instruction information and information indicating the second target is a request regarding the second target. In these cases, the language model is generated for each category, for example, but is not limited to such examples.

[0055] The information processing apparatus 1 can also determine, by natural language processing that does not use the second language model instead of or in addition to the second language model, whether the request indicated by the request information is a request regarding the second target set as the response target.

[0056] The natural language processing that does not use the second language model is, for example, keyword-based natural language processing. The information processing apparatus 1 has, for example, a second target dictionary including a plurality of keywords directly or indirectly indicating each second target, and determines whether the keywords included in the second target dictionary are included in the request information.

[0057] When the keywords included in the second target dictionary are included in the request information, the information processing apparatus 1 determines that the request indicated by the request information is a request regarding the second target set as the response target, and otherwise determines that the request indicated by the request information is not a request regarding the second target set as the response target.

[0058] In addition, keyword-based natural language processing may be natural language processing using a model other than a large language model. In this case, the model is a model generated by machine learning using learning information including input information and information (label information) indicating whether or not the request indicated by the input information is a request related to the second target for each input information. Such models include, for example, GBDT, neural networks, etc., but are not limited to such examples.

[0059] Subsequently, based on the determination result as to whether or not the request indicated by the request information in step S2-2 is a request related to the second target, the information processing apparatus 1 determines whether or not to generate response information which is information indicating a response to the request indicated by the request information received in step S1 (step S2-3).

[0060] For example, when it is determined in step S2-2 that the request indicated by the request information is a request related to the second target, the information processing apparatus 1 determines to generate information indicating a response to the request indicated by the request information.

[0061] In addition, when it is determined in step S2-2 that the request indicated by the request information is not a request related to the second target, the information processing apparatus 1 determines not to generate information indicating a response to the request indicated by the request information.

[0062] In this way, the information processing apparatus 1 first performs explicit non-response determination processing, and when the explicit non-response determination criteria are not satisfied, performs implicit non-response determination processing, and when the implicit non-response determination criteria are not satisfied, makes a response. Thereby, the information processing apparatus 1 can perform response availability determination with low cost and high accuracy, and can suppress inappropriate responses.

[0063] Note that the information processing apparatus 1 can perform explicit non-response determination processing and implicit non-response determination processing in parallel. In this case, the information processing apparatus 1 determines whether or not to generate information indicating a response to the request indicated by the request information based on the determination result of the explicit non-response determination processing and the determination result of the implicit non-response determination processing.

[0064] In this case, when the request indicated by the request information is not a request related to the first target and the request indicated by the request information is a request related to the second target, the information processing apparatus 1 determines to generate information indicating a response to the request indicated by the request information, and otherwise determines not to generate information indicating a response to the request indicated by the request information. Also by this, the information processing apparatus 1 can suppress inappropriate responses.

[0065] Also, the information processing apparatus 1 can perform the processing in the order of the implicit non-response determination process and the explicit non-response determination process. In this case, the explicit non-response determination process can be performed only when it is determined in the implicit non-response determination process that the request indicated by the request information is a request related to the second target. Also by this, the information processing apparatus 1 can suppress inappropriate responses.

[0066] In this way, the information processing apparatus 1 can determine whether to generate information indicating a response to the request indicated by the request information based on whether the request indicated by the request information is a request related to the first target set as the non-response target and a request related to the second target set as the response target.

[0067] Subsequently, when it is determined in step S2 that information indicating a response to the request indicated by the request information is to be generated, the information processing apparatus 1 generates response information using a generation AI as the information indicating a response to the request indicated by the request information (step S3).

[0068] For example, the information processing apparatus 1 inputs, as input information to the generation AI, information including instruction information for instructing the output of information indicating a response to the request indicated by the request information and information indicating the request included in the request information, and causes the generation AI to generate response information.

[0069] The request indicated by the request information is, for example, a question in the category of economy in a Q&A service, and is assumed to be the information of the character string "What is inflation? How does it affect the economy?". In this case, the information processing apparatus 1 includes, as the instruction information, the information of the character string "As an expert in {category}, you accept and answer questions regarding {category}.", and inputs, as input information to the generation AI, the information including the information of the character string "What is inflation? How does it affect the economy?" as the information indicating the request included in the request information.

[0070] Note that {category} includes information indicating the designated category. Further, the instruction information may include guideline information indicating the guidelines for the answer. The guideline information includes, for example, information instructing the clarity and neutrality of the answer.

[0071] Also, in the above-described example, an example in which the process of step S2 and the process of step S3 are performed separately is shown, but a part or all of the process of step S2 may be included in the process of step S3.

[0072] For example, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is not a request regarding the first target, and is a request regarding the second target, by inputting, as input information to the generation AI, the information including the instruction information instructing the generation of the information indicating the response to the request indicated by the request information, the processes of step S2-2, step S2-3, and step S3 can be performed collectively like the process described in the speech bubble in FIG. 1. In this case, the above-described second language model is the generation AI, and the processes of step S2-2, step S2-3, and step S3 can be performed collectively by inputting the input information (prompt) once to the generation AI.

[0073] In this case, the instruction information included in the input information is, for example, information such as the character string "#As an expert in {category}, you will accept and answer questions and consultations regarding {category}. Do not answer questions outside the {category} category." However, it is not limited to such an example. Note that {category} includes information indicating the specified category. Further, the instruction information may include guideline information.

[0074] Also, when the request indicated by the request information is a request regarding a second target instead of a request regarding a first target, the information processing apparatus 1 inputs, as input information to the generation AI, information including instruction information for instructing generation of information indicating a response to the request indicated by the request information, whereby the processes of steps S2 and S3 can be performed collectively. In this case, the first language model and the second language model described above are generation AIs, and the processes of steps S2 and S3 can be performed collectively by inputting the input information (prompt) once to the generation AI.

[0075] In this case, the instruction information included in the input information is, for example, information such as the character string "#You are an excellent AI assistant. Determine whether the given question corresponds to any of {category group}, and if the given question does not correspond to any of {category group}, as an expert in {category}, you will accept and answer questions regarding {category}. Do not answer questions outside the {category} category." However, it is not limited to such an example. Note that {category group} includes information indicating one or more first targets, and {category} includes information indicating the specified category. Further, the instruction information may include guideline information.

[0076] Subsequently, when the information processing apparatus 1 generates response information using the generation AI in step S3, the information processing apparatus 1 provides the response information generated using the generation AI to the user U (step S4). For example, the information processing apparatus 1 provides the response information generated using the generation AI to the user U by transmitting the response information generated using the generation AI in step S3 to the terminal device 2 that transmitted the utilization request.

[0077] For example, when the request indicated by the request information is a question of the Q&A service, if the user U accesses the page showing the answer to the user U's question, the information processing apparatus 1 can provide the user U with the page showing the answer to the user U's question.

[0078] Also, when the information processing apparatus 1 does not generate response information using the generation AI in step S3, the information processing apparatus 1 provides non-response information to the user U (step S5). For example, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is a request regarding the first target, or when the information processing apparatus 1 determines in step S2-2 that the request indicated by the request information is not a request regarding the second target, the information processing apparatus 1 provides non-response information to the user U.

[0079] The non-response information is, for example, information of the character information "Sorry. It is a non-response target and I cannot respond to the request." For example, when the request indicated by the request information is a request for the Q&A service, the non-response information is, for example, information of the character information "Sorry. I cannot answer because it is a question outside the scope of answers." Note that the non-response information may include information indicating that it is a request regarding the first target or not a request regarding the second target.

[0080] In the above-described explicit non-response determination criteria, in addition to the fixed first target that does not depend on the designated category, as the first target that depends on the designated category, by including categories other than the designated category that are preset for each designated category, the non-response accuracy for inappropriate requests can be improved.

[0081] On the one hand, the above-mentioned first target that depends on the specified category is a category that is preset as a non-response category for each specified category and is a non-response category that is a category other than the specified category. Due to becoming an abstract concept and other reasons, the coverage of non-response is low, and in the information processing apparatus 1, there is a possibility that inappropriate requests cannot be rejected.

[0082] For example, in the non-response category, the information processing apparatus 1 may not be able to reject requests that should not be responded to, such as requests that violate social morals and requests for responses that are slanderous. In the non-response category, there is a possibility that requests for which responses using generative AI are inappropriate cannot be rejected. Requests for which responses using generative AI are inappropriate are, for example, requests including topics such as medical care, health, and finance in the non-response category, but are not limited to such examples.

[0083] Therefore, the information processing apparatus 1 is configured to be able to perform a generation process of generating non-response target information indicating a new non-response target based on non-response category information indicating a non-response category associated with the specified category, whereby the non-response accuracy for inappropriate requests can be further improved. Hereinafter, the generation process executed by the information processing apparatus 1 will be specifically described.

[0084] The generation process executed by the information processing apparatus 1 includes a first generation process of generating risk information, which is information indicating a risk, as new non-response target information from non-response category information, which is information indicating a non-response category, and a second generation process of extracting characteristic words from the non-response category information and generating new non-response target information from the extracted characteristic words.

[0085] First, the first generation process will be described. In the first generation process, the information processing device 1 generates, for example, new non-response target information using a language model. Such a language model is an example of a third language model and may be referred to as the third language model hereinafter. The third language model is a large language model such as, for example, a transformer-based model or an RNN-based model. The third language model may be the same as one or more of the above-described generative AI, the first language model, and the second language model.

[0086] The information processing device 1 generates, for example, risk information, which is information indicating a risk in the non-response category from the non-response category information, as new non-response target information using the third language model. For example, the information processing device 1 can input information including non-response category information as input information into the third language model and cause the third language model to generate and output risk information, which is information indicating a risk in the non-response category, as new non-response target information. Hereinafter, information indicating a risk may be referred to as risk information.

[0087] The information processing device 1 inputs, for example, information including instruction information for instructing the output of risks in one or more non-response categories as input information into the third language model and causes the third language model to generate and output risk information for the non-response category. The instruction information includes, for example, information of the character string "Please output the risks considered based on the given category information.\n#Category Information\nAdult\nGambling" when the non-response categories are adult and gambling, etc.

[0088] The third language model may be a model learned to output non-response target information when information including non-response category information is input as input information. In this case, the third language model is learned using, for example, learning information including information including non-response category information and one or more non-response target information for each non-response category. The one or more non-response target information are used as teacher data (labels).

[0089] The information processing apparatus 1 can also generate, for example, one or more pieces of risk information for each non-response category using a third language model. Further, the information processing apparatus 1 can also include information specifying the number of non-response category information to be generated by the third language model in the specifying information.

[0090] Also, the information processing apparatus 1 can also cause the third language model to generate, as risk information, information indicating a risk considered as an administrator of a service that responds to requests for a specified category.

[0091] For example, assume that the request indicated by the request information is a question of a Q&A service, the specified category is television, and the non-response categories are adult and gambling. In this case, the instruction information included in the input information includes information of the character string "You are an administrator who answers questions regarding television in the Q&A service. Please output information regarding risks considered based on the given category information."

[0092] Also, when the information processing apparatus 1 obtains a plurality of pieces of risk information using the third language model, the information processing apparatus 1 aggregates the plurality of pieces of risk information into n or fewer pieces of risk information, which is a preset number, as non-response target information using the third language model. n is an integer of 1 or more. For example, the information processing apparatus 1 can input, as input information to the third language model, information including instruction information for grouping the plurality of pieces of risk information into n pieces of risk information, and cause the third language model to output the n pieces of aggregated risk information.

[0093] Further, when the information processing apparatus 1 causes the third language model to generate information indicating, for example, a risk considered as an administrator of the Q&A service described above as risk information, for example, the information processing apparatus 1 inputs instruction information including information of the character string "You are an administrator who answers questions regarding television in the Q&A service. Please aggregate into five risks that must be emphasized in continuing the service based on the given risk information." and a plurality of risk information into the third language model as input information, whereby the third language model can be caused to output the aggregated n pieces of risk information.

[0094] Next, the second generation process will be described. In the second generation process, the information processing apparatus 1 extracts characteristic words (for example, keywords or characteristic phrases) from the non-response category information, and generates new non-response target information from the extracted characteristic words.

[0095] The information processing apparatus 1 divides the non-response category information to extract characteristic words, and generates new non-response target information using such characteristic words. The information processing apparatus 1 can perform, for example, a process of extracting characteristic words from the non-response category information and generating information including each of such characteristic words as a plurality of new non-response target information using the third language model.

[0096] For example, assume that the request indicated by the request information is a question in the Q&A service and the non-response category information is the character string "Questions regarding the Internet". In this case, the information processing apparatus 1 can cause the third language model to generate a plurality of new non-response target information by inputting information of the character string "# Please extract characteristic words that are important words from the given keywords and create subdivided keywords.\n\n# Keywords\nQuestions regarding the Internet" into the third language model. The plurality of new non-response target information in this case is, for example, information of the character string "Internet technology", information of the character string "Internet service", etc., but is not limited to such examples.

[0097] In addition, for example, instead of the third language model, the information processing apparatus 1 can also generate a plurality of new non-response target information based on rules. For example, the information processing apparatus 1 has a feature word dictionary including a plurality of keywords or phrases for each feature word, and can extract a plurality of keywords or phrases associated with the feature words included in the feature word dictionary among the non-response category information as a plurality of new non-response target information.

[0098] When there are a plurality of non-response category information, the information processing apparatus 1 can perform the process of generating a plurality of non-response target information using the third language model for each non-response category, but can also perform the process of generating a plurality of non-response target information using the third language model collectively for a plurality of non-response categories.

[0099] The information processing apparatus 1 can select two or more non-response target information to be used in step S2-1 among the plurality of non-response target information generated as described above based on the non-response accuracy for each combination of two or more non-response target information among the plurality of non-response category information generated as described above. In this case, the information processing apparatus 1 determines, in step S2-1, whether the request indicated by the request information is a request regarding a non-response target based on the selected two or more response target information.

[0100] The information processing apparatus 1 can evaluate, for example, the non-response accuracy for each combination of two or more non-response target information. For example, the information processing apparatus 1 can evaluate the non-response accuracy for each combination of two or more response target information using a plurality of evaluation request information. Hereinafter, a combination of two or more non-response target information may be referred to as a non-response target information combination.

[0101] Each evaluation request information is information indicating a first request that is a request regarding a non-response target in a specified category, information indicating a second request that is a request regarding a response target in the specified category, or information indicating a third request that is a request regarding a non-response target regardless of the specified category.

[0102] The information processing apparatus 1 determines whether the request indicated by the evaluation request information is determined to be a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination. The explicit non-response determination process using the non-response target information combination is, for example, a process of using each of a plurality of non-response target information included in the non-response target information combination as a first target.

[0103] When the information processing apparatus 1 determines that the first request or the third request indicated by the evaluation request information is a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the information processing apparatus 1 sets the evaluation of the non-response target information combination to 1, and when it is not the case, sets the evaluation of the non-response target information combination to 0.

[0104] Further, when the information processing apparatus 1 determines that the second request indicated by the evaluation request information is not a request regarding a non-response target in the explicit non-response determination process using a prompt including the non-response target information combination, the information processing apparatus 1 sets the evaluation of the non-response target information combination to 1, and when it is not the case, sets the evaluation of the non-response target information combination to 0.

[0105] The information processing apparatus 1 can calculate the non-response accuracy by aggregating the evaluation results and dividing the aggregated result by the number of evaluations. Further, the information processing apparatus 1 can also change the weight according to which of the first request, the second request, and the third request the request indicated by the evaluation request information is, and aggregate the evaluation results by weighted addition.

[0106] Further, the evaluation request information may be limited to the information indicating the first request, or may be limited to two or more pieces of information among the information indicating the first request, the information indicating the second request, and the information indicating the third request. Further, the evaluation is not limited to 0 and 1, and may be, for example, -1 and 1, or other combinations of values.

[0107] In addition, when the information processing apparatus 1 determines whether a request indicated by request information from each user U is a request regarding a non-response target in an explicit non-response determination process using non-response target information combinations by using keyword-based natural language processing, the information processing apparatus 1 can also evaluate the non-response accuracy of the non-response target information combinations by using a language model.

[0108] For example, the information processing apparatus 1 can evaluate the non-response accuracy of non-response target information combinations for each non-response target information combination by using a third language model based on information indicating a request included in request information from each user U and the determination result of keyword-based natural language processing.

[0109] For example, the information processing apparatus 1 inputs, as input information, information including instruction information for instructing determination as to whether any of two or more non-response target information indicated by a non-response target information combination is not included in request information from the user U into the third language model, so that the third language model outputs information indicating the non-response accuracy of the two or more response target information indicated by the non-response target information combination.

[0110] The information processing apparatus 1 selects, as the non-response target information combination to be used in step S2-1, the non-response target information combination having the highest non-response accuracy among the non-response accuracies for each non-response target information combination.

[0111] In addition, the information processing apparatus 1 can also select, as combinations of two or more non-response target information to be used in step S2-1 according to a random or predetermined rule, non-response target information combinations whose non-response accuracy for each non-response target information combination is equal to or higher than a threshold value.

[0112] As described above, the information processing apparatus 1 according to the embodiment receives information indicating a request of the user U, determines whether to generate information indicating a response to the request indicated by the received information, and if it is determined to generate the information indicating the response, provides the user U with response information generated using a generation AI as the information indicating the response to the request. The information processing apparatus 1 determines whether to generate information indicating a response to the request indicated by the received information based on whether the request indicated by the received information is a request regarding a first target set as a non-response target and a request regarding a second target set as a response target. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0113] Further, the information processing apparatus 1 according to the embodiment generates non-response target information indicating a non-response target, receives information indicating a request of the user U, and determines whether the request indicated by the received information is a request regarding the non-response target based on the plurality of generated non-response target information. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0114] Hereinafter, the configuration of the information processing system including the information processing apparatus 1 and the terminal device 2 that perform such processing will be described in detail.

[0115] [2. Configuration of Information Processing System] FIG. 2 is a diagram showing an example of the configuration of the information processing system according to the embodiment. As shown in FIG. 2, the information processing system 100 according to the embodiment includes an information processing apparatus 1, a plurality of terminal devices 2, and a terminal device 3.

[0116] The plurality of terminal devices 2 are used by different users U. The terminal device 3 is used, for example, by an employee O of the operating company of the information processing apparatus 1. The terminal devices 2 and 3 are, for example, a notebook PC (Personal Computer), a desktop PC, a smartphone, a tablet PC, or a wearable device. The wearable device is, for example, smart glasses or a smartwatch, but is not limited to such examples.

[0117] Each of the information processing device 1, the terminal device 2, and the terminal device 3 is connected to be communicable with each other by wire or wirelessly via the network N. Note that the information processing system 100 shown in FIG. 2 may include a plurality of information processing devices 1 and the like.

[0118] The network N includes, for example, a WAN (Wide Area Network) such as the Internet and a mobile communication network such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation: the 5th generation mobile communication system), but is not limited to such examples.

[0119] The terminal devices 2 and 3 are connected to the network N via a mobile communication network, Bluetooth (registered trademark), or short-range wireless communication such as a wireless LAN (Local Area Network), and can communicate with the information processing device 1 and the like.

[0120] 〔3. Configuration of Information Processing Device 1〕 FIG. 3 is a diagram showing an example of the configuration of the information processing device 1 according to the embodiment. As shown in FIG. 3, the information processing device 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0121] 〔3.1. Communication Unit 10〕 The communication unit 10 is realized by, for example, a communication module or a NIC (Network Interface Card). Then, the communication unit 10 is connected to the network N by wire or wirelessly and transmits and receives information to and from other various devices. For example, the communication unit 10 transmits and receives information to and from each of the terminal device 2 and the terminal device 3 via the network N.

[0122] 〔3.2. Storage Unit 11〕 The storage unit 11 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 includes a user information storage unit 20, a non-response target information storage unit 21, and an evaluation request information storage unit 22.

[0123] [3.2.1. User Information Storage Unit 20] The user information storage unit 20 stores user information including information related to the user U. FIG. 4 is a diagram showing an example of a user information table stored in the user information storage unit 20 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 4, the user information table stored in the user information storage unit 20 includes items such as "user ID" and "attribute information".

[0124] The "user ID" is identification information for identifying the user U. The "attribute information" is attribute information of the user U corresponding to the "user ID", and includes, for example, information on psychographic attributes and demographic attributes. Demographic attributes are, for example, gender, age, place of residence, and occupation, and psychographic attributes are objects of interest such as travel, clothing, cars, and religion, lifestyle, thoughts, and trends of thoughts.

[0125] [3.2.2. Non-Response Target Information Storage Unit 21] The non-response target information storage unit 21 stores information indicating various non-response targets. FIG. 5 is a diagram showing an example of a non-response target table stored in the non-response target information storage unit in the information processing apparatus 1 according to the embodiment.

[0126] In the example shown in FIG. 5, the non-response target table stored in the non-response target information storage unit 21 includes information on items such as "non-response target ID", "target category", and "non-response target information". The "non-response target ID" is an identifier for identifying a non-response target and is information assigned to each non-response target. The non-response targets include the first target described above.

[0127] "Target Category" is information indicating the category to which a non-response target is associated. When there is no category to which the non-response target is associated, it is set to blank (null). "Non-response Target Information" is information indicating a non-response target. For example, it includes non-response category information indicating a non-response category pre-associated by an employee O or the like as a non-response category for a target category, non-response target information generated by the processing unit 12, etc., but is not limited to such examples.

[0128] [3.2.3. Evaluation Requirement Information Storage Unit 22] The evaluation requirement information storage unit 22 stores various evaluation requirement information. The evaluation requirement information is, for example, information indicating a first requirement that is a requirement regarding a non-response target in a specified category, information indicating a second requirement that is a requirement regarding a response target in a specified category, or information indicating a third requirement that is a requirement regarding a non-response target regardless of the specified category.

[0129] [3.3. Processing Unit 12] The processing unit 12 is a controller and is realized, for example, by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing apparatus 1 using a RAM or the like as a work area.

[0130] Also, the processing unit 12 is a controller and may be partially or entirely realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPGPU (General Purpose Graphic Processing Unit).

[0131] As shown in FIG. 3, the processing unit 12 includes an acquisition unit 30, a reception unit 31, a generation unit 32, an evaluation unit 33, a selection unit 34, a determination unit 35, and a provision unit 36, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in FIG. 3, and may be any other configuration as long as it can perform the information processing described later.

[0132] [3.3.1. Acquisition Unit 30] The acquisition unit 30 acquires various information from external information processing devices, terminal devices 2, 3, etc. via the network N and the communication unit 10.

[0133] For example, the acquisition unit 30 acquires the information of the user U from an external information processing device or terminal device 2, and stores the acquired information of the user U in the user information storage unit 20. Also, the acquisition unit 30 acquires non-response category information from an external information processing device or terminal device 3, and stores the acquired non-response category information in the non-response target information storage unit 21. Further, the acquisition unit 30 acquires evaluation request information from an external information processing device or terminal device 3, and stores the acquired evaluation request information in the evaluation request information storage unit 22.

[0134] In addition, the acquisition unit 30 acquires various information from the storage unit 11. For example, the acquisition unit 30 acquires the information of the user U from the user information storage unit 20. Also, the acquisition unit 30 acquires non-response target information corresponding to a specified category from the non-response target information storage unit 21. Further, the acquisition unit 30 acquires evaluation request information from the evaluation request information storage unit 22.

[0135] [3.3.2. Reception Unit 31] The reception unit 31 receives various requests and information. For example, the reception unit 31 receives request information that is information indicating the request of the user U. The reception unit 31 receives, for example, the request information included in the usage request by receiving the usage request. The request information includes, for example, information indicating a question of the user U, information indicating an instruction of the user U, or information indicating a desire of the user U, but is not limited to such examples.

[0136] The request information includes designation information that designates the category of the request. The reception unit 31 identifies the designated category based on the designation information included in the received usage request. For example, when the question of the user U is a question in a Q&A service or a customer support service, the usage request includes designation information that designates the category of the question.

[0137] Also, when the instruction of the user U is a content creation instruction in a content creation service, the usage request includes designation information that designates the category of the content to be created. Also, when the instruction of the user U is an instruction for interactive learning in a learning support service, the usage request includes designation information that designates a textbook as the category of learning.

[0138] The designation information is, for example, information indicating the category of the request or information corresponding to the category of the request. The information corresponding to the category of the request is, for example, part or all of the URL or domain name of the category in the service provided by the information processing apparatus 1, but is not limited to such an example.

[0139] [3.3.3. Generation unit 32] The generation unit 32 generates various information. For example, the generation unit 32 generates non-response target information as information indicating a first target. The non-response target information is information indicating a non-response target that is a target for which no response is given to the request indicated by the request information received by the reception unit 31.

[0140] The generation unit 32 generates non-response target information indicating a new non-response target. For example, the generation unit 32 generates non-response target information indicating a new non-response target based on non-response category information indicating a non-response category that is different from the designated category designated by the user U and that is associated in advance as a non-response category with respect to the designated category.

[0141] For example, the generation unit 32 can generate risk information, which is information indicating risk, from non-response category information, which is information indicating a non-response category, as new non-response target information. Further, the generation unit 32 can extract characteristic words from the non-response category information and generate new non-response target information from the extracted characteristic words. For example, the generation unit 32 generates, as non-response target information, information including some of the characteristic words extracted from the non-response category information.

[0142] For example, the generation unit 32 generates non-response target information as information indicating a first target using a third language model. For example, the generation unit 32 inputs information including non-response category information indicating a non-response category into the third language model and causes the third language model to generate non-response target information.

[0143] Further, when it is determined by the determination unit 35 that the generation unit 32 generates information indicating a response to a request indicated by the request information received by the reception unit 31, the generation unit 32 generates response information using a generation AI as information indicating a response to the request indicated by the request information received by the reception unit 31.

[0144] FIG. 6 is a diagram showing an example of non-response categories preset for each category in the information processing apparatus 1 according to the embodiment. In the example shown in FIG. 6, an example of non-response categories preset for each category when the request indicated by the request information is a question of a Q&A service is shown. Regardless of the category of the question of the user U, categories such as finance and law are set as non-response categories.

[0145] Also, in the example shown in FIG. 6, when the category of the question of the user U is finance or law, all categories other than that category are set as non-response categories. When the category of the question of the user U is TV, anime is set as the non-response category. When the category of the question of the user U is anime, TV is set as the non-response category. Note that the non-response categories preset for each category are not limited to the example shown in FIG. 6.

[0146] As shown in FIG. 3, the generation unit 32 includes a first generation processing unit 40 that generates a plurality of pieces of risk information indicating risks in the non-response category, and a second generation processing unit 41 that aggregates the plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information. Further, the generation unit 32 includes a third generation processing unit 42 that generates, as new non-response target information, information including a part of the characteristic words extracted from the non-response category information, and a fourth generation processing unit 43 that causes the generation AI to generate response information, which is information indicating a response to the request indicated by the request information.

[0147] [3.3.3.1. First Generation Processing Unit 40] The first generation processing unit 40 generates risk information indicating risks in the non-response category using the third language model. The third language model is a large language model such as, for example, a transformer-based model or an RNN-based model. The third language model may be the same as one or more of the above-described generation AI, the first language model, and the second language model.

[0148] The first generation processing unit 40 generates, for example, risk information, which is information indicating risks in the non-response category, from the non-response category information as new non-response target information using the third language model. For example, the first generation processing unit 40 can input information including the non-response category information as input information to the third language model, and cause the third language model to generate and output risk information, which is information indicating risks in the non-response category, as new non-response target information.

[0149] The first generation processing unit 40 inputs, for example, information including instruction information for instructing the output of risks in one or more non-response categories as input information to the third language model, and causes the third language model to generate and output risk information for the non-response category.

[0150] FIG. 7 is a diagram showing an example of instruction information used in the first generation processing unit 40 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model. In FIG. 7, the instruction information includes, for example, when the non-response category is adult and gambling, etc., the information of the character string "# Instruction\nYou are a risk manager in the {category} field of the Q&A service.\nPlease output the risks that can be considered based on the given category information.\n\n# Category information\nAdult\nGambling ···". In FIG. 7, {category} is information indicating the specified category.

[0151] As shown in FIG. 7, the first generation processing unit 40 can cause the third language model to generate, as risk information, information indicating risks considered as an administrator of a service that responds to requests for a specified category.

[0152] The third language model may be a model trained to output non-response target information when information including non-response category information is input as input information. In this case, the third language model is trained using, for example, learning information including information including non-response category information and one or more non-response target information for each non-response category. The one or more non-response target information are used as teacher data (labels).

[0153] The first generation processing unit 40 can also generate, for example, one or more risk information for each non-response category using the third language model. In addition, the first generation processing unit 40 can also include information specifying the number of non-response category information to be generated by the third language model in the specification information.

[0154] 〔3.3.3.2. Second Generation Processing Unit 41〕 The second generation processing unit 41 aggregates, using a language model, a plurality of risk information generated by the first generation processing unit 40 into risk information equal to or less than a preset number n as non-response target information. n is an integer of 1 or more.

[0155] When, for example, more than n pieces of risk information are generated by the first generation processing unit 40, the second generation processing unit 41 aggregates a plurality of pieces of risk information generated by the first generation processing unit 40 into n or fewer pieces of risk information, which is a preset number, using a third language model as non-response target information.

[0156] For example, the second generation processing unit 41 can input information including instruction information for grouping a plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information as input information to the third language model, and cause the third language model to output the aggregated n pieces of risk information.

[0157] FIG. 8 is a diagram showing an example of instruction information used in the second generation processing unit 41 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model. In FIG. 8, the instruction information is information of the character string "# Instruction\nYou are a risk manager in the {category} field of the Q&A service.\nPlease aggregate the risks that must be emphasized in continuing the service based on the given risk information into 5.\n\n# Risk Information\nLegal violation: The possibility that the service violates the law.\nTax issues: Risks due to inappropriate tax processing...". By inputting information including such instruction information and information indicating the requests included in the request information as input information to the third language model, the third language model can be caused to output the aggregated n pieces of risk information.

[0158] FIG. 9 is a diagram showing the relationship between the processing by the first generation processing unit 40 and the second generation processing unit 41 in the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 9, the first generation processing unit 40 generates a plurality of pieces of risk information based on the non-response category information stored in the non-response target information storage unit 21. The second generation processing unit 41 aggregates a plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information, and stores the aggregation result in the non-response target information storage unit 21 as n pieces of non-response target information.

[0159] 〔3.3.3.3. Third generation processing unit 42〕 The third generation processing unit 42 generates, as new non-response target information, information including a feature word (for example, a keyword or a characteristic phrase) extracted from the non-response category information.

[0160] For example, the third generation processing unit 42 divides the non-response category information to extract a feature word, and generates new non-response target information using such a feature word. The third generation processing unit 42 can perform, for example, a process of extracting a feature word from the non-response category information and generating, as a plurality of new non-response target information, information each including a part of such a feature word, using a third language model.

[0161] FIG. 10 is a diagram showing an example of instruction information used in the third generation processing unit 42 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model, and is an example in a case where the request indicated by the request information is a question in a Q&A service and the non-response category information is the character string "questions about the Internet".

[0162] In this case, as shown in FIG. 10, the third generation processing unit 42 can cause the third language model to generate a plurality of new non-response target information, for example, by inputting information of the character string "#Instruction\nExtract important words from the given keyword and create a refined keyword.\n\n #Keyword\nQuestions about the Internet" into the third language model. The plurality of new non-response target information in this case is, for example, information of the character string "Internet technology", information of the character string "Internet service", etc. as shown in FIG. 9, but is not limited to such examples.

[0163] Further, the third generation processing unit 42 can also generate a plurality of new non-response target information based on rules, for example, instead of the third language model. For example, the third generation processing unit 42 has a feature word dictionary including a plurality of keywords or phrases for each feature word, and can extract, as a plurality of new non-response target information, a plurality of keywords or phrases associated with the feature words included in the feature word dictionary among the non-response category information.

[0164] When there is a plurality of non-response category information, the third generation processing unit 42 can perform the process of generating a plurality of non-response target information using the third language model for each non-response category, but it is also possible to perform the process of generating a plurality of non-response target information using the third language model collectively for a plurality of non-response categories.

[0165] 〔3.3.3.4. Fourth generation processing unit 43〕 The fourth generation processing unit 43 generates response information, which is information indicating a response to the request indicated by the request information, using a generation AI.

[0166] For example, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information instructing the output of information indicating a response to the request indicated by the request information and information indicating the request included in the request information, and causes the generation AI to generate response information.

[0167] Suppose the request indicated by the request information is, for example, a question in the category of economy in a Q&A service and is information of the character string "What is inflation? How does it affect the economy?". In this case, the fourth generation processing unit 43 includes, as instruction information, information of the character string "As an expert in {category}, you receive and answer questions regarding {category}.", and inputs, as input information to the generation AI, information including information of the character string "What is inflation? How does it affect the economy?" as information indicating the request included in the request information.

[0168] Note that {category} includes information indicating a specified category. Further, the instruction information may include guideline information indicating a guideline for an answer. The guideline information includes, for example, information instructing clarity or neutrality of the answer.

[0169] The process of the fourth generation processing unit 43 may include part or all of the functions of the determination unit 35. In this case, the fourth generation processing unit 43 functions as part or all of the determination unit 35 in addition to the function of the generation unit 32 that generates response information using the generation AI.

[0170] For example, the fourth generation processing unit 43 may have a process of determining whether the request indicated by the request information received by the reception unit 31 is a request related to the second target set as the response target. In this case, the fourth generation processing unit 43 includes the processing of the generation unit 32 and the processing of the determination unit 35.

[0171] For example, when the request indicated by the request information received by the reception unit 31 is a request related to the second target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information, so that when the request indicated by the request information is a request related to the second target, the generation AI can generate response information.

[0172] For example, when the determination unit 35 determines that the request indicated by the request information is not a request related to the first target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information when it is a request related to the second target, whereby the determination process of whether it is a request related to the second target and the generation process of response information can be performed together.

[0173] FIG. 11 is a diagram showing an example of instruction information used for the explicit non-response determination process performed by the fourth generation processing unit 43 of the processing unit 12 in the information processing apparatus 1 according to the embodiment as part of the processing of the determination unit 35.

[0174] In the example shown in FIG. 11, the instruction information included in the input information is, for example, information such as the character string "# You are an expert in {category}. We accept questions and consultations regarding {category}. Please do not answer questions outside the {category} category.", but is not limited to such an example. Note that {category} includes information indicating the specified category. Further, as guidelines for answers, additional conditions that are not limited to the further specified category may be included.

[0175] In addition, the processing by the fourth generation processing unit 43 may include processing for determining whether the request indicated by the request information received by the reception unit 31 is a request regarding the first target and whether it is a request regarding the second target. Also in this case, the fourth generation processing unit 43 includes the processing of the generation unit 32 and the processing of the determination unit 35.

[0176] For example, when the request indicated by the request information received by the reception unit 31 is not a request regarding the first target and is a request regarding the second target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information. By doing so, when the request indicated by the request information is not a request regarding the first target and is a request regarding the second target, the generation AI can also be made to generate response information.

[0177] In this case, the input information includes, in addition to the instruction information, the request information, information indicating the first target, and information indicating the second target. The instruction information is, for example, information of the character string "#You are an excellent AI assistant. Determine whether the given question corresponds to the first target shown below. If the given question does not correspond to the first target, as an expert in {category}, receive and answer questions regarding {category}. Do not answer questions other than those in the {category} category." However, it is not limited to such an example. Note that {category} includes information indicating the specified category. The instruction information may also include guideline information.

[0178] 〔3.3.4. Evaluation Unit 33〕 The evaluation unit 33 evaluates the non-response accuracy for each non-response target information combination that is a combination of two or more non-response target information among the plurality of non-response target information generated by the generation unit 32.

[0179] For example, the evaluation unit 33 can evaluate the non-response accuracy for each non-response target information combination by using a plurality of pieces of evaluation requirement information stored in the evaluation requirement information storage unit 22 and acquired by the acquisition unit 30.

[0180] As described above, each piece of evaluation requirement information is information indicating a first requirement that is a requirement regarding a non-response target in the specified category, information indicating a second requirement that is a requirement regarding a response target in the specified category, or information indicating a third requirement that is a requirement regarding a non-response target regardless of the specified category.

[0181] The evaluation unit 33 determines whether or not the requirement indicated by the evaluation requirement information is determined to be a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination. The explicit non-response determination process using the non-response target information combination is, for example, a process of using each of a plurality of pieces of non-response target information included in the non-response target information combination as a first target.

[0182] When the evaluation unit 33 determines that the first requirement or the third requirement indicated by the evaluation requirement information is a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation of the non-response target information combination is set to 1, and otherwise, the evaluation of the non-response target information combination is set to 0.

[0183] Also, when the evaluation unit 33 determines that the second requirement indicated by the evaluation requirement information is not a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation of the non-response target information combination is set to 1, and otherwise, the evaluation of the non-response target information combination is set to 0.

[0184] The evaluation unit 33 can calculate the non-response accuracy by aggregating the evaluation results and dividing the aggregation result by the number of evaluations. Further, the evaluation unit 33 can also change the weight according to which of the first requirement, the second requirement, and the third requirement the requirement indicated by the evaluation requirement information is, and aggregate the evaluation results by weighted addition.

[0185] Further, the evaluation request information may be limited to the information indicating the first request, or may be limited to two or more pieces of information among the information indicating the first request, the information indicating the second request, and the information indicating the third request. Further, the evaluation is not limited to 0 and 1, and may be, for example, -1 and 1, or other combinations of values.

[0186] Further, even when the determination unit 35 determines whether or not a request indicated by the request information from each user U is a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation unit 33 can evaluate the non-response accuracy of the non-response target information combination using a language model by using keyword-based natural language processing.

[0187] For example, the evaluation unit 33 can evaluate the non-response accuracy of the non-response target information combination for each non-response target information combination by using a third language model based on the information indicating the request included in the request information from each user U and the determination result by the determination unit 35 by keyword-based natural language processing.

[0188] For example, the evaluation unit 33 inputs, as input information, information including instruction information for instructing determination as to whether or not any of two or more pieces of non-response target information indicated by the non-response target information combination is not included in the request information from the user U to the third language model, so that the third language model outputs information indicating the non-response accuracy of two or more pieces of response target information indicated by the non-response target information combination.

[0189] FIG. 12 is a diagram showing the flow of processing by the evaluation unit 33 of the processing unit 12 in the information processing apparatus 1 according to the embodiment. As shown in FIG. 12, the evaluation unit 33 determines a combination of two or more pieces of non-response target information from among a plurality of non-response target information (step S50). The combination of two or more pieces of non-response target information is determined randomly or according to a predetermined rule. Further, the combination of two or more pieces of non-response target information may be, for example, a combination including m pieces of non-response target information for each non-response category. m is an integer of 1 or more.

[0190] Subsequently, the evaluation unit 33 creates a prompt (input information) including instruction information including two or more pieces of non-response target information of the combination determined in step S50 (step S51). Then, the evaluation unit 33 performs prompt evaluation to evaluate the non-response accuracy of the prompt generated in step S51 as the non-response accuracy of the combination of two or more pieces of non-response target information determined in step S51 (step S52).

[0191] In step S52, the evaluation unit 33 inputs information including the prompt generated in step S51 and the evaluation request information into the first language model as input information for each piece of evaluation request information, and determines the non-response accuracy of the combination of two or more pieces of non-response target information determined in step S50 based on information indicating whether the first target output from the first language model for each piece of evaluation request information is a request indicating the first target or not.

[0192] Each piece of evaluation request information is, in the example shown in FIG. 12, non-response required information (category-dependent), response required information (category-dependent), or non-response required information (category-independent). The non-response required information (category-dependent) is information indicating a first request that is a request regarding a non-response target in the specified category. The response required information (category-dependent) is information indicating a second request that is a request regarding a response target in the specified category. The non-response required information (category-independent) is information indicating a third request that is a request regarding a non-response target regardless of the specified category.

[0193] Then, a selection unit 34, which will be described later, determines whether the non-response accuracy obtained in step S51 satisfies a predetermined condition based on the evaluation result in step S52 (step S53). The predetermined condition is, for example, a condition that the non-response accuracy is the highest when the process of step S50 is completed for all combinations of two or more pieces of non-response target information, or a condition that the non-response accuracy is equal to or higher than a threshold value, but is not limited to such examples.

[0194] When the selection unit 34 determines that the non-response accuracy obtained in step S51 satisfies a predetermined condition (step S53: Yes), the selection unit 34 selects, as two or more pieces of non-response target information to be used by the determination unit 35, combinations of two or more pieces of non-response target information for which the non-response accuracy obtained in step S51 satisfies the predetermined condition.

[0195] When the evaluation unit 33 determines, based on the selection by the selection unit 34, that the non-response accuracy obtained in step S51 does not satisfy the predetermined condition (step S53: No), the evaluation unit 33 transfers the process to step S50.

[0196] 〔3.3.5. Selection Unit 34〕 Based on the non-response accuracy for each combination of two or more pieces of non-response target information among a plurality of pieces of non-response target information, the selection unit 34 selects two or more pieces of non-response target information to be used by the determination unit 35 from among the plurality of pieces of non-response target information. The plurality of pieces of non-response target information are, for example, non-response category information indicating a non-response category, non-response target information generated by the generation unit 32, and the like.

[0197] For example, the selection unit 34 selects two or more pieces of non-response target information to be used by the determination unit 35 based on the evaluation result by the evaluation unit 33. For example, the selection unit 34 selects, as the combination of two or more pieces of non-response target information to be used by the determination unit 35, the combination of non-response target information for which the non-response accuracy is the highest among the non-response accuracies for each combination of non-response target information.

[0198] Further, the selection unit 34 can also select, as the combination of two or more pieces of non-response target information to be used by the determination unit 35, combinations of non-response target information for which the non-response accuracy for each combination of non-response target information is equal to or higher than a threshold value, randomly or according to a predetermined rule.

[0199] 〔3.3.6. Determination Unit 35〕 The determination unit 35 performs various determinations. For example, the determination unit 35 determines whether to generate response information, which is information indicating a response to a request indicated by the request information received by the reception unit 31.

[0200] Based on whether the request indicated by the request information received by the reception unit 31 is a request related to a first target set as a non-response target and a request related to a second target set as a response target, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information received by the reception unit 31.

[0201] The first target is, for example, a target set as a non-response target for a specified category that is a category specified by the user U, a non-response category pre-associated with the specified category, a non-response target indicated by non-response target information generated by the generation unit 32, and the like.

[0202] The non-response category is, for example, another category (a category other than the specified category) whose boundary with the specified category is ambiguous and the probability of an appropriate response is equal to or lower than a threshold value, and is set in advance for each specified category, but is not limited to such an example.

[0203] For example, based on two or more pieces of non-response target information selected by the selection unit 34, the determination unit 35 performs an explicit non-response determination process of determining whether the request indicated by the request information received by the reception unit 31 is a request related to a first target that is a non-response target.

[0204] The two or more pieces of non-response target information selected by the selection unit 34 are two or more pieces of information among a plurality of non-response category information each indicating a non-response category preset for the specified category and a plurality of pieces of non-response target information generated by the generation unit 32.

[0205] The first target set as a non-response target includes, for example, in addition to the target set as a non-response target for the specified category, a target that does not depend on the specified category. The target that does not depend on the specified category is, for example, a violation of the law, a violation of social morals, slander, defamation, etc., but is not limited to such examples.

[0206] In addition, the objects that depend on the specified category and the objects that do not depend on the specified category may include objects for which a response using generative AI is not appropriate. Examples of objects for which a response using generative AI is not appropriate include, but are not limited to, medical, health, finance, etc.

[0207] The determination unit 35 can determine, for example, using a first language model, whether the request indicated by the request information received by the reception unit 31 is a request regarding a first object set as a non-response target. The first language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples.

[0208] The determination unit 35 inputs, as input information to the first language model, information including the instruction information instructing the output of information indicating whether the request indicated by the request information received by the reception unit 31 is a request regarding the first object and information indicating the request indicated by the request information included in the usage request. In this case, the first language model outputs information indicating whether the request is a request regarding the first object. In this way, the determination unit 35 can determine, using the first language model, whether the request indicated by the request information is a request regarding the first object set as a non-response target.

[0209] The instruction information includes information indicating the output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the first object and information to be output when the request indicated by the request information is not a request regarding the first object.

[0210] The information to be output when the request indicated by the request information is a request regarding the first object is, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the first object is, for example, "false", but is not limited to such examples.

[0211] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the first target. For example, the information indicating the output format is information such as the character string "When the request indicated by the request information is a request regarding the first target, output 'applicable', and if not, output nothing", but is not limited to such an example. Also, the information indicating the output format may be information indicating a sample of the output.

[0212] FIG. 13 is a diagram showing an example of instruction information used in the explicit non-response determination process by the determination unit 35 of the processing unit 12 in the information processing apparatus 1 according to the embodiment. In the instruction information shown in FIG. 13, although details are omitted, the information corresponding to "Condition 1" includes, as information indicating a question regarding the first target, information indicating a question regarding the non-response category, and the information corresponding to "Condition 2" includes, as information indicating a question regarding the first target, non-response target information generated by the generation unit 32, but is not limited to such an example.

[0213] In the above-described example, the input information input to the first language model includes instruction information, but the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information. Also, the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information and the information indicating the first target. In these cases, the language model is generated for each category, for example, but is not limited to such an example.

[0214] The determination unit 35 can also determine, in natural language processing that does not use the first language model, instead of or in addition to the first language model, whether the request indicated by the request information is a request regarding a first target set as a non-response target. Natural language processing that does not use the first language model is, for example, keyword-based natural language processing. The determination unit 35 has, for example, a first target dictionary including a plurality of keywords that directly or indirectly indicate each of the first targets, and determines whether the keywords included in the first target dictionary are included in the request information.

[0215] When the keywords included in the first target dictionary are included in the request information, the determination unit 35 determines that the request indicated by the request information is a request regarding the first target set as a non-response target, and otherwise determines that the request indicated by the request information is not a request regarding the first target set as a non-response target.

[0216] Also, the keyword-based natural language processing may be natural language processing using a model. The model in this case is a model generated by machine learning using learning information including input information and information (label information) indicating whether the request indicated by the input information is a request regarding the first target for each input information. Such a model is, for example, GBDT, a neural network, etc., but is not limited to such examples.

[0217] When the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is not a request regarding the first target, the determination unit 35 performs implicit non-response determination processing to determine whether the request indicated by the request information is a request regarding a second target set as a response target.

[0218] As described above, the second target is a target set as a response target. By determining whether or not it is a request regarding the second target, it is determined whether or not it is a non-response target, and it can be said that it is a target that implicitly indicates a non-response target. Therefore, as described above, the second target that is a response target can be said to implicitly indicate a non-response determination criterion, and can be called an implicit non-response determination criterion. The second target includes, for example, a specified category if there is a category specified by the user U.

[0219] For example, the determination unit 35 can determine whether or not the request indicated by the request information is a request regarding the second target set as the response target using the second language model. The second language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples. Note that the second language model may be the same language model as the first language model.

[0220] The determination unit 35 inputs, as input information, information including instruction information for instructing the output of information indicating whether or not the request indicated by the request information is a request regarding the second target and information indicating the request indicated by the request information included in the usage request to the second language model.

[0221] In this case, information indicating whether or not it is a request regarding the second target is output from the second language model. In this way, the determination unit 35 can determine whether or not the request indicated by the request information is a request regarding the second target set as the response target using the second language model.

[0222] The instruction information includes information indicating the second target, but the information indicating the second target may be included in the input information separately from the instruction information. Further, the instruction information may include information indicating the request included in the request information.

[0223] The indication information includes information indicating an output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the second target, and information to be output when the request indicated by the request information is not a request regarding the second target.

[0224] The information to be output when the request indicated by the request information is a request regarding the second target is, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the second target is, for example, "false", but is not limited to such examples.

[0225] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the second target. For example, the information indicating the output format is information in the character string "When the request indicated by the request information is a request regarding the second target, output 'corresponding', and otherwise, do not output anything", but is not limited to such examples. Also, the information indicating the output format may be information indicating a sample of the output.

[0226] In the above example, the indication information is included in the input information input to the second language model, but the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the indication information.

[0227] Also, the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the indication information and the information indicating the second target.

[0228] The determination unit 35 can also determine, instead of or in addition to the second language model, by natural language processing without using the second language model, whether the request indicated by the request information is a request regarding the second target set as the response target.

[0229] Natural language processing without using a second language model is, for example, keyword-based natural language processing. The determination unit 35 has, for example, a second target dictionary including a plurality of keywords each directly or indirectly indicating a second target, and determines whether a keyword included in the second target dictionary is included in the request information.

[0230] When a keyword included in the second target dictionary is included in the request information, the determination unit 35 determines that the request indicated by the request information is a request regarding the second target set as a response target, and otherwise determines that the request indicated by the request information is not a request regarding the second target set as a response target.

[0231] Also, keyword-based natural language processing may be natural language processing using a model. The model in this case is a model generated by machine learning using learning information including input information and information (label information) indicating whether the request indicated by the input information is a request regarding the second target for each input information. Such a model is, for example, GBDT, a neural network, etc., but is not limited to such examples.

[0232] Subsequently, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information received by the reception unit 31 based on the determination result as to whether the request indicated by the request information received by the reception unit 31 is a request regarding the second target.

[0233] For example, when the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is a request regarding the second target, it determines to generate information indicating a response to the request indicated by the request information.

[0234] Also, when the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is not a request regarding the second target, it determines not to generate information indicating a response to the request indicated by the request information.

[0235] In this way, the determination unit 35 first performs explicit non-response determination processing. When the explicit non-response determination criteria are not satisfied, it performs implicit non-response determination processing. When the implicit non-response determination criteria are not satisfied, it makes a response. As a result, the determination unit 35 can perform response availability determination with low cost and high accuracy, and can suppress inappropriate responses.

[0236] Note that the determination unit 35 can perform the explicit non-response determination processing and the implicit non-response determination processing in parallel. In this case, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information based on the determination result of the explicit non-response determination processing and the determination result of the implicit non-response determination processing.

[0237] In this case, when the request indicated by the request information is not a request related to the first target and the request indicated by the request information is a request related to the second target, the determination unit 35 determines to generate information indicating a response to the request indicated by the request information. Otherwise, the determination unit 35 determines not to generate information indicating a response to the request indicated by the request information. Also by this means, the information processing apparatus 1 can suppress inappropriate responses.

[0238] Also, the determination unit 35 can perform the processing in the order of implicit non-response determination processing and explicit non-response determination processing. In this case, the explicit non-response determination processing can be performed only when it is determined in the implicit non-response determination processing that the request indicated by the request information is a request related to the second target. Also by this means, the information processing apparatus 1 can suppress inappropriate responses.

[0239] In this way, the determination unit 35 can determine whether to generate information indicating a response to the request indicated by the request information based on whether the request indicated by the request information is a request related to the first target set as a non-response target and a request related to the second target set as a response target.

[0240] 〔3.3.7. Providing Unit 36〕 The providing unit 36 provides various types of information to the user U and the employee O. For example, the providing unit 36 provides various types of information to the user U by transmitting the various types of information to the terminal device 2 via the communication unit 10 and the network N. Also, the providing unit 36 provides various types of information to the employee O by transmitting the various types of information to the terminal device 3 via the communication unit 10 and the network N.

[0241] When it is determined by the determination unit 35 that the response information is to be generated, the providing unit 36 provides the user U with the response information generated using the generation AI as information indicating the response to the request indicated by the request information received by the reception unit 31.

[0242] For example, the providing unit 36 provides the user U with the response information generated using the generation AI by transmitting the response information generated using the generation AI generated by the generation unit 32 to the terminal device 2 that transmitted the usage request.

[0243] For example, when the request indicated by the request information is a question of the Q&A service and the user U accesses the page showing the answer to the user U's question, the providing unit 36 can provide the user U with the page showing the answer to the user U's question.

[0244] Also, when the response information is not generated using the generation AI in the generation unit 32, the providing unit 36 provides the user U with non-response information. For example, when it is determined by the determination unit 35 that the request indicated by the request information is a request regarding the first target, or when it is determined by the determination unit 35 that the request indicated by the request information is not a request regarding the second target, the providing unit 36 provides the user U with non-response information.

[0245] The non-response information is, for example, the information of the character string "Sorry. This is a non-response target and I cannot respond to the request." For example, when the request indicated by the request information is a request for a Q&A service, the non-response information is, for example, the information of the character string "Sorry. I cannot answer this question as it is outside the scope of the answers." Note that the non-response information may include information indicating that the request is related to the first target and not related to the second target.

[0246] [4. Processing Procedure] Next, the procedure of information processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment will be described. FIG. 14 is a flowchart showing an example of information processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0247] As shown in FIG. 14, the processing unit 12 of the information processing apparatus 1 determines whether or not it has received request information (step S10). When the processing unit 12 determines that it has received the request information (step S10: Yes), it performs request processing (step S11). The request processing in step S11 is the processing of steps S20 to S24 shown in FIG. 15, which will be described in detail later.

[0248] When the processing in step S11 is completed, or when the processing unit 12 determines that it has not received the request information (step S10: No), it determines whether or not it is the timing to generate non-response target information (step S12). The timing to generate non-response target information is, for example, the timing when a generation request comes from the terminal device 3 or the timing that arrives at a predetermined cycle, but is not limited to such an example.

[0249] When the processing unit 12 determines that it is the timing to generate non-response target information (step S12: Yes), it performs non-response target information generation processing (step S13). The non-response target information generation processing in step S13 is the processing of steps S30 to S33 shown in FIG. 16, which will be described in detail later.

[0250] When the process in step S13 ends, or when it is determined that it is not the non-response target information generation timing (step S12: No), the processing unit 12 determines whether it is the operation end timing (step S14). For example, the processing unit 12 determines that it is the operation end timing when the power of the information processing apparatus 1 is turned off.

[0251] When it is determined that it is not the operation end timing (step S14: No), the processing unit 12 shifts the process to step S10. When it is determined that it is the operation end timing (step S14: Yes), the processing shown in FIG. 14 ends.

[0252] FIG. 15 is a flowchart showing an example of the request process by the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 15, the processing unit 12 of the information processing apparatus 1 determines whether the request indicated by the request information received in step S10 is a request regarding the first target (step S20).

[0253] Subsequently, when it is determined that the request indicated by the request information is not a request regarding the first target (step S20: No), the processing unit 12 determines whether the request indicated by the request information is a request regarding the second target (step S21). When it is determined that the request indicated by the request information is a request regarding the second target (step S21: Yes), the processing unit 12 generates response information which is information indicating a response corresponding to the request indicated by the request information (step S22). Then, the response information generated in step S22 is provided (step S23).

[0254] When it is determined that the request indicated by the request information is a request regarding the first target (step S20: Yes), or when it is determined that the request indicated by the request information is not a request regarding the second target (step S21: No), the processing unit 12 provides non-response information (step S24).

[0255] FIG. 16 is a flowchart showing an example of non-response target information generation processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 16, the processing unit 12 determines a combination of two or more pieces of non-response target information among a plurality of pieces of non-response target information (step S30).

[0256] Subsequently, the processing unit 12 determines the non-response accuracy of the combination of two or more pieces of non-response target information (step S31). Then, the processing unit 12 determines whether or not all combinations of the combination of two or more pieces of non-response target information have been determined (step S32).

[0257] When the processing unit 12 determines that not all combinations have been determined (step S32: No), it proceeds to step S30. Also, when the processing unit 12 determines that all combinations have been determined (step S32: Yes), it selects the combination of two or more pieces of non-response target information with the highest non-response accuracy as information to be used in the explicit non-response determination processing (step S33), and ends the processing shown in FIG. 16.

[0258] 〔5. Modification Example〕 In the above-described example, the generation AI has been described as a text generation AI. However, the generation AI is not limited to the text generation AI, and may be an image generation AI, a multimodal generation AI, or the like. The image generation AI is an AI that generates an image from text. For example, it includes, but is not limited to, StackGAN (Generative Adversarial Networks), AttnGAN, T2I (Text-to-Image) with Transformers, Diffusion models, etc. Examples of Diffusion models include DALL-E and Stable-Diffusion.

[0259] The multimodal generation AI is a generation AI that generates at least one of text, images, and audio from at least one of text, images, and audio. The multimodal generation AI includes, for example, GPT-4 Turbo with vision, gemini, CM3Leon (Chameleon Multimodal Model), etc., but is not limited to such examples.

[0260] In addition, in the above example, an example in which a non-response category corresponding to a specified category is preset as the first target has been described. However, the present invention is not limited to such an example, and instead of the non-response category corresponding to the specified category, a non-response target corresponding to the specified category may be preset as the first target. Also in this case, the generation unit 32 can generate non-response target information by the same process as in the case of the non-response category based on the information indicating the non-response target preset as the first target for the specified category.

[0261] 〔6. Hardware Configuration〕 The information processing apparatus 1 according to the above-described embodiment is realized by a computer 80 configured as shown in FIG. 17, for example. FIG. 17 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing apparatus 1 according to the embodiment. The computer 80 includes a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0262] The CPU 81 operates based on a program stored in the ROM 83 or the HDD 84 and controls each part. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 is started up, a program dependent on the hardware of the computer 80, and the like.

[0263] The HDD 84 stores programs executed by the CPU 81, data used by such programs, and the like. The communication interface 85 receives data from other devices via the network N (see FIG. 2), sends it to the CPU 81, and transmits the data generated by the CPU 81 to other devices via the network N.

[0264] The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards or mice, via the input / output interface 86. The CPU 81 acquires data from the input device via the input / output interface 86. Also, the CPU 81 outputs the data generated via the input / output interface 86 to the output device.

[0265] The media interface 87 reads a program or data stored in the recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads such a program from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0266] For example, when the computer 80 functions as the information processing apparatus 1 according to the embodiment, the CPU 81 of the computer 80 realizes the functions of the processing unit 12 by executing the program loaded onto the RAM 82. Also, the data in the storage unit 11 is stored in the HDD 84. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be acquired from other devices via the network N.

[0267] 〔7. Others〕 Also, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0268] Also, each component of each device shown in the drawings is a functional concept and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the distribution and integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0269] For example, the above-described information processing apparatus 1 may be realized by a terminal device and a server computer, or may be realized by a plurality of server computers. Also, depending on the function, the configuration can be flexibly changed, such as by calling an external platform or the like using an API or network computing.

[0270] Also, the above-described embodiments and modifications can be appropriately combined as long as the processing contents do not conflict.

[0271] 〔8. Effects〕 As described above, the information processing apparatus 1 according to the embodiment includes a reception unit 31, a determination unit 35, and a provision unit 36. The reception unit 31 receives information indicating a request from the user U. The determination unit 35 determines whether to generate information indicating a response to the request indicated by the information received by the reception unit 31. The provision unit 36 includes a provision unit 36 that, when it is determined by the determination unit 35 that information indicating a response is to be generated, provides the user U with response information generated using a generation AI as information indicating a response to the request. The determination unit 35 determines whether to generate information indicating a response to the request indicated by the information received by the reception unit 31 based on whether the request indicated by the information received by the reception unit 31 is a request related to a first target set as a non-response target and a request related to a second target set as a response target. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0272] Further, the first target includes a target set as a non-response target for a category specified by the user U. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0273] Further, the second target includes a category specified by the user U. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0274] Further, the determination unit 35 determines whether the request indicated by the information received by the reception unit 31 is a request related to the first target using a first language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0275] Further, the determination unit 35 determines whether the request indicated by the information received by the reception unit 31 is a request related to the second target using a second language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0276] The information processing apparatus 1 also includes a generation unit 32 that generates information indicating a first target using a third language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0277] Also, the generation unit 32 inputs information including non-response category information indicating a non-response category that is different from the specified category specified by the user U and is associated in advance as a non-response category to the specified category, into the language model, and causes the language model to generate information indicating risk as information indicating the first target. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0278] As described above, the embodiments of the present application have been described in detail with reference to the drawings. However, this is an example, and the present invention can be implemented in other forms in which various modifications and improvements are made based on the knowledge of those skilled in the art, including the aspects described in the column of the disclosure of the invention.

[0279] Also, the "section (section, module, unit)" described above can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

Description of Reference Numerals

[0280] 1 Information processing apparatus 2, 3 Terminal device 10 Communication unit 11 Storage unit 12 Processing unit 20 User information storage unit 21 Non-response target information storage unit 22 Evaluation request information storage unit 30 Acquisition unit 31 Reception unit 32 Generation unit 33 Evaluation unit 34 Selection unit 35 Determination unit 36 Provision unit 40 First generation processing unit 41 Second generation processing unit 42 Third generation processing unit 43 Fourth generation processing unit 100 Information processing system N Network

Claims

1. a receiving unit that receives information indicating a user's request; a determination unit that determines whether to generate information indicating a response to the request indicated by the information received by the receiving unit; a providing unit that, when it is determined by the determination unit that information indicating the response is to be generated, provides the user with response information generated using a generation AI as information indicating a response to the request; and the determination unit determines whether to generate information indicating a response to the request indicated by the information received by the receiving unit based on whether the request indicated by the information received by the receiving unit is a request related to a first target set as a non-response target and a request related to a second target set as a response target An information processing apparatus characterized by the above.

2. The first target includes a target set as a non-response target for a category specified by the user The information processing apparatus according to claim 1, characterized by the above.

3. The second target includes the category specified by the user The information processing apparatus according to claim 1, characterized by the above.

4. The determination unit determines whether the request indicated by the information received by the receiving unit is a request related to the first target using a first language model The information processing apparatus according to any one of claims 1 to 3, characterized by the above.

5. The determination unit determines whether the request indicated by the information received by the receiving unit is a request related to the second target using a second language model The information processing apparatus according to any one of claims 1 to 3, characterized by the above.

6. A generation unit that generates information indicating the first target using a third language model The information processing apparatus according to claim 4, characterized by the above.

7. The generation unit inputs information including non-response category information indicating a non-response category that is a category different from the specified category specified by the user and is associated in advance as a non-response category to the specified category into a language model, and causes the language model to generate information indicating a risk as information indicating the first target The information processing apparatus according to claim 6, characterized by the above.

8. An information processing method executed by a computer, comprising: a receiving step of receiving information indicating a user's request; A determination step of determining whether to generate information indicating a response to a request indicated by the information received in the reception step; When it is determined in the determination step that information indicating the response is to be generated, a provision step of providing, to the user, response information generated using a generation AI as information indicating a response to the request, including: The determination step is: Based on whether the request indicated by the information received in the reception step is a request related to a first target set as a non-response target and a request related to a second target set as a response target, it is determined whether to generate information indicating a response to the request indicated by the information received in the reception step An information processing method characterized by the above.

9. A reception procedure for receiving information indicating a user's request; A determination procedure for determining whether to generate information indicating a response to a request indicated by the information received in the reception procedure; When it is determined in the determination procedure that information indicating the response is to be generated, causing a computer to execute a provision procedure of providing, to the user, response information generated using a generation AI as information indicating a response to the request; The determination procedure is: Based on whether the request indicated by the information received in the reception procedure is a request related to a first target set as a non-response target and a request related to a second target set as a response target, it is determined whether to generate information indicating a response to the request indicated by the information received in the reception procedure An information processing program characterized by the above.

Images