Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device capable of suppressing an inappropriate response, an information processing method, and an information processing program.SOLUTION: An information processing device comprises a generation unit, an acceptance unit, and a determination unit. The generation unit generates non-response target information indicative of a non-response target. The acceptance unit accepts information indicative of a user's request. The determination unit determines whether or not the request indicated by the information accepted by the acceptance unit is a request on the non-response target on the basis of the plurality of pieces of non-response target information generated by the generation unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, technologies for responding to requests from users using language models such as transformer models have been developed. For example, Patent Document 1 discloses a technology for generating an answer to a question using a language model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, no consideration is given to how to handle cases where responses such as answers are not appropriate, and there is room for improvement.

[0005] The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of suppressing inappropriate responses.

Means for Solving the Problems

[0006] The information processing apparatus according to the present application includes a generation unit, a reception unit, and a determination unit. The generation unit generates non-response target information indicating a non-response target. The reception unit receives information indicating a request from a user. The determination unit determines, based on a plurality of pieces of non-response target information generated by the generation unit, whether the request indicated by the information received by the reception unit is a request related to a non-response target.

Effects of the Invention

[0007] According to one aspect of the embodiment, it has an effect of suppressing inappropriate responses.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments for implementing an information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. Also, each embodiment can be appropriately combined as long as the processing contents do not conflict. In addition, in the following embodiments, the same parts are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] 〔1. Example of Information Processing〕 First, an example of information processing according to an embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram for explaining the information processing according to the embodiment.

[0011] The information processing apparatus 1 shown in FIG. 1 is an information processing apparatus that cooperates with each terminal device 2 of the user U and provides various services to the user U online, and is realized by, for example, one or more servers or a cloud system, etc. The terminal device 2 is, for example, a smartphone, a tablet, or a personal computer, etc.

[0012] The services provided by the information processing apparatus 1 are, for example, various services such as Q&A (Question and Answer) services like Yahoo! Chiebukuro, content creation services, customer support services, learning support services, etc., but are not limited to such examples.

[0013] In various services, the information processing apparatus 1 generates response information, which is information indicating a response corresponding to the request of the user U, using a generation AI (Artificial Intelligence), and provides the generated response information to the user U.

[0014] The generation AI is, for example, a text generation AI. The text generation AI is, for example, a large language model trained to estimate and output the next token from the input token sequence, and is, for example, a Transformer-based model, an RNN (Recurrent Neural Network)-based model, etc., but may also be a hybrid model of these, etc. Also, the text generation AI may be a composite system combined with an identification mechanism for preventing unauthorized use, etc.

[0015] The Transformer-based model is, for example, GPT (Generative Pre-trained Transformer) (registered trademark), PaLM2 (Pathways Language Model Version 2), or LLaMA (Large Language Model Meta AI), etc., but is not limited to such examples. The RNN-based model is, for example, RWKV (Receptance Weighted Key Value), etc., but is not limited to such examples.

[0016] Note that it is desirable for the generative AI to be trained so as not to include personal information or the like in its generation results. The generative AI is arranged in an external information processing device, and the information processing device 1 uses the generative AI via an API (Application Programming Interface). However, the generative AI may be arranged within the information processing device 1.

[0017] As shown in FIG. 1, the information processing device 1 receives request information, which is information indicating the request of the user U (step S1). The information processing device 1 receives request information, which is information indicating the request of the user U, for example, by receiving a usage request from the terminal device 2.

[0018] The usage request includes request information. The request information includes, for example, information indicating a question of the user U, information indicating an instruction of the user U, or information indicating a desire of the user U, but is not limited to such examples.

[0019] Further, the request information includes designation information for designating the category of the request. For example, when the question of the user U is a question in a Q&A service or a customer support service, the usage request includes designation information for designating the category of the question.

[0020] Further, when the instruction of the user U is a content creation instruction in a content creation service, the usage request includes designation information for designating the category of the content to be created. Further, when the instruction of the user U is an instruction for interactive learning in a learning support service, the usage request includes designation information for designating a subject as the category of learning.

[0021] The designation information is, for example, information indicating the category of the request or information corresponding to the category of the request. The information corresponding to the category of the request is, for example, information that is part or all of the URL (Uniform Resource Locator) or domain name of the category in the service provided by the information processing device 1, but is not limited to such examples.

[0022] Subsequently, the information processing apparatus 1 determines whether to generate information indicating a response to the request indicated by the request information received in step S1 (step S2). In the following, it will be described on the assumption that the request indicated by the request information received in step S1 is a question in the Q&A service.

[0023] For example, the information processing apparatus 1 performs an explicit non-response determination process which is a process of determining whether the request indicated by the request information received in step S1 is a request regarding a first target set as a non-response target (step S2-1). The first target is a target explicitly set as a non-response target.

[0024] The non-response target is a target that does not give a response corresponding to the request indicated by the request information. A plurality of first targets may be set as non-response targets, or there may be one. Such a first target is a target explicitly indicated as a non-response target, and thus, the first target can be said to be a determination criterion for specifying a non-response target, and can be called an explicit non-response determination criterion.

[0025] The first target which is an explicit exclusion determination criterion includes, for example, a target set as a non-response target for a specified category which is a category specified by the user U. The target set as a non-response target for the specified category is, for example, a target that depends on the specified category. The information processing apparatus 1 specifies the specified category based on the specified information included in the usage request.

[0026] The target that depends on the specified category is, for example, another category (a category other than the specified category) for which a response is not appropriate in the specified category, another category (a category other than the specified category) where the boundary with the specified category is ambiguous and the probability of an appropriate response is equal to or less than a threshold value, etc., and may be described as a non-response category in the following. The non-response category is preset for each specified category.

[0027] In addition, the first target set as a non-response target includes, for example, targets that are independent of the specified category in addition to the targets set as non-response targets for the specified category. Targets that are independent of the specified category are, for example, but not limited to, violations of laws, violations of social morals, slander, and defamation.

[0028] In addition, the targets that depend on the specified category and the targets that do not depend on the specified category may include targets for which responses using generative AI are not appropriate. Targets for which responses using generative AI are not appropriate are, for example, but not limited to, medical, health, finance, etc.

[0029] The information processing device 1 can determine, for example, using a language model, whether the request indicated by the request information is a request related to the first target set as a non-response target. Such a language model is an example of a first language model and may be described as the first language model hereinafter. The first language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples.

[0030] The information processing device 1 inputs, as input information, information including instruction information for instructing the output of information indicating whether the request indicated by the request information included in the usage request is a request related to the first target and information indicating the request included in the request information to the first language model.

[0031] In this case, information indicating whether the request is a request related to the first target is output from the first language model. In this way, the information processing device 1 can determine, using the first language model, whether the request indicated by the request information is a request related to the first target set as a non-response target.

[0032] The input information input into AI such as a language model is called a prompt. In the following, the input information input into the language model may be described as a prompt. A prompt is, for example, information indicating instructions, requests, etc. given to an AI such as a language model to execute a specific task for the AI such as a language model.

[0033] For example, when the request indicated by the request information is a question in a specific category (designated category) in a Q&A service, the instruction information is information such as the string "You are an excellent filter. Please judge whether it meets the conditions for the given question.\n#Conditions\nReturn 'true' when it meets the following, and return 'false' when it does not.\nQuestions regarding {category1}\nQuestions regarding {category2},...". {category1} and {category2} are, for example, information indicating non-response categories set for the designated category.

[0034] The instruction information includes information indicating a first target, but the information indicating the first target may be included in the input information separately from the instruction information. Also, the instruction information may include information indicating the request included in the request information.

[0035] In addition, the instruction information includes information indicating an output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the first target, and information to be output when the request indicated by the request information is not a request regarding the first target.

[0036] The information to be output when the request indicated by the request information is a request regarding the first target is information indicating a non-response determination, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the first target is information indicating a response determination, for example, "false", but is not limited to such examples.

[0037] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the first target. In this case, the specific information is information indicating a non-response determination, and the fact that the specific information is not output becomes information indicating a response determination. For example, the information indicating the output format is information in the form of the string "When the request indicated by the request information is a request regarding the first target, output 'corresponding', and if not, do not output anything", but is not limited to such an example. Also, the information indicating the output format may be information indicating a sample of the output.

[0038] In the above example, the input information input to the first language model includes the instruction information, but the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information. Also, the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information and the information indicating the first target. In these cases, the language model is generated for each category, for example, but is not limited to such an example.

[0039] Instead of or in addition to the first language model, the information processing apparatus 1 can also determine whether the request indicated by the request information is a request regarding the first target set as a non-response target by natural language processing that does not use the first language model. The natural language processing that does not use the first language model is, for example, keyword-based natural language processing. The information processing apparatus 1 has, for example, a first target dictionary including a plurality of keywords each directly or indirectly indicating the first target, and determines whether the keywords included in the first target dictionary are included in the request information.

[0040] When the keyword included in the first target dictionary is included in the request information, the information processing apparatus 1 determines that the request indicated by the request information is a request regarding the first target set as a non-response target, and otherwise determines that the request indicated by the request information is not a request regarding the first target set as a non-response target.

[0041] In addition, keyword-based natural language processing may also be natural language processing using a model other than a large language model. In this case, the model is a model generated by machine learning using learning information that includes input information and information (label information) indicating whether the request indicated by the input information is a request related to the first target for each input information. Such models include, for example, GBDT (Gradient Boosting Decision Tree), neural networks, etc., but are not limited to such examples.

[0042] Subsequently, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is not a request related to the first target, the information processing apparatus 1 performs an implicit non-response determination process, which is a process of determining whether the request indicated by the request information is a request related to the second target set as the response target (step S2-2).

[0043] The second target is the target set as the response target. By determining whether it is not a request related to the second target, it is determined whether it is not a non-response target, and it can be said that it is a target implicitly indicating a non-response target. Therefore, it can be said that the second target that becomes the response target implicitly indicates the non-response determination criterion, and can be called the implicit non-response determination criterion. The second target includes, for example, the designated category if there is a category designated by the user U.

[0044] The information processing apparatus 1 can determine, for example, using a language model, whether the request indicated by the request information is a request related to the second target set as the response target. Such a language model is an example of a second language model, and may be referred to as the second language model hereinafter. The second language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples. Note that the second language model may be the same language model as the first language model.

[0045] The information processing apparatus 1 inputs, as input information, information including instruction information for instructing output of information indicating whether the request indicated by the request information included in the usage request is a request regarding a second target, and information indicating the request indicated by the request information included in the usage request, to a second language model.

[0046] In this case, the second language model outputs information indicating whether the request is a request regarding the second target. In this way, the information processing apparatus 1 can determine, using the second language model, whether the request indicated by the request information is a request regarding the second target set as the response target.

[0047] The instruction information is, for example, when the request indicated by the request information is a question in a specific category (designated category) in a Q&A service, information such as the character string "You are an expert in {category}. When it corresponds to a question regarding {category}, return "true", and when it does not correspond to a question regarding {category}, do not return "false".". {category} is the designated category.

[0048] The instruction information includes information indicating the second target, but the information indicating the second target may be included in the input information separately from the instruction information. Further, the instruction information may include information indicating the request included in the request information.

[0049] Also, the instruction information includes information indicating an output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the second target, and information to be output when the request indicated by the request information is not a request regarding the second target.

[0050] The information to be output when the request indicated by the request information is a request regarding the second target is information indicating a response determination, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the second target is information indicating a non-response determination, for example, "false", but is not limited to such examples.

[0051] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the second target. In this case, the specific information is information indicating a response determination, and the non-output of the specific information becomes information indicating a non-response determination.

[0052] For example, the information indicating the output format may be information in the form of the character string "When the request indicated by the request information is a request regarding the second target, output 'corresponding'; otherwise, output nothing", but is not limited to such an example. Also, the information indicating the output format may be information indicating a sample of the output.

[0053] In the example described above, the input information input to the second language model includes the instruction information, but the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the instruction information.

[0054] Also, the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the instruction information and the information indicating the second target. In these cases, the language model is generated for each category, for example, but is not limited to such an example.

[0055] Instead of or in addition to the second language model, the information processing apparatus 1 can also determine whether the request indicated by the request information is a request regarding the second target set as the response target by natural language processing that does not use the second language model.

[0056] The natural language processing that does not use the second language model is, for example, keyword-based natural language processing. The information processing apparatus 1 has, for example, a second target dictionary including a plurality of keywords that directly or indirectly indicate the second target, and determines whether the keywords included in the second target dictionary are included in the request information.

[0057] When the keyword included in the second target dictionary is included in the request information, the information processing apparatus 1 determines that the request indicated by the request information is a request related to the second target set as the response target, and if not, determines that the request indicated by the request information is not a request related to the second target set as the response target.

[0058] Also, the keyword-based natural language processing may be natural language processing using a model other than a large language model. In this case, the model is a model generated by machine learning using learning information including, for each input information, the input information and information (label information) indicating whether the request indicated by the input information is a request related to the second target. Such a model is, for example, GBDT, a neural network, etc., but is not limited to such examples.

[0059] Subsequently, the information processing apparatus 1 determines whether to generate response information, which is information indicating a response to the request indicated by the request information received in step S1, based on the determination result as to whether the request indicated by the request information in step S2-2 is a request related to the second target (step S2-3).

[0060] For example, when the information processing apparatus 1 determines in step S2-2 that the request indicated by the request information is a request related to the second target, it determines to generate information indicating a response to the request indicated by the request information.

[0061] Also, when the information processing apparatus 1 determines in step S2-2 that the request indicated by the request information is not a request related to the second target, it determines not to generate information indicating a response to the request indicated by the request information.

[0062] In this way, the information processing apparatus 1 first performs explicit non-response determination processing, performs implicit non-response determination processing when the explicit non-response determination criteria are not satisfied, and performs a response when the implicit non-response determination criteria are not satisfied. Thereby, the information processing apparatus 1 can perform response availability determination with low cost and high accuracy, and can suppress inappropriate responses.

[0063] Note that the information processing apparatus 1 can perform explicit non-response determination processing and implicit non-response determination processing in parallel. In this case, the information processing apparatus 1 determines whether to generate information indicating a response to the request indicated by the request information based on the determination result of the explicit non-response determination processing and the determination result of the implicit non-response determination processing.

[0064] In this case, when the request indicated by the request information is not a request regarding the first target and the request indicated by the request information is a request regarding the second target, the information processing apparatus 1 determines to generate information indicating a response to the request indicated by the request information, and otherwise determines not to generate information indicating a response to the request indicated by the request information. Also by this, the information processing apparatus 1 can suppress inappropriate responses.

[0065] Further, the information processing apparatus 1 can also perform the processing in the order of implicit non-response determination processing and explicit non-response determination processing. In this case, the explicit non-response determination processing can be performed only when it is determined in the implicit non-response determination processing that the request indicated by the request information is a request regarding the second target. Also by this, the information processing apparatus 1 can suppress inappropriate responses.

[0066] In this way, the information processing apparatus 1 can determine whether to generate information indicating a response to the request indicated by the request information based on whether the request indicated by the request information is a request regarding the first target set as a non-response target and a request regarding the second target set as a response target.

[0067] Subsequently, in step S2, when it is determined that the information processing apparatus 1 generates information indicating a response to the request indicated by the request information, the information processing apparatus 1 generates response information using the generation AI as the information indicating a response to the request indicated by the request information (step S3).

[0068] For example, the information processing apparatus 1 inputs, as input information to the generation AI, information including instruction information for instructing the output of information indicating a response to a request indicated by the request information and information indicating the request included in the request information, and causes the generation AI to generate response information.

[0069] The request indicated by the request information is, for example, a question in the category of economy in the Q&A service, and is assumed to be information of the character string "What is inflation? How does it affect the economy?". In this case, the information processing apparatus 1 includes, as the instruction information, information of the character string "As an expert in {category}, you accept and answer questions regarding {category}.", and inputs, as input information to the generation AI, information including the information of the character string "What is inflation? How does it affect the economy?" as the information indicating the request included in the request information.

[0070] Note that {category} includes information indicating the designated category. Further, the instruction information may include guideline information indicating a guideline for an answer. The guideline information includes, for example, information instructing clarity or neutrality of the answer.

[0071] In the above-described example, an example in which the processing of step S2 and the processing of step S3 are performed separately is shown, but a part or all of the processing of step S2 may be included in the processing of step S3.

[0072] For example, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is not a request regarding the first target, by inputting, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information in the case of being a request regarding the second target, the processing of step S2-2, step S2-3, and step S3 can be performed collectively like the processing described in the balloon in FIG. 1. In this case, the above-described second language model is the generation AI, and the processing of step S2-2, step S2-3, and step S3 can be performed collectively by inputting the input information (prompt) once to the generation AI.

[0073] In this case, the instruction information included in the input information is, for example, information such as the character string "#As an expert in {category}, you will receive and answer questions and consultations regarding {category}. Do not answer questions outside the {category} category." However, it is not limited to such an example. Note that {category} includes information indicating the specified category. Also, the instruction information may include guideline information.

[0074] Also, when the request indicated by the request information is a request regarding the second target instead of the first target, the information processing apparatus 1 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information, whereby the processes of steps S2 and S3 can be performed collectively. In this case, the first language model and the second language model described above are generation AIs, and the processes of steps S2 and S3 can be performed collectively by inputting the input information (prompt) to the generation AI once.

[0075] In this case, the instruction information included in the input information is, for example, information such as the character string "#You are an excellent AI assistant. Determine whether the given question corresponds to any of {category group}, and if the given question does not correspond to any of {category group}, as an expert in {category}, you will receive and answer questions regarding {category}. Do not answer questions outside the {category} category." However, it is not limited to such an example. Note that {category group} includes information indicating one or more first targets, and {category} includes information indicating the specified category. Also, the instruction information may include guideline information.

[0076] Subsequently, when the information processing apparatus 1 generates response information using the generation AI in step S3, the information processing apparatus 1 provides the response information generated using the generation AI to the user U (step S4). For example, the information processing apparatus 1 provides the response information generated using the generation AI to the user U by transmitting the response information generated using the generation AI in step S3 to the terminal device 2 that transmitted the utilization request.

[0077] For example, when the request indicated by the request information is a question of the Q&A service, and when the user U accesses the page showing the answer to the question of the user U, the information processing apparatus 1 can provide the user U with the page showing the answer to the question of the user U.

[0078] Also, when the information processing apparatus 1 does not generate response information using the generation AI in step S3, the information processing apparatus 1 provides non-response information to the user U (step S5). For example, when the information processing apparatus 1 determines in step S2-1 that the request indicated by the request information is a request related to the first target, or when the information processing apparatus 1 determines in step S2-2 that the request indicated by the request information is not a request related to the second target, the information processing apparatus 1 provides non-response information to the user U.

[0079] The non-response information is, for example, information of the character information "Sorry. It is a non-response target and I cannot respond to the request." For example, when the request indicated by the request information is a request of the Q&A service, the non-response information is, for example, information of the character information "Sorry. I cannot answer because it is a question outside the answer range." Note that the non-response information may include information indicating that it is a request related to the first target or not a request related to the second target.

[0080] In the above-described explicit non-response determination criteria, in addition to the fixed first target that does not depend on the specified category, as the first target that depends on the specified category, by including a category other than the specified category and preset for each specified category, the non-response accuracy for inappropriate requests can be improved.

[0081] On the one hand, the above-mentioned first target that depends on the specified category is a category that is preset as a non-response category for each specified category and is a non-response category that is a category other than the specified category. Due to becoming an abstract concept, etc., the coverage of non-response is low, and in the information processing apparatus 1, there is a possibility that inappropriate requests cannot be rejected.

[0082] For example, in the non-response category, the information processing apparatus 1 may not be able to reject requests that should not be responded to, such as requests that violate social morals or requests for defamatory responses. In the non-response category, there is a possibility that requests for which responses using generative AI are inappropriate cannot be rejected. Requests for which responses using generative AI are inappropriate include, for example, requests including topics such as medical care, health, and finance in the non-response category, but are not limited to such examples.

[0083] Therefore, the information processing apparatus 1 is configured to be able to perform a generation process of generating non-response target information indicating a new non-response target based on non-response category information indicating a non-response category associated with the specified category, whereby the non-response accuracy for inappropriate requests can be further improved. Hereinafter, the generation process executed by the information processing apparatus 1 will be specifically described.

[0084] The generation process executed by the information processing apparatus 1 includes a first generation process of generating risk information, which is information indicating a risk, as new non-response target information from non-response category information, which is information indicating a non-response category, and a second generation process of extracting characteristic words from the non-response category information and generating new non-response target information from the extracted characteristic words.

[0085] First, the first generation process will be described. In the first generation process, the information processing device 1 generates, for example, new non-response target information using a language model. Such a language model is an example of a third language model and may be referred to as the third language model hereinafter. The third language model is a large language model such as, for example, a transformer-based model or an RNN-based model. The third language model may be the same as one or more of the above-described generative AI, the first language model, and the second language model.

[0086] The information processing device 1 generates, for example, risk information, which is information indicating a risk in the non-response category from the non-response category information, as new non-response target information using the third language model. For example, the information processing device 1 can input information including non-response category information as input information into the third language model and cause the third language model to generate and output risk information, which is information indicating a risk in the non-response category, as new non-response target information. Hereinafter, information indicating a risk may be referred to as risk information.

[0087] The information processing device 1 inputs, for example, information including instruction information for instructing the output of risks in one or more non-response categories as input information into the third language model and causes the third language model to generate and output risk information for the non-response category. The instruction information includes, for example, information of the character string "Please output the risks considered based on the given category information.\n#Category Information\nAdult\nGambling" when the non-response categories are adult and gambling, etc.

[0088] The third language model may be a model learned to output non-response target information when information including non-response category information is input as input information. In this case, the third language model is learned using, for example, learning information including information including non-response category information and one or more non-response target information for each non-response category. The one or more non-response target information are used as teacher data (labels).

[0089] The information processing apparatus 1 can also generate, for example, one or more pieces of risk information for each non-response category using a third language model. Further, the information processing apparatus 1 can also include information specifying the number of non-response category information to be generated by the third language model in the specified information.

[0090] Also, the information processing apparatus 1 can, for example, cause the third language model to generate, as risk information, information indicating a risk considered as an administrator of a service that makes a response according to a request for a specified category.

[0091] For example, assume that the request indicated by the request information is a question of a Q&A service, the specified category is television, and the non-response categories are adult and gambling. In this case, the instruction information included in the input information includes information of the character string "You are an administrator who answers questions regarding television in the Q&A service. Please output regarding the risks considered based on the given category information."

[0092] Also, when the information processing apparatus 1 obtains a plurality of pieces of risk information using the third language model, the information processing apparatus 1 aggregates the plurality of pieces of risk information into n or fewer pieces of risk information, which is a preset number, as non-response target information using the third language model. n is an integer of 1 or more. For example, the information processing apparatus 1 can input, as input information to the third language model, information including instruction information for grouping a plurality of pieces of risk information into n pieces of risk information, and cause the third language model to output the n pieces of aggregated risk information.

[0093] Further, when the information processing apparatus 1 causes the third language model to generate information indicating a risk considered as, for example, an administrator of the above-described Q&A service as risk information, for example, the information processing apparatus 1 inputs, as input information to the third language model, instruction information including information of the character string "You are an administrator who answers questions regarding television in the Q&A service. Please aggregate into five risks that must be emphasized in continuing the service based on the given risk information." and a plurality of pieces of risk information, whereby the third language model can be caused to output n pieces of aggregated risk information.

[0094] Next, the second generation process will be described. In the second generation process, the information processing apparatus 1 extracts characteristic words (for example, keywords or characteristic phrases) from the non-response category information, and generates new non-response target information from the extracted characteristic words.

[0095] The information processing apparatus 1 divides the non-response category information to extract characteristic words, and generates new non-response target information using such characteristic words. The information processing apparatus 1 can perform, for example, a process of extracting characteristic words from the non-response category information and generating, as a plurality of new non-response target information, information each including a part of such characteristic words, using the third language model.

[0096] For example, assume that the request indicated by the request information is a question in the Q&A service and the non-response category information is the character string "Questions regarding the Internet". In this case, the information processing apparatus 1 can cause the third language model to generate a plurality of new non-response target information by, for example, inputting information of the character string "# Please extract characteristic words that are important words from the given keywords and create subdivided keywords.\n\n# Keywords\nQuestions regarding the Internet" to the third language model. The plurality of new non-response target information in this case is, for example, information of the character string "Internet technology", information of the character string "Internet service", etc., but is not limited to such examples.

[0097] In addition, for example, instead of the third language model, the information processing device 1 can also generate a plurality of new non-response target information based on rules. For example, the information processing device 1 has a feature word dictionary containing a plurality of keywords or phrases for each feature word, and can extract a plurality of keywords or phrases associated with the feature words included in the feature word dictionary among the non-response category information as a plurality of new non-response target information.

[0098] When there are a plurality of non-response category information, the information processing device 1 can perform the process of generating a plurality of non-response target information using the third language model for each non-response category, but can also perform the process of generating a plurality of non-response target information using the third language model collectively for a plurality of non-response categories.

[0099] Based on the non-response accuracy for each combination of two or more non-response target information among the plurality of non-response category information generated as described above, the information processing device 1 can select two or more non-response target information to be used in step S2-1 among the plurality of non-response target information generated as described above. In this case, based on the selected two or more response target information, the information processing device 1 determines in step S2-1 whether the request indicated by the request information is a request regarding a non-response target.

[0100] For example, the information processing device 1 can evaluate the non-response accuracy for each combination of two or more non-response target information. For example, the information processing device 1 can evaluate the non-response accuracy for each combination of two or more response target information using a plurality of evaluation request information. Hereinafter, a combination of two or more non-response target information may be referred to as a non-response target information combination.

[0101] Each evaluation request information is information indicating a first request that is a request regarding a non-response target in a specified category, information indicating a second request that is a request regarding a response target in a specified category, or information indicating a third request that is a request regarding a non-response target regardless of the specified category.

[0102] The information processing apparatus 1 determines whether the request indicated by the request information for evaluation is determined to be a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination. The explicit non-response determination process using the non-response target information combination is, for example, a process of using each of a plurality of non-response target information included in the non-response target information combination as a first target.

[0103] When the information processing apparatus 1 determines that the first request or the third request indicated by the request information for evaluation is a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the information processing apparatus 1 sets the evaluation of the non-response target information combination to 1, and when it is not the case, sets the evaluation of the non-response target information combination to 0.

[0104] Also, when the information processing apparatus 1 determines that the second request indicated by the request information for evaluation is not a request regarding a non-response target in the explicit non-response determination process using a prompt including the non-response target information combination, the information processing apparatus 1 sets the evaluation of the non-response target information combination to 1, and when it is not the case, sets the evaluation of the non-response target information combination to 0.

[0105] The information processing apparatus 1 can calculate the non-response accuracy by aggregating the evaluation results and dividing the aggregated result by the number of evaluations. Also, the information processing apparatus 1 can change the weight according to which of the first request, the second request, and the third request the request indicated by the request information for evaluation is, and aggregate the evaluation results by weighted addition.

[0106] Also, the request information for evaluation may be limited to the information indicating the first request, or may be limited to two or more pieces of information among the information indicating the first request, the information indicating the second request, and the information indicating the third request. Also, the evaluation is not limited to 0 and 1, and may be, for example, -1 and 1, or other combinations of values.

[0107] In addition, the information processing apparatus 1 can also use a keyword-based natural language processing to evaluate the non-response accuracy of the non-response target information combination using a language model, for example, when determining whether a request indicated by request information from each user U is a request regarding a non-response target in an explicit non-response determination process using a non-response target information combination.

[0108] For example, the information processing apparatus 1 can evaluate the non-response accuracy of the non-response target information combination for each non-response target information combination using a third language model based on information indicating a request included in the request information from each user U and the determination result of the keyword-based natural language processing.

[0109] For example, the information processing apparatus 1 inputs, as input information, information including instruction information for instructing a determination as to whether any of two or more non-response target information indicated by the non-response target information combination is not included in the request information from the user U into the third language model, so as to cause the third language model to output information indicating the non-response accuracy of the two or more response target information indicated by the non-response target information combination.

[0110] The information processing apparatus 1 selects, as the non-response target information combination to be used in step S2-1, the non-response target information combination having the highest non-response accuracy among the non-response accuracies for each non-response target information combination.

[0111] In addition, the information processing apparatus 1 can also select, as combinations of two or more non-response target information to be used in step S2-1 according to a random or predetermined rule, non-response target information combinations having a non-response accuracy for each non-response target information combination equal to or higher than a threshold value.

[0112] As described above, the information processing apparatus 1 according to the embodiment receives information indicating a request from the user U, determines whether to generate information indicating a response to the request indicated by the received information, and if it is determined to generate information indicating a response, provides the user U with response information generated using a generation AI as information indicating a response to the request. The information processing apparatus 1 determines whether to generate information indicating a response to the request indicated by the received information based on whether the request indicated by the received information is a request regarding a first target set as a non-response target and a request regarding a second target set as a response target. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0113] Also, the information processing apparatus 1 according to the embodiment generates non-response target information indicating a non-response target, receives information indicating a request from the user U, and determines whether the request indicated by the received information is a request regarding a non-response target based on the plurality of generated non-response target information. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0114] Hereinafter, the configuration of an information processing system including the information processing apparatus 1 and the terminal device 2 that perform such processing will be described in detail.

[0115] [2. Configuration of Information Processing System] FIG. 2 is a diagram showing an example of the configuration of an information processing system according to the embodiment. As shown in FIG. 2, the information processing system 100 according to the embodiment includes an information processing apparatus 1, a plurality of terminal devices 2, and a terminal device 3.

[0116] The plurality of terminal devices 2 are used by different users U. The terminal device 3 is used by, for example, an employee O of the operating company of the information processing apparatus 1. The terminal devices 2 and 3 are, for example, a notebook PC (Personal Computer), a desktop PC, a smartphone, a tablet PC, or a wearable device. The wearable device is, for example, smart glasses or a smartwatch, but is not limited to such examples.

[0117] Each of the information processing apparatus 1, the terminal apparatus 2, and the terminal apparatus 3 is connected to be communicable with each other by wire or wirelessly via the network N. Note that the information processing system 100 shown in FIG. 2 may include a plurality of information processing apparatuses 1 and the like.

[0118] The network N includes, for example, a WAN (Wide Area Network) such as the Internet and a mobile communication network such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation: 5th generation mobile communication system), but is not limited to such examples.

[0119] The terminal apparatuses 2 and 3 are connected to the network N via a mobile communication network, short-range wireless communication such as Bluetooth (registered trademark), or a wireless LAN (Local Area Network), and can communicate with the information processing apparatus 1 and the like.

[0120] 〔3. Configuration of Information Processing Apparatus 1〕 FIG. 3 is a diagram showing an example of the configuration of the information processing apparatus 1 according to the embodiment. As shown in FIG. 3, the information processing apparatus 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0121] 〔3.1. Communication Unit 10〕 The communication unit 10 is realized by, for example, a communication module or a NIC (Network Interface Card). The communication unit 10 is connected to the network N by wire or wirelessly and transmits and receives information to and from other various devices. For example, the communication unit 10 transmits and receives information to and from each of the terminal apparatus 2 and the terminal apparatus 3 via the network N.

[0122] 〔3.2. Storage Unit 11〕 The storage unit 11 is implemented by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 includes a user information storage unit 20, a non-response target information storage unit 21, and an evaluation request information storage unit 22.

[0123] [3.2.1. User Information Storage Unit 20] The user information storage unit 20 stores user information including information about the user U. FIG. 4 is a diagram showing an example of a user information table stored in the user information storage unit 20 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 4, the user information table stored in the user information storage unit 20 includes items such as "user ID" and "attribute information".

[0124] The "user ID" is identification information for identifying the user U. The "attribute information" is attribute information of the user U corresponding to the "user ID", and includes, for example, information on psychographic attributes or demographic attributes. Demographic attributes are, for example, gender, age, place of residence, and occupation, and psychographic attributes are objects of interest such as travel, clothing, cars, and religion, lifestyle, thoughts, and trends of thoughts.

[0125] [3.2.2. Non-Response Target Information Storage Unit 21] The non-response target information storage unit 21 stores information indicating various non-response targets. FIG. 5 is a diagram showing an example of a non-response target table stored in the non-response target information storage unit 21 of the information processing apparatus 1 according to the embodiment.

[0126] In the example shown in FIG. 5, the non-response target table stored in the non-response target information storage unit 21 includes information on items such as "non-response target ID", "target category", and "non-response target information". The "non-response target ID" is an identifier for identifying the non-response target and is information assigned to each non-response target. The non-response targets include the first target described above.

[0127] "Target Category" is information indicating the category to which a non-response target is associated. If there is no category to which the non-response target is associated, it is set to blank (null). "Non-response Target Information" is information indicating a non-response target. For example, it includes non-response category information indicating a non-response category pre-associated by an employee O or the like as a non-response category for a target category, non-response target information generated by the processing unit 12, etc., but is not limited to such examples.

[0128] 〔3.2.3. Evaluation Requirement Information Storage Unit 22〕 The evaluation requirement information storage unit 22 stores various evaluation requirement information. The evaluation requirement information is, for example, information indicating a first requirement that is a requirement regarding a non-response target in a specified category, information indicating a second requirement that is a requirement regarding a response target in a specified category, or information indicating a third requirement that is a requirement regarding a non-response target regardless of the specified category.

[0129] 〔3.3. Processing Unit 12〕 The processing unit 12 is a controller and is realized, for example, by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing apparatus 1 with a RAM or the like as a work area.

[0130] Also, the processing unit 12 is a controller and may be partially or entirely realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPGPU (General Purpose Graphic Processing Unit).

[0131] As shown in FIG. 3, the processing unit 12 includes an acquisition unit 30, a reception unit 31, a generation unit 32, an evaluation unit 33, a selection unit 34, a determination unit 35, and a provision unit 36, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in FIG. 3, and any other configuration may be used as long as it can perform the information processing described later.

[0132] 〔3.3.1. Acquisition Unit 30〕 The acquisition unit 30 acquires various types of information from external information processing devices, terminal devices 2 and 3, etc. via the network N and the communication unit 10.

[0133] For example, the acquisition unit 30 acquires the information of the user U from an external information processing device or terminal device 2, and stores the acquired information of the user U in the user information storage unit 20. Also, the acquisition unit 30 acquires non-response category information from an external information processing device or terminal device 3, and stores the acquired non-response category information in the non-response target information storage unit 21. Further, the acquisition unit 30 acquires evaluation request information from an external information processing device or terminal device 3, and stores the acquired evaluation request information in the evaluation request information storage unit 22.

[0134] In addition, the acquisition unit 30 acquires various types of information from the storage unit 11. For example, the acquisition unit 30 acquires the information of the user U from the user information storage unit 20. Also, the acquisition unit 30 acquires non-response target information corresponding to a specified category from the non-response target information storage unit 21. Further, the acquisition unit 30 acquires evaluation request information from the evaluation request information storage unit 22.

[0135] 〔3.3.2. Reception Unit 31〕 The reception unit 31 receives various requests and information. For example, the reception unit 31 receives request information, which is information indicating the request of the user U. For example, by receiving a usage request, the reception unit 31 receives the request information included in the usage request. The request information includes, for example, information indicating a question of the user U, information indicating an instruction of the user U, or information indicating a desire of the user U, etc., but is not limited to such examples.

[0136] The request information includes designation information that designates the category of the request. The reception unit 31 identifies the designated category based on the designation information included in the received usage request. For example, when the question of the user U is a question in a Q&A service or a customer support service, the usage request includes designation information that designates the category of the question.

[0137] Also, when the instruction of the user U is a content creation instruction in a content creation service, the usage request includes designation information that designates the category of the content to be created. Further, when the instruction of the user U is an instruction for interactive learning in a learning support service, the usage request includes designation information that designates a subject as the category of learning.

[0138] The designation information is, for example, information indicating the category of the request or information corresponding to the category of the request. The information corresponding to the category of the request is, for example, part or all of the URL or domain name of the category in the service provided by the information processing apparatus 1, but is not limited to such an example.

[0139] [3.3.3. Generation unit 32] The generation unit 32 generates various information. For example, the generation unit 32 generates non-response target information as information indicating a first target. The non-response target information is information indicating a non-response target that is a target for which no response is given to the request indicated by the request information received by the reception unit 31.

[0140] The generation unit 32 generates non-response target information indicating a new non-response target. For example, the generation unit 32 generates non-response target information indicating a new non-response target based on non-response category information indicating a non-response category that is different from the designated category designated by the user U and is associated in advance as a non-response category to the designated category.

[0141] For example, the generation unit 32 can generate risk information, which is information indicating risk, from non-response category information, which is information indicating a non-response category, as new non-response target information. Further, the generation unit 32 can extract characteristic words from the non-response category information and generate new non-response target information from the extracted characteristic words. For example, the generation unit 32 generates, as non-response target information, information including some of the characteristic words extracted from the non-response category information.

[0142] For example, the generation unit 32 generates non-response target information as information indicating a first target using a third language model. For example, the generation unit 32 inputs information including non-response category information indicating a non-response category into the third language model and causes the third language model to generate non-response target information.

[0143] Further, when it is determined by the determination unit 35 that the generation unit 32 generates information indicating a response to a request indicated by the request information received by the reception unit 31, the generation unit 32 generates response information using a generation AI as information indicating a response to the request indicated by the request information received by the reception unit 31.

[0144] FIG. 6 is a diagram showing an example of non-response categories preset for each category in the information processing apparatus 1 according to the embodiment. In the example shown in FIG. 6, an example of non-response categories preset for each category when the request indicated by the request information is a question of a Q&A service is shown. Regardless of the category of the question of the user U, categories such as finance and law are set as non-response categories.

[0145] Also, in the example shown in FIG. 6, when the category of the question of the user U is finance or law, all categories other than that category are set as non-response categories. When the category of the question of the user U is TV, anime is set as the non-response category. When the category of the question of the user U is anime, TV is set as the non-response category. Note that the non-response categories preset for each category are not limited to the example shown in FIG. 6.

[0146] As shown in FIG. 3, the generation unit 32 includes a first generation processing unit 40 that generates a plurality of pieces of risk information indicating risks in the non-response category, and a second generation processing unit 41 that aggregates the plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information. Further, the generation unit 32 includes a third generation processing unit 42 that generates, as new non-response target information, information including a keyword extracted from the non-response category information, and a fourth generation processing unit 43 that causes the generation AI to generate response information, which is information indicating a response to a request indicated by the request information.

[0147] [3.3.3.1. First Generation Processing Unit 40] The first generation processing unit 40 generates risk information indicating risks in the non-response category using a third language model. The third language model is a large language model such as, for example, a transformer-based model or an RNN-based model. The third language model may be the same as one or more of the above-described generation AI, first language model, and second language model.

[0148] The first generation processing unit 40 generates, for example, risk information, which is information indicating risks in the non-response category, from the non-response category information as new non-response target information using the third language model. For example, the first generation processing unit 40 can input information including the non-response category information as input information to the third language model, and cause the third language model to generate and output risk information, which is information indicating risks in the non-response category, as new non-response target information.

[0149] The first generation processing unit 40 inputs, for example, information including instruction information for instructing the output of risks in one or more non-response categories as input information to the third language model, and causes the third language model to generate and output risk information for the non-response category.

[0150] FIG. 7 is a diagram showing an example of instruction information used in the first generation processing unit 40 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model. In FIG. 7, for example, when the non-response category is adult and gambling, the instruction information includes information of the character string "# Instruction\nYou are a risk manager in the {category} field of the Q&A service.\nPlease output the risks considered based on the given category information.\n\n# Category information\nAdult\nGambling ···". In FIG. 7, {category} is information indicating the specified category.

[0151] As shown in FIG. 7, the first generation processing unit 40 can cause the third language model to generate, as risk information, information indicating risks considered as an administrator of a service that responds to requests for a specified category.

[0152] The third language model may be a model learned to output non-response target information when information including non-response category information is input as input information. In this case, the third language model is learned using learning information including information including non-response category information and one or more non-response target information for each non-response category. The one or more non-response target information are used as teacher data (labels).

[0153] The first generation processing unit 40 can also generate, for example, one or more pieces of risk information for each non-response category using the third language model. Further, the first generation processing unit 40 can also include, in the specified information, information specifying the number of non-response category information to be generated by the third language model.

[0154] 〔3.3.3.2. Second generation processing unit 41〕 The second generation processing unit 41 aggregates, using a language model, a plurality of pieces of risk information generated by the first generation processing unit 40 into risk information equal to or less than a preset number n as non-response target information. n is an integer of 1 or more.

[0155] When, for example, more than n pieces of risk information are generated by the first generation processing unit 40, the second generation processing unit 41 aggregates a plurality of pieces of risk information generated by the first generation processing unit 40 into n or fewer pieces of risk information, which is a preset number, using a third language model as non-response target information.

[0156] For example, the second generation processing unit 41 can input information including instruction information for grouping a plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information as input information to the third language model, and cause the third language model to output n pieces of aggregated risk information.

[0157] FIG. 8 is a diagram showing an example of instruction information used in the second generation processing unit 41 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model. In FIG. 8, the instruction information is information of the character string "# Instruction\nYou are a risk manager in the {category} field of the Q&A service.\nPlease aggregate the risks that must be emphasized in continuing the service based on the given risk information into 5.\n\n# Risk Information\nLaw violation: The possibility that the service violates the law.\nTax issues: Risks due to inappropriate tax processing...", and by inputting information including such instruction information and information indicating the requirements included in the request information as input information to the third language model, the third language model can be caused to output n pieces of aggregated risk information.

[0158] FIG. 9 is a diagram showing the relationship between the processing by the first generation processing unit 40 and the second generation processing unit 41 in the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 9, the first generation processing unit 40 generates a plurality of pieces of risk information based on the non-response category information stored in the non-response target information storage unit 21. The second generation processing unit 41 aggregates a plurality of pieces of risk information generated by the first generation processing unit 40 into n pieces of risk information, and stores the aggregation result in the non-response target information storage unit 21 as n pieces of non-response target information.

[0159] 〔3.3.3.3. Third generation processing unit 42〕 The third generation processing unit 42 generates, as new non-response target information, information including a feature word (for example, a keyword or a characteristic phrase) extracted from the non-response category information.

[0160] For example, the third generation processing unit 42 divides the non-response category information to extract feature words, and generates new non-response target information using such feature words. The third generation processing unit 42 can perform, for example, a process of extracting feature words from the non-response category information and generating, as a plurality of new non-response target information, information each including a part of such feature words, using a third language model.

[0161] FIG. 10 is a diagram showing an example of instruction information used in the third generation processing unit 42 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and an output example of the third language model, and is an example in a case where the request indicated by the request information is a question in a Q&A service and the non-response category information is the character string "questions about the Internet".

[0162] In this case, as shown in FIG. 10, the third generation processing unit 42 can cause the third language model to generate a plurality of new non-response target information, for example, by inputting information of the character string "#Instruction\nExtract important words from the given keyword and create a refined keyword.\n\n #Keyword\nQuestions about the Internet" into the third language model. The plurality of new non-response target information in this case is, for example, information of the character string "Internet technology", information of the character string "Internet service", etc. as shown in FIG. 9, but is not limited to such examples.

[0163] Further, the third generation processing unit 42 can also generate a plurality of new non-response target information based on rules, for example, instead of the third language model. For example, the third generation processing unit 42 has a feature word dictionary including a plurality of keywords or phrases for each feature word, and can extract, as a plurality of new non-response target information, a plurality of keywords or phrases associated with the feature words included in the feature word dictionary among the non-response category information.

[0164] When there are multiple pieces of non-response category information, the third generation processing unit 42 can perform the process of generating a plurality of pieces of non-response target information using the third language model for each non-response category, but the process of generating a plurality of pieces of non-response target information using the third language model can also be performed collectively for a plurality of non-response categories.

[0165] [3.3.3.4. Fourth Generation Processing Unit 43] The fourth generation processing unit 43 generates response information, which is information indicating a response to the request indicated by the request information, using a generation AI.

[0166] For example, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing an output of information indicating a response to the request indicated by the request information and information indicating the request included in the request information, and causes the generation AI to generate response information.

[0167] Suppose the request indicated by the request information is, for example, a question in the category of economy in a Q&A service and is information of the character string "What is inflation? How does it affect the economy?". In this case, the fourth generation processing unit 43 includes, as instruction information, information of the character string "As an expert in {category}, you receive and answer questions regarding {category}.", and inputs, as input information to the generation AI, information including information of the character string "What is inflation? How does it affect the economy?" as information indicating the request included in the request information.

[0168] Note that {category} includes information indicating a specified category. Also, the instruction information may include guideline information indicating a guideline for answering. The guideline information includes, for example, information instructing the clarity and neutrality of the answer.

[0169] The process of the fourth generation processing unit 43 may include part or all of the functions of the determination unit 35. In this case, the fourth generation processing unit 43 functions as part or all of the determination unit 35 in addition to the function of the generation unit 32 that generates response information using the generation AI.

[0170] For example, the fourth generation processing unit 43 may have a process of determining whether the request indicated by the request information received by the reception unit 31 is a request regarding the second target set as the response target. In this case, the fourth generation processing unit 43 includes the processing of the generation unit 32 and the processing of the determination unit 35.

[0171] For example, when the request indicated by the request information received by the reception unit 31 is a request regarding the second target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information. Thus, when the request indicated by the request information is a request regarding the second target, the generation AI can be made to generate response information.

[0172] For example, when the determination unit 35 determines that the request indicated by the request information is not a request regarding the first target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information in the case where it is a request regarding the second target. Thus, the determination process as to whether it is a request regarding the second target and the generation process of the response information can be performed together.

[0173] FIG. 11 is a diagram showing an example of instruction information used for an explicit non-response determination process that the fourth generation processing unit 43 of the processing unit 12 in the information processing apparatus 1 according to the embodiment performs as part of the determination unit 35.

[0174] In the example shown in FIG. 11, the instruction information included in the input information is, for example, information such as the character string "# You are an expert in {category}. I accept questions and consultations regarding {category}. Please do not answer questions outside the {category} category.", but is not limited to such an example. Note that {category} includes information indicating the specified category. Also, as guidelines for answers, additional conditions that are not limited to the further specified category may be included.

[0175] Further, the processing by the fourth generation processing unit 43 may include processing for determining whether the request indicated by the request information received by the reception unit 31 is a request regarding the first target and whether it is a request regarding the second target. Also in this case, the fourth generation processing unit 43 includes the processing of the generation unit 32 and the processing of the determination unit 35.

[0176] For example, when the request indicated by the request information received by the reception unit 31 is not a request regarding the first target and is a request regarding the second target, the fourth generation processing unit 43 inputs, as input information to the generation AI, information including instruction information for instructing the generation of information indicating a response to the request indicated by the request information. Thus, when the request indicated by the request information is not a request regarding the first target and is a request regarding the second target, the generation AI can also be caused to generate response information.

[0177] In this case, the input information includes, in addition to the instruction information, the request information, information indicating the first target, and information indicating the second target. The instruction information is, for example, information of the character string "#You are an excellent AI assistant. Determine whether the given question corresponds to the first target shown below. If the given question does not correspond to the first target, as an expert in {category}, you accept and answer questions regarding {category}. Do not answer questions other than those in the {category} category.", but is not limited to such an example. Note that {category} includes information indicating the designated category. The instruction information may also include guideline information.

[0178] 〔3.3.4. Evaluation Unit 33〕 The evaluation unit 33 evaluates the non-response accuracy for each non-response target information combination that is a combination of two or more non-response target information among the plurality of non-response target information generated by the generation unit 32.

[0179] For example, the evaluation unit 33 can evaluate the non-response accuracy for each non-response target information combination by using a plurality of pieces of evaluation requirement information stored in the evaluation requirement information storage unit 22 and acquired by the acquisition unit 30.

[0180] As described above, each piece of evaluation requirement information is information indicating a first requirement that is a requirement regarding a non-response target in a specified category, information indicating a second requirement that is a requirement regarding a response target in the specified category, or information indicating a third requirement that is a requirement regarding a non-response target regardless of the specified category.

[0181] The evaluation unit 33 determines whether or not the requirement indicated by the evaluation requirement information is determined to be a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination. The explicit non-response determination process using the non-response target information combination is, for example, a process of using each of a plurality of pieces of non-response target information included in the non-response target information combination as a first target.

[0182] When the evaluation unit 33 determines that the first requirement or the third requirement indicated by the evaluation requirement information is a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation of the non-response target information combination is set to 1, and otherwise, the evaluation of the non-response target information combination is set to 0.

[0183] Also, when the evaluation unit 33 determines that the second requirement indicated by the evaluation requirement information is not a requirement regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation of the non-response target information combination is set to 1, and otherwise, the evaluation of the non-response target information combination is set to 0.

[0184] The evaluation unit 33 can calculate the non-response accuracy by aggregating the evaluation results and dividing the aggregated result by the number of evaluations. Further, the evaluation unit 33 can also change the weight according to whether the requirement indicated by the evaluation requirement information is any of the first requirement, the second requirement, and the third requirement, and aggregate the evaluation results by weighted addition.

[0185] Further, the evaluation request information may be limited to the information indicating the first request, or may be limited to two or more pieces of information among the information indicating the first request, the information indicating the second request, and the information indicating the third request. Further, the evaluation is not limited to 0 and 1, and may be, for example, -1 and 1, or other combinations of values.

[0186] Also, even when it is determined by the determination unit 35 whether a request indicated by the request information from each user U is a request regarding a non-response target in the explicit non-response determination process using the non-response target information combination, the evaluation unit 33 can evaluate the non-response accuracy of the non-response target information combination using a language model by using keyword-based natural language processing.

[0187] For example, the evaluation unit 33 can evaluate the non-response accuracy of the non-response target information combination for each non-response target information combination by using a third language model based on the information indicating the request included in the request information from each user U and the determination result by the determination unit 35 by keyword-based natural language processing.

[0188] For example, the evaluation unit 33 inputs, as input information, information including instruction information for instructing determination as to whether any of two or more non-response target information indicated by the non-response target information combination is not included in the request information from the user U to the third language model, so that the third language model outputs information indicating the non-response accuracy of two or more response target information indicated by the non-response target information combination.

[0189] FIG. 12 is a diagram showing a processing flow by the evaluation unit 33 of the processing unit 12 in the information processing apparatus 1 according to the embodiment. As shown in FIG. 12, the evaluation unit 33 determines a combination of two or more non-response target information from among a plurality of non-response target information (step S50). The combination of two or more non-response target information is determined randomly or according to a predetermined rule. Further, the combination of two or more non-response target information may be, for example, a combination including m non-response target information for each non-response category. m is an integer of 1 or more.

[0190] Subsequently, the evaluation unit 33 creates a prompt (input information) including instruction information including two or more pieces of non-response target information of the combination determined in step S50 (step S51). Then, the evaluation unit 33 performs prompt evaluation to evaluate the non-response accuracy of the prompt generated in step S51 as the non-response accuracy of the combination of two or more pieces of non-response target information determined in step S51 (step S52).

[0191] In step S52, the evaluation unit 33 inputs information including the prompt generated in step S51 and the evaluation request information into the first language model as input information for each piece of evaluation request information, and determines the non-response accuracy of the combination of two or more pieces of non-response target information determined in step S50 based on information indicating whether the first target output from the first language model for each piece of evaluation request information is a request indicating a response or not.

[0192] In the example shown in FIG. 12, each piece of evaluation request information is non-response required information (category-dependent), response required information (category-dependent), or non-response required information (category-independent). The non-response required information (category-dependent) is information indicating a first request that is a request regarding a non-response target in a specified category. The response required information (category-dependent) is information indicating a second request that is a request regarding a response target in a specified category. The non-response required information (category-independent) is information indicating a third request that is a request regarding a non-response target regardless of the specified category.

[0193] Then, a selection unit 34, which will be described later, determines whether the non-response accuracy obtained in step S51 satisfies a predetermined condition based on the evaluation result in step S52 (step S53). The predetermined condition is, for example, a condition that the non-response accuracy is the highest when the process of step S50 is completed for all combinations of two or more pieces of non-response target information, or a condition that the non-response accuracy is equal to or higher than a threshold value, but is not limited to such examples.

[0194] When the selection unit 34 determines that the non-response accuracy obtained in step S51 satisfies a predetermined condition (step S53: Yes), the selection unit 34 selects, as two or more pieces of non-response target information to be used by the determination unit 35, a combination of two or more pieces of non-response target information for which the non-response accuracy obtained in step S51 satisfies the predetermined condition.

[0195] When the evaluation unit 33 determines, based on the selection by the selection unit 34, that the non-response accuracy obtained in step S51 does not satisfy the predetermined condition (step S53: No), the process proceeds to step S50.

[0196] [3.3.5. Selection Unit 34] Based on the non-response accuracy for each combination of two or more pieces of non-response target information among a plurality of pieces of non-response target information, the selection unit 34 selects two or more pieces of non-response target information to be used by the determination unit 35 from among the plurality of pieces of non-response target information. The plurality of pieces of non-response target information are, for example, non-response category information indicating a non-response category, non-response target information generated by the generation unit 32, and the like.

[0197] For example, the selection unit 34 selects two or more pieces of non-response target information to be used by the determination unit 35 based on the evaluation result by the evaluation unit 33. For example, the selection unit 34 selects, as the combination of two or more pieces of non-response target information to be used by the determination unit 35, the combination of non-response target information having the highest non-response accuracy among the non-response accuracies for each combination of non-response target information.

[0198] Alternatively, the selection unit 34 can select, as the combination of two or more pieces of non-response target information to be used by the determination unit 35, a combination of non-response target information for which the non-response accuracy for each combination of non-response target information is equal to or higher than a threshold value, randomly or according to a predetermined rule.

[0199] [3.3.6. Determination Unit 35] The determination unit 35 performs various determinations. For example, the determination unit 35 determines whether to generate response information, which is information indicating a response to a request indicated by the request information received by the reception unit 31.

[0200] Based on whether the request indicated by the request information received by the reception unit 31 is a request regarding a first target set as a non-response target and a request regarding a second target set as a response target, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information received by the reception unit 31.

[0201] The first target is, for example, a target set as a non-response target for a specified category that is a category specified by the user U, a non-response category pre-associated with the specified category, a non-response target indicated by non-response target information generated by the generation unit 32, and the like.

[0202] The non-response category is, for example, another category (a category other than the specified category) whose boundary with the specified category is ambiguous and the probability of an appropriate response is equal to or less than a threshold value, and is set in advance for each specified category, but is not limited to such an example.

[0203] The determination unit 35 performs an explicit non-response determination process of determining whether the request indicated by the request information received by the reception unit 31 is a request regarding a first target that is a non-response target, based on, for example, two or more pieces of non-response target information selected by the selection unit 34.

[0204] The two or more pieces of non-response target information selected by the selection unit 34 are two or more pieces of information among a plurality of non-response category information each indicating a non-response category preset for the specified category and a plurality of non-response target information generated by the generation unit 32.

[0205] The first target set as a non-response target includes, for example, in addition to the target set as a non-response target for the specified category, a target that does not depend on the specified category. The target that does not depend on the specified category is, for example, a violation of law, a violation of social morality, slander, libel, etc., but is not limited to such an example.

[0206] In addition, the objects that depend on the specified category and the objects that do not depend on the specified category may include objects for which a response using generative AI is not appropriate. Objects for which a response using generative AI is not appropriate include, for example, but are not limited to, medical, health, finance, etc.

[0207] The determination unit 35 can determine, for example, using a first language model, whether the request indicated by the request information received by the reception unit 31 is a request regarding a first object set as a non-response target. The first language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples.

[0208] The determination unit 35 inputs, as input information to the first language model, information including instruction information instructing an output of information indicating whether the request indicated by the request information received by the reception unit 31 is a request regarding the first object, and information indicating the request indicated by the request information included in the usage request. In this case, information indicating whether the request is a request regarding the first object is output from the first language model. In this way, the determination unit 35 can determine, using the first language model, whether the request indicated by the request information is a request regarding the first object set as a non-response target.

[0209] The instruction information includes information indicating an output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the first object, and information to be output when the request indicated by the request information is not a request regarding the first object.

[0210] The information to be output when the request indicated by the request information is a request regarding the first object is, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the first object is, for example, "false", but is not limited to such examples.

[0211] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the first target. For example, the information indicating the output format is information such as the string "When the request indicated by the request information is a request regarding the first target, output 'applicable', and if not, do not output anything", but is not limited to such an example. Also, the information indicating the output format may be information indicating a sample of the output.

[0212] FIG. 13 is a diagram showing an example of instruction information used in the explicit non-response determination process by the determination unit 35 of the processing unit 12 in the information processing apparatus 1 according to the embodiment. In the instruction information shown in FIG. 13, although details are omitted, the information corresponding to "Condition 1" includes information indicating a question regarding the non-response category as information indicating a question regarding the first target, and the information corresponding to "Condition 2" includes non-response target information generated by the generation unit 32 as information indicating a question regarding the first target, but is not limited to such an example.

[0213] In the above example, the input information input to the first language model includes instruction information, but the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information. Also, the first language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the first target from input information that does not include the instruction information and the information indicating the first target. In these cases, the language model is generated for each category, for example, but is not limited to such an example.

[0214] The determination unit 35 can also determine, in natural language processing without using the first language model, instead of or in addition to the first language model, whether the request indicated by the request information is a request regarding a first target set as a non-response target. Natural language processing without using the first language model is, for example, keyword-based natural language processing. The determination unit 35 has, for example, a first target dictionary including a plurality of keywords each directly or indirectly indicating the first target, and determines whether the keywords included in the first target dictionary are included in the request information.

[0215] When the keywords included in the first target dictionary are included in the request information, the determination unit 35 determines that the request indicated by the request information is a request regarding the first target set as a non-response target, and otherwise determines that the request indicated by the request information is not a request regarding the first target set as a non-response target.

[0216] Also, the keyword-based natural language processing may be natural language processing using a model. The model in this case is a model generated by machine learning using learning information including input information and information (label information) indicating whether the request indicated by the input information is a request regarding the first target for each input information. Such a model is, for example, GBDT, a neural network, etc., but is not limited to such examples.

[0217] When the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is not a request regarding the first target, the determination unit 35 performs implicit non-response determination processing to determine whether the request indicated by the request information is a request regarding a second target set as a response target.

[0218] As described above, the second target is a target set as a response target, and by determining whether it is not a request related to the second target, it is determined whether it is not a non-response target, and it can be said that it is a target that implicitly indicates a non-response target. Therefore, as described above, the second target that becomes a response target can be said to implicitly indicate a non-response determination criterion, and can be called an implicit non-response determination criterion. The second target includes, for example, a specified category if there is a category specified by the user U.

[0219] For example, the determination unit 35 can determine whether the request indicated by the request information is a request related to the second target set as the response target by using the second language model. The second language model is, for example, a large language model such as a transformer-based model or an RNN-based model, but is not limited to such examples. Note that the second language model may be the same language model as the first language model.

[0220] The determination unit 35 inputs, as input information, information including instruction information instructing the output of information indicating whether the request indicated by the request information is a request related to the second target and information indicating the request indicated by the request information included in the usage request to the second language model.

[0221] In this case, information indicating whether it is a request related to the second target is output from the second language model. In this way, the determination unit 35 can determine whether the request indicated by the request information is a request related to the second target set as the response target by using the second language model.

[0222] The instruction information includes information indicating the second target, but the information indicating the second target may be included in the input information separately from the instruction information. Further, the instruction information may include information indicating the request included in the request information.

[0223] The indication information includes information indicating the output format. The information indicating the output format includes, for example, information to be output when the request indicated by the request information is a request regarding the second target, and information to be output when the request indicated by the request information is not a request regarding the second target.

[0224] The information to be output when the request indicated by the request information is a request regarding the second target is, for example, "true", and the information to be output when the request indicated by the request information is not a request regarding the second target is, for example, "false", but is not limited to such examples.

[0225] The information indicating the output format may be information in a format that causes specific information to be output only when the request indicated by the request information is a request regarding the second target. For example, the information indicating the output format is information in the character string "when the request indicated by the request information is a request regarding the second target, output 'applicable', and otherwise, output nothing", but is not limited to such examples. Also, the information indicating the output format may be information indicating a sample of the output.

[0226] In the above example, the indication information is included in the input information input to the second language model, but the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the indication information.

[0227] Also, the second language model may be a language model learned to output information indicating whether the request indicated by the request information is a request regarding the second target from input information that does not include the indication information and the information indicating the second target.

[0228] The determination unit 35 can also determine, instead of or in addition to the second language model, in natural language processing without using the second language model, whether the request indicated by the request information is a request regarding the second target set as the response target.

[0229] Natural language processing that does not use a second language model is, for example, keyword-based natural language processing. The determination unit 35 has, for example, a second target dictionary including a plurality of keywords each directly or indirectly indicating a second target, and determines whether a keyword included in the second target dictionary is included in the request information.

[0230] When a keyword included in the second target dictionary is included in the request information, the determination unit 35 determines that the request indicated by the request information is a request regarding the second target set as a response target, and otherwise determines that the request indicated by the request information is not a request regarding the second target set as a response target.

[0231] Also, keyword-based natural language processing may be natural language processing using a model. The model in this case is a model generated by machine learning using learning information including input information and information (label information) indicating whether the request indicated by the input information is a request regarding a second target for each input information. Such a model is, for example, GBDT, a neural network, etc., but is not limited to such examples.

[0232] Subsequently, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information received by the reception unit 31 based on the determination result as to whether the request indicated by the request information received by the reception unit 31 is a request regarding a second target.

[0233] For example, when the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is a request regarding a second target, it determines to generate information indicating a response to the request indicated by the request information.

[0234] Also, when the determination unit 35 determines that the request indicated by the request information received by the reception unit 31 is not a request regarding a second target, it determines not to generate information indicating a response to the request indicated by the request information.

[0235] In this way, the determination unit 35 first performs an explicit non-response determination process. When the explicit non-response determination criteria are not satisfied, it performs an implicit non-response determination process. When the implicit non-response determination criteria are not satisfied, it makes a response. As a result, the determination unit 35 can perform a response availability determination with low cost and high accuracy, and can suppress inappropriate responses.

[0236] Note that the determination unit 35 can perform the explicit non-response determination process and the implicit non-response determination process in parallel. In this case, the determination unit 35 determines whether to generate information indicating a response to the request indicated by the request information based on the determination result of the explicit non-response determination process and the determination result of the implicit non-response determination process.

[0237] In this case, when the request indicated by the request information is not a request related to the first target and the request indicated by the request information is a request related to the second target, the determination unit 35 determines to generate information indicating a response to the request indicated by the request information. Otherwise, it determines not to generate information indicating a response to the request indicated by the request information. Also by this means, the information processing apparatus 1 can suppress inappropriate responses.

[0238] Also, the determination unit 35 can perform the processes in the order of the implicit non-response determination process and the explicit non-response determination process. In this case, the explicit non-response determination process can be performed only when it is determined in the implicit non-response determination process that the request indicated by the request information is a request related to the second target. Also by this means, the information processing apparatus 1 can suppress inappropriate responses.

[0239] In this way, the determination unit 35 can determine whether to generate information indicating a response to the request indicated by the request information based on whether the request indicated by the request information is a request related to the first target set as a non-response target and a request related to the second target set as a response target.

[0240] 〔3.3.7. Providing Unit 36〕 The providing unit 36 provides various types of information to the user U and the employee O. For example, the providing unit 36 provides various types of information to the user U by transmitting the various types of information to the terminal device 2 via the communication unit 10 and the network N. Also, the providing unit 36 provides various types of information to the employee O by transmitting the various types of information to the terminal device 3 via the communication unit 10 and the network N.

[0241] When it is determined by the determination unit 35 that the response information is to be generated, the providing unit 36 provides the user U with the response information generated using the generation AI as information indicating the response to the request indicated by the request information received by the reception unit 31.

[0242] For example, the providing unit 36 provides the user U with the response information generated using the generation AI by transmitting the response information generated using the generation AI generated by the generation unit 32 to the terminal device 2 that transmitted the utilization request.

[0243] For example, when the request indicated by the request information is a question of the Q&A service and the user U accesses the page showing the answer to the user U's question, the providing unit 36 can provide the user U with the page showing the answer to the user U's question.

[0244] Also, when the response information is not generated using the generation AI in the generation unit 32, the providing unit 36 provides the non-response information to the user U. For example, when it is determined by the determination unit 35 that the request indicated by the request information is a request regarding the first target, or when it is determined by the determination unit 35 that the request indicated by the request information is not a request regarding the second target, the providing unit 36 provides the non-response information to the user U.

[0245] The non-response information is, for example, the information of the character string "Sorry. This is a non-response target and I cannot respond to the request." For example, when the request indicated by the request information is a request for a Q&A service, the non-response information is, for example, the information of the character string "Sorry. I cannot answer because it is a question outside the scope of answers." Note that the non-response information may include information indicating that the request is related to the first target or not related to the second target.

[0246] [4. Processing Procedure] Next, the information processing procedure by the processing unit 12 of the information processing apparatus 1 according to the embodiment will be described. FIG. 14 is a flowchart showing an example of information processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0247] As shown in FIG. 14, the processing unit 12 of the information processing apparatus 1 determines whether or not it has received request information (step S10). When the processing unit 12 determines that it has received the request information (step S10: Yes), it performs request processing (step S11). The request processing in step S11 is the processing of steps S20 to S24 shown in FIG. 15, which will be described in detail later.

[0248] When the processing in step S11 is completed, or when it is determined that the request information has not been received (step S10: No), the processing unit 12 determines whether or not it is the timing for generating non-response target information (step S12). The timing for generating non-response target information is, for example, the timing when a generation request comes from the terminal device 3 or the timing that arrives at a predetermined cycle, but is not limited to such an example.

[0249] When the processing unit 12 determines that it is the timing for generating non-response target information (step S12: Yes), it performs non-response target information generation processing (step S13). The non-response target information generation processing in step S13 is the processing of steps S30 to S33 shown in FIG. 16, which will be described in detail later.

[0250] When the process in step S13 ends, or when it is determined that it is not the non-response target information generation timing (step S12: No), the processing unit 12 determines whether it is the operation end timing (step S14). The processing unit 12 determines that it is the operation end timing, for example, when the power of the information processing apparatus 1 is turned off.

[0251] When it is determined that it is not the operation end timing (step S14: No), the processing unit 12 shifts the process to step S10. When it is determined that it is the operation end timing (step S14: Yes), the processing shown in FIG. 14 ends.

[0252] FIG. 15 is a flowchart showing an example of the request process by the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 15, the processing unit 12 of the information processing apparatus 1 determines whether the request indicated by the request information received in step S10 is a request regarding the first target (step S20).

[0253] Subsequently, when it is determined that the request indicated by the request information is not a request regarding the first target (step S20: No), the processing unit 12 determines whether the request indicated by the request information is a request regarding the second target (step S21). When it is determined that the request indicated by the request information is a request regarding the second target (step S21: Yes), the processing unit 12 generates response information which is information indicating a response corresponding to the request indicated by the request information (step S22). Then, the response information generated in step S22 is provided (step S23).

[0254] When it is determined that the request indicated by the request information is a request regarding the first target (step S20: Yes), or when it is determined that the request indicated by the request information is not a request regarding the second target (step S21: No), the processing unit 12 provides non-response information (step S24).

[0255] FIG. 16 is a flowchart showing an example of non-response target information generation processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 16, the processing unit 12 determines a combination of two or more non-response target information among a plurality of non-response target information (step S30).

[0256] Subsequently, the processing unit 12 determines the non-response accuracy of the combination of two or more non-response target information (step S31). Then, the processing unit 12 determines whether or not all combinations of the combination of two or more non-response target information have been determined (step S32).

[0257] When the processing unit 12 determines that not all combinations have been determined (step S32: No), it proceeds to step S30. Also, when the processing unit 12 determines that all combinations have been determined (step S32: Yes), it selects the combination of two or more non-response target information with the highest non-response accuracy as information to be used in the explicit non-response determination processing (step S33), and ends the processing shown in FIG. 16.

[0258] [5. Modification Example] In the above example, the generation AI has been described as a text generation AI. However, the generation AI is not limited to the text generation AI, and may be an image generation AI, a multimodal generation AI, or the like. The image generation AI is an AI that generates an image from text. For example, it includes, but is not limited to, StackGAN (Generative Adversarial Networks), AttnGAN, T2I (Text-to-Image) with Transformers, Diffusion models, etc.. Examples of Diffusion models include DALL-E and Stable-Diffusion.

[0259] The multimodal generation AI is a generation AI that generates at least one of text, images, and audio from at least one of text, images, and audio. The multimodal generation AI includes, for example, GPT-4 Turbo with vision, gemini, CM3Leon (Chameleon Multimodal Model), etc., but is not limited to such examples.

[0260] In addition, in the above example, an example in which a non-response category corresponding to a specified category is preset as the first target was described. However, the present invention is not limited to such an example. Instead of the non-response category corresponding to the specified category, a non-response target corresponding to the specified category may be preset as the first target. Also in this case, the generation unit 32 can generate non-response target information by the same process as in the case of the non-response category based on the information indicating the non-response target preset as the first target for the specified category.

[0261] [6. Hardware Configuration] The information processing apparatus 1 according to the above-described embodiment is realized by a computer 80 having a configuration as shown in FIG. 17, for example. FIG. 17 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing apparatus 1 according to the embodiment. The computer 80 includes a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0262] The CPU 81 operates based on a program stored in the ROM 83 or the HDD 84 and controls each part. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 is started up, a program dependent on the hardware of the computer 80, and the like.

[0263] The HDD 84 stores programs executed by the CPU 81, data used by such programs, and the like. The communication interface 85 receives data from other devices via the network N (see FIG. 2), sends it to the CPU 81, and transmits the data generated by the CPU 81 to other devices via the network N.

[0264] The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards or mice, via the input / output interface 86. The CPU 81 acquires data from the input device via the input / output interface 86. Also, the CPU 81 outputs the data generated via the input / output interface 86 to the output device.

[0265] The media interface 87 reads a program or data stored in the recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads such a program from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0266] For example, when the computer 80 functions as the information processing apparatus 1 according to the embodiment, the CPU 81 of the computer 80 realizes the functions of the processing unit 12 by executing the program loaded on the RAM 82. Also, the data in the storage unit 11 is stored in the HDD 84. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be acquired from other devices via the network N.

[0267] 〔7. Others〕 Also, among the respective processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, regarding the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0268] Further, each component of each device shown in the drawings is a functional concept, and it is not necessarily physically configured as shown in the drawings. That is, the specific form of the distribution and integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0269] For example, the information processing apparatus 1 described above may be realized by a terminal device and a server computer, or may be realized by a plurality of server computers. Also, depending on the function, the configuration can be flexibly changed, such as by calling an external platform or the like through an API or network computing.

[0270] In addition, the above-described embodiments and modification examples can be appropriately combined within a range that does not conflict with the processing content.

[0271] 〔8. Effects〕 As described above, the information processing apparatus 1 according to the embodiment includes a generation unit 32, a reception unit 31, and a determination unit 35. The generation unit 32 generates non-response target information indicating a non-response target. The reception unit 31 receives information indicating a request from the user U. The determination unit 35 determines, based on a plurality of non-response target information generated by the generation unit 32, whether the request indicated by the information received by the reception unit 31 is a request regarding the non-response target. Thereby, the information processing apparatus 1 can suppress inappropriate responses.

[0272] Further, the generation unit 32 generates non-response target information based on non-response category information indicating a non-response category that is different from the designated category, which is the category designated by the user U, and is associated in advance with respect to the designated category. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0273] Further, the generation unit 32 generates non-response target information using a language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0274] Further, the generation unit 32 generates, as non-response target information, information indicating the risk in the non-response category using a language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0275] Further, the generation unit 32 includes a first generation processing unit 40 that generates risk information indicating the risk in the non-response category using a language model, and a second generation processing unit 41 that aggregates, using a language model, the risk information generated by the first generation processing unit 40 into risk information equal to or less than a preset number as non-response target information. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0276] Further, the generation unit 32 generates, as non-response target information, information partially including characteristic words extracted from the non-response category information using a language model. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0277] Further, the information processing apparatus 1 includes a selection unit 34 that selects two or more pieces of non-response target information to be used by the determination unit 35 from among the plurality of pieces of non-response target information generated by the generation unit 32, based on the non-response accuracy for each combination of two or more pieces of non-response target information. The determination unit 35 determines whether or not a request indicated by the information received by the reception unit 31 is a request related to a non-response target, based on the two or more pieces of non-response target information selected by the selection unit 34. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0278] Further, the information processing apparatus 1 includes an evaluation unit 33 that evaluates the non-response accuracy for each combination of two or more pieces of non-response target information, and the selection unit 34 selects two or more pieces of non-response target information to be used by the determination unit 35, based on the evaluation result by the evaluation unit 33. Thereby, the information processing apparatus 1 can further suppress inappropriate responses.

[0279] As described above, the embodiments of the present application have been described in detail based on the drawings, but this is an example, and the present invention can be implemented in other forms in which various modifications and improvements are made based on the knowledge of those skilled in the art, including the aspects described in the column of the disclosure of the invention.

[0280] Also, the above-described "section (section, module, unit)" can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

Explanation of Reference Numerals

[0281] 1 Information processing apparatus 2, 3 Terminal device 10 Communication unit 11 Storage unit 12 Processing unit 20 User information storage unit 21 Non-response target information storage unit 22 Evaluation request information storage unit 30 Acquisition unit 31 Reception unit 32 Generation unit 33 Evaluation Unit 34 Selection Unit 35 Determination Unit 36 Provision Unit 40 First Generation Processing Unit 41 Second Generation Processing Unit 42 Third Generation Processing Unit 43 Fourth Generation Processing Unit 100 Information Processing System N Network

Claims

1. A generation unit that generates non-response target information indicating non-response targets; A reception unit that receives information indicating a user's request; A determination unit that determines whether or not the request indicated by the information received by the reception unit is a request related to the non-response target based on the non-response target information generated by the generation unit. An information processing apparatus comprising: An information processing apparatus characterized by the above.

2. The generation unit: Based on non-response category information indicating a non-response category that is different from a designated category designated by the user and is associated in advance as a non-response category to the designated category, generates the non-response target information. The information processing apparatus according to claim 1, characterized by the above.

3. The generation unit: Generates the non-response target information using a language model. The information processing apparatus according to claim 2, characterized by the above.

4. The generation unit: Generates information indicating a risk in the non-response category as the non-response target information using the language model. The information processing apparatus according to claim 3, characterized by the above.

5. The generation unit: A first generation processing unit that generates risk information indicating a risk in the non-response category as the non-response target information using the language model; A second generation processing unit that aggregates, using the language model, the risk information generated by the first generation processing unit into risk information equal to or less than a preset number as the non-response target information. An information processing apparatus comprising: The information processing apparatus according to claim 4, characterized by the above.

6. The generation unit: Generates, using the language model, information including a part of characteristic words extracted from the non-response category information as the non-response target information. The information processing apparatus according to claim 3, characterized by the above.

7. A selection unit that selects two or more pieces of the non-response target information among the plurality of pieces of non-response target information used by the determination unit based on the non-response accuracy for each combination of two or more pieces of the non-response target information among the plurality of pieces of non-response target information generated by the generation unit; The determination unit: Determines whether or not the request indicated by the information received by the reception unit is a request related to the non-response target based on two or more pieces of the non-response target information selected by the selection unit. The information processing apparatus according to any one of claims 2 to 6, characterized by the above.

8. An evaluation unit that evaluates the non-response accuracy for each combination of two or more pieces of the non-response target information. The selection unit selects two or more pieces of the non-response target information used by the determination unit based on the evaluation result by the evaluation unit. The information processing apparatus according to claim 7, characterized in that.

9. An information processing method executed by a computer, comprising: a generation step of generating non-response target information indicating a non-response target; a reception step of receiving information indicating a user's request; a determination step of determining whether or not the request indicated by the information received in the reception step is a request regarding the non-response target based on the non-response target information generated in the generation step. The information processing method, characterized in that.

10. a generation procedure of generating non-response target information indicating a non-response target; a reception procedure of receiving information indicating a user's request; causing a computer to execute a determination procedure of determining whether or not the request indicated by the information received in the reception procedure is a request regarding the non-response target based on the non-response target information generated in the generation procedure. The information processing program, characterized in that.

Images