Information processing system, information processing method, and program

Abstract

To provide an information processing system, method and program for enabling a service provider to efficiently answer.SOLUTION: In the information processing system 1, an information processing method by the server apparatus includes a data accepting step of accepting an answered question related to security of a service provided by a service provider, an inputted answer of the service provider to the answered question, and designation of a category to be given to the inputted answer, and adding answer generation data including the answered question, the inputted answer, and the category to a database, a designation accepting step of accepting designation of a question for security and designation of a category, and an answer generating step of extracting, as a reference answer, an inputted answer to an answered question corresponding to the designated category and similar to the designated question among inputted answers included in accumulated answer generation data, and generating an answer to the question and a correlation based on the reference answer and reference information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 describes a system audit support system that includes a checklist database that stores checkpoint array information data in a regular order to identify each checkpoint when auditing an audit target, and also stores evidence array information data corresponding to the checkpoint array information data to identify the evidence required to evaluate each checkpoint. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-44658 Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable to provide a system that allows the person answering the question to answer efficiently.

[0005] In view of the above circumstances, the present invention provides an information processing system and the like that enables service providers to respond efficiently. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing system comprising at least one processor, the processor being configured to execute the following steps by reading a program: a data receiving step receiving answered questions regarding the security of a service provided by a service provider, answers entered by the service provider to the answered questions, and a designation of a category to be assigned to the entered answers; answer generation data including the answered questions, entered answers, and categories being added to a database; a designation receiving step receiving designations of security questions and designations of categories; an answer generation step extracting, from among the entered answers included in the accumulated answer generation data, input answers to answered questions that correspond to the designated category and are similar to the designated question as reference answers; and generating an answer to the question based on the reference answers and reference information; wherein the reference information includes a correlation between the reference answers and the answers to the questions.

[0007] According to this aspect, combinations of answered questions and answers are stored together with the category of the answer, so that by specifying a category for an unanswered question, a highly accurate answer can be generated by referencing answers to similar past questions about the same type of service, allowing service providers to efficiently create answers to security-related questions. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. FIG. [Figure 2] 2 is a block diagram showing the hardware configuration of the server device 10 and the provider terminal 20. FIG. [Figure 3] FIG. 2 is a block diagram showing the hardware configuration of an evaluator terminal 30 and a user terminal 40. [Figure 4] 1 is a block diagram showing functions realized by a server device 10 (controller 11), a provider terminal 20 (controller 21), an evaluator terminal 30 (controller 31), and a user terminal 40 (controller 41). [Figure 5] FIG. 10 is a diagram showing an example of a data entry screen DD displayed on the provider terminal 20. [Figure 6] 10 is a diagram showing an example of an evaluation information selection screen ED displayed on the provider terminal 20. FIG. [Figure 7] 10 is a diagram showing an example of a management screen MD of data for generating an answer displayed on the provider terminal 20. FIG. [Figure 8] FIG. 10 is a diagram showing an example of a question input screen QD displayed on the provider terminal 20. [Figure 9] FIG. 10 is a diagram showing an example of a service selection screen SD displayed on the provider terminal 20. [Figure 10] FIG. 10 is a diagram showing an example of a question input screen QD in a state where a service name has been selected and a question has been input. [Figure 11] FIG. 10 is a diagram showing an example of an answer display screen AD displayed on the provider terminal 20. [Figure 12] 1 is an activity diagram showing an example of the flow of information processing (answer generation processing) executed by the information processing system 1. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, various information processing according to an embodiment may realize input and output corresponding to the input. Here, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited as long as an output is obtained as a result of the input. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression formula constructed using a statistical method), a trained model that has previously trained the correlation between input and output, or a large-scale language model that can output a desired result by inputting a prompt.

[0012] In one embodiment, a "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on a circuit in the broad sense.

[0013] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, processor, memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0014] 1. Hardware Configuration This section explains the hardware configuration.

[0015] <Information Processing System 1> 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes a communication line 2, a server device 10, a plurality of provider terminals 20, at least one evaluator terminal 30, and a plurality of user terminals 40. The server device 10, the provider terminal 20, the evaluator terminal 30, and the user terminal 40 are configured to be able to communicate with each other via the communication line 2. The connection between the server device 10, the provider terminal 20, the evaluator terminal 30, and the user terminal 40 may be wired or wireless.

[0016] The information processing system 1 constitutes, for example, at least a part of a security evaluation system for Internet services used by multiple service providers (a first service provider U1 and a second service provider U2), at least one evaluator U3, and multiple service users (a first service user U4 and a second service user U5). The information processing system 1 mainly evaluates the security of services provided by service providers and provides security evaluations, and for example, the information processing system 1 mainly provides a security evaluation service by evaluators. In one embodiment, the information processing system 1 is composed of one or more devices or components. These components will be described below.

[0017] <Server device 10> Fig. 2 is a block diagram showing the hardware configuration of the server device 10 and the provider terminal 20. As shown in Fig. 2A, the server device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a communication bus 14. The control unit 11, the storage unit 12, and the communication unit 13 are electrically connected via the communication bus 14 inside the server device 10.

[0018] <Control unit 11> The control unit 11 processes and controls the overall operations related to the server device 10. The control unit 11 is, for example, a central processing unit (CPU). The control unit 11 realizes various functions related to the server device 10 by reading out predetermined programs stored in the storage unit 12. In other words, information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. These will be described in more detail in the next section. Note that the control unit 11 is not limited to being single, and there may be multiple control units 11 for each function. Furthermore, the control unit 11 may be a combination of these.

[0019] <Storage section 12> The memory unit 12 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the server device 10 executed by the control unit 11, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program operations. The memory unit 12 stores various programs, variables, etc. related to the server device 10 executed by the control unit 11.

[0020] <Communications Department 13> The communication unit 13 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), or wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, or BLUETOOTH (registered trademark) communication as needed. That is, it is more preferable to implement it as a collection of multiple communication means. That is, the server device 10 may communicate various information from the outside via the communication unit 13 and the network.

[0021] The server device 10 may be an on-premise server or a cloud server. The cloud server device 10 may provide the above-described functions and processes in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0022] <Provider terminal 20> 2B, the provider terminal 20 includes a control unit 21, a memory unit 22, a communication unit 23, an input unit 24, an output unit 25, and a communication bus 26. The control unit 21, the memory unit 22, the communication unit 23, the input unit 24, and the output unit 25 are electrically connected via the communication bus 26 inside the provider terminal 20. The provider terminal 20 is an information processing terminal used by a service provider that provides IT services. The explanation of the control unit 21, the memory unit 22, and the communication unit 23 is omitted because they are the same as the explanation of each unit in the server device 10.

[0023] <Input section 24> The input unit 24 accepts operation inputs made by the user. The operation inputs are transferred as command signals to the control unit 21 via the communication bus 26. The control unit 21 can execute predetermined control or calculations based on the transferred command signals as necessary. The input unit 24 may be included in the housing of the provider terminal 20 or may be externally attached. For example, the input unit 24 may be implemented as a touch panel integrated with the output unit 25. When the input unit 24 is implemented as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 24. Instead of a touch panel, a switch button, a mouse, a trackpad, a QWERTY keyboard, etc. can be used as the input unit 24.

[0024] <Output section 25> The output unit 25 displays a screen of a graphical user interface (GUI) that can be operated by the user. The output unit 25 may be included in the housing of the donor terminal 20 or may be attached externally. Specifically, the output unit 25 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. It is preferable that these display devices are implemented by selectively using them depending on the type of donor terminal 20.

[0025] <Evaluator terminal 30> 3A is a block diagram showing the hardware configuration of the evaluator terminal 30 and the user terminal 40. As shown in FIG. 3A, the evaluator terminal 30 includes a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, an output unit 35, and a communication bus 36. The control unit 31, the storage unit 32, the communication unit 33, the input unit 34, and the output unit 35 are electrically connected within the evaluator terminal 30 via the communication bus 36.

[0026] The evaluator terminal 30 is an information processing terminal used by an evaluator who evaluates the answers of service providers. The evaluator is an organization such as a company that evaluates services provided by service providers (for example, a company that evaluates security of cloud services, etc.) or a person in charge thereof. The evaluator evaluates the service, for example, from the evaluator terminal 30 via the information processing system 1. The description of the control unit 31, memory unit 32, communication unit 33, input unit 34, and output unit 35 of the evaluator terminal 30 is omitted because they are the same as the description of each unit in the provider terminal 20.

[0027] <User terminal 40> 3B, the user terminal 40 includes a control unit 41, a memory unit 42, a communication unit 43, an input unit 44, an output unit 45, and a communication bus 46. The control unit 41, the memory unit 42, the communication unit 43, the input unit 44, and the output unit 45 are electrically connected via the communication bus 46 inside the user terminal 40. The user terminal 40 is an information processing terminal used by a user of the security evaluation service provided by the server device 10. The description of the control unit 41, the memory unit 42, the communication unit 43, the input unit 44, and the output unit 45 will be omitted as they are the same as the description of each unit in the provider terminal 20.

[0028] 2. Functional configuration This section describes the functional configuration of this embodiment. Information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11 (at least one processor included in the information processing system 1).

[0029] FIG. 4 is a block diagram showing functions realized by the server device 10 (controller 11), provider terminal 20 (controller 21), evaluator terminal 30 (controller 31), and user terminal 40 (controller 41).

[0030] As shown in FIG. 4A, the server device 10 (control unit 11) includes a basic display control unit 111, an answer registration unit 112, an evaluation receiving unit 113, a data receiving unit 114, a designation receiving unit 115, an answer generation unit 116, a report output unit 117, and an artificial intelligence unit 120. As shown in FIG. 4B, the provider terminal 20 (control unit 21) includes a display unit 211 and an operation acquisition unit 212. As shown in FIG. 4C, the evaluator terminal 30 (control unit 31) includes a display unit 311 and an operation acquisition unit 312. As shown in FIG. 4D, the user terminal 40 (control unit 41) includes a display unit 411 and an operation acquisition unit 412.

[0031] <Basic display control unit 111> The basic display control unit 111 is configured to display various information on the provider terminal 20, the evaluator terminal 30, and the user terminal 40. For example, the basic display control unit 111 displays an input form in which the service provider inputs answers, answers registered by the service provider, evaluation results of the answers by the evaluators, etc. on the display unit 211 of the provider terminal 20, the display unit 311 of the evaluator terminal 30, or the display unit 411 of the user terminal 40.

[0032] <Answer Registration Unit 112> The answer registration unit 112 is configured to register information related to the security of the service provided by the service provider in order to create a report or a report to be presented to the service user. Specifically, the answer registration unit 112 displays a predetermined input form for accepting input of answers to security-related questions from the service provider, and registers the answers input in the input form as evaluation information.

[0033] The service provider inputs a response according to service attributes such as the type of target service, the destination of the service, the type of plan included in the service, etc. In other words, a service provider that offers multiple services and plans inputs a response for each of these service attributes, and the response registration unit 112 registers evaluation information for each service attribute.

[0034] "Services" provided by a service provider include services provided via the Internet, such as SaaS, IaaS (Infrastructure as a Service), PaaS (Platform as a Service), and cloud services, and "service providers" include cloud service providers, etc. Furthermore, "services" provided by a service provider may be services provided to business partners, customers, etc. "Questions" are matters for which a service provider is requested to respond, and may include, for example, matters regarding the security of the service provider itself and the security of the services provided by the service provider. Furthermore, "questions" may include matters regarding the security of other businesses in the service provider's supply chain. "Other businesses" include, for example, business partners of the service provider, such as businesses to which the service provider outsources its own business (including contractors, subcontractors, sub-subcontractors, etc.). "Questions" may include so-called security checklists.

[0035] "Evaluation information" consists of the service provider's answers to multiple questions regarding the security of the service, etc., and is the information that is the subject of evaluators' content checks and evaluations. Evaluation information may be evaluated only once or multiple times. In other words, evaluations of evaluation information may include initial evaluations (so-called reviews), interim evaluations, final evaluations, etc. Initial evaluations and interim evaluations are evaluations in which further evaluations are conducted after the initial evaluations and interim evaluations. Multiple evaluations may be performed by the same evaluator, or by different evaluators (so-called reviewers, etc.). The final evaluation creates an evaluation that is used in a report or report provided to the service user. Furthermore, evaluation information may be evaluated again by an evaluator when the service provider makes corrections, updates, etc.

[0036] The evaluation results of the evaluation information by the evaluator are used as part of a report or a written report, or as reference information. Note that the "service user" in the information processing system 1 includes not only individuals or organizations that use the service to be evaluated, but also individuals or organizations that request an evaluation of a specific service, such as a company that wants to evaluate the security of a cloud service or the like that it uses, or a company that wants to confirm that evaluation.

[0037] The evaluation information may include answers and questions corresponding to the answers, and may also include service attributes (information indicating the type of service, the recipient of the service, the type of plan included in the service, or a combination thereof).

[0038] The evaluation information is registered, for example, in an answer database in the storage unit 12. The answers to each question included in the evaluation information are answers entered by the service provider into each question presented in the input form, and are registered in the answer database. The answers to one question may be hierarchical.

[0039] "Registering evaluation information" includes both registration with answers entered for all questions (full registration) and registration with answers entered for only some questions (with some answers left blank) (saving as a draft). "Saving as a draft" may also include registration with answers entered for all questions but before submitting the answers to the evaluator.

[0040] Questions in the evaluation information include, for example, questions to confirm (request answers from the service provider) security certification or evaluation, service level, scope of responsibility, security measures within the service provider, handling of data created by service users, and service development, maintenance, or operation policies (account management, etc.).

[0041] <Evaluation Reception Department 113> The evaluation receiving unit 113 is configured to receive evaluations from evaluators on the evaluation information registered by the response registration unit 112, and to register the evaluated evaluation information together with the evaluation results in a response database or the like. "Evaluation" refers to the act of checking whether or not there are any deficiencies in the answers included in the evaluation information, making comments on the content of the answers, assessing the content of the answers, scoring the answers based on predetermined criteria, etc. As described above, evaluation may be performed by multiple evaluators, or one evaluator may request other evaluators to evaluate the registered evaluation information. In this case, the evaluation receiving unit 113 may receive evaluations from other evaluators after one evaluator has performed their evaluation.

[0042] For example, if evaluation is performed in two stages, the initial evaluation (review) involves checking for deficiencies and creating comments. Subsequently, the evaluation information after the initial evaluation is subjected to a final evaluation, which involves assessing and scoring the content (answers) based on predetermined criteria, creating comments on the entire evaluation information, and creating a report or report (e.g., a security report) including the results. The report or report is created by the final evaluator or an evaluation unit included in the control unit 11. The evaluation unit is configured to generate a part or the whole of the report or report based on the input evaluation information. The evaluation results of the evaluation information may be included in the report or report as part of the report or report or as reference information. The report or report is transmitted to the user terminal 40 of the service user who requested the report or report.

[0043] The evaluation receiving unit 113 receives input of characters, sentences, etc. used for evaluation from the evaluator terminal 30. Evaluation is typically performed on evaluation information in which all questions have been answered. Evaluation information that has unanswered questions (questions for which answers are not registered) may not be subject to evaluation, and evaluations may not be received by the evaluation receiving unit 113. Furthermore, even if evaluation information includes unanswered questions (questions for which answers are not registered), only the answered questions may be subject to partial evaluation.

[0044] <Data Reception Unit 114> The data accepting unit 114 is configured to acquire and record data for automatically generating answers to questions in the evaluation information. Specifically, the data accepting unit 114 accepts answered questions regarding the security of services provided by the service provider, answers already entered by the service provider to the answered questions, and a category designation to be assigned to the entered answers, and adds answer generation data including the answered questions, the entered answers, and the categories to the generation database. One piece of answer generation data includes at least data on the answered question, data on the entered answers, and information indicating the category. Note that the data on the answered questions and the data on the entered answers may each be text data, vector data (described below), or both.

[0045] The category assigned to the input answer is information indicating the type of service, the service provider (the service user's attributes, such as the service user's organization name), the type of plan included in the service, or a combination of these. This allows the input answer to be labeled using the type of service or plan provided by the service provider. As a result, the accuracy of the answer generated by the answer generation unit 116 is improved. It also becomes possible to generate an answer according to the type of service or plan used by the service user.

[0046] "Service type" may be interpreted as the service name, and indicates the type or name of the service that the input answer targets. For example, if a service provider offers Service A and Service B, at least one of "Service A" and "Service B" may be assigned to the input answer as the service type. "Type of plan included in the service" refers to one or more types of contract plans set for the same service (e.g., Service A), such as a "Standard Plan" or a "Premium Plan." Multiple plans set for the same service may differ from each other in, for example, the available functions, usage period, usage fees, etc.

[0047] Answered questions included in answer generation data may not be assigned a category, or may be assigned a category (hereinafter, question category) different from the category assigned to the input answer. Also, a question category may be assigned to a question before it is answered. The question category is used as a filtering condition for answer generation data, for example, when displaying answer generation data or generating an answer using the answer generation data.

[0048] Question categories classify the content of answered questions or pre-answer questions. For example, an answered question or pre-answer question about a contract is assigned the question category "contract," and an answered question or pre-answer question about company policies or rules is assigned the question category "policy / rule." In other words, the categories assigned to entered answers and the question categories assigned to answered questions or pre-answer questions are based on different perspectives.

[0049] When a question category is assigned to an answered question or an unanswered question, for example, the data accepting unit 114 may accept a selection of a question category from a predefined group of categories at the provider terminal 20. This group of categories may consist of categories different from the categories assigned to the entered answer. Alternatively, the data accepting unit 114 may automatically assign a question category by referring to the questions and their question categories included in the generation database. For example, the data accepting unit 114 may extract stored questions similar to the received question and assign the question category assigned to the extracted question to the received question.

[0050] Furthermore, the data accepting unit 114 may input a question to a question categorization model and cause the question categorization model to output a question category. The question categorization model is a learning model that is trained so that it can take a question as input and output a question category.

[0051] The question categorization model is a learning model trained using a combination of a question and a question category corresponding to the question as training data. The question categorization model may also be a generative AI including a large-scale language model. In this case, the data accepting unit 114 receives a question as input, inputs a prompt including an instruction to output a question category to be assigned to the question to the question categorization model, and causes the question categorization model to output the question category. The data accepting unit 114 may generate a prompt that instructs the question categorization model to output a question category to be assigned to the question, and input the prompt to the question categorization model. The data accepting unit 114 may also input a prompt into the question categorization model, in which, in addition to the question and the instruction to output the question category, one or more question samples and one or more corresponding question category samples are inserted as input and output samples. The question categorization model outputs a question category to be assigned to the question according to the input prompt.

[0052] The answer generation data may include first answer generation data based on the answered question and the entered answer input by the service provider without using the input form provided by the answer registration unit 112, and second answer generation data based on the evaluation information. Furthermore, the answer generation data may include third answer generation data based on the answer generated by the answer generation unit 116. Therefore, the data accepting unit 114 may acquire answer generation data through at least one of a first data acquisition flow for acquiring first answer generation data and a second data acquisition flow for acquiring second answer generation data, and a third data acquisition flow for acquiring third answer generation data, and may be configured to be able to execute all of the first data acquisition flow, the second data acquisition flow, and the third data acquisition flow.

[0053] In the first data acquisition flow, the data accepting unit 114 accepts input of data combining an answered question and an entered answer from the provider terminal 20. For example, the data accepting unit 114 displays a UI including a question input field, an answer input field, etc. on the provider terminal 20 and accepts input of the answered question and the entered answer. Furthermore, the data accepting unit 114 accepts input of a category for the entered entered answer from the provider terminal 20. The data accepting unit 114 generates first answer generation data in which the accepted category is assigned to the entered answer. This allows entered answers to which no category is assigned to be registered as first answer generation data to be used by the answer generation unit 116. Note that the input order (acceptance order) of the answered question, entered answer, and category does not matter.

[0054] For example, the data accepting unit 114 may refer to the types of services and plans provided that are registered in advance as user information by the service provider in the information processing system 1, and present these to the provider terminal 20 as candidates for "type of service" or "type of plan included in service," which is a type of category, and accept a category selection from the provider terminal 20. Furthermore, the data accepting unit 114 may present registered service users, such as organizations to which the service provider has provided reports or documents in the information processing system 1 in the past, as candidates for "service recipient," which is a type of category.

[0055] Furthermore, the data accepting unit 114 may accept input (designation) of multiple categories for one input answer. That is, the data accepting unit 114 may assign multiple categories to one input answer. For example, in the case of an answer that is common to multiple services, multiple service names are assigned to the answer as categories.

[0056] Furthermore, the service provider may not input a category for an input answer that does not depend on a category (i.e., it is not classified into a specific category and applies to any service, plan, recipient, etc.). Also, the data accepting unit 114 may accept a designation (input) of an "uncategorized category" for an input answer that does not depend on a category. For example, the data accepting unit 114 may accept a designation (input) of the "common" category as an uncategorized category.

[0057] The data accepting unit 114 may determine whether an answered question similar to the received answered question already exists in the generation database. For example, if answer generation data including an answered question similar to the received answered question is stored, the data accepting unit 114 may display on the provider terminal 20 a message indicating that the answered questions are duplicates. The data accepting unit 114 may also display on the provider terminal 20 the number of duplicately registered answered questions (the number of duplicates) or the answers already entered to the duplicately registered answered questions. Note that the determination of similarity between answered questions can be performed using the same means as for determining similarity between an answered question and a specified question, which will be described later. The data accepting unit 114 may also extract words contained in the answered question and determine that the two questions are similar if the degree of word duplication, the degree of similarity in word order, or the like, is equal to or exceeds a threshold.

[0058] 5 is a diagram showing an example of a data entry screen DD displayed on the provider terminal 20. The data entry screen DD includes a registration check field CC, a question entry field QC1, an answer entry field AC1, a memo field MC, a service selection field SC, and a past data comparison field PC.

[0059] The registration check box CC displays check boxes for selecting which of the answered questions and answers entered by the service provider in the question input field QC1 and answer input field AC1 to register as first answer generation data. One check box is provided per line (i.e., one check box for each question).

[0060] The service provider inputs a pair of an answered question and an input answer into the question input field QC1 and the answer input field AC1 of the same row, respectively. The question and answer are input, for example, by copying and pasting from reference data (e.g., an Excel file, a CSV file such as Google Spreadsheet) containing the question and answer prepared by the service provider on the provider terminal 20. The data accepting unit 114 may also accept uploads of reference data, extract the question and answer from the reference data, and display them in the question input field QC1 and the answer input field AC1. Furthermore, the data accepting unit 114 may also accept operations such as repeatedly copying cells on a matrix including the question input field QC1, the answer input field AC1, etc. on the data input screen DD.

[0061] The memo field MC is a field where the service provider can write comments. Information entered in the memo field MC is registered as part of the first answer generation data, for example, but is not referenced when generating an answer.

[0062] The service selection field SC is a field where the service provider inputs the service that is the target of the answer entered in the answer input field AC1. A selection button SB is arranged in the service selection field SC for selecting a service from a list. When an input operation is performed on the selection button SB, service candidates are displayed in the form of, for example, a pull-down menu. When the service provider selects a service type from the candidates, the service selected by the service provider (corresponding to the "category to be assigned to the entered answer") is displayed in the service selection field SC. In the case of an answer that corresponds to multiple services, the multiple services are displayed within a single service selection field SC. Note that in the example of FIG. 5, the service selection field SC displays the service name (type of service), but the service provider (for example, the name of the service user) and plan name (type of plan) may also be displayed. The service name may also be entered by text by the service provider.

[0063] If there are services or plans that provide different answers to the same question, the service provider may create and input data with the same question but different answers and service names. In other words, answered questions can be registered with the same content as long as the entered answers are different.

[0064] The past data comparison field PC displays the results of a comparison between the answered questions and input answers contained in the answer generation data already registered in the generation database (either only the first answer generation data, or any of the first answer generation data, second answer generation data, and third answer generation data). The comparison results include, for example, inconsistencies (differences) in the content of input answers for related answered questions, overlapping answered questions, etc. The display of overlapping answered questions in the past data comparison field PC may include links to display other overlapping answered questions, the number of overlapping answered questions, etc.

[0065] In the second data acquisition flow, the data accepting unit 114 creates second answer generation data from the evaluation information registered by the answer registration unit 112 and adds the data to the generation database. This allows answer generation data to be created using evaluation information created in advance, thereby reducing the effort required for the service provider to input answered questions, entered answers, etc.

[0066] The first answer generation information and the second answer generation information may be added to a single common generation database, or may be added to different sub-databases included in the generation database. That is, the data accepting unit 114 may register the first answer generation information in a first sub-database included in the generation database, and may register the second answer generation information in a second sub-database included in the generation database.

[0067] The data accepting unit 114, for example, presents a list of registered evaluation information to the provider terminal 20 and accepts a selection of evaluation information to be added to the generation database as second answer generation data from the provider terminal 20. The data accepting unit 114 accepts the selection of evaluation information by the provider terminal 20 as a specification of multiple answered questions and entered answers included in the evaluation information.

[0068] If the evaluation information includes information on service attributes, the data accepting unit 114 acquires the service attributes as the category of the input answer. That is, the data accepting unit 114 accepts the selection of evaluation information as a category designation. On the other hand, if the evaluation information does not include information on service attributes, the data accepting unit 114 accepts input of a category for the input answer (evaluation information) from the provider terminal 20.

[0069] The evaluation information that the data accepting unit 114 uses as source information for second answer generation data is registered (i.e., registered permanently or saved as a draft), regardless of whether it has been rated or not. That is, the data accepting unit 114 accepts both pre-evaluation evaluation information and evaluated evaluation information as source information for second answer generation data. Note that the data accepting unit 114 may accept only evaluated evaluation information as source information for second answer generation data. Note that in the case of evaluation information saved as a draft, the data accepting unit 114 uses only questions for which answers have been input to create answer generation data.

[0070] 6 is a diagram showing an example of the selection screen ED for evaluation information displayed on the provider terminal 20. The selection screen ED displays a list of evaluation information EI that has been registered in the answer database by the answer registration unit 112 after the service provider has answered the question. The evaluation information EI displayed on the selection screen ED displays information such as the identification name of the evaluation information (the service name in this example), whether or not there is an evaluation, the service name, and the service site URL.

[0071] The evaluation information EI listed on the selection screen ED includes both officially registered information and information saved as drafts. The officially registered evaluation information EI includes information that has been evaluated (reviewed) by an evaluator and information that has not yet been evaluated. "Evaluated" refers to a state in which an evaluation (review) has been performed by an evaluator at least once, and in cases where evaluations have been performed multiple times, this also includes information for which a final evaluation has not yet been performed. The data accepting unit 114 creates second answer generation data from one or more pieces of evaluation information EI selected from the selection screen ED and adds the data to the generation database.

[0072] In the third data acquisition flow, the data accepting unit 114 sets the specified question accepted by the specification accepting unit 115 (described later) as the answered question, and adds the answer to the specified question, which has been generated by the answer generating unit 116 and edited by the service provider, to the generation database as the entered answer. This allows the data generated by the answer generating unit 116 and further modified by the service provider to be used as new answer generation data, thereby improving the answer accuracy of the answer generating unit 116.

[0073] Furthermore, the data accepting unit 114 may add to the generation database third answer generation data in which the specified question for which the answer generating unit 116 was unable to generate an answer is treated as an answered question and the answer to the specified question input by the service provider is treated as an input answer. This makes it possible to add answer generation data for generating an answer to a question for which the answer generating unit 116 was unable to generate one.

[0074] The data accepting unit 114 may convert at least a part of the text data (answered questions and input answers) accepted as the source information of the answer generation data into vector data, and add the obtained vector data to the answer generation data. This enables the answer generating unit 116 to generate answers by referring to vector data prepared in advance (search for reference answers), thereby increasing the processing speed of answer generation. Specifically, the data accepting unit 114 may convert at least the answered questions into vector data.

[0075] 7 is a diagram showing an example of a management screen MD for answer generation data displayed on the provider terminal 20. The management screen MD includes an evaluation information display field EF, a knowledge display field NF, an add knowledge button B11, and an answer generation button B12.

[0076] The evaluation information display field EF displays a list of evaluation information received by the data receiving unit 114. Specifically, the evaluation information display field EF displays information such as the identification name of the evaluation information (the service name in this example), the last registration date and time, and the status. In the status, second answer generation data based on the evaluation information that has been added to the generation database (data that can be searched by the answer generating unit 116, which will be described later) is displayed as "registered." In addition, second answer generation data based on the evaluation information that has not yet been generated (for example, data that has not yet been converted into vector data) is displayed as "registering."

[0077] The knowledge display field NF includes a condition setting field CF, a knowledge edit button B13, and a knowledge list NL. The knowledge list NL displays a list of first answer generation data (knowledge) that the data receiving unit 114 added to the generation database. The condition setting field CF accepts input of conditions for narrowing down the knowledge to be displayed in the knowledge list NL. In the example of FIG. 7, the condition setting field CF accepts selection of the last update date and the category of the entered answer (related services, etc.). As shown in FIG. 7, when the category is set to "all," knowledge of all categories (service names) (including knowledge not assigned a category) is displayed. Note that in the knowledge display field NF, knowledge may be displayed in groups grouped by category.

[0078] When the knowledge edit button B13 in the knowledge display field NF is operated, a knowledge editing screen conforming to the data input screen DD of Figure 5 is displayed on the provider terminal 20, and the data receiving unit 114 receives editing of knowledge (answered questions, entered answers, or categories) from the provider terminal 20.

[0079] When the Add Knowledge button B11 is pressed on the management screen MD, the data entry screen DD in Fig. 5 is displayed on the provider terminal 20. Note that when no knowledge has been registered, the Add Knowledge button B11 may be displayed in the knowledge display field NF. When the Generate Answer button B12 is pressed, a screen (described later) for causing the answer generation unit 116 to generate an answer is displayed.

[0080] <Designation Reception Section 115> The specification receiving unit 115 is configured to receive specification of a security question and specification of a category from the provider terminal 20. The questions received by the specification receiving unit 115 include, for example, new questions to which the service provider has not yet answered, and questions that have been answered in another category (for example, a service name) but have not yet been answered in a new category (specified category).

[0081] The specification receiving unit 115 receives specification of a question, for example, by inputting a question from the provider terminal 20. Alternatively, the specification receiving unit 115 may receive specification of a question by calling up an input form for registering evaluation information in the provider terminal 20 (selection of a service, plan, etc. for which evaluation information is to be created). In this case, multiple questions to be included in the input form are specified by calling up the input form.

[0082] The specification receiving unit 115 may present a list of categories included in the answer generation data registered in the generation database and receive a category specification from the list from the provider terminal 20. This reduces the effort required for the service provider to specify a category. Note that the order in which the question and category are specified does not matter.

[0083] When a question is specified by calling an input form, if the input form is called after a category is entered, the specification receiving unit 115 accepts the category specification by inputting (selecting) a category (service name, plan name, etc.) when the input form is called (when creating evaluation information).

[0084] 8 is a diagram showing an example of a question input screen QD displayed on the provider terminal 20. The question input screen QD includes a create button B21, a service selection button B22, and a question specification field QI. The create button B21 is an object that instructs the generation of an answer to a question entered in the question specification field QI according to the service name (category) selected by the service selection button B22. The create button B21 does not accept input when a service name has not been selected or when no question has been entered.

[0085] When an input operation is performed on the service selection button B22, a UI for selecting a service is displayed on the provider terminal 20. FIG. 9 is a diagram showing an example of a service selection screen SD displayed on the provider terminal 20. The service selection screen SD includes a selection button SB, a cancel button B31, and an enter button B32. A selection button SB is provided for each option for the service name. In the example of FIG. 9, multiple-choice radio buttons are arranged as the selection buttons SB. When an input operation is performed on the cancel button B31, the selection of the service name is canceled. When an input operation is performed on the enter button B32, the selected service name is accepted as a category of answer generation data to be referenced when generating a question, and is displayed on the question input screen QD of FIG. 8.

[0086] 8 accepts input of a question from the provider terminal 20. A service provider can simultaneously input multiple questions for the same category (service name, etc.) into the question specification field QI.

[0087] Fig. 10 is a diagram showing an example of the question input screen QD in a state where a service name has been selected and a question has been input. In the question input screen QD of Fig. 10, the service selection button B22 displays the selected service name. As shown in Fig. 10, when a service name has been selected and at least one question has been input, and an input operation is performed on the create button B21, the specification receiving unit 115 receives the specification of the security question and the specification of the category, and then the answer generating unit 116 generates an answer.

[0088] The specification receiving unit 115 may further receive specification of the answer format for the question from the provider terminal 20. The answer format for the question is the format of the answer created by the answer generating unit 116, and includes, for example, a multiple choice format in which an answer is selected from predetermined options (for example, a combination of "yes" and "no"), a free description format, a specified description format in which keywords or sentences are created according to a predetermined sentence pattern, and the like.

[0089] <Answer generation unit 116> The answer generation unit 116 is configured to extract, from the input answers included in the accumulated answer generation data, input answers to answered questions that correspond to a designated category (designated category) whose designation has been accepted by the designation accepting unit 115 and are similar to a designated question (designated question) whose designation has been accepted by the designation accepting unit 115, as reference answers. Furthermore, the answer generation unit 116 is configured to generate an answer to the question based on the reference answer and the reference information.

[0090] An "already-entered answer corresponding to a specified category" is an already-entered answer that has been assigned the same category as the specified category. The answer generation unit 116 uses the specified category to narrow down the answer generation data to be searched, and then searches the generation database for answer generation data that includes answered questions similar to the specified question.

[0091] The answer generation unit 116 may extract, as reference answers, answers that have not been assigned a category or that have been assigned an unclassified category, indicating that they are not classified into a specific category, and that are similar to the specified question, regardless of the specified category. This reduces the work required for the service provider to assign individual categories to answers that are common to multiple categories (such as service names).

[0092] The answer generation unit 116 may extract a reference answer by referring to only any answer generation data among the first answer generation data, the second answer generation data, and the third answer generation data. Alternatively, the answer generation unit 116 may extract a reference answer by referring to the first answer generation data and the second answer generation data. This allows the answer generation unit 116 to generate an answer based on both the answer entered in the input form provided by the answer registration unit 112 and other answers (e.g., answers entered in a form other than the input form, answers prepared by the service provider, etc.), thereby improving the accuracy of the answer. Alternatively, if third answer generation data is accumulated, the answer generation unit 116 may extract a reference answer by referring to the third answer generation data in addition to the first answer generation data and the second answer generation data.

[0093] The answer generation unit 116 may determine the similarity between an answered question and a specified question by comparing vector data obtained by vectorizing the content of the answered question included in the answer generation data with vector data obtained by vectorizing the content of the specified question. This allows for rapid and accurate extraction of reference answers. The criterion for similarity between questions is a threshold value for the difference in vector data (vector distance) between two questions, and, for example, questions whose vector data difference is less than the threshold value (e.g., cosine similarity is equal to or greater than the threshold value) are determined to be similar to each other. Note that "answered questions similar to the specified question" also include answered questions that are identical or substantially identical to the specified question.

[0094] The answer generation unit 116 may input the answered question and the designated question into a similarity determination model of the artificial intelligence unit 120, have the similarity determination model output the similarity between them, and determine the similarity between the answered question and the designated question based on the similarity. The similarity determination model is a learning model that is trained so that it can input two questions and output the similarity between the two questions.

[0095] The similarity determination model is a learning model trained using a combination of a set of training questions and a similarity corresponding to the set of training questions as training data. The similarity determination model may also be a generative AI including a large-scale language model. In this case, the answer generation unit 116 receives the answered question and the designated question as input, inputs a prompt including an instruction to output the similarity between the answered question and the designated question to the similarity determination model, and causes the similarity determination model to output the similarity. The answer generation unit 116 may generate a prompt that instructs the similarity determination model to output the similarity between the answered question and the designated question, and input the prompt to the similarity determination model. The answer generation unit 116 may also input a prompt to the similarity determination model that includes, for example, one or more samples of the answered question and the designated question and one or more corresponding similarity samples as input and output samples, in addition to the instruction to output the similarity, the answered question, and the designated question. The similarity determination model outputs the similarity between the answered question and the designated question according to the input prompt.

[0096] The similarity determination model may be a learning model configured to input a specified question, refer to answer generation data (database for generation), and output a reference answer. In this case, the similarity determination model extracts answered questions contained in the database for generation that have a high degree of similarity to the specified question (for example, the cosine similarity of vector data is equal to or greater than a threshold), and outputs the input answer to the answered question as a reference answer.

[0097] The answer generation unit 116 may extract, as a reference answer, an input answer to an answered question similar to the specified question based on a specified question category (specified question category). For example, the answer generation unit 116 extracts a reference answer from input answers to answered questions assigned a question category identical or similar to the specified question category. Here, the specified question category may be specified based on an input by a user such as a service provider, or may be obtained by inputting a specified question into the above-mentioned question category assignment model and causing the question category assignment model to output a question category corresponding to the specified question. This improves the accuracy of matching between the specified question and the answered question or the reference answer. Note that the extraction of a reference answer based on a question category may be performed simultaneously with the extraction of a reference answer based on a specified category whose designation has been accepted by the designation accepting unit 115.

[0098] When the generation database contains multiple answered questions similar to the specified question, the answer generation unit 116 may extract, as a reference answer, the input answers to each of the multiple answered questions. Alternatively, the answer generation unit 116 may extract, as a reference answer, only the input answer to the answered question with the highest similarity among multiple answered questions similar to the specified question.

[0099] The answer generation unit 116 generates an answer to the specified question by arranging the extracted reference answer using the reference information in accordance with the content of the specified question. Arranging the reference answer includes, for example, replacing words included in the reference answer with synonyms, hypernyms, or hyponyms, organizing the context (such as changing the order of sentences), deleting unnecessary information, adding missing information, etc.

[0100] The reference information includes a correlation between the reference answer and the answer to the specified question. The reference information is stored, for example, in the storage unit 12. The reference information is an estimator constructed so as to be able to input the reference answer and output an answer to the specified question. The reference information may include, for example, a table, a function, a simple algorithm, or the like, which indicates a correlation between a feature (vector data) extracted from the reference answer and a feature (vector data) extracted from the specified question. The correlation included in the reference information can be constructed, for example, by statistically analyzing data recording questions to answers generated based on the reference answer.

[0101] The reference information may include an answer generation model that has been trained to use the reference answer as input and output an answer to the specified question. In this case, the answer generation unit 116 inputs the reference answer into the answer generation model and causes the answer generation model to output an answer to the question. This makes it possible to generate answer generation examples for many questions or answers based on natural language processing.

[0102] The answer generation model may be a learning model that is trained using pairs of reference answers and questions for learning, and combinations of the reference answers and answers corresponding to the questions, as training data. In the answer generation model, parameters calculated, tuned, etc. by learning constitute correlations of the reference information.

[0103] The answer generation model may also be a generation AI. In particular, the answer generation model may be a large-scale language model. In this case, the answer generation unit 116 receives the reference answer and the specified question as input, inputs an instruction to output an answer to the specified question to the answer generation model, and causes the answer generation model to output the answer. This makes it possible to generate an answer to the specified question without learning using teacher data for a large number of answers and questions, thereby reducing the construction cost of the information processing system 1.

[0104] The answer generation unit 116 may generate a prompt that instructs the answer generation model to generate an answer to the specified question based on the reference answer, and input the prompt to the answer generation model. Furthermore, the answer generation unit 116 may input a prompt to the answer generation model that includes, in addition to the answer generation / output instruction, the reference answer, and the specified question, input and output samples, for example, one or more sample reference answers and specified questions, and one or more corresponding answer samples. The answer generation model generates an answer that references the reference answer in accordance with the input prompt.

[0105] Furthermore, the answer generation unit 116 outputs information that an answer could not be generated for a specified question for which no reference answer was extracted (that is, a specified question for which no similar answered answer existed in the generation database).

[0106] When the designation receiving unit 115 receives a designation of an answer format, the answer generating unit 116 may generate an answer to the designated question according to the designated answer format. For example, when an alternative format of "yes" or "no" is designated, the answer generating unit 116 generates either "yes" or "no" as the answer. Also, for example, when a free-form format is designated, the answer generating unit 116 generates a freely written sentence as the answer. For example, the answer generating unit 116 inserts an instruction to generate an answer in the designated answer format into a prompt for the answer generation model.

[0107] The answer generation unit 116 may accept edits to the generated answers to the questions. Furthermore, when an answer to a specified question cannot be generated, the answer generation unit 116 may present a message that an answer to the specified question has not been generated, and may accept input of an answer to the specified question.

[0108] 11 is a diagram showing an example of an answer display screen AD displayed on the provider terminal 20. The answer display screen AD includes a designated question display field QC2, a generated answer display field AC2, a remarks field RC, and an answer output button B41. The designated question display field QC2 displays the designated question entered on the question input screen QD of FIG. 10.

[0109] In the generated answer display field AC2, the answers generated by the answer generation unit 116 are displayed in association with the specified questions (in the same row). The answer generation unit 116 may accept edits of the answers by the service provider in the generated answer display field AC2. Furthermore, for specified questions for which an answer could not be generated, information indicating this (in the example of FIG. 11, a standard phrase saying "An answer could not be generated") is displayed.

[0110] The remarks column RC displays action buttons, related question information RI, etc. The action buttons include, for example, an input acceptance button B42, an evaluation button B43, etc. The input acceptance button B42 is displayed only for specified questions for which an answer could not be generated, and accepts an input operation for accepting input of an answer (displaying an answer input screen). The evaluation button B43 accepts an input operation for evaluating the generated answer. The evaluation input from the evaluation button B43 is used, for example, for fine-tuning the answer generation model. The related question information RI presents questions whose content is related. Furthermore, the answer generation unit 116 may accept an input operation for the related question information RI and switch the display of the answer display screen AD so that the corresponding specified question is displayed (that is, it may jump the display range of the answer display screen AD to the position of the corresponding specified question).

[0111] When the answer output button B41 is operated, the answer generation unit 116 outputs the question and answer combination data (matrix) displayed in the specified question display field QC2 and the generated answer display field AC2 as CSV data. In addition, the service provider can copy answers individually from the generated answer display field AC2 and paste them into any location (for example, a file in the provider terminal 20, an input field for another service, etc.).

[0112] The answer generation unit 116 may display, on the provider terminal 20, the contents of the answer generation data (specifically, the already-input answer) that served as the basis for the generated answer, or a link for displaying the answer generation data, in association with the generated answer. For example, the answer generation unit 116 may display, on the provider terminal 20, the reference answer and the already-input answer that corresponds to the reference answer (or a link to these) as the basis for answer generation. For example, the answer generation unit 116 inserts, into a prompt to the answer generation model, an instruction to output the reference answer and the already-input answer that corresponds to the reference answer (or a link to these), in addition to the generated answer.

[0113] The answer generation unit 116 may determine whether the answer generated by the answer generation model includes information that should not be made public, and if information that should not be made public is included, may display that fact on the provider terminal 20. Examples of "information that should not be made public" include personal information, private information, links to private data, etc. Examples of personal information include names and phone numbers of employees of the service provider, etc. Examples of private information include information intended for internal use by the service provider, such as information in fields such as "Notes" and "Additional Information." Examples of links to private data include links (links that allow reference to files, posts, etc.) issued by storage services, messaging services, etc. external to the information processing system 1 that are used by the service provider. In addition, the answer generation unit 116 may determine whether an input answer used for answer generation includes "information that should not be made public," and may not use an input answer that includes "information that should not be made public" for answer generation (e.g., not input it into the answer generation model).

[0114] The answer generation unit 116 may generate answers to questions included in the input form presented by the answer registration unit 112. This allows answers required for the evaluation information to be automatically generated from the answer history held by the service provider, thereby reducing the effort required by the service provider to create the evaluation information. This improves convenience, particularly for service providers who are using the information processing system 1 (e.g., a security evaluation system) for the first time.

[0115] When generating answers to an input form for registering evaluation information, as described above, the service provider inputs a category (service, plan, etc.) when calling up the input form or after calling up the input form, thereby specifying the category for extracting reference answers, and answers to the questions included in the input form are automatically generated. This eliminates the need for the service provider to input questions individually.

[0116] When the data accepting unit 114 accepts input of answered questions and entered answers from a service provider, the answer generating unit 116 may generate answers to unanswered questions by the service provider in advance for input forms of a category in which the answer generation data has been accumulated, at a predetermined timing (for example, the timing when the data accepting unit 114 generates answer generation data based on the input information). In other words, the answer generating unit 116 may generate some or all of the evaluation information in the background. This significantly reduces the effort required for the service provider to generate evaluation information from the input form, thereby promoting the use of input forms by service providers.

[0117] <Report Output Section 117> The report output unit 117 is configured to output a report or a report created based on the evaluation received by the evaluation receiving unit 113 to the user terminal 40. The report or report includes at least all or part of the evaluation information input by the service provider and the evaluation results by the evaluator (e.g., evaluation value, comments, etc.).

[0118] <Artificial Intelligence Department 120> The artificial intelligence unit 120 is configured to receive input from each functional unit and return the instructed output. The artificial intelligence used by the server device 10 in each functional unit may be a common one, or may be prepared individually for each functional unit.

[0119] The artificial intelligence unit 120 is an AI (Artificial Intelligence) equipped with learning models such as Transformers including GPT (Generative Pretrained Transformer, including GPT-1, GPT-2, GPT-3, and GPT-4), BERT (Bidirectional Encoder Representations from Transformers), BART (Bidirectional and Auto-regressive Transformer), etc., and language models such as Recurrent Neural Networks (RNNs), and may include generative AI.

[0120] The language model is an example of a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor methods, naive Bayes methods, decision trees, support vector machines, and deep learning using neural networks. The artificial intelligence unit 120 can apply the above algorithms as appropriate.

[0121] The artificial intelligence unit 120 may have a trained model constructed by a learning method such as supervised learning, unsupervised learning, or self-supervised learning. In supervised learning, machine learning is performed using training data (training data). The training data consists of pairs of input data for learning and output data (correct answer data). Furthermore, the language model may not only be trained for a specific task, but also be a general-purpose model that can be used for a wide range of tasks.

[0122] The artificial intelligence unit 120 may be a general-purpose natural language processing learning model, such as a large-scale language model (LLM), trained on a huge amount of data. LLMs are learning models that have previously trained on a large amount of data, such as text data (e.g., (i) web content on the Internet, or (ii) data stored in a specified database). They can execute various language processing tasks when given tasks, and can perform a wide range of natural language processing tasks, such as grasping sentence patterns and contexts, answering questions, and generating sentences, according to given prompts. Such general-purpose learning models include language models that can handle various tasks without fine-tuning, such as through one-shot learning or few-shot learning. Furthermore, general-purpose learning models may also be configured to handle various tasks through zero-shot learning. The artificial intelligence used in each functional unit of the control unit 11 may be a separate learning model, or a common general-purpose learning model.

[0123] The learning model (answer generation model, etc.) included in the artificial intelligence unit 120 can undergo additional learning as transfer learning or fine tuning. For example, each time new evaluation information, reports, or reports are registered, the artificial intelligence unit 120 may perform additional learning and fine tuning using these as new training data. This improves the accuracy of the information output from the learning model.

[0124] The learning model included in the artificial intelligence unit 120 may be a learning model (distilled model) obtained by knowledge distillation using an original learning model. In knowledge distillation, a trained model such as a large-scale language model is used as a teacher model, and the parameters of the student model are adjusted to reduce the output loss (Soft Target Loss) of the student model (distilled model) relative to the output (Soft Target) of the teacher model, thereby learning the student model, which becomes the distilled model. Alternatively, the student model may be learned to reduce the output loss (Hard Target Loss) of the student model relative to the correct label (Hard Target) of the teacher data (combination of input data and output data of the learning model). Compared to the original learning model (teacher model), the distilled model has a smaller number of parameters and a smaller processing load while maintaining performance similar to the learning model. Therefore, using a distilled model can reduce the cost of the information processing system 1.

[0125] For example, the answer generation model etc. may be a distilled model trained using a combination of input data and output data in a large-scale language model as training data. Furthermore, when the information processing system 1 is introduced, a large-scale language model may be used as the answer generation model etc., and once training data from the large-scale language model has been accumulated, a distilled model obtained by knowledge distillation using the training data may be used as the answer generation model etc.

[0126] <Display> The display unit 211 of the provider terminal 20, the display unit 311 of the evaluator terminal 30, and the display unit 411 of the user terminal 40 each display a screen indicated by the screen data transmitted from the server device 10.

[0127] <Operation acquisition section> The operation acquisition unit 212 of the provider terminal 20 accepts operations by a service provider who uses the provider terminal 20. The operation acquisition unit 312 of the evaluator terminal 30 accepts operations by an evaluator who uses the evaluator terminal 30. The operation acquisition unit 412 of the user terminal 40 accepts operations by a service user who uses the user terminal 40.

[0128] 3. Information Processing Method This section describes an information processing method of the server device 10. This information processing method is executed by a computer, with each unit of the server device 10 acting as each step.

[0129] This information processing includes a data receiving step, a designation receiving step, and an answer generating step. In the data receiving step, answered questions related to the security of the service provided by the service provider, answers already entered by the service provider to the answered questions, and a designation of a category to be assigned to the entered answers are received, and answer generation data including the answered questions, the entered answers, and the categories is added to a database. In the designation receiving step, a designation of a security question and a designation of a category are received. In the answer generating step, an entered answer to an answered question that corresponds to the designated category and is similar to the designated question is extracted as a reference answer from among the entered answers included in the accumulated answer generation data, and an answer to the question is generated based on the reference answer and the reference information.

[0130] 12 is an activity diagram showing an example of the flow of information processing (answer generation processing) executed by the information processing system 1. Below, the information processing will be described along with each activity in this activity diagram.

[0131] The answer generation process begins with the specification of previously entered answers, etc. The service provider specifies the answered question, previously entered answer, and category by inputting the answered question, previously entered answer, and category, or by selecting registered evaluation information, on the provider terminal 20 (activity A101). The server device 10 receives the specification from the provider terminal 20 and accepts the answered question, previously entered answer, and category (activity A102). Next, the server device 10 generates answer generation data from the specified answered question, previously entered answer, and category, and adds it to the generation database (activity A103). Activities A101 to A103 are performed appropriately at any timing.

[0132] After answer generation data has been generated, the service provider specifies a question and a category for generating an answer on the provider terminal 20 (activity A201). The question and category may be specified by directly inputting the question and category into the provider terminal 20, or by calling up an input form for creating evaluation information. The server device 10 extracts a reference answer from the generation database based on the specified question and category (activity A202). After extracting the reference answer, the server device 10 generates an answer to the specified question by referring to the reference answer (activity A203). Next, the server device 10 outputs the generated answer to the provider terminal 20 (activity A204). As a result, the answer generated by the server device 10 is output to the provider terminal 20 (activity A205).

[0133] 4. Effect The operation of this embodiment can be summarized as follows: Since combinations of answered questions and answers are stored along with the categories of those answers, specifying a category for an unanswered question generates a highly accurate answer that references answers to similar past questions about the same type of service. This allows service providers to efficiently create answers to security-related questions.

[0134] Although the embodiment of the present invention has been described above, the present invention is not limited to this and can be modified as appropriate within the scope of the technical idea of ​​the invention.

[0135] 5.Other In the above embodiment, the server device 10 performs various storage and control functions. However, multiple external devices may be used instead of the server device 10. That is, various information and programs may be distributed and stored in multiple external devices using blockchain technology or the like. In particular, the artificial intelligence unit 120 may be an external component of the server device 10. In this case, the external artificial intelligence unit 120 may be provided, for example, by an artificial intelligence service server and configured to receive inputs from each functional unit of the server device 10, receive requests to execute artificial intelligence services, and return the instructed output as a processing result to the server device 10. The artificial intelligence service server may be a server that provides services using a language model as a learning model, or a server that executes language processing tasks using a language model. The artificial intelligence service server may be constructed using LLM. The artificial intelligence service server receives inputs of prompts such as text, images, and voice, and generates and responds to the prompts.

[0136] The control unit 11 does not necessarily have to include the response registration unit 112 and the evaluation receiving unit 113. In other words, the information processing system 1 does not necessarily have to create evaluation information and evaluate the evaluation information based on input into a predetermined input form.

[0137] The aspect of this embodiment is not limited to the information processing system 1, and may be an information processing method or a program. The search support method includes steps executed by the information processing system 1. The program causes a computer to execute the steps of the information processing system 1.

[0138] It may be provided in the following manner.

[0139] (1) An information processing system including at least one processor configured to execute each of the following steps by reading a program: a data receiving step receiving answered questions regarding the security of a service provided by a service provider, answers entered by the service provider to the answered questions, and a designation of a category to be assigned to the entered answers; and adding answer generation data including the answered questions, the entered answers, and the categories to a database; a designation receiving step receiving a designation of a question regarding the security and a designation of the category; and an answer generation step extracting, from the entered answers included in the accumulated answer generation data, the entered answers to the answered questions that correspond to the designated category and are similar to the designated question as reference answers; and generating an answer to the question based on the reference answers and reference information; wherein the reference information includes a correlation between the reference answers and the answers to the questions.

[0140] (2) In the information processing system described in (1) above, the reference information includes an answer generation model that has been trained to use the reference answer as input and to be able to output an answer to the question, and in the answer generation step, the reference answer is input to the answer generation model and the answer generation model is caused to output an answer to the question.

[0141] (3) In the information processing system described in (2) above, the answer generation model is a large-scale language model, and in the answer generation step, the reference answer and the question are input, and an instruction to output an answer to the question is input to the answer generation model.

[0142] (4) In the information processing system described in any one of (1) to (3) above, the category is information indicating the type of the service, the recipient of the service, the type of plan included in the service, or a combination thereof.

[0143] (5) In the information processing system described in any one of (1) to (4) above, in the answer generation step, the input answer to the answered question that is not assigned a category or is assigned an unclassified category indicating that it is not classified into a specific category, and that is similar to the specified question, is extracted as the reference answer regardless of the specified category.

[0144] (6) In the information processing system described in any one of (1) to (5) above, in the designation receiving step, a list of the categories included in the answer generation data is presented, and a designation of the category is received from the list.

[0145] (7) In the information processing system described in any one of (1) to (6) above, in the answer generation step, the similarity between the answered question and the question is determined by comparing vector data obtained by vectorizing the content of the answered question with vector data obtained by vectorizing the content of the specified question.

[0146] (8) In the information processing system described in any one of (1) to (7) above, in the answer generation step, edits to the generated answer to the question are accepted, and in the data acceptance step, the question is designated as the answered question, and the edited answer to the question is designated as the entered answer, and the answer generation data is added to the database.

[0147] (9) In the information processing system described in (8) above, in the answer generation step, if an answer to the question cannot be generated, a message is displayed indicating that an answer to the question has not been generated and an answer to the question is accepted as input, and in the data acceptance step, the question is designated as the answered question and the answer generation data, in which the input answer to the question is designated as the entered answer, is added to the database.

[0148] (10) In the information processing system according to any one of (1) to (9) above, the data receiving step receives input of the category for the input answer.

[0149] (11) In the information processing system described in any one of (1) to (10) above, the processor is configured to further execute the following steps: in the answer registration step, a predetermined input form is displayed that accepts input of answers to the security questions from the service provider, and the answers entered in the input form are registered as evaluation information; in the evaluation reception step, an evaluation of the registered evaluation information is accepted from an evaluator, and the evaluated evaluation information is registered together with the evaluation result; and in the data reception step, the answer generation data is created from the evaluation information.

[0150] (12) In the information processing system described in (11) above, in the answer generating step, an answer to the question included in the input form is generated.

[0151] (13) In the information processing system described in (11) or (12) above, the answer generation data includes first answer generation data based on the answered question and the input answer input by the service provider, and second answer generation data based on the evaluation information, and in the answer generation step, the reference answer is extracted by referring to the first answer generation data and the second answer generation data.

[0152] (14) An information processing method, comprising steps executed by the information processing system according to any one of (1) to (13) above.

[0153] (15) A program for causing a computer to execute each step of the information processing system described in any one of (1) to (13) above. Of course, this is not the case.

[0154] Finally, while various embodiments of the present disclosure have been described, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0155] 1: Information processing system 2: Communication line 10: Server device 11: Control section 12: Storage section 13: Communications Department 14: Communication bus 20: Provider terminal 21: Control unit 22: Storage section 23: Communications Department 24: Input section 25: Output section 26: Communication bus 30: Evaluator terminal 31: Control unit 32: Storage section 33: Communications Department 34: Input section 35: Output section 36: Communication bus 40: User terminal 41: Control unit 42: Storage section 43: Communications Department 44: Input section 45: Output section 46: Communication bus 111: Basic display control section 112: Response registration section 113: Evaluation Reception Department 114: Data reception section 115: Designated Reception Department 116: Answer generation part 117: Report output section 120: Artificial Intelligence Department 211:Display section 212: Operation acquisition section 311: Display section 312: Operation acquisition section 411: Display section 412: Operation acquisition section AC1: Answer input field AC2: Generated answer display field AD:Answer display screen B11: Add Knowledge button B12: Answer generation button B13: Knowledge Edit Button B21: Create button B22: Service selection button B31: Cancel button B32: Decision button B41: Answer output button B42: Input acceptance button B43: Rating button CC: Registration check box CF: Condition setting field DD: Data entry screen ED: Selection screen EF: Evaluation information display field EI: Evaluation Information MC: Memo section MD: Management screen NF: Knowledge display column NL: Knowledge List PC: Past data comparison column QC1: Question entry field QC2:Specified question display field QD: Question input screen QI:Question specification field RC: Remarks column RI: Related Questions SB:Select button SC: Service selection column SD: Service selection screen

Claims

1. An information processing system, at least one processor; The processor is configured to execute the following steps by reading the program: In the data receiving step, answered questions related to the security of a service provided by a service provider, answers input by the service provider to the answered questions, and designation of a category to be assigned to the input answers are received, and answer generation data including the answered questions, the input answers, and the categories is added to a database; In the specification receiving step, a specification of the security question and a specification of the category are received, In the answer generating step, an input answer to an answered question that corresponds to the specified category and is similar to the specified question is extracted as a reference answer from the input answers included in the accumulated answer generation data, and an answer to the question is generated based on the reference answer and reference information; Here, the reference information includes a correlation between the reference answer and the answer to the question.

2. 2. The information processing system according to claim 1, the reference information includes an answer generation model that is trained to use the reference answer as an input and to be able to output an answer to the question; In the answer generating step, the reference answer is input to the answer generation model, and the answer generation model is caused to output an answer to the question.

3. 3. The information processing system according to claim 2, the answer generation model is a large-scale language model, In the answer generation step, the reference answer and the question are input, and an instruction to output an answer to the question is input to the answer generation model.

4. 2. The information processing system according to claim 1, An information processing system, wherein the category is information indicating the type of the service, the destination of the service, the type of plan included in the service, or a combination thereof.

5. 2. The information processing system according to claim 1, In the answer generation step, the input answers to the answered questions that have not been assigned a category or have been assigned an unclassified category indicating that they are not classified into a specific category, and that are similar to the specified question, are extracted as the reference answers regardless of the specified category.

6. 2. The information processing system according to claim 1, In the designation receiving step, an information processing system presents a list of the categories included in the answer generating data and receives designation of the category from the list.

7. 2. The information processing system according to claim 1, In the answer generation step, the information processing system determines the similarity between the answered question and the question by comparing vector data obtained by vectorizing the content of the answered question with vector data obtained by vectorizing the content of the specified question.

8. 2. The information processing system according to claim 1, In the answer generating step, edits to the generated answers to the questions are accepted; In the data receiving step, the question is treated as the answered question, and the edited answer to the question is treated as the input answer, and the answer generation data is added to the database.

9. 9. The information processing system according to claim 8, In the answer generating step, when an answer to the question cannot be generated, a message is displayed indicating that an answer to the question has not been generated, and an input of an answer to the question is accepted; In the data receiving step, the question is treated as the answered question, and the answer generation data in which the input answer to the question is treated as the input answer is added to the database.

10. 2. The information processing system according to claim 1, In the data receiving step, an input of the category for the input answer is received.

11. 2. The information processing system according to claim 1, The processor is further configured to perform the steps of: In the answer registration step, a predetermined input form is displayed for receiving input of an answer to the security question from the service provider, and the answer input in the input form is registered as evaluation information; In the evaluation receiving step, an evaluation on the registered evaluation information is received from an evaluator, and the evaluated evaluation information is registered together with an evaluation result. In the data receiving step, the answer generation data is created from the evaluation information.

12. 12. The information processing system according to claim 11, In the answer generating step, an answer to the question included in the input form is generated.

13. 12. The information processing system according to claim 11, the answer generation data includes first answer generation data based on the answered question and the input answer input by the service provider, and second answer generation data based on the evaluation information; In the answer generating step, the first answer generating data and the second answer generating data are referenced to extract the reference answer.

14. An information processing method, comprising: An information processing method comprising the steps executed by the information processing system according to any one of claims 1 to 13.

15. A program, A program for causing a computer to execute each step of the information processing system according to any one of claims 1 to 13.

Images