Program, method, and information processing device
The system addresses the challenge of updating security check sheet answers by using a processor to manage question lists, identify similar questions, and generate answers with AI, ensuring accurate and timely responses to security check items.
Patent Information
- Application Number
- JP2024042272
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-03-18
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2044-03-18
AI Technical Summary
Service providers face challenges in updating security check sheet answers due to changes in security requirements, provider status, or environmental changes, making it difficult to create or update answers that reflect these changes using past responses.
A system that includes a processor to manage question lists, identify similar questions, determine answers based on accumulated data, and display answer accumulation status, utilizing artificial intelligence to generate answers using trained models and vectorization techniques.
Enables users to easily create updated answers to security check questions by leveraging past responses and AI-generated answers, ensuring relevance and accuracy in security check sheet updates.
Smart Images

Figure 0007595204000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a program, a method, and an information processing device. [Background technology]
[0002] Service providers offering IT services (such as SaaS (Software as a Service)) may be requested by service users who are considering introducing or using the IT services to answer a security checklist.
[0003] The following Patent Document 1 describes, with the aim of "accelerating system audits and reducing the burden on the audited department," "arranging and storing checkpoint array information data for identifying each checkpoint when auditing an audited object in a rule-based order," "storing trail array information data for identifying trails required for evaluation of each checkpoint in correspondence with the checkpoint array information data," "rearranging the stored trail array information data in a rule-based order and rearranging the checkpoint array information data in correspondence with the trail array information data," and "creating a trail ledger for managing trails based on the rearranged array converted data." According to the technology described in Patent Document 1, "since the trail array information data is arranged in a rule-based order, the trails can be used as indexes, and the checkpoints corresponding to the trails can be easily grasped, thereby realizing a rapid system audit." [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2003-44658 A Summary of the Invention [Problem to be solved by the invention]
[0005] In response to requests from service users, service providers create security checklists that include answers to questions. However, as time passes, the level of security required may change, and the security response status may be updated due to changes in the service provider itself or changes in the circumstances surrounding the service provider. Therefore, simply referring to the content of answers in security checklists created in the past may not be enough to create or update answers that reflect the above changes.
[0006] Therefore, there is a need for technology that allows users to easily create answers to questions while making use of security checklists that have been created in the past. [Means for solving the problem]
[0007] According to one embodiment of the present disclosure, there is provided a program for operating a computer having a processor, the program causing the processor to execute the steps of: storing in a memory unit, each time a list of questions to be asked to a service provider, the list of questions including at least questions corresponding to security check items; identifying similar questions by referring to one or more list of questions stored in the memory unit; and displaying, for each question, the accumulation status of answers to that question based on the identified results. Effect of the Invention
[0008] According to the present disclosure, a user can refer to the accumulation status of answers to similar questions, making it even easier to answer questions. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a diagram showing the configuration of a system 1. [Diagram 2] FIG. 2 is a diagram showing the configuration of the server 20. As shown in FIG. [Diagram 3]FIG. 3 is a diagram showing the configuration of a terminal 10 of a user of a service provider. [Figure 4] FIG. 4 is a diagram showing the data structure of the business database 211, which is a database of users of a service provider. [Diagram 5] FIG. 5 is a diagram showing the data structure of the user database 212, which is a database of service users. [Figure 6] FIG. 6 is a diagram showing the data structure of the question list acquisition database 213. As shown in FIG. [Figure 7] FIG. 7 is a diagram showing the data structure of the response database 214. As shown in FIG. [Figure 8] FIG. 8 is a diagram showing the data structure of the question group database 215 for managing related questions as question groups. [Figure 9] FIG. 9 is a diagram showing the data structure of the trained model database 216 for managing trained models that output answers to input questions. [Figure 10] FIG. 10 is a diagram showing the data structure of the vector database 217 for managing vectors of a data set consisting of questions and answers. [Figure 11] FIG. 11 is a diagram showing the data structure of the LLM usage history database 218 that indicates the usage history of the service server of the large-scale language model. [Figure 12] FIG. 12 is a diagram showing the data structure of the investigation request database 221 for managing investigation requests regarding security from users. [Figure 13] FIG. 13 is a diagram showing the data structure of the response transmission database 222 for managing responses from service providers to survey requests from users. [Figure 14] FIG. 14 is a flowchart showing the flow of processing for acquiring a question list including security questions from a user of a service provider, in particular a question list to which answers have been previously input for all or some of the questions. [Figure 15]FIG. 15 is a flowchart showing the flow of a process for presenting a list of stored questions to a user and accepting input of answers. [Figure 16] FIG. 16 is a flow chart showing the flow of the process of reflecting the questions and answers specified by the user in the question list in the answer database 214. [Figure 17] FIG. 17 is a flowchart showing the process flow for generating a trained model that outputs an answer to an input question through a learning process. [Figure 18] FIG. 18 is a flowchart showing a process for generating answers to other questions by inputting an answer to one question when the multiple questions form a group of related questions. [Figure 19] This is a flowchart showing the process of generating a new answer to a question using a trained model managed in the trained model database 216. [Figure 20] FIG. 20 is a flow chart showing the flow of processing for vectorizing a question and generating an answer by the artificial intelligence service server 92 by referring to the question and answer that have been previously vectorized. [Figure 21] FIG. 21 shows an example of a screen for accepting upload of a question list from a service provider's user. [Figure 22] FIG. 22 is an example of a screen that displays the category, answers, and comparison results with previously registered data for an uploaded question list. [Figure 23] FIG. 23 is an example of a response database screen. [Figure 24] FIG. 24 shows an example of a screen that is displayed when an answer is generated without the user having to input an answer. [Diagram 25] FIG. 25 is an example of a screen that accepts editing of the generated answer and accepts an operation to save it. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.
[0011] <Outline of the embodiment> <1.1 Overall system configuration> 1 is a diagram showing the configuration of a system 1. In the system 1, a business providing a security check service (server 20) provides the above-mentioned security check service to a service provider (terminal 10) and a service user (terminal 30).
[0012] The system 1 shown in Fig. 1 includes a security check service server 20, a service provider user terminal 10, a service consumer user terminal 30, an artificial intelligence service server 92, and a service provider web server 94. Each of these devices is connected for communication via a network 80.
[0013] In the illustrated example, the terminals used by users of the security check service provided by server 20 are shown as one each of a terminal 10 of a service provider user and a terminal 30 of a service user user, with each user operating their own terminal.
[0014] In this embodiment, each device (terminal device, server, etc.) can also be regarded as an information processing device. That is, a collection of the devices can be regarded as one "information processing device", and the system 1 may be formed as a collection of multiple devices. The method of allocating multiple functions required to realize the system 1 according to this embodiment to one or multiple pieces of hardware can be appropriately determined in consideration of the processing capacity of each piece of hardware and / or the specifications required for the system 1.
[0015] The artificial intelligence and trained models used in each functional unit described below in this embodiment may be separate models or may be a common, general-purpose trained model.
[0016] The terminal 10 is a device operated by a user. The terminal 10 is realized, for example, as follows. Desktop PCs (Personal Computers), laptop PCs · Handheld mobile devices such as smartphones and tablets Wearable devices worn by users (wristwatches, glasses, etc.) The terminal 10 includes a communication IF (Interface) 12 , an input device 13 , an output device 14 , a memory 15 , a storage 16 , and a processor 19 .
[0017] The communication IF 12 is an interface for inputting and outputting signals so that the terminal 10 can communicate with an external device.
[0018] The input device 13 is a device for receiving an input operation from a user (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).
[0019] The output device 14 is a device (such as a display and a speaker) for presenting information to a user.
[0020] The memory 15 is for temporarily storing programs and data processed by the programs, etc., and is, for example, a volatile memory such as a DRAM (Dynamic Random Access Memory). The programs may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the programs are started on an external computer and their functions are realized on a client terminal (so-called cloud computing).
[0021] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).
[0022] The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.
[0023] The server 20 is a device for providing each user with a security check service that makes it easy for service providers and service users to ask questions and answer them using a security check sheet.
[0024] The server 20 includes a communication IF 22 , an input / output IF 23 , a memory 25 , a storage 26 , and a processor 29 .
[0025] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.
[0026] The input / output IF 23 functions as an interface with an input device for receiving an input operation from a user and an output device for presenting information to the user.
[0027] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).
[0028] The storage 26 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).
[0029] The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.
[0030] The artificial intelligence service server 92 is an AI (Artificial Intelligence) equipped with a learning model constructed by a learning process, and may include a generation AI, and executes a task by the learning model. The artificial intelligence service server 92 may provide a service using a language model as the learning model, and may be a server that executes a language processing task using a language model. The artificial intelligence service server 92 accepts a request to execute an artificial intelligence service from an external device and responds with the processing result. The artificial intelligence service server 92 may be constructed as an LLM (Large Language Model), and executes a language processing task in response to a request from an external device and responds with the processing result. The LLM is a trained model that has previously learned a large amount of large-scale data (text data, etc.), for example, (i) web content on the Internet, or (ii) a trained model that has previously learned a large amount of data accumulated in a specified database, and can execute various language processing tasks by giving a task. In addition, a general-purpose learning model such as an LLM may be fine-tuned by performing additional learning for a specific task, and includes a language model that can handle various tasks without fine-tuning by One-shot Learning, Few-shot Learning, etc. Furthermore, the learning model may be configured to be compatible with various tasks using zero-shot learning.
[0031] The artificial intelligence service server 92 accepts input of a prompt in the form of text, image, voice, etc., and generates and responds to the prompt. Examples of LLM include GPT-3 and GPT-4 developed by OpenAI, Inc., and BERT developed by Google, Inc. In this embodiment, the artificial intelligence service server 92 generates answers to questions corresponding to security check items in the following manner.
[0032] (i) The LLM is constructed as a trained model that has learned a large amount of large-scale data, not limited to the security field. The artificial intelligence service server 92 acquires some (or all) of the questions and answers from a database that accumulates questions corresponding to security check items and answers to those questions (for example, data to be referenced is specified in a prompt), and generates an answer to the input question. In this case, the artificial intelligence service server 92 receives a question corresponding to the check item as input, acquires a prompt including an instruction to output an answer to the input question, inputs the prompt into the trained model, and causes the trained model to output the answer to the question. For example, the artificial intelligence service server 92 searches for an answer corresponding to the input question sentence by referring to some questions and answers extracted from the database, and summarizes the search results in a predetermined number of characters to generate a new answer corresponding to the input question sentence.
[0033] (ii) The LLM is constructed as a trained model that has learned a large amount of data in the security field. The artificial intelligence service server 92 constructs a large-scale language model by learning a large amount of information on the Internet (such as the terms of use of various services) as training data, for example, as data in the security field. The artificial intelligence service server 92 accepts questions according to check items about security and generates answers to those questions.
[0034] In this manner, the artificial intelligence service server 92 accepts questions based on security check items and generates answers to the questions using the trained model.
[0035] The service provider's web server 94 provides the user with a service as a web service. The service provider's web server 94 holds information that users should refer to in order to use the service, such as the terms of the service, Q&A, an operation manual, and a bulletin board for users to communicate with each other about the service in text form, etc. The server 20 can obtain information about the service, such as the terms of the service and Q&A, by communicating with the service provider's web server 94.
[0036] (iii) The artificial intelligence service server 92 may not be constructed as an LLM, but may provide a service that accepts learning data, generates a trained model, and responds to requests from other devices with the output results that the trained model outputs for the input data.
[0037] In this embodiment, each device may have a trained model constructed by a learning method such as supervised learning, unsupervised learning, or self-supervised learning. In supervised learning, machine learning is performed using training data (training data). The training data is composed of a pair of input data for learning and output data (correct answer data). In addition, the language model may not only be one trained for a specific task, but also a general-purpose model that can be used for a wide range of tasks.
[0038] <1.2 Functional configuration of server 20> Fig. 2 is a diagram showing the configuration of the server 20. As shown in Fig. 2, the server 20 fulfills the functions of a communication unit 201, a storage unit 202, and a control unit 203.
[0039] The communication unit 201 performs processing for the server 20 to communicate with external devices.
[0040] The memory unit 202 stores various databases such as a business operator database 211, a user database 212, a question list acquisition database 213, an answer database 214, a question group database 215, a trained model database 216, a vector database 217, an LLM usage history database 218, a survey request database 221, and an answer transmission database 222.
[0041] The business database 211 is a database for managing users of the service provider.
[0042] The business operator database 211 includes identification information of each service provider, identification information of the service, and other information, which will be described in detail later.
[0043] The user database 212 is a database for managing users who use the service.
[0044] The user database 212 includes identification information and other information of service users, as will be described in detail later.
[0045] The question list acquisition database 213 is a database for managing data that includes questions corresponding to security check items as data uploaded to the server 20 by the service provider.
[0046] The question list acquisition database 213 includes information such as the upload date and time, the contents of the uploaded data, etc. Details will be described later.
[0047] The answer database 214 is a database for managing answers to security questions.
[0048] The answer database 214 includes questions, answers, and other information. The server 20 uses the list of questions and answers as learning data to generate a trained model that outputs answers to input questions. Details will be described later.
[0049] The question group database 215 is a database for managing related questions as question groups.
[0050] The question group database 215 manages the question group by assigning an order or a superior-subordinate relationship to the multiple questions contained in the question group, as will be described in detail later.
[0051] The trained model database 216 is a database for managing trained models that output answers to input questions.
[0052] The trained model database 216 manages information such as the time when the trained model was generated, as will be described in detail later.
[0053] The vector database 217 is a database for managing vectors (question vectors) that are generated when a new question is entered in order to match the questions and answers managed in the answer database 214, etc. with questions that are similar in content to a certain extent.
[0054] The vector database 217 contains vectors and other information for each question, as described in more detail below.
[0055] The LLM usage history database 218 is a database for managing the history of answers generated to questions using the artificial intelligence service server 92 .
[0056] The LLM usage history database 218 includes information such as input questions, extracted questions, answers generated by the artificial intelligence service server 92, and other information, as will be described in detail later.
[0057] The investigation request database 221 is a database for managing the history of inquiries about security received as investigation requests from service users to service providers.
[0058] The investigation request database 221 includes information on the service for which an investigation has been requested and other information, which will be described in detail later.
[0059] The answer transmission database 222 is a database for managing the history of answers given by the service provider to the service users regarding security questions.
[0060] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 operates in accordance with the program to perform functions indicated as a reception control module 2041, a transmission control module 2042, a user management module 2043, a question and answer management module 2044, an answer generation module 2045, a learning processing module 2046, and a survey request processing module 2047.
[0061] The reception control module 2041 controls the process in which the server 20 receives a signal from an external device in accordance with a communication protocol.
[0062] The transmission control module 2042 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.
[0063] The user management module 2043 accepts registration of users who use the services provided by the server 20 and performs processing to update the business database 211 and the user database 212 .
[0064] The question and answer management module 2044 accepts operations from the user regarding the questions and answers managed in the question list acquisition database 213 and the questions and answers managed in the answer database 214 (uploading a question, inputting an answer, etc.), and performs processing to update the question list acquisition database 213 and the answer database 214.
[0065] The answer generation module 2045 performs a process of generating answers to questions using a trained model.
[0066] The learning processing module 2046 performs processing to generate a trained model using a data set consisting of questions and answers as learning data, and to update the trained model database 216.
[0067] The investigation request processing module 2047 accepts inquiries about security from service users as investigation requests, updates the investigation request database 221, and performs processing to provide replies from the service provider to the service users by referring to the response sending database 222.
[0068] <1.3 Configuration of Terminal 10> FIG. 3 is a diagram showing the configuration of a terminal 10 of a user of a service provider.
[0069] As shown in FIG. 3, the terminal 10 includes a plurality of antennas (antenna 111, antenna 112), communication units (first communication unit 120, second communication unit 121) corresponding to the respective antennas, an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a storage unit 180, and a control unit 190. The terminal 10 also has functions and configurations (e.g., a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not particularly shown in FIG. 3. As shown in FIG. 3, each block included in the terminal 10 is electrically connected by a bus or the like.
[0070] The antenna 111 radiates a signal generated by the terminal 10 as a radio wave. The antenna 111 also receives the radio wave from space and provides the received signal to the first communication unit 120.
[0071] The antenna 112 radiates a signal generated by the terminal 10 as a radio wave. The antenna 112 also receives the radio wave from space and provides the received signal to the second communication unit 121.
[0072] The first communication unit 120 performs modulation and demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal 10 can communicate with other wireless devices. The second communication unit 121 performs modulation and demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal 10 can communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are communication modules including a tuner, a Received Signal Strength Indicator (RSSI) calculation circuit, a Cyclic Redundancy Check (CRC) calculation circuit, a high-frequency circuit, and the like. The first communication unit 120 and the second communication unit 121 perform modulation and demodulation, frequency conversion, and the like of wireless signals transmitted and received by the terminal 10, and provide the received signals to the control unit 190.
[0073] The input device 130 has a mechanism for accepting an input operation by a user. Specifically, the input device 130 is configured as a touch screen and includes a touch-sensitive device 131. The touch-sensitive device 131 accepts an input operation by a user of the terminal 10. The touch-sensitive device 131 detects a touch position of the user on the touch panel by using, for example, a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the touch position of the user detected by the touch panel to the control unit 190 as an input operation.
[0074] Display 132 displays data such as images, videos, and text under the control of control unit 190. Display 132 is realized by, for example, an LCD, an organic EL display, or the like.
[0075] The audio processing unit 140 modulates and demodulates an audio signal. The audio processing unit 140 modulates a signal provided from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized by, for example, a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal 10.
[0076] The position information sensor 150 is a sensor that detects the position of the terminal 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. In the satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal 10 equipped with the GPS module is detected based on the received signals.
[0077] The camera 160 is a device that receives light with a light receiving element and outputs the received light as a captured image. The camera 160 is, for example, a depth camera that can detect the distance from the camera 160 to a subject being photographed.
[0078] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10 .
[0079] The storage unit 180 is configured, for example, from a flash memory or the like, and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.
[0080] The control unit 190 reads a program stored in the storage unit 180 and executes instructions included in the program to control the operation of the terminal 10. The control unit 190 is, for example, an application processor. The control unit 190 operates according to the program to fulfill the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a storage control unit 195.
[0081] The operation reception unit 191 performs a process of receiving a user's input operation to an input device such as the touch-sensitive device 131. The operation reception unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation, based on information about the coordinates where the user touches the touch-sensitive device 131 with a finger or the like.
[0082] The transmission / reception unit 192 performs processing for the terminal 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.
[0083] The data processing unit 193 performs calculations on the data that the terminal 10 accepts as input in accordance with a program, and outputs the calculation results to a memory or the like.
[0084] The notification control unit 194 performs processing for displaying a display image on the display 132, processing for outputting sound from the speaker 142, and processing for generating vibrations.
[0085] The storage control unit 195 controls the storage of data in the storage unit 180 .
[0086] The following describes various types of information stored in storage unit 180. In one aspect, storage unit 180 stores various types of information such as user information 181, question and answer list 182, and answer transmission history 183.
[0087] The user information 181 is information on the user of the service provider who uses the service of the server 20 .
[0088] The question and answer list 182 is information on a security check sheet in which the service provider answers security questions posed by the service user.
[0089] The answer transmission history 183 is information on the history of answers given to security questions by providing a security check sheet to a service user.
[0090] <2 Data Structure> FIG. 4 is a diagram showing the data structure of the business database 211, which is a database of users of a service provider.
[0091] The business entity database 211 includes the following items: "user ID", "service provider ID", "service ID", "user name", "department", "job title", and "email address".
[0092] The item "user ID" is information for identifying each user.
[0093] Specifically, the item "user ID" includes information for identifying a user of the service provider.
[0094] The item "service provider ID" is information for identifying each service provider.
[0095] Specifically, the item "service provider ID" includes identification information about the following businesses as service providers: Businesses developing services Businesses that operate or sell services Businesses that have capital ties with businesses that develop and operate the services (parent companies, etc.) The item "service ID" is information for identifying each service.
[0096] Specifically, the item "service ID" includes identification information assigned to each service that is the subject of questions and answers in the security checklist.
[0097] The server 20 may assign identification information to each version of a service or each major update of a service in response to the operation of a user of the service provider. For example, when a service has a major update, the server 20 may assign an identification information different from the identification information assigned to the service before the update to the service after the update and manage the service with the different identification information.
[0098] The item "user name" is information indicating the name of a user who belongs to the service provider.
[0099] The item "department" is information about the department to which the user belongs.
[0100] The item "position" is information indicating the position of the user.
[0101] The item "email address" is information on the user's email address as the user's contact information.
[0102] FIG. 5 is a diagram showing the data structure of the user database 212, which is a database of service users.
[0103] The user database 212 includes the following items: "user ID", "service user ID", "user name", "department", "job title", and "email address".
[0104] The item "user ID" is information for identifying each user.
[0105] Specifically, the item "user ID" includes information for identifying a user of the service.
[0106] The item "service user ID" is information for identifying each service user.
[0107] Specifically, the item "service user ID" includes identification information about the following businesses as service users: Businesses using the service Businesses that act as sales agents for services Businesses that have capital ties with businesses using the service (parent companies, etc.) The item "user name" is information indicating the name of a user who belongs to the service user.
[0108] The item "department" is information about the department to which the service user belongs.
[0109] The item "position" is information indicating the position of the user.
[0110] The item "email address" is information on the user's email address as the user's contact information.
[0111] FIG. 6 is a diagram showing the data structure of the question list acquisition database 213. As shown in FIG.
[0112] The question list acquisition database 213 includes the items "list ID", "service provider ID", "service ID", "upload date and time", "uploader", "upload data", and "referring link".
[0113] The item “list ID” is information for identifying each question list received by the server 20.
[0114] The item "service provider ID" is information for identifying each service provider.
[0115] The item "service ID" is information for identifying each service.
[0116] The item "upload date and time" is information indicating the time when the server 20 accepted the upload of the question list.
[0117] The item “uploader” is information for identifying the user who uploaded the question list to the server 20 .
[0118] The item "upload data" is data that the server 20 has accepted for upload.
[0119] Specifically, the item "upload data" includes the following as data that the server 20 accepts for upload: - Security checklists created by the service provider in the past (such as a list of questions and answers) Information shown on internet web pages (URLs) related to the service or the service provider (such as files for browsing websites) · Terms and conditions documents related to the service or service provider (privacy policy, terms of use, etc.) The data accepted by the server 20 includes data in a table format in which answers are associated with questions. The data accepted by the server 20 may include data in which questions are set but answers are not set. When the server 20 uploads structured data (such as a general-purpose file format such as CSV format, a file format output by a specific application, etc.) output as a file by a spreadsheet application (such as an application installed and executed on a terminal, or an application provided as a cloud service) as tabular data, the server 20 can display the table in its original format and provide the user with a screen on which the user can select rows and columns, copy, and paste. Details will be described later.
[0120] The item "referring source link" is information for referencing the source from which the server 20 obtained data.
[0121] Specifically, the item "Referrer Link" may include links to the following pages: Web page Links set up to share files stored in online storage services FIG. 7 is a diagram showing the data structure of the response database 214. As shown in FIG.
[0122] The answer database 214 includes an item "answer list ID," an item "service provider ID," an item "service ID," an item "answer creation date and time," an item "answer creator," an item "answer approver," an item "answer approval date and time," an item "question ID," an item "question category," an item "question text," and an item "answer text."
[0123] The item "answer list ID" is information for identifying an answer list set by a service provider. The answer list can also be held as a question and answer list 182 in the terminal 10, as described later.
[0124] The item "service provider ID" is information for identifying each service provider.
[0125] The item "service ID" is information for identifying each service.
[0126] The item "answer creation date and time" is information indicating the timing when the answer list was created or saved.
[0127] The item "answer creator" is information indicating the user who saved the answer list.
[0128] The item "answer approver" is information indicating the user who approved the answer list.
[0129] The item "answer approval date and time" is information indicating the timing when the user who approved the answer list gave approval.
[0130] The item "Question ID" is information that identifies each question.
[0131] The item "Question Category" is information indicating a category assigned to a question.
[0132] Specifically, the item "question category" includes information on the category assigned to each question by the user or the category inferred by the server 20 from the question text.
[0133] The category to be assigned to the question may be selected from a group of categories defined in advance by the business providing the service through the server 20 or by the business providing the security check service through the server 20 .
[0134] In this way, if a category is assigned to a question sentence and the question sentence is stored as an answer list, when a new question is asked and an answer is generated, the server 20 can further improve the accuracy of the answer by identifying a question sentence that has a similar category to the new question.
[0135] The item "Question" is information indicating the content of the question.
[0136] The item "answer message" is information indicating the content of the answer message.
[0137] FIG. 8 is a diagram showing the data structure of the question group database 215 for managing related questions as question groups.
[0138] The question group database 215 includes an item "question group ID," an item "question ID of main question," an item "answer to main question," an item "question ID of secondary question," and an item "answer to secondary question."
[0139] The item "Question Group ID" is information for identifying a question group consisting of multiple related questions.
[0140] The item "Question ID of main question" is information for identifying a question (main question) among questions included in a question group, for which an answer can be generated to another question by setting an answer.
[0141] The item "Content of Answer to Main Question" is information indicating the content of the answer to the main question.
[0142] The item "Question ID of Sub-Question" is information that identifies a question (sub-question) among questions included in a question group that can generate an answer to a question by setting an answer to another question (main question).
[0143] The item "content of answer to subordinate question" is information indicating the content of the answer to the subordinate question.
[0144] FIG. 9 is a diagram showing the data structure of the trained model database 216 for managing trained models that output answers to input questions.
[0145] The trained model database 216 includes an item "trained model ID", an item "service provider ID", an item "service ID", an item "learning date and time", an item "answer list ID", an item "availability", and an item "availability period".
[0146] The item “Trained Model ID” is information that identifies a trained model that is trained to output an answer to an input question.
[0147] The item “Service Provider ID” is information that identifies the service provider of the service to which the trained model is applied to generate answers to security questions.
[0148] The item “Service ID” is information that identifies the service to which the trained model is applied to generate answers to security questions.
[0149] The item “learning date and time” is information indicating the time when the trained model was trained.
[0150] The item "answer list ID" is information that identifies each answer list that serves as training data for training a trained model.
[0151] The item "Availability" is information indicating whether or not the trained model is permitted to output an answer to an input question.
[0152] Specifically, the item "Availability" may store the result of receiving a setting from a user of the service provider regarding whether or not the trained model can be used.
[0153] For example, by updating the questions and answers stored in the answer database 214, it may be desirable to stop using a trained model that was trained using data before the update.
[0154] The server 20 may be configured to register the trained model obtained by training using the learning processing module 2046 in the trained model database 216 when the questions and answers stored in the answer database 214 are updated, while suspending the use of the trained model trained based on the data before the update.
[0155] The item “Availability” is information indicating the period during which the trained model can be used.
[0156] Specifically, the item "Available Period" may store the result of accepting a setting of a period for using the trained model from a user of the service provider. If the trained model is available during the period, the server 20 can generate an answer using the trained model.
[0157] For example, if the user specifies the time to update the questions and answers stored in the answer database 214, the user may want to disable the use of a trained model trained based on data before the update after the update. The server 20 may set the availability of the trained model to "unavailable" when the period during which the trained model can be used arrives.
[0158] In addition, the server 20 can, based on the settings, make the trained model unavailable before the period during which the trained model can be used arrives.
[0159] FIG. 10 is a diagram showing the data structure of the vector database 217 for managing vectors of a data set consisting of questions and answers.
[0160] The vector database 217 includes an item "question ID," an item "service provider ID," an item "service ID," an item "answer list ID," an item "question text after adding words," an item "vector," an item "question text," an item "answer text," and an item "answer usability."
[0161] The item "Question ID" is information that identifies each question.
[0162] The item "service provider ID" is information for identifying each service provider.
[0163] The item "service ID" is information for identifying each service.
[0164] The item "answer list ID" is information for identifying an answer list set by a service provider.
[0165] The item "Question after adding words" is information showing the question with synonyms, related words, and abbreviations added to the words included in the question using dictionary data for security check sheets. The process of adding words using dictionary data will be described later in the explanation of FIG. 20 etc.
[0166] The item "vector" is information indicating a vector obtained by vectorizing data consisting of questions and answers to which words have been added based on dictionary data. In other words, the vector database 217 stores the original question sentence and the corresponding vector as a set.
[0167] Specifically, the item "vector" stores vectors obtained by vectorizing sentences consisting of questions and answers using a method such as Word2Vec.
[0168] The item "Question" is information about the question indicated by the question ID.
[0169] The item "Answer Sentence" is information indicating an answer to a question indicated by a question ID.
[0170] The item "Can answers be used?" is information indicating whether or not an answer is permitted to be generated by the artificial intelligence service server 92 in response to the input of a new question, based on the question and answer stored in the vector database 217.
[0171] Specifically, the item "answer availability" receives a setting from the service provider's user as to whether or not an answer is permitted to be generated, and stores the result.
[0172] For example, as the content of a question or answer is updated, it may no longer be used to generate an answer.
[0173] In addition, if the content of the question or answer is expected to change due to future operational changes by the service provider, even if the question and answer are registered in advance, it may not yet be appropriate to use them to generate an answer.
[0174] FIG. 11 is a diagram showing the data structure of the LLM usage history database 218 that indicates the usage history of the artificial intelligence service server 92.
[0175] The LLM usage history database 218 includes the items "usage history ID", "question ID", "input question text", "answer list ID", "LLM usage date and time", "question ID of the question text used as the basis", "prompt", and "LLM output result".
[0176] The item “Usage History ID” is information for identifying the history of generating answers using the artificial intelligence service server 92 .
[0177] The item "Question ID" is information that identifies each question.
[0178] The item "input question" is information indicating the content input as a question.
[0179] The item "answer list ID" is information for identifying an answer list set by a service provider.
[0180] The item “LLM use date and time” is information indicating the timing when the answer was generated using the artificial intelligence service server 92.
[0181] The item "Question ID of the question sentence used as the basis" is information that indicates the question ID of the question that was used as input to the artificial intelligence service server 92 to generate an answer.
[0182] Specifically, the item "Question ID of the question sentence used as the basis" includes the question ID of the question obtained by the server 20 vectorizing the content entered as a question and matching it with the vectors of each question and answer stored in the vector database 217 (determining whether the questions are of the same type).
[0183] The item "prompt" is information indicating a prompt sent to the artificial intelligence service server 92 to generate an answer.
[0184] Specifically, the "prompt" item holds prompts including instructions such as instructing the system to generate an answer to a question received from a user, data to be referenced when generating the answer (such as data contained in answer database 214), and generating the answer by considering or summarizing the question and answer extracted from answer database 214.
[0185] The item “LLM output result” is information indicating the content of the answer generated by the artificial intelligence service server 92.
[0186] FIG. 12 is a diagram showing the data structure of the investigation request database 221 for managing investigation requests regarding security from users.
[0187] The investigation request database 221 includes an item "request ID", an item "service ID", an item "service user ID", an item "user ID", and an item "request date and time".
[0188] The item "request ID" is information for identifying each request made by a service user to a service provider for answers to security questions.
[0189] The item "service ID" is information for identifying the service for which a request is made to answer security questions.
[0190] The item "service user ID" is information for identifying the service user making the request.
[0191] The item "user ID" is information for identifying the user of the service making the request.
[0192] The item "Request Date and Time" is information indicating the timing of the request.
[0193] Although not shown, the investigation request database 221 stores data on the contents of requests made by service users (for example, a security checklist including questions regarding security) in association with a request ID.
[0194] FIG. 13 is a diagram showing the data structure of the response transmission database 222 for managing responses from service providers to survey requests from users.
[0195] The reply sending database 222 includes an item "reply sending ID", an item "request ID", an item "reply date and time", an item "replyer", and an item "reply content".
[0196] The item "answer transmission ID" is information for identifying the answer given by the service provider in response to a request for an answer to a security question.
[0197] The item "Request ID" is information for identifying the request of the service user to which a reply is to be given.
[0198] The item "answer date and time" is information indicating the timing of the answer.
[0199] The item "answer" is information for identifying the user who made the answer.
[0200] The item "answer content" is information indicating the content of the answer.
[0201] Specifically, the item "answer content" may include the following as the contents of the answer: -Answers to security questions on the security checklist <3 operations> FIG. 14 is a flowchart showing the flow of processing for acquiring a question list including security questions from a user of a service provider, in particular a question list to which answers have been previously input for all or some of the questions.
[0202] In step S1421, the question and answer management module 2044 of the server 20 communicates with the terminal 10 to present the service provider user with the upload status of a question list such as a security checklist, and also presents a screen that accepts the user's specification of data consisting of questions and answers.
[0203] The question and answer management module 2044 of the server 20 presents the user with a screen for accepting uploading of files such as a security checklist and an answer manual, and specification of a reference link (such as a URL) where the terms and conditions can be viewed, as data specification.
[0204] In step S1411, the terminal 10 accepts from the user an upload of a spreadsheet or other file (e.g., a security checklist file for which answers have previously been entered) as data consisting of questions and answers regarding security-related check items, and a URL (e.g., a URL indicating the location on a cloud service where the security checklist file for which answers have previously been entered is stored).
[0205] The terminal 10 transmits to the server 20 data related to the designation received from the user.
[0206] In step S1423, the question and answer management module 2044 of the server 20 updates the question list acquisition database 213 based on the data received from the user.
[0207] In step S1425, the question and answer management module 2044 of the server 20 refers to the question list acquisition database 213 and presents the accumulation status of data such as security checklists that the user has accumulated in the service of the server 20.
[0208] The question and answer management module 2044 of the server 20 displays the history of data such as files uploaded by the user, and presents a screen for accepting operations such as deleting and downloading each file.
[0209] This allows the user of the terminal 10 to check the history of security-related question and answer information stored in the server 20, such as a security checklist.
[0210] FIG. 15 is a flowchart showing the flow of a process for presenting a list of stored questions to a user and accepting input of answers.
[0211] In step S1521, the question and answer management module 2044 of the server 20 refers to the question list acquisition database 213, and determines whether or not each acquired question has overlapping questions based on the content of the question. The question and answer management module 2044 determines whether or not the content of the questions has overlapping questions as follows. The questions and answers are vectorized using a method such as Word2Vec, and the vectors of each question are compared. If the distance between the two vectors is closer than a certain threshold, it is determined that the questions are of the same type and have overlapping content. Cosine similarity may be used as an index to measure the similarity of the vectors. Extract words from the question text, and judge the degree of word duplication and the degree of agreement in the word order. If the degree of overlap reaches a certain level (e.g., the degree of word duplication is equal to or greater than a certain percentage), it is determined that the content of the question is duplicated. The question and answer management module 2044 of the server 20 counts the number of overlapping questions for each question.
[0212] In step S1523, the question and answer management module 2044 of the server 20 associates, for each question, the category, the question text, the answer text, whether the question is duplicated, the number of duplicated questions, etc., and presents them to the user in a tabular format so that answer input can be accepted.
[0213] In step S1511, the terminal 10 displays, for each question, the category, the question text, the answer, and the number of overlaps as a comparison situation with the accumulated data, and accepts the user's specification of a category and input of an answer.
[0214] In step S1525, the question and answer management module 2044 of the server 20 updates the question list acquisition database 213 based on the answer received from the user.
[0215] In this way, the user can input answers to questions, which can also be used as learning data, as described below.
[0216] FIG. 16 is a flow chart showing the flow of the process of reflecting the questions and answers specified by the user in the question list in the answer database 214.
[0217] In step S1621, the question and answer management module 2044 of the server 20 refers to the question list acquisition database 213, and displays the category, question text, answer text, etc. for each question, and presents a screen for accepting designation as to whether or not each question should be used as learning data.
[0218] In step S1611, the terminal 10 accepts, for each question, a designation as to whether or not to use the data for learning, and accepts an operation to confirm the designation for each question.
[0219] In this way, the terminal 10 receives an operation from the user as to whether or not to register each question in the answer database 214.
[0220] In step S1623, the question and answer management module 2044 of the server 20 updates the answer database 214 based on the question specified by the user.
[0221] In this way, when the question and answer management module 2044 of the server 20 receives from the user a specification of a question to be registered in the answer database, it registers the received question and answer in the answer database 214.
[0222] If there is a user who approves the answer registered by the user in the answer database (the user who registered the answer and the user who approves may be the same or different), the server 20 presents the user with a screen for approving the answer.
[0223] In step S1613, the terminal 10 accepts an operation from the user who approves the answer to have the answer approved and reflected in the answer database.
[0224] In step S1625, the server 20 updates the answer database 214 based on the user's approval of the answer.
[0225] FIG. 17 is a flowchart showing the process flow for generating a trained model that outputs an answer to an input question through a learning process.
[0226] In step S1721, the learning processing module 2046 of the server 20 generates a trained model that is trained to output an answer to an input question, using the questions and answers stored in the answer database 214 as learning data. The learning processing module 2046 updates the trained model database 216 based on the generated trained model. The learning processing module 2046 presents the generation status of the trained model to the user based on the trained model database 216.
[0227] In step S1711, the terminal 10 displays information such as the date and time of learning as the generation status of the trained model. The terminal 10 accepts an operation for managing the trained model from the user, such as specifying whether or not the trained model can be used.
[0228] In step S1723, the learning processing module 2046 of the server 20 updates the trained model database 216 in response to the user's operation.
[0229] Here, the learning processing module 2046 may generate a new trained model without receiving a designation from the user as to whether or not the trained model can be used, and update the trained model database 216 so that trained models generated in the past cannot be used.
[0230] FIG. 18 is a flowchart showing a process for generating answers to other questions by inputting an answer to one question when the multiple questions form a group of related questions.
[0231] In step S1823, the question and answer management module 2044 of the server 20 refers to the question group database 215 and determines whether the question for which an answer has been received is part of a related question group (whether the question is considered to be a main question in the question group database 215). If it is determined that the question is part of a related question group, the question and answer management module 2044 acquires the answer to the related question (sub-question) from the question group database 215 if the content of the user's answer to the sub-question is registered in the question group database 215.
[0232] In step S1825, the question and answer management module 2044 of the server 20 presents the obtained answers to the group of related questions to the user.
[0233] In step S1815, the terminal 10 displays the answers acquired for the group of related questions, and accepts an operation from the user to confirm the answers or edit the answers.
[0234] In step S1827, the question and answer management module 2044 of the server 20 updates the question list acquisition database 213 based on the answer received from the user.
[0235] As explained above, examples of generating answers to questions for a set of related questions include the following:
[0236] The main question was, "In managing the subcontractor, have you agreed with the subcontractor on security measures appropriate to the content of the contract?", and the answer to that was, "The data is stored on a domestic server owned by a third party (XX Company's cloud service),...", which included the impression that an external service was being used.
[0237] The secondary question is assumed to be a question that assumes the answer to the main question, such as "If outsourcing and external services are used, have you established a policy for managing the outsourcing company?" In this case, the server 20 generates a new answer based on the answer to the main question, such as an answer that assumes the use of an external service, for example, "We use a cloud service provided by XX company as an external service, and regularly check reports from XX company."
[0238] Also, for example, assume that the main question is "Are you using XX Company's cloud services?" and the answer is "Yes, we are using them." If the secondary question is "Is physical security being addressed?", the answer "Yes, it is," which is based on the assumption that XX Company's cloud services are being used, and takes into account that XX Company's cloud services are able to address physical security, is automatically generated.
[0239] FIG. 19 is a flowchart showing the process of generating a new answer to a question using a trained model managed in the trained model database 216.
[0240] In step S1913, the terminal 10 accepts input of a question sentence for which an answer is to be generated. For example, the terminal 10 accepts an upload of a security checklist in which answers to all or some of the questions have not been entered, or accepts input of a question sentence by the user himself / herself (including input by copy and paste, input by keyboard operation, etc.) for each question sentence, and transmits the input to the server 20. The terminal 10 accepts an operation from the user to generate an answer using a trained model for each question for which an answer is to be generated, regardless of an operation of the user inputting the answer.
[0241] In step S1927, the answer generation module 2045 of the server 20 refers to the trained model database 216 to identify a trained model that can be used to generate an answer. For example, the answer generation module 2045 identifies a trained model that has been trained for a service related to the question, has a learning date and time close to the date and time at which an answer is to be generated (e.g., the current time), and has a usable setting.
[0242] The answer generation module 2045 generates an answer to the question by inputting the question to the identified trained model and presents it to the user.
[0243] In step S1915, the terminal 10 accepts, from the user, an operation to confirm the answer generated by the trained model or an operation to edit the answer.
[0244] In step S1929, the server 20 updates the question list acquisition database 213 based on the answer received from the user.
[0245] 19, as an example of using a trained model to generate an answer to a question, an example has been described in which a trained model managed in the trained model database 216, i.e., a trained model trained to output a corresponding answer by inputting a question, with questions and corresponding answers stored in the server 20 as training data (teacher data), is used. In addition, a large-scale language model may be used as the trained model, and an example of generating an answer using a large-scale language model will be described later with reference to FIG. 20 etc.
[0246] FIG. 20 is a flow chart showing the flow of processing for vectorizing a question and generating an answer by the artificial intelligence service server 92 by referring to the question and answer that have been previously vectorized.
[0247] In step S2021, the answer generation module 2045 of the server 20 adds synonyms, similar words, related words, and abbreviations to the questions and answers stored in the question list acquisition database 213 or the answer database 214 based on dictionary data related to security.
[0248] The answer generation module 2045 extracts words included in a question, and references dictionary data to obtain synonyms, similar words, related words, and abbreviations of the words, thereby generating a question sentence with the above synonyms, etc. added. Dictionary data may be prepared for each security question. Dictionary data may also be prepared for each industry, occupation, service user, or service provider.
[0249] As dictionary data, for a specific keyword, related vocabulary is predefined, for example, as follows: - A specific service name is established for the implementation of security measures such as "threat intelligence." For example, there is a name for a service provided by a specific company, such as "Amazon GuardDuty." "Portable storage media" and other commonly recognized terms for media that store information. For example, "USB memory" Other terms and similar terms that refer to tools implemented for security measures, such as "antivirus software." For example, "Endpoint Detection and Response" (EDR), "antivirus software," etc. A term that refers to another name for an organization or team that responds when a security incident occurs, such as an "incident response team." For example, "CSIRT" (Computer Security Incident Response Team) Other terms or similar terms that refer to communication methods to improve security, such as "encrypted communication." For example, "HTTPS" (Hypertext Transfer Protocol Secure) - A term that indicates the name of a specific standard for security measures, such as "third-party certification." Examples include "ISO27001," "SOC2 (Service Organization Control Type 2)," and "ISMAP (Information system Security Management and Assessment Program)." The answer generation module 2045 of the server 20 vectorizes the question sentence with the added vocabulary. For example, the sentence is vectorized using a method such as Word2Vec. The answer generation module 2045 of the server 20 associates the question sentence vector with the question and stores it in the vector database 217.
[0250] In step S2011, the terminal 10 newly accepts input of a question such as a security checklist.
[0251] In step S2023, the answer generation module 2045 of the server 20 vectorizes the question newly received from the user, and compares it with the vector of each question sentence stored in the vector database 217. An example of the process when a category is set for the newly received question will be described later.
[0252] In step S2025, the answer generation module 2045 of the server 20 acquires a question and an answer that match the vector of the newly accepted question sentence (the vector is close to a certain degree) from the vector database 217 based on the result of the vector comparison. The answer generation module 2045 associates the acquired information (question and answer) with the question sentence newly accepted from the user and inputs them to the artificial intelligence service server 92 to generate an answer to the question sentence newly accepted from the user. In this case, the answer generation module 2045 inputs the question sentence newly accepted from the user, and inputs a prompt including an instruction to output an answer to the question sentence while referring to the question and answer that match the question sentence (the vector is close to a certain degree) to the artificial intelligence service server 9, and causes the artificial intelligence service server to output the answer to the question sentence. The answer generation module 2045 acquires an answer to the question sentence newly accepted from the user by accepting the answer generated by the artificial intelligence service server 92.
[0253] In step S2013, the terminal 10 displays the answer generated by the artificial intelligence service server 92. The terminal 10 accepts an operation to reflect the generated answer in a security checklist or the like stored in the question list acquisition database 213 or the like.
[0254] In response to the above operation, the server 20 updates the security checklist etc. stored in the question list acquisition database 213.
[0255] <4 Screen example> 21 shows an example of a screen for accepting upload of a question list from a user of a service provider, which corresponds to the process of step S1411 in FIG.
[0256] The screen 2000 is a screen for the server 20 to provide the user with a function for generating answers to a security check sheet in the security check service.
[0257] In the illustrated example, a screen 2000 displays a screen that appears when a user of a service provider logs into the service and starts up a function for generating answers to a security checklist.
[0258] The menu display area 2001 is an area for displaying a menu of operations in the security check service and receiving menu designations from the user.
[0259] The content display area 2002 is an area for displaying the content corresponding to the menu and accepting user operations.
[0260] The investigation request menu 2011 is an area for accepting designation of a menu for managing each investigation request for which a security question is accepted from a service user via the security check service. When the server 20 accepts a user's operation on the investigation request menu 2011, it provides the service provider's user with a screen displaying cases for which security investigation requests are accepted from service users. In response to the operation of the service provider's user, the server 20 accepts an operation to start answering a security check sheet for the case, and transmits the created answer to the service user.
[0261] The answer sharing menu 2012 is an area for accepting designation of a menu for managing a history of answers given to questions from a service user using a security check sheet or the like in the security check service. When the server 20 accepts a user's operation on the answer sharing menu 2012, it provides the user of the service provider with a screen for accepting an operation to share with the service user an answer master registered as an answer that has been reviewed in the security check service provided by the server 20. The server 20 accepts input of, for example, an email address, a name, and the like as information of the other party to share with, and transmits the designated answer master.
[0262] The answer master menu 2013 is an area for accepting designation of a menu for managing the answer master as a menu for managing the contents of answers to a security check sheet. When the server 20 accepts a user's operation on the answer master menu 2013, the server 20 provides the user with a screen for managing the answer master. On this screen, the server 20 accepts an operation from the user to save the answers that have been reviewed in the security check service provided by the server 20 as the answer master.
[0263] The answer generation support menu 2014 is an area for receiving the designation of a menu for editing answers to the security check sheet.
[0264] In the illustrated example, the answer generation support menu 2014 is displayed highlighted compared to the other menus, indicating that the answer generation support menu has been specified by the user.
[0265] The answer upload tab 2021 is an area for accepting an operation to switch to a screen that accepts uploading of a question list such as a security checklist to which the user has already entered answers to all or some of the questions.
[0266] The answer database tab 2022 is an area for accepting an operation to switch to a screen for displaying an answer list managed in the answer database 214 .
[0267] The AI answer generation receiving section 2023 is an area for receiving an operation to switch to a screen that generates an answer using a trained model.
[0268] In the illustrated example, the AI answer generation receiving unit 2023 switches to a screen shown in FIG. 23, which will be described later, when the server 20 receives a user operation.
[0269] The upload reception area 2030 is an area for receiving the specification of data to be uploaded.
[0270] The URL acceptance section 2031 is an area for accepting, from the user, a URL designated as a storage location where the data to be uploaded is to be stored.
[0271] The file reception section 2032 is an area for receiving, from the user, the specification of a file to be uploaded.
[0272] The history display area 2040 is an area for displaying the history of data that the user has uploaded in the past.
[0273] As shown in the figure, the history display area 2040 displays the following and accepts user operations. Whether the uploaded data is a file or retrieved based on a URL Upload date and time (registration date and time) Uploaded by Possible operations for uploaded data include "Delete" to delete data and "Download" to download data. 22 shows an example of a screen displaying the category, questions, answers, and comparison results with previously registered data for an uploaded question list, which corresponds to the process of step S1511 in FIG.
[0274] A screen 2100 is a screen for displaying a list of questions and receiving input of answers to the questions from the user.
[0275] The addition reception section 2101 is an area for receiving an operation to reflect the question and answer designated by the user in the answer list managed in the answer database 214.
[0276] In the illustrated example, the addition receiving unit 2101 specifies five of the six questions displayed as questions, and indicates that it accepts an operation to add the five questions and their answers to the answer list in the answer database 214. In the illustrated example, in the item "Register or Not" in the list shown in the question list display area 2110, it accepts a specification for each question as to whether or not to add it to the answer list, and adds a check mark to the specified question.
[0277] The reacquisition receiving section 2102 is an area for receiving an operation to reacquire data including a question.
[0278] The re-acquisition receiving unit 2102 may, upon receiving a user operation, return to the screen 2000 in FIG. 20, for example, and receive a file upload, or receive a designation of data to be edited by the user from the history of uploaded data.
[0279] The question list display area 2110 is an area for displaying questions and the like included in the data specified by the user and accepting edits by the user. In the example shown in the figure, a list of acquired questions is displayed, including the items "Registerable / Not Registered," "Category," "Question," "Answer," "Internal Memo," and "Comparison with Past Data."
[0280] The server 20 accepts category settings for uploaded questions as follows, and associates the category with each question managed in the question list acquisition database 213, or associates the category with the question in the answer database 214 and updates the item "question category."
[0281] (i) Accept the user's request for a category of questions Accept any category setting from the user for each question For each question, set a category from a set of predefined categories according to the user's preference
[0282] (ii) Guess the question category and set it without the user's input It is assumed that data stored in the question list acquisition database 213 or the answer database 214 is referenced and questions or answers are stored in association with categories. By analyzing the text of a newly uploaded question, a similar question from among questions already stored is identified, and a category associated with that question is assigned to the newly uploaded question. For example, by performing a morphological analysis on the question, it is possible to extract words contained in the question. Similar questions from among the stored questions may be identified based on the words and word order that appear in these questions. Use a trained model that outputs a question category for a question or an answer to a question. The trained model is generated by training the model to output a category for at least one of the inputs of a question or an answer, using questions with a specified category or answers with a specified category as training data. The trained model that outputs a category may be a generative AI including a large-scale model. In this case, a question or answer is input, and a prompt including an instruction to output a category corresponding to the question or answer is input to the trained model, and the trained model outputs the category. Here, in addition to the instruction to output the category, the prompt may include, as input and output samples, for example, one or more sample questions or answers and one or more categories corresponding thereto, which are then input to the trained model. The server 20 holds the category assigned to the question as described above in association with the question in the question list acquisition database 213 or the answer database 214, and displays the category in association with the question, as shown in FIG.
[0283] The warning object 2111 is an object for alerting the user that an answer to a question has not been input.
[0284] The warning object 2111 may be drawn by the server 20 when, with reference to the question list acquisition database 213, no answer is entered to a question.
[0285] The reference link 2112 is a link (a link to each question in the question list managed in the question list acquisition database 213) for referencing overlapping questions and answers when there are overlapping questions and answers that the user has previously stored in the server 20.
[0286] The reference link 2112 may include information on a link to refer to the answer list while displaying the question number included in the answer list when the server 20 determines by referring to the answer database 214 that a similar question exists.
[0287] The duplication indication object 2113 is an object that notifies the user of the existence of duplicate questions and answers that the user has previously stored in the server 20, if any.
[0288] In the illustrated example, the duplication indication object 2113 notifies the user that there are duplicate questions and answers, but the server 20 may also display the number of duplicates.
[0289] The edited display 2114 notifies that the user has edited the answer to the question. For example, when the server 20 generates an answer using a trained model and receives an operation in which the user confirms and edits the answer, the edited display 2114 is displayed.
[0290] FIG. 23 is an example of a response database screen.
[0291] The answer list display area 2230 is an area for displaying the answer database managed in the answer database 214. As shown in the figure, the server 20 accepts operations on questions and answers while having the user confirm the questions and answers being managed. The items "Number", "Category", "Question", "Answer" and "Action" contained in the answer database are presented to the user.
[0292] For each question, the server 20 displays an area 2231 for receiving an operation for the question in the item "action," and receives the following operations. Edit: Allows you to edit the answer content. Archive: Do not use the trained model to generate answers, but accept the operation to include it in the list of answers. · Delete: Accepts the operation to delete a question or answer from the list of answers. The server 20 displays update times 2232A, 2232B, and 2232C indicating the time when the answer to each question was registered, in the answer list display area 2230. In the example shown, the update time 2232A notifies the user that a certain period of time has passed since the answer was registered, and urges the user to review the answer.
[0293] When there is an update to the data acquired from the user (FIG. 20), such as when the server 20 receives a registration from the user that a security policy has been changed, the server 20 may notify the user to confirm the content of the answer in association with the question on a screen such as that shown in FIG. 22.
[0294] 24 shows an example of a screen when an answer is generated without the user having to input an answer, which corresponds to the processing in steps S1913, S1927, etc. in FIG.
[0295] The answer generation receiving section 2311 is an area for receiving operations from a user to generate an answer to a question using a trained model managed in the trained model database 216 or the artificial intelligence service server 92.
[0296] In the illustrated example, the answer generation receiving unit 2311 shows an example in which the question text is input by copy and paste or manually input, but the user may specify the history of data uploaded by the user (such as questions managed in the question list acquisition database 213) and have it reflected in the answer generation receiving unit 2311.
[0297] The server 20 accepts a user's operation to the answer generation accepting unit 2311, and generates an answer to a question input to the question accepting unit 2321, using a trained model managed in the trained model database 216 or the artificial intelligence service server 92. An example of a screen on which an answer is generated will be described later with reference to FIG.
[0298] The question receiving section 2321 is an area for receiving input of a question from the user.
[0299] In the illustrated example, the question receiving unit 2321 receives the following specifications for questions and answers: Question category: Accepts operations to specify a predefined category using a pull-down menu, etc. Question · Response format for questions: Should responses be "yes / no" or free text? In addition, in the illustrated example, the question receiving unit 2321 displays the following regarding the generated answer: · Response format: yes / no or free text ·Answer text For example, if a "yes / no" alternative format is specified as the format of the answer to a question, a "yes / no" answer is generated in accordance with this specification. For example, an answer that conforms to the answer format is generated by instructing the artificial intelligence service server 92 to generate an answer in accordance with the answer format (specifying the answer format in a prompt and instructing the generation of an answer). Reference information: When data on which an answer is generated is output, such as when an answer is generated by the artificial intelligence service server 92, the reference information may include the data on which the answer is generated and a link to refer to the data. Here, the reference information may be a question and an answer that match the vector of a newly accepted question (the vector is close to a certain degree) based on the result of vector comparison by the answer generation module 2045 of the server 20. In other words, the reference information may be a past question and an answer that are determined to be close to the question and input to the artificial intelligence service server in order to generate an answer to a newly accepted question from a user. In this case, the reference information may be a link to the vector database 217 that indicates the matched question and answer.
[0300] FIG. 25 is an example of a screen that accepts editing of the generated answer and accepts an operation to save it.
[0301] The answer storage unit 2312 is an area for storing the generated answers and accepting an operation from the user to reflect them in the answer list of the answer database 214. In the illustrated example, the server 20 presents the answers generated by the trained model to the user, and accepts an operation for the user to edit the answers and save the contents of the answers, and accepts an operation to save the answers in the answer list of the answer database 214 while displaying the number of questions for which the answers have been edited in the answer storage unit 2312. Here, in order to prompt the user to confirm the contents of the answers generated by the trained model, when the answers to each question are generated by the trained model, the answers are stored in the answer database 214 by accepting an operation for confirmation by the user or an operation for editing and saving the contents by the user, whereas if the generated answers have not been confirmed by the user or have not been edited, the answers may not be stored in the answer database 214.
[0302] The warning object 2322 is an object that warns the user that the answer could not be generated by the trained model managed in the trained model database 216 or the artificial intelligence service server 92 when a matching question and answer do not exist in the answer database 214, and that the user should input the answer himself.
[0303] As shown in the figure, if the artificial intelligence service server 92 is unable to generate an answer, a prompt is defined to respond to that effect, and an answer is generated by the artificial intelligence service server 92. For example, a prompt defined as follows is sent to the artificial intelligence service server 92: Generate answers to questions that need answers, giving reference questions and answers If an answer to a question that requires an answer cannot be generated from the question and answer to be referenced (for example, if there are no questions and answers corresponding to the question that requires an answer), the answer should not be guessed, but a response should be issued stating that an answer could not be generated. The report object 2323 corresponds to a question group managed in the question group database 215, and is an object for notifying the user that an answer has been generated by inputting an answer to another question.
[0304] The edited result display 2324 is an object for notifying the user that the answer has been edited by the user inputting the answer.
[0305] The reference link 2325 may include link information (link to the question and answer in the answer database 214) that references the data on which the answer is generated when the server 20 generates an answer using the artificial intelligence service server 92. By specifying in the prompt sent to the artificial intelligence service server 92 that an answer is to be generated that includes a link to the data on which the answer is generated, an answer including the link information as described above can be generated together with the proposed answer.
[0306] <Modification> In addition to the aspects described in the above embodiment, the following may be adopted: The following will be described along with a chronological experience when a user uses a service.
[0307] (1) The server 20 obtains a list of questions from the user In the above description of the embodiment, the following has been described with reference to FIG. 21 etc. as a mode in which the server 20 acquires a list of questions from a user. Data to be acquired: Security checklists filled out in the past, answer manuals, etc. -Method of specifying the location of data: Accepting URL specification, accepting upload of file to be saved on user's device, etc. In addition, when the server 20 acquires a list of questions by uploading or the like, if there are questions already stored in the question list acquisition database 213, the answer database 214, etc. (if it determines that there are duplicate questions and answers, as in step S1521), the server 20 may update the questions and answers in the answer database 214, etc. with the newly acquired questions and answers, or may accept instructions from the user about updating as described above.
[0308] In addition, in the service provided by server 20, when a survey request is received from a service user (survey request database 221), the file attached to the survey request may be acquired as a list of questions, and the question list acquisition database 213 may be updated. At this time, server 20 may retain the list of questions in question list acquisition database 213 by associating the identification information of the service user related to the survey request (user database 212) with the identification information of the survey request (survey request database 221).
[0309] The server 20 may manage whether the acquired list of questions has been edited by the user or not. In this way, upon receiving a survey request from a service user, the user of the service provider can easily grasp the list of questions for which answers should be created by displaying whether the list of questions has been edited by the user or not. In addition, the user of the service provider can easily respond to the survey request using a security checklist by confirming that the user has edited the questions while generating answers using a trained model.
[0310] (2) Obtaining a list of questions and presenting the accumulated answers to the questions to the user In the above embodiment, the following example has been described with reference to FIGS. 22 and 25, in which a list of questions is acquired and the accumulation status of answers to the questions is presented to the user. - Detects whether questions have been duplicated and displays a message indicating that the questions are duplicated Show number of duplicate questions Show previous answers (links to previous answers) that have already been created for the duplicate question Shows the category set for the question A set of related questions, where an answer entered in one question generates an answer in other related questions In addition, the accumulation status of answers to a question may be presented to the user as follows. When questions are aggregated into similar questions (when they are determined to be duplicate questions), the answers to those questions are displayed to show changes over time. That is, the server 20 determines that the content of the answer to the question has been changed, and displays that the content of the answer has been changed. This makes it easier for the user to determine whether the content of the answer is consistent with the past content. For example, if the content of the answer has been changed (the content of the answer may change due to the service provider's strengthened response to security as the business progresses), the user can decide not to use the answer before the change as learning data for the trained model.
[0311] At this time, the server 20 may distinguish between answers to questions that have been changed and those that have not been changed, for example by changing the display color. Also, when there has been a change in the content of the answer, it may display "there is a contradiction in the content of the answer" or "there is a discrepancy in the content of the answer" in association with the question and the answer. This allows the service provider's users to narrow down the areas that need to be checked to see if the content of the answer is appropriate, reducing the burden of checking the content of the answer. See when an answer was generated for a question The user can easily grasp the time when the accumulated answer was generated, and for example, when a certain period of time has passed since the answer was generated (the answer has become outdated, etc.), the user can easily determine whether the answer needs to be revised because of the passage of the certain period of time. The server 20 may notify the user of the fact that a certain period of time has passed since the answer was generated. This can encourage regular review.
[0312] (3) The situation where answers to questions are received from users In the above embodiment, the following example has been described with reference to FIGS. 22, 24, and 25, assuming that an answer to a question is accepted from a user. Displaying categories, questions, and answer formats, and allowing users to enter answers or generate answers using a trained model Additionally, to make it easier for the user to input an answer or generate an answer using a trained model, the following may be done: Extract questions from a list of questions based on criteria such as category, answer format, and whether an answer has already been entered. This allows users to easily identify the questions they need to answer, and makes it easy to create answers to those questions. Extract questions from a list of questions based on the answer accumulation status (a certain number of duplicates, the answer content has changed, the answer could not be generated by the trained model, etc.). For example, if a question has a certain number of duplicates, it can be assumed that it is a frequently asked question and questions can be extracted to prioritize the input of answers to it.
[0313] (4) The phase where answers to questions are generated using a trained model In the above embodiment, the following example has been described as an example in which the answer is generated automatically without relying on the user's input operation. A trained model is prepared that is trained to output an answer to an input question using a history of questions and answers as training data (such as the answer database 214). The question for which an answer is to be generated is input to the trained model, and an answer is output. Prepare a trained model (large-scale language model in the above example) that has been trained on a large amount of data, not limited to the security field, and extract data consisting of questions and answers to be given to the trained model according to the degree of matching between the question vector for which an answer is to be generated and the question vector from among the combinations of questions and answers already stored, and generate an answer by having the trained model create a summary, etc.
[0314] (4-1) How to extract reference questions and answers In addition, for a question requiring an answer, a question and an answer to be given to the artificial intelligence service server 92 may be extracted and the artificial intelligence service server 92 may generate an answer, as described below. Based on keywords contained in the question that requires an answer, questions and answers already stored are searched for to extract questions and answers to be given to the AI service server 92. For example, the question sentence that requires an answer is broken down into words by morphological analysis, and the broken down words are used for searching. A knowledge graph, which defines the semantic closeness between words using a graph structure, and keywords included in the question that requires an answer may be used to extract questions and answers to be given to the artificial intelligence service server 92. For example, data containing security-related words can be collected, related information about the collected data can be grouped, and these words can be expressed in a graph structure by defining nodes and edges. Based on the category assigned to the question, a question and answer to be given to the artificial intelligence service server 92 are extracted. For example, the server 20 infers the category of the question that requires an answer. From the questions and answers for which a category has already been set, the server 20 extracts questions and answers that are in the same category from the answer database 214, etc., and generates an answer by providing the extracted questions and answers to the artificial intelligence service server 92 and having it create a summary. For example, if a category has been set for the question that requires an answer as described above, vectors may be searched for from questions and answers that belong to the same category among the questions and answers stored. This may further improve the accuracy of matching.
[0315] (4-2) How to avoid giving answers that are not suitable for disclosure In addition, the following may be done to avoid generating answers that are not suitable for disclosure to the service user. · Information that should not be made public to service users is excluded from the questions and answers given to the trained model, i.e., the data referenced to generate answers to new questions. Here, the following information may be considered as information that should not be made public to service users:
[0316] Personal information related to the service provider (personal name, phone number, etc.) Links to access data stored in systems used by service providers (such as links issued by storage services, messaging services, etc. that allow access to files or posts) Information entered for internal use by the service provider (for example, information entered in fields designated for internal use by the service provider, such as "Notes" or "Additional Information" in a table-format list containing answers to questions) In this way, by providing the artificial intelligence service server 92 with data that has had information that should not be made public removed and then having it generate answers to questions, it may be possible to generate more appropriate answers.
[0317] (4-3) How to generate multiple answers using a trained model When generating an answer using the trained model, the trained model may generate multiple answer proposals for a question. For example, the large-scale language model may be provided with a prompt that defines the number of answers to be generated, the generation of answers with different contents, etc., to generate multiple answer proposals using the large-scale language model.
[0318] In the case of a question such as the one below, the server 20 may generate a plurality of answer proposals as described above. User-specified questions Questions in categories specified by the user Questions containing specific predefined words A question that has a certain number of similar questions stored
[0319] (5) Adding security-related vocabulary to questions In addition to the example described above in FIG. 20 etc., security-related vocabulary may be added to a question etc. by accepting an operation from a user to save an answer entered in response to a question (FIG. 16 etc.).
[0320] (6) The user checks the answer generated by the trained model. In addition, the answer generated by the trained model may be confirmed by the user as follows. Determine whether the generated response contains any information that should not be disclosed to the service user. Information that should not be disclosed to the service user may include the following:
[0321] Personal information related to the service provider (personal name, phone number, etc.) Links to access data stored in systems used by service providers Information entered for internal use by the service provider When the server 20 determines that the generated answer contains information that should not be made public, as described above, it notifies the user by displaying a message to that effect on the screen, for example.
[0322] (7) When responding to a request for investigation from a service user by using a security checklist In the above, it has been explained that the server 20 may receive a survey request from a service user (survey request database 221, survey request menu 2011), obtain questions related to the survey request, and manage them as a question list acquisition database 213. The server 20 may generate answers to the questions related to the survey request using a trained model, and when the user completes the answers by inputting the answers, etc., send the answers to the survey request to the service user (answer transmission database 222). For example, when the answers to each question related to the survey request are set without omission by generating the answers using a trained model, etc., an operation reception unit (displayed on the screen as an operation button, etc.) for receiving an operation to share the answers with the service user may be displayed in an operable manner for the user, and sharing of answers with the service user may be restricted (for example, the operation reception unit may not be enabled) while the answers to each question are not completed.
[0323] Although several embodiments of the present disclosure have been described above, these embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are within the scope of the invention and its equivalents as described in the claims, as well as the scope and spirit of the invention.
[0324] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general purpose processors, application specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the functions described. Processors include transistors and other circuits and are considered to be circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in a memory.
[0325] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed in this specification or any hardware known to be programmed to realize or perform the described functions.
[0326] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.
[0327] <Additional Notes> The matters explained in the above embodiment will be supplemented below.
[0328] (Appendix 1) A program for operating a computer having a processor, the program executing the steps of: storing the received question list in a memory unit each time the processor receives a question list for a service provider, the question list including at least questions corresponding to security check items (S1423, 2030, 2031, 2032); identifying similar questions by referring to one or more question lists stored in the memory unit (S1521); and displaying, for each question, the accumulation status of answers to that question based on the identified results (S1523, 2112, 2113).
[0329] (Appendix 2) The program further causes the processor to, in the identifying step, identify that the same type of question has been accumulated in duplicate in one or more accumulated question lists (S1521), and, in the displaying step, display the accumulation status by displaying that the same type of question has been accumulated in duplicate (S1523, 2112, 2113).
[0330] (Appendix 3) The program according to appended claim 2, wherein in the display step, the accumulation status is displayed by at least one of displaying the number of duplicate questions of the same type and displaying past answers to the questions of the same type (S1523, 2112, 2113).
[0331] (Appendix 4) The program further causes the processor to execute a step of accepting a category designation for a question, and associating the question with the category accepted for the question and storing the associated information in a memory unit (214, S1511), and in a display step, displays, for each question, a category associated with the question (2110), a program described in any one of Appendices 1 to 3.
[0332] (Appendix 5) The program further causes the processor to execute a step of identifying categories to be assigned to the questions by analyzing the text of the questions and assigning the identified categories to the questions, and in the display step, displays, for each question, the assigned category in association with the question (2110).
[0333] (Appendix 6) The program further causes the processor to execute the steps of: identifying a group of questions from among a plurality of questions stored in the memory unit, where answers to other questions are determined by an answer to one question (215, S1823); accepting input of answers to the questions stored in the memory unit from the user, and storing the questions and the answers to the questions in association with each other in the memory unit (S1511, S1823); and, in accepting input of answers to questions, generating answers to the other questions without the user's input operation by accepting input of an answer to one question in the group of questions (S1823).In the displaying step, the generated answers to the other questions are displayed to the user (S1825, 2323).A program described in any of Appendices 1 to 5.
[0334] (Appendix 7) The program further causes the processor to execute the steps of accepting input of an answer to a question stored in the memory unit from the user, and storing the question and the answer to the question in association with each other in the memory unit (214, 213, S1525), accepting a designation from the user as to whether the question and answer should be used as training data for a trained model that is trained to output an answer to an input question (S1621, S1611), and training the trained model based on the specified training data (S1721).
[0335] (Appendix 8) A program according to any one of Appendices 1 to 7, wherein in the display step, the accumulation status is displayed by displaying, for each question, at least one of the number of answers to that question or link information indicating a reference for the answer to that question (2112, 2113).
[0336] (Appendix 9) The program further causes the processor to, when a plurality of answers are accumulated when a similar question identified in one or a plurality of question lists accumulated in the memory unit is aggregated, determine whether there are any changes in the content of the plurality of accumulated answers and display the result of the determination whether there are any changes in the answers to the aggregated questions of the same type, thereby displaying the accumulation status.
[0337] (Appendix 10) 10. The program according to claim 9, wherein in the displaying step, questions whose answers have changed are displayed separately from questions whose answers have not changed.
[0338] (Appendix 11) 11. The program according to any one of appendices 1 to 10, wherein in the displaying step, the accumulation status is displayed by displaying, for each question, information indicating when the question was answered.
[0339] (Appendix 12) The program further causes the processor to execute a step of generating an answer to the question for which an answer is to be generated by the trained model by using the question for which an answer is to be generated as input to the trained model that outputs an answer in response to the question input (S1913, S1927, S2025).
[0340] (Appendix 13) The program described in Appendix 12, further comprising a processor that receives, from a user, a designation of an answer format for each question for which an answer is to be generated, and generates answers to the questions for which an answer is to be generated using the trained model in accordance with the designated answer format (2321).
[0341] (Appendix 14) A program described in any one of Appendices 12 to 13, in which, in the displaying step, for each question, questions for which an answer has been generated by the trained model are displayed separately from questions for which an answer has not been generated by the trained model (2322).
[0342] (Appendix 15) A program described in any one of Appendices 12 to 14, wherein in the step of generating an answer using a trained model, at least one of the contents of the data on which the answer was generated or a link to the data is generated together with the answer, and in the step of displaying the answer, at least one of the data on which the answer was generated or the link is displayed together with the generated answer (2325).
[0343] (Appendix 16) The program further causes the processor to execute a step of accepting input of answers to questions stored in the memory unit from the user, and storing the questions and the answers to the questions in association with each other in the memory unit, and in the display step, displays for each question whether an answer to the question has been input by the user (2111), and manages the questions to which answers have been input and the answers in the memory unit as learning data for a trained model that is trained to output answers to input questions (214).
[0344] (Appendix 17) The program described in any of Appendices 1 to 16, further causing the processor to execute the steps of generating answers to questions stored in the memory unit by referencing a trained model that is trained to output answers to input questions (S1913, S1927), determining whether the answer generated by the trained model includes information that should not be made public as an answer, and notifying the user of the result of the determination.
[0345] (Appendix 18) The program described in Appendix 17, in which in the determining step, it is determined that the answer contains information that should not be made public if the answer contains at least any of personal information such as a personal name of the service provider answering the question, and a link to reference data stored in a system used by the service provider.
[0346] (Appendix 19) The memory unit stores answers to the questions in association with the questions in the question list (213, 214), and the program further causes the processor to: add security-related vocabulary to the questions and answers (S2021); vectorize data consisting of the questions and answers to which the security-related vocabulary has been added (S2021); store the vector obtained by vectorization in association with the question list that was the subject of vectorization in the memory unit (S2021, 217); vectorize the input question in response to input of a question and compare it with the vector stored in the memory unit (S2023); identify a question list stored in the memory unit based on the comparison result in the comparison step, and input the identified question list and the input question into the trained model, thereby causing the trained model to generate answers to the input questions (S2025, S2013).
[0347] (Appendix 20) A method executed by a computer having a processor, the method performing the steps of: storing the received question list in a memory unit each time the processor receives a question list for a service provider, the question list including at least questions corresponding to security check items; referring to one or more question lists stored in the memory unit to identify similar questions; and displaying, for each question, the accumulation status of answers to that question based on the identified results.
[0348] (Appendix 21) An information processing device having a control unit, wherein the control unit executes the steps of: storing the received question list in a memory unit each time the control unit receives a question list for a service provider, the question list including at least questions corresponding to security check items; identifying similar questions by referring to one or more question lists stored in the memory unit; and displaying, for each question, the accumulation status of answers to that question based on the identified results.
Claims
1. A program for operating a computer having a processor, the processor comprising: receiving a list of questions to be asked to a service provider, the list including at least questions corresponding to security check items, and storing the received list of questions in a storage unit; Identifying a group of questions, among the plurality of questions stored in the storage unit, in which an answer to one question determines an answer to another question; receiving, from a user, an input of an answer to a question to be stored in the storage unit, and storing, in the storage unit, the question and the answer to the question in association with each other; generating an answer to the other question without relying on an input operation by the user when the question to which an answer has been received from the user is the one question included in the group of questions; and displaying to the user the generated answer to the other question.
2. A step of identifying similar questions by referring to one or more of the question lists stored in the memory unit; 2. The program according to claim 1, wherein the display step includes a step of displaying, for each of the questions, a status of accumulation of answers to the questions based on the identified questions of the same type.
3. The program further causes the processor to In the step of identifying, it is identified that the same kind of question is stored in duplicate in the one or more stored question lists; 3. The program according to claim 2, wherein in said displaying step, displaying the accumulation status includes displaying that the same type of question is accumulated in duplicate.
4. 4. The program according to claim 3, wherein in the displaying step, the display of the accumulation status includes at least one of displaying the number of overlapping questions of the same type and displaying past answers to the questions of the same type.
5. The program further causes the processor to receiving a category for the question, and storing the question and the received category in the storage unit in association with each other; 3. The program product according to claim 2, wherein in the displaying step, for each question, the category associated with the question is displayed.
6. The program further causes the processor to analyzing the text of the question to identify the category to be assigned to the question and assigning the identified category to the question; 6. The program according to claim 5, wherein in the displaying step, the assigned category is displayed in association with each question.
7. The program further causes the processor to receiving, from a user, an input of an answer to the question to be stored in the storage unit, and storing the question and the answer in the storage unit in association with each other; receiving, from a user, a designation as to whether or not the question and the answer to the question are to be used as learning data for a trained model that is trained to output an answer to an input of a question; The program according to claim 1 , further comprising: a step of training the trained model based on the specified training data.
8. 3. The program according to claim 2, wherein in the display step, the accumulation status is displayed by displaying, for each question, at least one of the number of answers to that question or link information indicating a reference for the answer to that question.
9. The program further causes the processor to when a plurality of answers are accumulated in the one or a plurality of question lists accumulated in the storage unit when the identified questions of the same type are aggregated, determining whether there is a change in the contents of the plurality of accumulated answers; 3. The program according to claim 2, wherein in said displaying step, the display of said accumulation status comprises displaying a result of determining whether there has been a change in said answers to said aggregated questions of the same type.
10. 10. The program according to claim 9, wherein in the displaying step, questions of the same type for which the answers have been changed are displayed in a distinguished manner from questions of the same type for which the answers have not been changed.
11. 3. The program according to claim 2, wherein in said displaying step, the display of said accumulation status comprises displaying, for each of said questions, information indicating when said question was answered.
12. The program further causes the processor to The program of claim 1, further comprising: a step of causing a question for which an answer is to be generated to be input to a trained model that outputs an answer in response to a question input, thereby causing an answer to be generated by the trained model.
13. The program further causes the processor to accepting, from a user, a designation of an answer format for each question for which an answer is to be generated; The program according to claim 12 , further comprising: causing the trained model to generate an answer to a question for which an answer is to be generated, in accordance with the specified format of the answer.
14. 14. The program according to claim 13, wherein in the displaying step, for each of the questions, a question for which the answer has been generated by the trained model is displayed in a distinguished manner from a question for which the answer has not been generated by the trained model.
15. In the step of generating the answer using the trained model, at least one of the contents of data on which the answer was generated or a link to the data is generated together with the answer, 14. The program according to claim 13, wherein in the displaying step, at least one of the data on which the answer is based and the link is displayed together with the generated answer.
16. The program further causes the processor to receiving, from a user, an input of an answer to a question to be stored in the storage unit, and storing, in the storage unit, the question and the answer to the question in association with each other; In the display step, for each of the questions, displaying whether or not an answer to the question has been input; The program according to claim 1 , wherein the question for which the answer has been input and the answer are managed in the memory unit as learning data for a trained model that is trained to output an answer in response to an input of a question.
17. The program further causes the processor to generating an answer to a question stored in the storage unit by referring to a trained model that is trained to output an answer to an input of the question; A step of determining whether the answer generated by the trained model includes information that should not be made public as an answer; The program according to claim 1 , further comprising a step of notifying a user of the result of the determination.
18. In the step of determining, determining that the answer includes information that should not be made public, Personal information, including the name of the service provider who will respond to the question; A link for referencing data stored in the system used by the service provider; 18. The program according to claim 17, further comprising: determining that the answer contains information that should not be made public when at least one of the above is included.
19. the storage unit stores answers to the questions in the question list in association with the questions, The program further causes the processor to adding security related vocabulary to the questions and answers; vectorizing data consisting of the questions and answers to which the security vocabulary has been added; storing the vector obtained by the vectorization and the list of questions that were the subject of the vectorization in the storage unit in association with each other; In response to input of a question, vectorizing the input question and comparing the vector with the vector stored in the storage unit; 2. The program according to claim 1, further comprising: identifying the question list stored in the memory unit based on the results of the comparison in the comparing step; inputting the identified question list and the input question into a trained model; and causing the trained model to generate an answer to the input question.
20. 1. A computer-implemented method comprising: receiving a list of questions to be asked to a service provider, the list including at least questions corresponding to security check items, and storing the received list of questions in a storage unit; Identifying a group of questions, among the plurality of questions stored in the storage unit, in which an answer to one question determines an answer to another question; receiving, from a user, an input of an answer to a question to be stored in the storage unit, and storing, in the storage unit, the question and the answer to the question in association with each other; generating an answer to the other question without relying on an input operation by the user when the question to which an answer has been received from the user is the one question included in the group of questions; displaying to the user the generated answers to the other questions.
21. An information processing device including a control unit, receiving a list of questions to be asked to a service provider, the list including at least questions corresponding to security check items, and storing the received list of questions in a storage unit; Identifying a group of questions, among the plurality of questions stored in the storage unit, in which an answer to one question determines an answer to another question; receiving, from a user, an input of an answer to a question to be stored in the storage unit, and storing, in the storage unit, the question and the answer to the question in association with each other; generating an answer to the other question without relying on an input operation by the user when the question to which an answer has been received from the user is the one question included in the group of questions; and displaying to the user the generated answer to the other question.
Citation Information
Patent Citations
System audit support system and support method, data recording medium and program, check sheet and audit trail ledger
JP2003044658A
Generation device and method and organization device and method for FAQ data, and FAQ answer system
JP2006119991A
Information processing device, fault cause identification method, and program
JP2017054202A
Method, device, and program for managing question list
JP2022109743A
JPP7448293B
Cited By
Support device, program, recording medium, and method for creating documents
JP2026141960A
Support device, program, recording medium, and method for creating documents
JP2026142489A
Programs, methods, information processing devices, graph-type data structures
JP2026144193A
Information processing system, information processing method, and program
JP7713610B1
