Information processing device, information processing method, and information processing program

Abstract

The present invention provides an information processing device, an information processing method, and an information processing program that can quickly detect inappropriate input from users to a generating AI and warn or restrict access to the responses. [Solution] The information processing device 10 includes an access control unit 11 that receives input information from a user terminal 20 to the generation AI server, an evaluation information setting unit 12 that sets evaluation information to evaluate the input information, an input determination unit 13 that determines whether the input information from the user terminal is inappropriate based on the evaluation information, and an answer transmission unit 14 that edits the answer from the generation AI server and sends it to the user terminal when the input information is determined to be inappropriate.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, AI (Artificial Intelligence) has been utilized in various fields, and in particular, the use of AI that can generate content, called generative AI, has been rapidly spreading. Some generative AIs can use large-scale learning models that have learned a large amount of natural language and programming languages to generate articles, images, summarize articles, etc. with high accuracy in response to user instructions. By using generative AI, when a user searches for and summarizes information on the Internet, the necessary information can be accurately obtained.

[0003] Conventionally, a system has been disclosed that generates a plurality of scenarios of cyberattacks using generative AI and evaluates security risks according to the scenarios.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] While various generative AIs have been developed and useful information acquisition and problem-solving consultations can be quickly performed using generative AI, for example, there is a risk that information inappropriate for education may be easily provided to children and students in a form that seems to interact with humans. Since the answers of generative AI may have an adverse impact on users such as children and students, it may be necessary to give attention and warnings about the answers of generative AI to inappropriate inputs (questions), or to restrict the viewing of the answers.

[0006] This invention has been made in consideration of these circumstances, and aims to provide an information processing device, an information processing method, and an information processing program that can quickly detect inappropriate input by a user to a generating AI and warn or restrict viewing of the response. [Means for solving the problem]

[0007] An information processing apparatus according to an embodiment of the present invention is characterized by comprising: an access control unit that receives input information from a user of a user terminal to a generation AI server; an evaluation information setting unit that sets evaluation information for evaluating the input information; an input determination unit that determines whether the input information from the user of the user terminal is inappropriate based on the evaluation information; and a response transmission unit that, when the input information is determined to be inappropriate, edits the response of the generation AI server and transmits it to the user terminal. [Effects of the Invention]

[0008] Embodiments of the present invention provide an information processing device, an information processing method, and an information processing program that can quickly detect inappropriate input by a user to a generating AI and warn or restrict viewing of the response. [Brief explanation of the drawing]

[0009] [Figure 1] A configuration diagram showing an example of the configuration of the information processing device according to this embodiment. [Figure 2] This schematic diagram shows an example of how the AI-generated response is edited when user input information is inappropriate in the information processing device according to this embodiment. [Figure 3] A schematic diagram illustrating another example of how the AI-generated response is edited when user input information is inappropriate in the information processing device according to this embodiment. [Figure 4] A flowchart showing an example of the information processing method according to this embodiment. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present invention will be described based on the attached drawings. Figure 1 shows an example of the configuration of the information processing device 10 according to this embodiment.

[0011] The information processing device 10 is a server that acts as an intermediary for the user terminal 20 to access the external network. The information processing device 10 is positioned between the user terminal 20 and the AI ​​generation server 50 on the external network, for example, as shown in Figure 1.

[0012] User terminals 20 refer to devices such as PCs, smartphones, and tablet devices owned by each user that are capable of communicating with an external network and can communicate with the information processing device 10 via the network. Although Figure 1 shows three user terminals 20 (20a, 20b, 20c) owned by each user, the configuration is not limited to this.

[0013] The administrator terminal 30 refers to a terminal such as a PC, smartphone, or tablet owned by an administrator user (for example, a system administrator, officer, or teacher in an organization) who has been set as an administrator in the information processing device 10, and which is capable of communicating with an external network and can communicate with the information processing device 10 via the network.

[0014] The generation AI server 50 is a server that automatically generates responses to user input, equipped with a learning model that has learned information, programming languages, images, etc., from the network. The generation AI server 50 and the information processing device 10 may be configured as a single unit.

[0015] The information processing device 10 receives an access request from the user terminal 20 to the generation AI server 50 and, on behalf of the user terminal 20, performs input (input of a question) to the generation AI server 50. The information processing device 10 then returns the content, such as the answer obtained from the generation AI server 50, to the user terminal 20.

[0016] The information processing apparatus 10 according to the embodiment determines whether the input information from the user of the user terminal 20 to the generation AI server 50 is inappropriate using preset evaluation information, and when the input information is inappropriate, edits the response of the generation AI server 50 and transmits it to the user terminal 20.

[0017] A specific configuration of the information processing apparatus 10 according to the present embodiment will be described. The information processing apparatus 10 includes an access control unit 11, an evaluation information setting unit 12, an input determination unit 13, and a response transmission unit 14.

[0018] Note that the functions of each unit constituting the information processing apparatus 10 may be realized by executing a predetermined program code using a processor, and are not limited to such software processing. For example, they may be realized by hardware processing using an ASIC or the like, or may be realized by combining software processing and hardware processing.

[0019] The access control unit 11 controls external access by the user terminal 20, and receives a response from the generation AI server 50 regarding the input information from the user to the generation AI server 50. The input information means information such as text, files, images, etc. that the user inputs to request a response from the generation AI server 50.

[0020] The evaluation information setting unit 12 sets evaluation information for evaluating the input information. The evaluation information is information for determining whether the input information by the user to the generative AI is inappropriate. As the evaluation information, information is set that the answer of the generative AI to the user's input information is unfavorable to the user and may have an adverse effect. For example, when the user is a child or a student, malicious words or sentences related to bullying, words related to suicide, etc., words or sentences that may have an adverse effect on the answer of the generative AI for children or students are set as evaluation information. Also, when the user is a member belonging to an organization such as a company or a group, words or sentences in the work policy of the organization that the answer of the generative AI has an adverse effect on the user's work are set as evaluation information. Further, as the evaluation information, data such as files and images that may have an adverse effect on the user by viewing the answer of the generative AI may be set. The evaluation information is set by the administrator of the information processing device 10 or the like.

[0021] Further, the evaluation information setting unit 12 may classify and set words, sentences, etc. that become evaluation information according to the degree of inappropriateness of the input information to the generative AI into categories. For example, while setting highly offensive inappropriate words related to bullying in a category with a high degree of inappropriateness, setting inappropriate words with a relatively low degree of inappropriateness such as those used in exchanges between friends in a low category. In this way, by classifying and setting words, sentences, etc. that become evaluation information according to the degree of inappropriateness, the control level for the answer may be adjusted, such as not permitting the editing or transmission of the answer of the generative AI depending on the category of the user's input information. The method of not permitting the editing or transmission of the answer of the generative AI will be described later.

[0022] The input determination unit 13 determines whether the input information by the user of the user terminal 20 is inappropriate based on the evaluation information. Specifically, by comparing the input information with the evaluation information that is inappropriate words, sentences, etc., when the input information matches or is similar to the evaluation information (such as when the evaluation information is included in the input information), it is determined that the input information is inappropriate. On the other hand, when the input information does not match or is not similar to the evaluation information, it is determined that the input information is appropriate.

[0023] The response transmission unit 14 edits the response from the generation AI server 50 and sends it to the user terminal 20 when it determines that the input information is inappropriate. Specifically, it adds a warning or caution to the user regarding viewing the response from the generation AI server 50 and sends it to the user terminal 20. It may also include the reason why the input information was determined to be inappropriate in the response from the generation AI server 50 along with the warning.

[0024] Furthermore, when the input information is determined to be inappropriate, the response transmission unit 14 may edit the content of the response from the generating AI server 50 so that the user cannot view it, and then send it to the user terminal 20. Specifically, the response from the generating AI server 50 may be deleted or masked to prevent the user from viewing the content of the response. Alternatively, only the part of the response related to the content of the input information that was determined to be inappropriate may be deleted or masked. In addition, the reason why the input information was determined to be inappropriate may be included in the response from the generating AI server 50.

[0025] Furthermore, the response transmission unit 14 may refuse to transmit the response to the user terminal 20 if it determines that the input information is inappropriate. It may also refuse to transmit the response to the user terminal 20 if the input information matches or is similar to specific evaluation information (words or sentences with a high degree of inappropriateness). In this case, the user terminal 20 may be notified that it has refused to transmit the response.

[0026] Furthermore, when the response transmission unit 14 determines that the input information is inappropriate, it may notify the administrator user on the pre-configured administrator terminal 30 of the input information along with the user who made the input. This allows the administrator user to constantly monitor whether users within the organization are making inappropriate inputs.

[0027] Furthermore, if the response transmission unit 14 has categorized words, sentences, etc., that serve as evaluation information according to the degree of inappropriateness of the input information to the generating AI, it may adjust the level of control over the response, such as disallowing editing or transmission of the generating AI's response depending on the category. For example, if the input information falls into a category with a low degree of inappropriateness, only a warning is included in the response, while if it falls into a category with a higher degree of inappropriateness, the response may be deleted or masked. In the case of an even higher category with a higher degree of inappropriateness, the system may control the system to disallow the transmission of the response altogether.

[0028] Furthermore, the response transmission unit 14 may, at any time, record inappropriate input information in association with the user who entered it, if the input information is deemed inappropriate. By recording inappropriate input information as a log, it is possible to understand the user's school life, work status, etc.

[0029] Figure 2 is an explanatory diagram showing an example of how the information processing device 10 according to this embodiment edits the response of the generating AI when the user input is inappropriate (see Figure 1 as appropriate). Here, we consider the case where the user inputs a method related to an inappropriate item to the generating AI.

[0030] The information processing device 10 evaluates the user input information from the user terminal 20 as inappropriate based on evaluation information. Then, it adds a warning message to the response generated by the AI ​​server 50 to caution the user before viewing it, and sends it to the user terminal 20. In addition, it notifies the user on the pre-configured administrator terminal 30 of the input information along with the user's information. As a result, the user who made the input will be aware that it is inappropriate before viewing the response.

[0031] Figure 3 is an explanatory diagram showing another example of how the information processing device 10 according to this embodiment edits the AI's response when the user's input is inappropriate (see Figure 1 as appropriate). Here, similar to Figure 2, we consider the case where the user inputs a method related to an inappropriate matter to the AI.

[0032] The information processing device 10 evaluates the user input information from the user terminal 20 as inappropriate based on evaluation information. It then masks the response from the generating AI server 50 to prevent the user from viewing the response. In addition, it notifies the user on the pre-configured administrator terminal 30 of the input information along with the user's information. As a result, the user who made the input is unable to view the inappropriate input information and is prevented from accessing information that could have a negative impact on them.

[0033] Next, the operation of the information processing device 10 according to this embodiment will be described. Figure 4 is a flowchart of an example of the information processing method according to this embodiment (refer to Figure 1 as appropriate).

[0034] The access control unit 11 receives user input information (S10).

[0035] The input determination unit 13 determines whether the input information from the user terminal 20 is inappropriate based on the evaluation information (S11).

[0036] The response transmission unit 14, when it determines that the input information is inappropriate, edits the response from the generating AI server 50 and sends it to the user terminal 20 (S12:YES, S13). Then, it displays the edited response on the user terminal 20 (S14).

[0037] On the other hand, if the input information is not inappropriate, the answer is displayed on the user terminal 20 (S12: NO, S14).

[0038] According to the information processing device of the embodiment described above, the input information from the user terminal to the generation AI server is determined to be inappropriate or not using pre-set evaluation information. If the input information is inappropriate, the response from the generation AI server is edited and sent to the user terminal, thereby quickly detecting inappropriate input from the user to the generation AI and allowing for warnings or restrictions on viewing the response.

[0039] The program executed by the information processing device 10 is provided pre-installed in a storage circuit such as ROM. Alternatively, this program may be provided as an installable or executable file stored on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk. Furthermore, the program executed by the information processing device 10 may be stored on a computer connected to a network such as the Internet and provided for download via the network.

[0040] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0041] 10... Information processing device, 11... Access control unit, 12... Evaluation information setting unit, 13... Input judgment unit, 14... Response transmission unit, 20 (20a, 20b, 20c)... User terminal, 30... Management terminal, 50... Generation AI server.

Claims

1. An access control unit that receives input information from the user on the user terminal to the AI ​​server, An evaluation information setting unit sets evaluation information to evaluate the aforementioned input information, An input determination unit that determines whether the input information from the user of the user terminal is inappropriate based on the evaluation information, When the input information is determined to be inappropriate, the response transmission unit edits the response of the generating AI server and sends it to the user terminal. An information processing device characterized by comprising:

2. An access control unit that receives input information from the user on the user terminal to the AI ​​server, An evaluation information setting unit sets evaluation information to evaluate the aforementioned input information, An input determination unit that determines whether the input information from the user of the user terminal is inappropriate based on the evaluation information, A response transmission unit that, when the input information is determined to be inappropriate, denies the transmission of the AI ​​server's response to the user terminal, An information processing device characterized by comprising:

3. The response transmission unit, when it determines that the input information is inappropriate, adds a warning to the response from the generating AI server and transmits it to the user terminal. The information processing apparatus according to feature 1.

4. The response transmission unit, when it determines that the input information is inappropriate, edits the content of the response from the generating AI server so that the user cannot view it, and then transmits it to the user terminal. The information processing apparatus according to feature 1.

5. The response transmission unit, when it determines that the input information is inappropriate, will include the reason for the inappropriateness in the response. The information processing apparatus according to feature 1.

6. When the aforementioned input information is determined to be inappropriate, a notification is sent to a pre-configured administrator terminal. The information processing apparatus according to feature 1.

7. The aforementioned evaluation information setting unit sets evaluation information classified based on the degree of inappropriateness, The response transmission unit edits the response of the generating AI server according to the degree of inappropriateness of the input information determined based on the evaluation information, and transmits it to the user terminal. The information processing apparatus according to feature 1.

8. When the input information is determined to be inappropriate, the inappropriate input information is recorded in association with the user who entered it. The information processing apparatus according to feature 1.

9. The steps include: receiving input information from the user's terminal to the AI ​​server that generates data; A step of setting evaluation information to evaluate the aforementioned input information, The steps include determining whether the input information by the user of the user terminal is inappropriate based on the evaluation information, When the input information is determined to be inappropriate, the AI ​​server edits the response and sends it to the user terminal. An information processing method characterized by including

10. Computers, A function that accepts input information from the user terminal to the AI ​​server that generates data. A function to set evaluation information for evaluating the aforementioned input information. A function that determines whether the input information from the user of the user terminal is inappropriate based on the evaluation information. A function to edit the response of the generating AI server and send it to the user terminal when the input information is determined to be inappropriate. An information processing program characterized by being executed as such.

Images