Information processing apparatus

Abstract

To detect a change of an AI model.SOLUTION: An information processing apparatus comprises monitoring means configured to monitor operation related to an AI model, and determination means configured to determine, based on the monitoring result of the operation related to the AI model, whether the operation related to the AI model includes any operation deviating from a predetermined operation.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to the technical field of information processing devices. [Background technology]

[0002] As an AI (Artificial Intelligence) model used in this type of device, for example, an AI model has been proposed that rewrites sentences written in Japanese so that they are properly translated into English (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-129995 Summary of the Invention [Problem to be solved by the invention]

[0004] AI models may be subject to cyberattacks. Examples of cyberattacks include malware that rewrites AI models and / or contaminates training datasets (which may result in unintended behavior by the AI ​​model). However, there is a technical problem in that it is difficult to determine whether an AI model has been subjected to a cyberattack.

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide an information processing device that can detect changes in an AI model. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention comprises a monitoring means for monitoring operations related to an AI model, and a determination means for determining, based on the monitoring results of the operations related to the AI ​​model, whether the operations related to the AI ​​model include operations that deviate from predetermined operations. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating an example of a configuration of an information processing apparatus according to an embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of a configuration of a calculation device according to an embodiment. [Figure 3] 10 is a flowchart illustrating an example of an operation of the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An embodiment of an information processing device will be described with reference to Fig. 1 to Fig. 3. In Fig. 1, the information processing device 100 includes a calculation device 110, a storage device 120, a communication device 130, an input device 140, and an output device 150. The calculation device 110, the storage device 120, the communication device 130, the input device 140, and the output device 150 are connected via a data bus 160. Note that the information processing device 100 does not necessarily have to include at least one of the input device 140 and the output device 150.

[0009] The arithmetic device 110 may include a processor 1101. The arithmetic device 110 may include other processors in addition to the processor 1101. That is, the arithmetic device 110 may include one or more processors. The processor 1101 may be, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), and a tensor processing unit (TPU).

[0010] The storage device 120 may include a memory 1201. The storage device 120 may include other memories in addition to the memory 1201. That is, the storage device 120 may include one or more memories. The memory 1201 may be, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and an optical disk array.

[0011] The storage device 120 can store desired data. A computer program 1201a executed by the arithmetic device 110 may be stored in the memory 1201 of the storage device 120. The storage device 120 may temporarily store data that is temporarily used by the arithmetic device 110 when the arithmetic device 110 is executing the computer program 1201a.

[0012] The communication device 130 may be capable of communicating with devices external to the information processing device 100. The communication device 130 may perform wired communication or wireless communication.

[0013] The input device 140 is a device capable of accepting information input from outside to the information processing device 100. The input device 140 may include an operation device (for example, a keyboard, a mouse, a touch panel, etc.) that can be operated by a user of the information processing device 100.

[0014] The output device 150 is a device capable of outputting information to the outside of the information processing device 100. The output device 150 may output visual information such as characters or images, auditory information such as sound, or tactile information such as vibration, as the information. The output device 150 may include, for example, at least one of a display, a speaker, a printer, and a vibration motor.

[0015] The processor 1101 of the arithmetic device 110, together with the memory 1201 of the storage device 120 in which a computer program 1201a is stored, may execute the processing to be performed by the information processing device 100. For example, the processor 1101 may execute the computer program 1201a to realize logical functional blocks in the arithmetic device 110 for executing the processing to be performed by the information processing device 100.

[0016] 2, the arithmetic device 110 may have a monitoring unit 111 and a determining unit 112 as logically realized functional blocks or as physically realized processing circuits. At least one of the monitoring unit 111 and the determining unit 113 may be realized in a form in which a logical functional block and a physical processing circuit (i.e., hardware) are mixed.

[0017] The operation of the information processing device 100 will be described with reference to the flowchart of FIG. 3. In FIG. 3, the monitoring unit 111 of the information processing device 100 monitors operation related to the AI ​​model (step S101). "Operation related to the AI ​​model" is a concept that includes not only the operation of the AI ​​model itself but also the operation of a device that uses the AI ​​model. The monitoring unit 111 may monitor the AI ​​model, for example, by monitoring log data related to the AI ​​model. The monitoring unit 111 may monitor the AI ​​model, for example, by monitoring the operation of a device that uses the AI ​​model. Note that the monitoring unit 111 may have an AI model for monitoring operation related to the AI ​​model. In other words, the monitoring unit 111 may use the AI ​​model to monitor operation related to the AI ​​model.

[0018] The determination unit 112 of the information processing device 100 may acquire the monitoring results of the operations related to the AI ​​model by the monitoring unit 111. Based on the monitoring results of the operations related to the AI ​​model, the determination unit 112 determines whether the operations related to the AI ​​model include operations that deviate from predetermined operations (step S102).

[0019] For example, the device using the AI ​​model may be an autonomous mobile robot. For example, the AI ​​model may be an AI model for controlling the robot. For example, the AI ​​model may temporarily halt the robot when it detects an obstacle as a predetermined behavior. For example, the monitoring result may include a case where the robot did not temporarily halt even though an obstacle obstructing the robot's movement was present on the robot's path. In this case, it can be said that the behavior of the robot (in other words, the behavior of the AI ​​model) deviates from the predetermined behavior. In this case, the determination unit 112 may determine that the behavior related to the AI ​​model includes a behavior that deviates from the predetermined behavior.

[0020] In the process of step S102, if it is determined that the behavior related to the AI ​​model includes a behavior that deviates from the predetermined behavior (step S102: Yes), for example, the arithmetic device 110 may control the output device 150 to issue a warning. On the other hand, if it is determined in the process of step S102 that the behavior related to the AI ​​model does not include a behavior that deviates from the predetermined behavior (step S102: No), the operation shown in Fig. 3 may be terminated. Note that in this case, the arithmetic device 110 may control the output device 150 to notify that there is no change in the AI ​​model to be monitored.

[0021] A backup model may exist as a backup for the AI ​​model to be monitored. In this case, if it is determined in the processing of step S102 that the behavior related to the AI ​​model includes behavior that deviates from the predetermined behavior (step S102: Yes), the information processing device 100 may replace the AI ​​model to be monitored with the backup model. In other words, the information processing device 100 may restore the AI ​​model by replacing the changed AI model with a backup model that corresponds to the AI ​​model before the change.

[0022] (Technical Effects) AI models are often constructed to perform predetermined operations. When a constructed AI model (in other words, a trained model) performs an operation that deviates from the predetermined operation, there is a high possibility that the AI ​​model has changed from the AI ​​model at the time of construction. In the information processing device 100, the determination unit 112 determines whether the operation related to the AI ​​model includes an operation that deviates from the predetermined operation. Therefore, the information processing device 100 can detect changes in the AI ​​model.

[0023] Aspects of the invention derived from the above-described embodiments will be described below.

[0024] An information processing device according to one aspect of the invention includes a monitoring unit that monitors operations related to an AI model, and a determining unit that determines, based on the monitoring results of the operations related to the AI ​​model, whether the operations related to the AI ​​model include operations that deviate from predetermined operations. In the above-described embodiment, the "monitoring unit 111" corresponds to an example of the "monitoring unit," and the "determining unit 112" corresponds to an example of the "determining unit."

[0025] The present invention is not limited to the above-described embodiments, but can be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices that involve such modifications are also included in the technical scope of the present invention. [Explanation of symbols]

[0026] 100...information processing device, 111...monitoring unit, 112...determination unit

Claims

[Claim 1] monitoring means for monitoring operations relating to the AI ​​model; A determination means for determining whether or not the behavior of the AI ​​model includes a behavior that deviates from a predetermined behavior based on a monitoring result of the behavior of the AI ​​model; An information processing device comprising:

Images