Information processing program, information processing method and information processing unit

Abstract

To make it easier to precisely detect a patch attack.SOLUTION: An information processing unit 100 acquires a first classification result 111 by utilizing a first machine learning model 110. The information processing unit 100 acquires a second classification result 121 by utilizing a second machine learning model 120. The information processing unit 100 specifies a first frame based upon the acquired first classification result 111 and the acquired second classification result 121. The information processing unit 100 specifies a first class with the highest certainty factor among first classification results 111 for any reference frame. The information processing unit 100 specifies a second frame right before or right after the specified first frame. The information processing unit 100 determines that a series of frames are inaccurate when the relationship between the first certainty factor of the first class with the first classification results 111 for the first frame and the second certainty factor of the first class with the first classification results 111 for the second frame meet predetermined conditions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing program, an information processing method, and an information processing device. [Background technology]

[0002] Conventionally, there are machine learning models that recognize areas in video frames where objects are captured and calculate confidence levels that represent the likelihood of classifying the objects into multiple classes. Patch attacks exist that degrade the accuracy of machine learning models by capturing specific patches in the frames. These specific patches are also called adversarial patches. Patch attacks reduce the confidence levels of classes calculated by the machine learning model that are correct for classifying an object, while increasing the confidence levels of classes that are incorrect for classifying an object.

[0003] Prior art techniques include, for example, determining whether an abnormal pattern is included in a time series recognition score generated using multiple recognition scores calculated using multiple image data captured within a shooting range at different times within the shooting time range. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2022 / 264331 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional techniques have a problem in that it is difficult to accurately detect patch attacks. For example, it is not possible to distinguish whether a decrease in the confidence level of a class calculated by a machine learning model to correctly classify an object is due to a patch attack or a characteristic of the machine learning model.

[0006] In one aspect, the present invention aims to facilitate accurate detection of patch attacks. [Means for solving the problem]

[0007] According to one embodiment, a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes is used to obtain, for each of a series of frames, a first classification result that associates two or more classes into which an object in the frame has been classified with a confidence level for each of the two or more classes, and a second machine learning model different from the first machine learning model that calculates a confidence level for classifying an object in the frame into each of a plurality of classes is used to obtain, for each of the series of frames, a second classification result that associates two or more classes into which an object in the frame has been classified with a confidence level for each of the two or more classes, and based on the obtained first classification result and the obtained second classification result, a classification result for the series of frames with the highest confidence level for each of the first classification result and the second classification result is obtained. an information processing program, an information processing method, and an information processing device that identify a first frame for which the class with the highest confidence in the first classification result is different from any frame for which the class with the highest confidence in the first classification result is the same, and for which the class with the highest confidence in each of the first classification result and the second classification result is different; and, based on the first class with the highest confidence in the first classification result for any of the frames, determine that the series of frames is fraudulent if a relationship between a first confidence of the first class in the first classification result for the identified first frame and a second confidence of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition; and determine that the series of frames is not fraudulent if the relationship does not satisfy the predetermined condition. [Effects of the Invention]

[0008] According to one aspect, it is possible to facilitate accurate detection of patch attacks. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of an information processing method according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram illustrating an example of an information processing system 200. As shown in FIG. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. As shown in FIG. [Figure 4] FIG. 4 is a block diagram showing an example of the functional configuration of the information processing device 100. As shown in FIG. [Figure 5] FIG. 5 is an explanatory diagram (part 1) showing an example of the operation of the information processing device 100. [Figure 6] FIG. 6 is an explanatory diagram (part 2) showing an example of the operation of the information processing device 100. [Figure 7] FIG. 7 is an explanatory diagram showing an example of the effect. [Figure 8] FIG. 8 is a flowchart illustrating an example of the overall processing procedure. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an information processing program, an information processing method, and an information processing device according to embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0011] (An example of an information processing method according to an embodiment) 1 is an explanatory diagram illustrating an example of an information processing method according to an embodiment. An information processing device 100 is a computer for detecting patch attacks. The information processing device 100 is, for example, a server or a PC (Personal Computer).

[0012] Conventionally, there exists a machine learning model called an object detection model. The object detection model has the function of recognizing an area in a frame where an object is captured, identifying the type of object, and calculating a confidence score that indicates the likelihood of classifying the object into one of multiple classes. The confidence score is a value between 0 and 1. Examples of object detection models include YOLO and Faster RCNN. For information on YOLO, see, for example, Reference 1 below. For information on Faster RCNN, see, for example, Reference 2 below.

[0013] Reference 1: “YOLO: Real-Time Object Detection”, [online], [Retrieved June 20, 2023], Internet <URL: https: / / pjreddie.com / darknet / yolo / >

[0014] Reference 2: Ren, Shaoqing, et al. “Faster r-cnn: Towards real-time object detection with region proposal networks.” Advances in neural information processing systems 28 (2015).

[0015] A patch attack reduces the accuracy of a machine learning model by projecting an adversarial patch into a frame. For example, an adversarial patch may be projected into a frame showing an object by attaching an adversarial patch to the object. For example, an adversarial patch may be projected into a frame showing an object by synthesizing an adversarial patch into the frame showing the object. A patch attack reduces the confidence of the machine learning model in the correct class for classifying the object and increases the confidence of the incorrect class for classifying the object. Therefore, a patch attack results in an incorrect identification of the class to which an object belongs. For more information on patch attacks, see, for example, References 3 and 4 below.

[0016] Reference 3: Brown, Tom B., et al. “Adversarial patch.” arXiv preprint arXiv:1712.09665 (2017).

[0017] Reference 4: Liu, Xin, et al. “Dpatch: An adversarial patch attack on object detectors.” arXiv preprint arXiv:1806.02299 (2018).

[0018] However, conventionally, there is a problem in that it is difficult to detect patch attacks with high accuracy. For example, Method 1 can be considered, which attempts to detect patch attacks by using two types of object detection models. The two types of object detection models have, for example, different algorithms. For example, it is preferable that the two types of object detection models are in a state where an adversarial patch that is effective for one object detection model is not effective for the other object detection model. Specifically, Method 1 detects a patch attack when the classes with the highest confidence calculated by each of the two types of object detection models do not match.

[0019] It is difficult for Method 1 to accurately detect patch attacks. For example, even when there is no patch attack, the object detection model may calculate a higher confidence level for a class in which the object is incorrectly classified than for a class in which the object is correctly classified. In this case, Method 1 may erroneously detect a patch attack even though there is no patch attack. Therefore, as a result of erroneously detecting a patch attack, it is possible that a legitimate user may be determined to be an attacker.

[0020] Furthermore, for example, Method 2 can be considered to detect patch attacks when the number of frames in a series of frames in which the most certain classes calculated by two types of object detection models do not match is equal to or greater than a predetermined threshold. Even with Method 2, it is difficult to accurately detect patch attacks. As described above, for example, even in the absence of a patch attack, an object detection model may calculate a higher certainty for a class in which the object is incorrectly classified as an algorithm than for a class in which the object is correctly classified as an algorithm. In this case, Method 2 cannot distinguish whether the discrepancy between the most certain classes calculated by two types of object detection models for a certain frame is due to a patch attack or an algorithmic problem in the object detection models. Therefore, with Method 2, the number of frames in a series of frames in which the most certain classes calculated by two types of object detection models do not match may not be an appropriate indicator for detecting patch attacks.

[0021] Therefore, in this embodiment, an information processing method that can easily detect patch attacks with high accuracy will be described. Specifically, this embodiment can easily distinguish whether the cause of a discrepancy between the most certain classes calculated by two types of machine learning models is a patch attack or a problem with the algorithm of one of the machine learning models.

[0022] 1, the information processing device 100 stores a first machine learning model 110. The first machine learning model 110 has a function of receiving an input of a frame and calculating a confidence factor representing the likelihood of classifying an object in the input frame into each of a plurality of classes. The frame is image data. The first machine learning model 110 is, for example, YOLO.

[0023] The information processing device 100 stores a second machine learning model 120. The second machine learning model 120 has a function of receiving an input frame and calculating a confidence factor representing the likelihood of classifying an object in the input frame into each of a plurality of classes. The second machine learning model 120 preferably has an algorithm different from that of the first machine learning model 110. The second machine learning model 120 is, for example, Faster RCNN.

[0024] The information processing device 100 acquires a series of frames. The series of frames is, for example, a plurality of image data included in moving image data. The series of frames may be, for example, a plurality of image data captured consecutively, rather than moving image data. The information processing device 100 acquires the series of frames, for example, by receiving the series of frames from a predetermined image capturing device. The information processing device 100 may acquire the series of frames by, for example, accepting input of the series of frames based on an operational input by a user. In the example of FIG. 1, the information processing device 100 specifically acquires a series of frames including frames 101 to 104.

[0025] (1-1) The information processing device 100 uses the first machine learning model 110 to obtain, for each of a series of frames, a first classification result 111 that associates two or more classes into which the object in the frame has been classified with the confidence level for each of the two or more classes.

[0026] 1, the first classification result 111 specifically includes correspondence information "face: m1v1[1]" that associates the class "face" with the highest certainty into which the object in frame 101 has been classified and the certainty "m1v1[1]" of the class "face". The first classification result 111 may specifically include correspondence information (not shown) that associates each of the classes with the second and subsequent highest certainty into which the object in frame 101 has been classified and the certainty of the class.

[0027] Specifically, the first classification result 111 includes correspondence information "face: m1v1[2]" that associates the class "face" with the highest certainty into which the object in frame 102 has been classified and the certainty "m1v1[2]" of the class "face". Specifically, the first classification result 111 may include correspondence information (not shown) that associates each of the classes with the second and subsequent highest certainty into which the object in frame 102 has been classified and the certainty of the class.

[0028] Specifically, the first classification result 111 includes correspondence information "ball:m1v1[3]" that associates the class "ball" with the highest certainty into which the object in frame 103 has been classified, with the certainty "m1v1[3]" of the class "ball". Specifically, the first classification result 111 includes correspondence information "face:m1v2[3]" that associates the class "face" with the second highest certainty into which the object in frame 103 has been classified, with the certainty "m1v2[3]" of the class "face". Specifically, the first classification result 111 may include correspondence information (not shown) that associates each of the classes with the third and subsequent highest certainty into which the object in frame 103 has been classified, with the certainty of the class.

[0029] Specifically, the first classification result 111 includes correspondence information "face: m1v1[4]" that associates the class "face" with the highest certainty into which the object in frame 104 has been classified and the certainty of the class "face" with "m1v1[4]". Specifically, the first classification result 111 may include correspondence information (not shown) that associates each of the classes with the second highest certainty and subsequent certainty into which the object in frame 104 has been classified and the certainty of the class.

[0030] (1-2) The information processing device 100 uses the second machine learning model 120 to obtain, for each of a series of frames, a second classification result 121 that associates two or more classes into which the object in the frame has been classified with the confidence level for each of the two or more classes.

[0031] 1, the second classification result 121 specifically includes correspondence information "face: m2v1[1]" that associates the class "face" with the highest certainty into which the object in frame 101 has been classified and the certainty of the class "face" with "m2v1[1]." The second classification result 121 may specifically include correspondence information (not shown) that associates each of the classes with the second and subsequent highest certainty into which the object in frame 101 has been classified and the certainty of the class.

[0032] Specifically, the second classification result 121 includes correspondence information "face: m2v1[2]" that associates the class "face" with the highest certainty into which the object in frame 102 has been classified and the certainty "m2v1[2]" of the class "face". Specifically, the second classification result 121 may include correspondence information (not shown) that associates each of the classes with the second and subsequent highest certainty into which the object in frame 102 has been classified and the certainty of the class.

[0033] Specifically, the second classification result 121 includes correspondence information "face: m2v1[3]" that associates the class "face" with the highest certainty into which the object in frame 103 has been classified and the certainty of the class "face" with "m2v1[3]". Specifically, the second classification result 121 may include correspondence information (not shown) that associates each of the classes with the second highest certainty and subsequent certainty into which the object in frame 103 has been classified and the certainty of the class.

[0034] Specifically, the second classification result 121 includes correspondence information "face: m2v1[4]" that associates the class "face" with the highest certainty into which the object in frame 104 has been classified, with the certainty of the class "face" "m2v1[4]." Specifically, the second classification result 121 may include correspondence information (not shown) that associates each of the classes with the second and subsequent highest certainty into which the object in frame 104 has been classified, with the certainty of the class.

[0035] (1-3) The information processing device 100 identifies a first frame based on the acquired first classification result 111 and the acquired second classification result 121. The first frame is a frame in the series of frames in which the class with the highest confidence level in the first classification result 111 is different from that of any of the reference frames, and in which the class with the highest confidence level in each of the first classification result 111 and the second classification result 121 is different.

[0036] The reference frame is a frame in the series of frames that matches the class with the highest confidence in both the first classification result 111 and the second classification result 121. In the example of FIG. 1, the reference frame is specifically frame 101.

[0037] 1, the information processing device 100 specifically identifies frame 103, which has a class with a highest confidence level different from that of frame 101 in the first classification result 111, and which also has a class with a highest confidence level different from that of frame 101 in both the first classification result 111 and the second classification result 121. Specifically, the information processing device 100 treats frame 103 as the first frame. This allows the information processing device 100 to identify the first frame in which a patch attack may have been carried out.

[0038] (1-4) The information processing device 100 identifies the first class with the highest degree of certainty in the first classification result 111 for any of the reference frames. In the example of FIG. 1, the information processing device 100 specifically identifies the class "face" with the highest degree of certainty in the first classification result 111 for frame 101 as the first class. The information processing device 100 identifies a second frame immediately before or after the identified first frame. In the example of FIG. 1, the information processing device 100 specifically identifies frame 102 immediately before frame 103 as the second frame.

[0039] Based on the first class, the information processing device 100 determines whether the relationship between the first certainty factor of the first class in the first classification result 111 for the first frame and the second certainty factor of the first class in the first classification result 111 for the second frame satisfies a predetermined condition. The information processing device 100 determines, for example, whether the absolute value of the difference between the first certainty factor and the second certainty factor satisfies the predetermined condition. In the example of FIG. 1, the predetermined condition specifically indicates that the absolute value of the difference between the first certainty factor and the second certainty factor is equal to or greater than a threshold.

[0040] In the example of FIG. 1, the information processing device 100 specifically acquires, as the first certainty factor, the certainty factor "m1v2[3]" of the class "face" into which the object in frame 103 is classified. The information processing device 100 specifically acquires, as the second certainty factor, the certainty factor "m1v1[2]" of the class "face" into which the object in frame 102 is classified. The information processing device 100 specifically determines whether the absolute value of the difference between the first certainty factor and the second certainty factor, "|m1v2[3]-m1v1[2]|", is equal to or greater than a threshold. The threshold is, for example, set in advance by the user.

[0041] The information processing device 100 determines that a series of frames is fraudulent when the relationship satisfies a predetermined condition. A series of frames being fraudulent means, for example, that a patch attack has been carried out on at least one of the series of frames. Specifically, a series of frames being fraudulent may mean that a patch attack has been carried out on a first frame of the series of frames. The information processing device 100 determines that a series of frames is not fraudulent when the relationship does not satisfy a predetermined condition. The information processing device 100 may output the result of determining whether a series of frames is fraudulent so that the user can refer to it.

[0042] In the example of FIG. 1, the information processing device 100 specifically determines that the absolute difference value "|m1v2[3]-m1v1[2]|" is equal to or greater than a threshold. Therefore, the information processing device 100 specifically determines that the series of frames is fraudulent. Specifically, the information processing device 100 outputs a notification indicating that the series of frames has been determined to be fraudulent, so that the notification can be referenced by the user. This enables the information processing device 100 to accurately detect patch attacks.

[0043] For example, when an adversarial patch is attached to an object, patch attacks are particularly effective in frames in which the object is photographed from a specific shooting direction, and it is believed that the certainty factor for the correct class for classifying the object is likely to decrease. Therefore, the information processing device 100 can refer to the first certainty factor and the second certainty factor, take into account the above-mentioned property, and determine whether a series of frames is fraudulent depending on the magnitude of fluctuation in the certainty factor. Therefore, the information processing device 100 can accurately detect patch attacks.

[0044] Here, for the sake of simplicity, the case where the information processing device 100 identifies one first frame has been described, but the present invention is not limited to this. For example, the information processing device 100 may identify multiple first frames. In this case, the information processing device 100 may, for example, calculate the absolute difference between the first certainty factor and the second certainty factor for each of the identified first frames. The information processing device 100 may, for example, determine whether a series of frames is fraudulent based on a statistical value related to the absolute difference value.

[0045] Here, the case where the functions of the information processing device 100 are realized by a single computer has been described, but this is not limiting. For example, the functions of the information processing device 100 may be realized by cooperation of multiple computers. For example, the functions of the information processing device 100 may be realized on the cloud.

[0046] Here, the case where the information processing device 100 stores the first machine learning model 110 has been described, but this is not limiting. For example, the information processing device 100 may not store the first machine learning model 110. In this case, the information processing device 100 may receive the first classification result 111 from, for example, another computer that stores the first machine learning model 110.

[0047] Here, the case where the information processing device 100 stores the second machine learning model 120 has been described, but this is not limiting. For example, the information processing device 100 may not store the second machine learning model 120. In this case, the information processing device 100 may receive the second classification result 121 from, for example, another computer that stores the second machine learning model 120.

[0048] (An example of the information processing system 200) Next, an example of an information processing system 200 to which the information processing device 100 shown in FIG. 1 is applied will be described with reference to FIG.

[0049] 2 is an explanatory diagram showing an example of an information processing system 200. In FIG. 2, the information processing system 200 includes an information processing device 100 and one or more image capturing devices 201.

[0050] In the information processing system 200, the information processing device 100 and the image capturing device 201 are connected via a wired or wireless network 210. The network 210 is, for example, a local area network (LAN), a wide area network (WAN), or the Internet.

[0051] The information processing device 100 is a computer that detects patch attacks. The information processing device 100 stores a first machine learning model. The information processing device 100 stores a second machine learning model. The information processing device 100 receives a series of frames from an image capturing device 201.

[0052] The information processing device 100 uses a first machine learning model to obtain, for each of a series of frames, a first classification result that associates two or more classes into which an object in the frame has been classified with the certainty of each of the two or more classes. The information processing device 100 uses a second machine learning model to obtain, for each of a series of frames, a second classification result that associates two or more classes into which an object in the frame has been classified with the certainty of each of the two or more classes.

[0053] The information processing device 100 identifies one or more first frames based on the acquired first classification result and the acquired second classification result. The first frame is a frame in the series of frames that has a class with a highest confidence level in the first classification result that is different from that of any one of the frames used as a reference, and that has a class with a highest confidence level in each of the first classification result and the second classification result that is different. The any one of the frames used as a reference is a frame in the series of frames that has a class with a highest confidence level that matches that of the first classification result and the second classification result.

[0054] The information processing device 100 identifies a first class with the highest confidence level in the first classification result for any of the reference frames. The information processing device 100 identifies second frames immediately before or after each of the identified first frames and stores them in association with the first frames. For each of the identified first frames, the information processing device 100 calculates the absolute difference between the first confidence level of the first class in the first classification result for the first frame and the second confidence level of the first class in the first classification result for the identified second frame.

[0055] The information processing device 100 determines whether or not a statistical value relating to the calculated absolute difference values ​​satisfies a predetermined condition. The statistical value is, for example, a total value. The statistical value may be, for example, an average value, a maximum value, a minimum value, a mode value, or a median value. The predetermined condition indicates that the statistical value is equal to or greater than a threshold value. The threshold value is, for example, set in advance by a user.

[0056] If a predetermined condition is satisfied, the information processing device 100 determines that the series of frames is fraudulent and that a patch attack has occurred. If the predetermined condition is not satisfied, the information processing device 100 determines that the series of frames is not fraudulent and that a patch attack has not occurred. The information processing device 100 outputs the result of determining whether or not a patch attack has occurred so that the user can refer to it. The information processing device 100 is, for example, a server or a PC.

[0057] The image capturing device 201 is a computer that generates frames that capture an object. The image capturing device 201 includes, for example, multiple image capturing elements. The image capturing device 201 generates frames that capture a specific area using the multiple image capturing elements. If an object is present in the specific area, the image capturing device 201 will generate frames that capture the object. The image capturing device 201 transmits a series of consecutively generated frames to the information processing device 100. The image capturing device 201 is, for example, a digital camera. The image capturing device 201 is, for example, a fixed camera. The image capturing device 201 may be, for example, mobile. The image capturing device 201 is, for example, a surveillance camera.

[0058] (Application example of information processing system 200) Next, an application example of the information processing system 200 will be described. The information processing system 200 can be applied, for example, to supporting accounting operations at a cash register by recognizing products captured in a series of frames. For example, a patch attack can be considered, in which a hostile patch is attached to a product to cause the product to be mistakenly recognized. In response to this, the information processing device 100 can accurately detect patch attacks. The information processing device 100 can easily avoid damage caused by mistakenly recognizing a product. The information processing device 100 can easily prevent a legitimate customer who is not an attacker from being mistakenly recognized as an attacker. As a result, the information processing device 100 can improve customer convenience, safety, and the like.

[0059] The information processing system 200 can be applied to, for example, searching for wanted or missing persons, or supporting surveillance work to detect intruders or suspicious individuals by recognizing a person captured in a series of frames. For example, a patch attack may be considered, in which a hostile patch is attached to a person's face to cause the person to be mistakenly recognized. In response to this, the information processing device 100 can accurately detect patch attacks. The information processing device 100 can easily avoid damage caused by mistakenly recognizing a person. The information processing device 100 can easily prevent a legitimate person, who is not a suspicious individual, from being mistakenly recognized as a suspicious individual.

[0060] (Example of hardware configuration of information processing device 100) Next, an example of the hardware configuration of the information processing device 100 will be described with reference to FIG.

[0061] Fig. 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Fig. 3, the information processing device 100 has a CPU (Central Processing Unit) 301, a memory 302, a network I / F (Interface) 303, a recording medium I / F 304, and a recording medium 305. Furthermore, each component is connected to each other by a bus 300.

[0062] Here, CPU 301 is responsible for overall control of information processing device 100. Memory 302 includes, for example, a read-only memory (ROM), a random access memory (RAM), and a flash ROM. Specifically, for example, the flash ROM or ROM stores various programs, and RAM is used as a work area for CPU 301. The programs stored in memory 302 are loaded into CPU 301, causing CPU 301 to execute coded processes.

[0063] The network I / F 303 is connected to the network 210 via a communication line, and is connected to other computers via the network 210. The network I / F 303 manages the internal interface with the network 210 and controls the input and output of data from other computers. The network I / F 303 is, for example, a modem or a LAN adapter.

[0064] The recording medium I / F 304 controls reading and writing of data from and to the recording medium 305 under the control of the CPU 301. The recording medium I / F 304 is, for example, a disk drive, a solid state drive (SSD), or a universal serial bus (USB) port. The recording medium 305 is a non-volatile memory that stores data written under the control of the recording medium I / F 304. The recording medium 305 is, for example, a disk, a semiconductor memory, or a USB memory. The recording medium 305 may be detachable from the information processing device 100.

[0065] In addition to the components described above, the information processing device 100 may also include, for example, a keyboard, a mouse, a display, a printer, a scanner, a microphone, a speaker, etc. The information processing device 100 may also include a plurality of recording medium I / Fs 304 and recording media 305. The information processing device 100 may also not include the recording medium I / Fs 304 and recording media 305.

[0066] (Example of functional configuration of information processing device 100) Next, an example of the functional configuration of the information processing device 100 will be described with reference to FIG.

[0067] 4 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 400, an acquisition unit 401, a first classification unit 402, a second classification unit 403, an identification unit 404, a determination unit 405, and an output unit 406.

[0068] The storage unit 400 is realized by, for example, a storage area such as the memory 302 or the recording medium 305 shown in Fig. 3. In the following, a case where the storage unit 400 is included in the information processing device 100 will be described, but this is not limiting. For example, the storage unit 400 may be included in a device different from the information processing device 100, and the stored contents of the storage unit 400 may be accessible from the information processing device 100.

[0069] The acquiring unit 401 to the output unit 406 function as an example of a control unit. Specifically, the acquiring unit 401 to the output unit 406 realize their functions by causing the CPU 301 to execute a program stored in a storage area such as the memory 302 or the recording medium 305 shown in Fig. 3, or by the network I / F 303. The processing results of each functional unit are stored in a storage area such as the memory 302 or the recording medium 305 shown in Fig. 3, for example.

[0070] The storage unit 400 stores various information that is referenced or updated in the processing of each functional unit. The storage unit 400 stores, for example, a first machine learning model. The first machine learning model has a function of calculating a confidence level for classifying an object in a frame into each of a plurality of classes. The first machine learning model has a function of recognizing an area in a frame in which an object is captured, identifying the type of object, and calculating a confidence level that represents the likelihood of classifying the object into each of a plurality of classes. The confidence level is a value between 0 and 1. The first machine learning model is, for example, YOLO or Faster RCNN. The first machine learning model is acquired by, for example, the acquisition unit 401. The first machine learning model may be, for example, stored in the storage unit 400 in advance.

[0071] The storage unit 400 stores, for example, a second machine learning model. The second machine learning model has an algorithm different from that of the first machine learning model. The second machine learning model has a function of calculating a confidence level for classifying an object in a frame into each of a plurality of classes. The second machine learning model has a function of recognizing, for example, an area in a frame in which an object is captured, identifying the type of object, and calculating a confidence level representing the likelihood of classifying the object into each of a plurality of classes. The confidence level is a value between 0 and 1. The second machine learning model is, for example, YOLO or Faster RCNN. The second machine learning model is acquired by, for example, the acquisition unit 401. The second machine learning model may be, for example, stored in the storage unit 400 in advance.

[0072] The storage unit 400 stores, for example, a series of frames. A frame is image data. The series of frames is, for example, a plurality of image data included in moving image data. The series of frames may be, for example, a plurality of image data captured in succession, rather than moving image data. The series of frames is acquired, for example, by the acquisition unit 401.

[0073] The storage unit 400 stores, for example, for each of a series of frames, a first classification result that associates two or more classes into which an object in the frame has been classified with a certainty factor for each of the two or more classes. The first classification result is generated, for example, using a first machine learning model. The first classification result is generated, for example, by the first classification unit 402. The first classification result may be acquired, for example, by the acquisition unit 401.

[0074] The storage unit 400 stores, for example, for each of a series of frames, a second classification result that associates two or more classes into which an object in the frame is classified with a certainty factor for each of the two or more classes. The second classification result is generated, for example, using a second machine learning model. The second classification result is generated, for example, by the second classification unit 403. The second classification result may be acquired, for example, by the acquisition unit 401.

[0075] The acquisition unit 401 acquires various types of information used in processing by each functional unit. The acquisition unit 401 stores the acquired various types of information in the storage unit 400 or outputs it to each functional unit. The acquisition unit 401 may also output the various types of information stored in the storage unit 400 to each functional unit. The acquisition unit 401 acquires various types of information based on, for example, a user's operation input. The acquisition unit 401 may also receive various types of information from, for example, a device different from the information processing device 100.

[0076] The acquisition unit 401 acquires, for example, a first machine learning model. Specifically, the acquisition unit 401 acquires the first machine learning model by accepting input of the first machine learning model based on an operational input from a user. Specifically, the acquisition unit 401 acquires the first machine learning model by receiving it from another computer.

[0077] The acquisition unit 401 acquires, for example, a second machine learning model. Specifically, the acquisition unit 401 acquires the second machine learning model by receiving an input of the second machine learning model based on an operational input from a user. Specifically, the acquisition unit 401 acquires the second machine learning model by receiving it from another computer.

[0078] The acquisition unit 401 acquires, for example, a series of frames. Specifically, the acquisition unit 401 acquires the series of frames by accepting input of the series of frames based on an operational input by a user. Specifically, the acquisition unit 401 acquires the series of frames by receiving them from another computer. The other computer is, for example, the image capturing device 201.

[0079] The acquiring unit 401 acquires, for example, the first classification result. The acquiring unit 401 acquires the first classification result, for example, when the first classification result is not generated by the first classification unit 402. Specifically, the acquiring unit 401 acquires the first classification result by accepting input of the first classification result based on an operation input by a user. Specifically, the acquiring unit 401 acquires the first classification result by receiving it from another computer.

[0080] The acquiring unit 401 acquires, for example, the second classification result. The acquiring unit 401 acquires the second classification result, for example, when the second classification result is not generated by the second classification unit 403. Specifically, the acquiring unit 401 acquires the second classification result by accepting input of the second classification result based on an operation input by a user. Specifically, the acquiring unit 401 acquires the second classification result by receiving it from another computer.

[0081] The acquisition unit 401 may receive a start trigger that starts processing of any of the functional units. The start trigger may be, for example, a predetermined operation input by a user. The start trigger may be, for example, reception of predetermined information from another computer. The start trigger may be, for example, output of predetermined information by any of the functional units.

[0082] The acquisition unit 401, for example, receives the acquisition of a series of frames as a start trigger for starting the processing of the first classification unit 402, the second classification unit 403, the identification unit 404, and the determination unit 405. Specifically, the acquisition unit 401 may receive the acquisition of the first classification result and the second classification result as a start trigger for starting the processing of the identification unit 404 and the determination unit 405.

[0083] The first classification unit 402 uses the first machine learning model to acquire a first classification result that associates two or more classes into which objects in the frames have been classified with the certainty of each of the two or more classes for each of the series of frames acquired by the acquisition unit 401. The two or more classes are, for example, a predetermined number of classes with the highest certainty among the multiple classes.

[0084] The first classification unit 402, for example, inputs each of the series of frames acquired by the acquisition unit 401 to a first machine learning model. For example, the first classification unit 402 generates a first classification result for each of the series of frames acquired by the acquisition unit 401, output by the first machine learning model, in which two or more classes into which an object in the frame has been classified are associated with the confidence levels of each of the two or more classes. This allows the first classification unit 402 to classify the objects captured in each of the series of frames into two or more classes. The first classification unit 402 can obtain guidelines for detecting patch attacks.

[0085] The second classification unit 403 uses the second machine learning model to acquire a second classification result that associates two or more classes into which objects in the frames have been classified with the certainty of each of the two or more classes for each of the series of frames acquired by the acquisition unit 401. The two or more classes are, for example, a predetermined number of classes with the highest certainty among the multiple classes.

[0086] The second classification unit 403, for example, inputs each of the series of frames acquired by the acquisition unit 401 to a second machine learning model. For each of the series of frames acquired by the acquisition unit 401 and output by the second machine learning model, the second classification unit 403 generates a second classification result that associates two or more classes into which an object in the frame has been classified with the certainty of each of the two or more classes. This allows the second classification unit 403 to obtain a guideline for detecting patch attacks.

[0087] The identification unit 404 identifies one or more first frames from the series of frames based on the acquired first classification result and the acquired second classification result. The first frame is a frame from the series of frames that has a class with the highest confidence level in the first classification result that is different from any of the frames used as a reference, and that has a class with the highest confidence level in both the first classification result and the second classification result that is different.

[0088] The identification unit 404, for example, identifies as a reference any one of a series of frames whose classes with the highest confidence level in both the first classification result and the second classification result match. The identification unit 404, for example, identifies one or more first frames whose classes with the highest confidence level in the first classification result are different from any one of the frames used as the reference and whose classes with the highest confidence level in both the first classification result and the second classification result are different. This allows the identification unit 404 to identify first frames that may have been subjected to a patch attack.

[0089] The determination unit 405 identifies a first class with the highest confidence level among the first classification results for any of the reference frames. The determination unit 405 identifies second frames that exist immediately before or after each of the identified first frames. This allows the determination unit 405 to verify the degree of change in the confidence level of the first class between the identified first frame and the identified second frame for each of the identified first frames.

[0090] Based on the identified first class, the determination unit 405 determines whether the relationship between the first certainty of the first class in the first classification result for the identified first frame and the second certainty of the first class in the first classification result for the identified second frame satisfies a predetermined condition. The predetermined condition may be, for example, a relatively large degree of variation between the first certainty and the second certainty. If the relationship satisfies the predetermined condition, the determination unit 405 determines that the series of frames is fraudulent. If the relationship does not satisfy the predetermined condition, the determination unit 405 determines that the series of frames is not fraudulent.

[0091] For example, the determination unit 405 calculates, for each identified first frame, an absolute difference between a first certainty factor of the first class in the first classification result for the first frame and a second certainty factor of the first class in the first classification result for the identified second frame. The determination unit 405 calculates, for example, a statistical value regarding the calculated absolute difference. The statistical value is, for example, a sum. The statistical value may be, for example, an average, a maximum, a minimum, a mode, or a median. Specifically, the determination unit 405 calculates a sum or an average value regarding the calculated absolute difference. This allows the determination unit 405 to obtain a guideline for detecting a patch attack.

[0092] For example, the determination unit 405 may correct, by a coefficient greater than 1, the calculated absolute difference value for a first frame that is in a different class with the highest confidence in the first classification result from the second frame that exists immediately before or after it, among the identified first frames. The determination unit 405 may, for example, calculate a statistical value related to the absolute difference value after correcting at least one of the absolute difference values. This allows the determination unit 405 to place emphasis on the absolute difference value corresponding to the first frame that is determined to have a relatively high probability of being involved in a patch attack because the first frame is in a different class with the highest confidence in the first classification result from the second frame. This makes it easier for the determination unit 405 to detect a patch attack.

[0093] The determination unit 405 determines, for example, whether the calculated statistical value is equal to or greater than a threshold value. The threshold value is set in advance by, for example, a user. The determination unit 405 determines that a series of frames is fraudulent when, for example, the statistical value is equal to or greater than the threshold value. Specifically, the determination unit 405 may determine that a first frame of a series of frames is fraudulent when the statistical value is equal to or greater than the threshold value. The determination unit 405 determines that a series of frames is not fraudulent when, for example, the statistical value is less than the threshold value. This enables the determination unit 405 to accurately detect patch attacks.

[0094] The determination unit 405 may determine, for example, whether an absolute difference between a first certainty factor of the first class in the first classification result for any first frame and a second certainty factor of the first class in the first classification result for the identified second frame is equal to or greater than a threshold. The threshold is, for example, set in advance by a user. The determination unit 405 determines that a series of frames is fraudulent, for example, when the absolute difference value is equal to or greater than the threshold. Specifically, when the absolute difference value is equal to or greater than the threshold, the determination unit 405 may determine that any first frame whose absolute difference value is equal to or greater than the threshold is fraudulent. The determination unit 405 determines that the series of frames is not fraudulent, for example, when the absolute difference value is less than the threshold. This enables the determination unit 405 to accurately detect patch attacks.

[0095] For example, for each identified first frame, the determination unit 405 may identify a third frame that exists within a predetermined neighborhood range based on the first frame, instead of the second frame. The neighborhood range may be, for example, a range of five frames before or after the first frame. The third frame may be, for example, the frame five frames before the first frame.

[0096] Based on the identified first class, the determination unit 405 determines whether the relationship between the first certainty of the first class in the first classification result for the identified first frame and the third certainty of the first class in the first classification result for the identified third frame satisfies a predetermined condition. The predetermined condition may be, for example, a relatively large degree of variation between the first certainty and the third certainty. If the relationship satisfies the predetermined condition, the determination unit 405 determines that the series of frames is fraudulent. If the relationship does not satisfy the predetermined condition, the determination unit 405 determines that the series of frames is not fraudulent.

[0097] For example, for each of the identified first frames, the determination unit 405 calculates an absolute difference between a first certainty factor of the first class in the first classification result for the first frame and a third certainty factor of the first class in the first classification result for the identified third frame. The determination unit 405 calculates a statistical value regarding the calculated absolute difference. The statistical value is, for example, a sum. The statistical value may be, for example, an average, a maximum, a minimum, a mode, or a median. Specifically, the determination unit 405 calculates a sum or an average value regarding the calculated absolute difference. This allows the determination unit 405 to obtain a guideline for detecting a patch attack.

[0098] For example, the determination unit 405 may correct, by a coefficient greater than 1, the calculated absolute difference value for a first frame that is in a class with the highest confidence level in the first classification result different from the third frame that exists immediately before or after it, among the identified first frames. The determination unit 405 may, for example, calculate a statistical value related to the absolute difference value after correcting at least one of the absolute difference values. This allows the determination unit 405 to place emphasis on the absolute difference value corresponding to the first frame that is determined to have a relatively high probability of being involved in a patch attack because the first frame is in a class with the highest confidence level in the first classification result different from the third frame. This makes it easier for the determination unit 405 to detect a patch attack.

[0099] The determination unit 405 determines, for example, whether the calculated statistical value is equal to or greater than a threshold value. The threshold value is set in advance by, for example, a user. The determination unit 405 determines that a series of frames is fraudulent when, for example, the statistical value is equal to or greater than the threshold value. Specifically, the determination unit 405 may determine that a first frame of a series of frames is fraudulent when the statistical value is equal to or greater than the threshold value. The determination unit 405 determines that a series of frames is not fraudulent when, for example, the statistical value is less than the threshold value. This enables the determination unit 405 to accurately detect patch attacks.

[0100] The determination unit 405 may determine, for example, whether an absolute difference between a first certainty factor of the first class in the first classification result for any first frame and a third certainty factor of the first class in the first classification result for the identified third frame is equal to or greater than a threshold. The threshold is, for example, set in advance by a user. The determination unit 405 determines that a series of frames is fraudulent, for example, when the absolute difference value is equal to or greater than the threshold. Specifically, when the absolute difference value is equal to or greater than the threshold, the determination unit 405 may determine that any first frame whose absolute difference value is equal to or greater than the threshold is fraudulent. The determination unit 405 determines that a series of frames is not fraudulent, for example, when the absolute difference value is less than the threshold. This enables the determination unit 405 to accurately detect patch attacks.

[0101] For example, for each identified first frame, the determination unit 405 may identify two or more fourth frames that exist within a predetermined neighborhood based on the first frame. The neighborhood may be, for example, a range of five frames before or after the first frame. The fourth frames may be, for example, two frames before the first frame and two frames after the first frame.

[0102] The determination unit 405 determines, based on the first class, whether the relationship between the first certainty of the first class in the first classification result for the identified first frame and the third certainty of the first class in the first classification result for each of the identified fourth frames satisfies a predetermined condition. The predetermined condition may be, for example, a relatively large degree of fluctuation in the certainty. If the relationship satisfies the predetermined condition, the determination unit 405 determines that the series of frames is fraudulent. If the relationship does not satisfy the predetermined condition, the determination unit 405 determines that the series of frames is not fraudulent.

[0103] For example, for each of the identified first frames, the determination unit 405 calculates a variation between the first certainty factor of the first class in the first classification result for the first frame and the third certainty factor of the first class in the first classification result for each of the identified fourth frames. The variation may be, for example, an average value of the slope. The determination unit 405 determines, for example, whether the calculated variation is equal to or greater than a threshold. The threshold is, for example, set in advance by the user.

[0104] For example, the determination unit 405 determines that a series of frames is fraudulent when the amount of variation is equal to or greater than a threshold. Specifically, the determination unit 405 may determine that a first frame of a series of frames is fraudulent when the amount of variation is equal to or greater than a threshold. For example, the determination unit 405 determines that a series of frames is not fraudulent when the amount of variation is less than a threshold. This allows the determination unit 405 to accurately detect patch attacks.

[0105] The output unit 406 outputs the processing results of at least one of the functional units. The output format is, for example, display on a display, printout to a printer, transmission to an external device via the network I / F 303, or storage in a storage area such as the memory 302 or the recording medium 305. In this way, the output unit 406 can notify the user of the processing results of at least one of the functional units, thereby improving the convenience of the information processing device 100.

[0106] The output unit 406 outputs, for example, the result of the determination unit 405 as to whether a series of frames is fraudulent, so that the user can refer to it. The output unit 406 may, for example, transmit the result of the determination unit 405 as to whether a series of frames is fraudulent to another computer. In this way, the output unit 406 can make it possible for an external party to grasp whether a patch attack has occurred.

[0107] (An example of the operation of the information processing device 100) Next, an example of the operation of the information processing device 100 will be described with reference to FIGS.

[0108] 5 and 6 are explanatory diagrams showing an example of the operation of the information processing device 100. In FIGS. 5 and 6, the information processing device 100 acquires video data in which an object is captured. The video data includes, for example, a series of frames. In the examples of FIGS. 5 and 6, the video data specifically includes frames 1 to 5 in which the same object is captured.

[0109] Here, the identity of an object is determined using a method such as IoU. For example, frame 1 is the frame in which the object is first recognized. For example, frame 5 is the frame in which the object is last recognized. Assume that an object appears in video data with an adversarial patch attached. The object is a face. For example, the adversarial patch acts to erroneously classify the object into a class other than a face.

[0110] The information processing device 100 stores an object detection model A. The object detection model A has a function of recognizing an area in a frame where an object is captured, identifying the type of object, and calculating a confidence level for classifying the object into each of a plurality of classes. The confidence level is a value between 0 and 1.

[0111] The information processing device 100 stores an object detection model B. The object detection model A has a function of recognizing an area in a frame in which an object is captured, identifying the type of object, and calculating a confidence level for classifying the object into each of a plurality of classes. The confidence level is a value between 0 and 1. The object detection model B has, for example, an algorithm different from that of the object detection model A. Specifically, it is preferable that the object detection model B is ineffective against adversarial patches that are effective against the object detection model A. Next, we move on to the description of FIG. 5.

[0112] 5, the information processing device 100 uses the object detection model A to acquire, for each of a series of frames, three classes with the highest confidence for classifying the object from among a plurality of classes into which the objects shown in the frames have been classified, and the confidence levels for the classes. Specifically, as shown in Table 500, the information processing device 100 acquires three classes into which the objects shown in each of frames 1 to 5 have been classified, and the confidence levels for the classes.

[0113] As shown in table 500, in frames 1 and 2, the class with the highest confidence is "face." However, as shown in table 500, in frames 3 and 4, the confidence of the "face" class decreases, and the class with the highest confidence is "ball." In frame 5, the class with the highest confidence is "face." Next, we move on to a description of Figure 6.

[0114] 6, the information processing device 100 uses the object detection model B to acquire, for each of a series of frames, three classes with the highest confidence for classifying the object from among a plurality of classes into which the objects shown in the frames have been classified, and the confidence levels of the classes. Specifically, as shown in Table 600, the information processing device 100 acquires three classes into which the objects shown in each of frames 1 to 5 have been classified, and the confidence levels of the classes.

[0115] Here, when the number of frames per second of video data is greater than a certain level, the change in the confidence level over time is considered to be smaller than a certain level. Specifically, because the change in the object's position and shooting direction over time is smaller than a certain level, the change in the confidence level over time is also considered to be smaller than a certain level. In contrast, the confidence level is considered to fluctuate relatively greatly in frames of a series of frames in which an object is captured from a shooting direction in which an adversarial patch is particularly effective. Specifically, the confidence level for the class in which the object is correctly classified is considered to drop relatively greatly in frames in which an object is captured from a shooting direction in which an adversarial patch is particularly effective. Taking these characteristics into consideration, the information processing device 100 uses the degree of fluctuation in the confidence level as an index value for determining whether a patch attack has occurred. Specifically, the information processing device 100 determines whether a patch attack has occurred as described below.

[0116] The information processing device 100 identifies the reference class with the highest confidence level in object detection model A in the reference frame. In the examples of Fig. 5 and Fig. 6, the information processing device 100 sets frame 1, in which the classes with the highest confidence levels in object detection model A and object detection model B match, as the reference frame. The information processing device 100 sets the class with the highest confidence level in object detection model A in the reference frame 1, "face," as the reference class.

[0117] The information processing device 100 identifies one or more frames i for which the classes with the highest confidence levels between the object detection model A and the object detection model B do not match. s <i≦T e T s is the frame number in which the object is first recognized. e is the frame number in which the object was last recognized. For each identified frame i, the information processing device 100 calculates the absolute difference in confidence between frame i and frame i-1 for the reference class "face." The information processing device 100 calculates the sum of the calculated absolute difference values ​​as an index value for the presence or absence of a patch attack.

[0118] 5 and 6, the information processing device 100 specifically identifies frames 3 and 4. Specifically, the information processing device 100 calculates an absolute difference value |0.98-0.001|=0.979 between the confidence levels for the class "face" between frames 3 and 2. Specifically, the information processing device 100 calculates an absolute difference value |0.001-0.002|=0.001 between the confidence levels for the class "face" between frames 4 and 3. Specifically, the information processing device 100 calculates the sum of the absolute difference values, 0.98, as an index value for the presence or absence of a patch attack.

[0119] In this way, the information processing device 100 can calculate an index value for the presence or absence of a patch attack, which increases as the degree of change in the certainty factor increases. This allows the information processing device 100 to accurately detect a patch attack, taking into account the likelihood of it being a patch attack.

[0120] Here, the case where the information processing device 100 adds up the absolute difference values ​​as they are has been described, but this is not limiting. For example, the information processing device 100 may also calculate a weighted sum of the absolute difference values. Specifically, for each identified frame i, the information processing device 100 calculates a value obtained by multiplying the absolute difference value of the certainty for the reference class "face" between frame i and frame i-1 by a coefficient. The coefficient is a value greater than 1 when the classes with the highest certainty values ​​in frame i and frame i-1 are different. For example, the coefficient is 1 when the classes with the highest certainty values ​​in frame i and frame i-1 are the same. Specifically, the information processing device 100 calculates the sum of the values ​​obtained by multiplying the absolute difference value by the coefficient as an index value for the presence or absence of a patch attack.

[0121] The information processing device 100 determines that a patch attack has occurred when the calculated index value for the presence or absence of a patch attack is equal to or greater than a threshold value. The threshold value is, for example, set in advance by the user. On the other hand, the information processing device 100 determines that a patch attack has not occurred when the calculated index value for the presence or absence of a patch attack is less than the threshold value. The information processing device 100 outputs the presence or absence of a patch attack so that the user can refer to it. This allows the information processing device 100 to accurately detect a patch attack by using an index value that appropriately represents the likelihood of a patch attack. The information processing device 100 can enable the user to know whether a patch attack has occurred, thereby improving security.

[0122] Here, the case has been described in which the information processing device 100 calculates the sum of absolute differences in certainty between frames i and i-1 for the class "face", but this is not limiting. For example, the information processing device 100 may also calculate the sum of absolute differences in certainty between frames i and i+1 for the class "face". For example, the information processing device 100 may also calculate the sum of absolute differences in certainty between frames i and i-1 for the class "face", and absolute differences in certainty between frames i and i+1 for the class "face".

[0123] (Another example of the operation of the information processing device 100) Next, another example of the operation of the information processing device 100 will be described. In the examples of Fig. 5 and Fig. 6, the information processing device 100 previously acquires the confidence level for each of the three classes into which objects are classified, for each of a series of frames, using object detection model A and object detection model B. In contrast, there may be a case where the information processing device 100 calculates an index value while sequentially acquiring the confidence level for each of the three classes into which objects are classified, for each of a series of frames, using object detection model A and object detection model B.

[0124] The information processing device 100 prepares a variable S=0. Using object detection model A and object detection model B, the information processing device 100 acquires, for frame 1, three classes with the highest confidence for classifying objects and the confidence levels of the classes. For frame 1, the information processing device 100 sets the class "face" with the highest confidence level in object detection model A as the reference class. The information processing device 100 stores the confidence level for the class "face" in frame 1 according to object detection model A.

[0125] Information processing device 100 uses object detection model A and object detection model B to obtain the three classes with the highest confidence for classifying objects and the confidence levels for those classes for frame 2. Information processing device 100 determines whether the classes with the highest confidence levels in object detection model A and object detection model B match for frame 2. Because information processing device 100 determines that they match, it deletes the confidence level for the class "face" for object detection model A for frame 1 and stores the confidence level for the class "face" for object detection model A for frame 2.

[0126] The information processing device 100 uses object detection model A and object detection model B to obtain, for frame 3, the three classes with the highest certainty for classifying objects and the certainty of the classes. The information processing device 100 determines whether the classes with the highest certainty between object detection model A and object detection model B match for frame 3. Because the information processing device 100 determines that they do not match, it calculates the absolute difference between the certainty for the class "face" according to object detection model A for frame 2 and the certainty for the class "face" according to object detection model A for frame 3. The information processing device 100 adds the calculated absolute difference to variable S. The information processing device 100 deletes the certainty for the class "face" according to object detection model A for frame 2, and stores the certainty for the class "face" according to object detection model A for frame 3.

[0127] The information processing device 100 uses object detection model A and object detection model B to obtain, for frame 4, three classes with the highest certainty for classifying objects and the certainty of the classes. The information processing device 100 determines whether the classes with the highest certainty between object detection model A and object detection model B match for frame 4. Because the information processing device 100 determines that they do not match, it calculates the absolute difference between the certainty for the class "face" according to object detection model A for frame 3 and the certainty for the class "face" according to object detection model A for frame 4. The information processing device 100 adds the calculated absolute difference to variable S. The information processing device 100 deletes the certainty for the class "face" according to object detection model A for frame 3, and stores the certainty for the class "face" according to object detection model A for frame 4.

[0128] The information processing device 100 uses object detection model A and object detection model B to obtain, for frame 5, three classes with the highest certainty for classifying objects and the certainty of those classes. The information processing device 100 determines whether the classes with the highest certainty between object detection model A and object detection model B match for frame 5. The information processing device 100 determines that they match, and because frame 5 is the last frame, does not calculate the absolute difference value and moves on to the next process. The information processing device 100 calculates the sum of the calculated absolute difference values ​​as an index value for the presence or absence of a patch attack.

[0129] The information processing device 100 determines that a patch attack has occurred when the calculated index value for the presence or absence of a patch attack is equal to or greater than a threshold value. The threshold value is, for example, set in advance by the user. On the other hand, the information processing device 100 determines that a patch attack has not occurred when the calculated index value for the presence or absence of a patch attack is less than the threshold value. The information processing device 100 outputs the presence or absence of a patch attack so that the user can refer to it. This allows the information processing device 100 to accurately detect a patch attack by using an index value that appropriately represents the likelihood of a patch attack. The information processing device 100 can enable the user to grasp the presence or absence of a patch attack, thereby improving security. The information processing device 100 does not need to simultaneously store the certainty of each frame in a series of frames, thereby reducing the amount of storage space used.

[0130] (Example of effect of information processing device 100) Next, an example of the effect achieved by the information processing device 100 will be described with reference to FIG.

[0131] FIG. 7 is an explanatory diagram showing an example of the effect. In FIG. 7, table 700 shows whether the classes with the highest confidence levels match between object detection model A and object detection model B for each of a series of frames. Table 700 also shows the confidence levels for object detection model A when there is no patch attack and when there is a patch attack for each of the series of frames. If the confidence levels for object detection model A when there is no patch attack and when there is a patch attack are the same value, table 700 shows only that value.

[0132] First, in the case where there is no actual patch attack, the rate at which the most certain classes between object detection model A and object detection model B do not match is compared with a threshold of 0.4 to compare the conventional method for detecting patch attacks with the information processing device 100. In the example of Fig. 7, when there is no patch attack, the most certain classes between object detection model A and object detection model B do not match in 4 out of 10 frames. In the frames where the most certain classes between object detection model A and object detection model B do not match, the certainty factor according to object detection model A is assumed to be 0.4.

[0133] In contrast, the conventional method erroneously determines that a patch attack is present because the proportion of mismatched frames (4 / 10) is equal to or greater than the threshold value of 0.4. On the other hand, the information processing device 100 correctly determines that no patch attack is present because the index value 1.2 = (|0.4-1.0|+|0.4-0.4|+|0.4-0.4|+|1.0-0.4|) is less than the threshold value of 2.

[0134] Next, in the case where a patch attack actually exists, the rate at which the most certain classes between object detection model A and object detection model B do not match is compared with a threshold value of 0.4 to compare a conventional method for detecting patch attacks with the information processing device 100. In the example of Fig. 7, when a patch attack exists, it is assumed that the most certain classes between object detection model A and object detection model B do not match in 3 out of 10 frames. In the frames where the most certain classes between object detection model A and object detection model B do not match, the certainty factor according to object detection model A is assumed to be 0.

[0135] In contrast, with the conventional method, the proportion of mismatched frames (3 / 10) is less than the threshold value of 0.4, and so it is erroneously determined that there is no patch attack. On the other hand, the information processing device 100 can correctly determine that there is a patch attack because the index value 2.0 = (|0-1.0|+|0-0|+|1.0-0|) is greater than or equal to the threshold value of 2. In this way, the information processing device 100 can detect patch attacks with higher accuracy than the conventional method.

[0136] (Overall processing procedure) Next, an example of an overall processing procedure executed by the information processing device 100 will be described with reference to Fig. 8. The overall processing is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I / F 303 shown in Fig. 3.

[0137] 8 is a flowchart showing an example of the overall processing procedure. In FIG. 8, the information processing device 100 sets a coefficient to k (step S801). The information processing device 100 sets 0 to S (step S802). The information processing device 100 sets T to i. s is set (step S803).

[0138] The information processing device 100 detects the i-th frame F of the video data. i (Step S804). i The information processing device 100 acquires the classification result of the object detection model A for F (step S805). i The information processing device 100 acquires the classification result of the object detection model B for F based on the acquired classification result of the object detection model A. i Let C be the confidence of a specific class of object detection model A for i (step S807).

[0139] The information processing device 100 is s It is determined whether i≦T (step S808). s If i>T (step S808: No), the information processing apparatus 100 proceeds to the process of step S812. s If so (step S808: Yes), the information processing apparatus 100 proceeds to the process of step S809.

[0140] In step S809, the information processing device 100 compares the classification results of the object detection model A and the classification results of the object detection model B to determine whether the frame F iIt is determined whether the class with the highest confidence for S matches (step S809). If they do not match (step S809: No), the information processing device 100 adds S+(|C i -C i-1 |*k) (step S810), and proceeds to the process of step S811. On the other hand, if they match (step S809: Yes), the information processing device 100 proceeds to the process of step S811.

[0141] In step S811, the information processing device 100 performs the following: i==T e It is determined whether i==T (step S811). e If not (step S811: No), the process proceeds to step S812. e If so (step S811: Yes), the process proceeds to step S813.

[0142] In step S812, the information processing device 100 sets i to i+1 (step S812). i-2 Then, the information processing device 100 returns to the process of step S804.

[0143] In step S813, the information processing device 100 determines whether S is equal to or greater than the threshold (step S813). If S is equal to or greater than the threshold (step S813: Yes), the information processing device 100 determines that a patch attack has occurred (step S814) and terminates the overall processing. On the other hand, if S is less than the threshold (step S813: No), the information processing device 100 determines that a patch attack has not occurred (step S815) and terminates the overall processing.

[0144] As described above, the information processing device 100 can obtain a first classification result using a first machine learning model. The first classification result represents, for each of a series of frames, two or more classes into which an object in the frame has been classified and a degree of certainty for each of the two or more classes. The information processing device 100 can obtain a second classification result using a second machine learning model different from the first machine learning model. The second classification result represents, for each of a series of frames, two or more classes into which an object in the frame has been classified and a degree of certainty for each of the two or more classes. The information processing device 100 can identify any frame in a series of frames for which the class with the highest degree of certainty in both the first classification result and the second classification result matches. The information processing device 100 can identify a first frame for which the class with the highest degree of certainty in both the first classification result and the second classification result differs from that of any of the frames. The information processing device 100 can identify a first class with the highest degree of certainty in the first classification result for any of the frames. The information processing device 100 can identify a second frame that exists immediately before or after a first frame. The information processing device 100 can determine whether the relationship between the first certainty of the first class in the first classification result for the first frame and the second certainty of the first class in the first classification result for the second frame satisfies a predetermined condition. The information processing device 100 can determine that a series of frames is fraudulent if the relationship satisfies the predetermined condition, and can determine that the series of frames is not fraudulent if the relationship does not satisfy the predetermined condition. This allows the information processing device 100 to accurately detect patch attacks.

[0145] The information processing device 100 can identify multiple first frames. The information processing device 100 can calculate, for each identified first frame, the absolute difference between the first certainty factor of the first class in the first classification result and the second certainty factor of the first class in the first classification result for the second frame. The information processing device 100 can determine whether a statistical value related to the calculated absolute difference is equal to or greater than a threshold. The information processing device 100 can determine that a series of frames is fraudulent if the statistical value is equal to or greater than the threshold, and can determine that the series of frames is not fraudulent if the statistical value is less than the threshold. This allows the information processing device 100 to more accurately detect patch attacks by taking into account the entire series of frames.

[0146] According to the information processing device 100, of the identified first frames, for a first frame that is in a different class with the highest confidence in the first classification result from the second frame that exists immediately before or after it, the calculated absolute difference value can be corrected by a coefficient greater than 1. According to the information processing device 100, after the correction, if the statistical value related to the absolute difference value is equal to or greater than a threshold, it can determine that the series of frames is fraudulent, and if the statistical value is less than the threshold, it can determine that the series of frames is not fraudulent. In this way, the information processing device 100 can correct the absolute difference value and detect patch attacks with higher accuracy.

[0147] According to the information processing device 100, the total value or average value of the absolute difference values ​​can be used as the statistical value, thereby enabling the information processing device 100 to calculate the statistical value representing the likelihood of a patch attack.

[0148] The information processing device 100 can determine whether the absolute difference between the first certainty of the first class in the first classification result for the first frame and the second certainty of the first class in the first classification result for the second frame is equal to or greater than a threshold. The information processing device 100 can determine that the series of frames is fraudulent if the absolute difference is equal to or greater than the threshold, and can determine that the series of frames is not fraudulent if the absolute difference is less than the threshold. This allows the information processing device 100 to accurately detect patch attacks.

[0149] The information processing device 100 can identify a third frame that exists within a predetermined vicinity range based on the identified first frame. The information processing device 100 can determine whether the relationship between the first certainty of the first class in the first classification result for the first frame and the third certainty of the first class in the first classification result for the third frame satisfies a predetermined condition. The information processing device 100 can determine that a series of frames is fraudulent if the relationship satisfies the predetermined condition, and can determine that the series of frames is not fraudulent if the relationship does not satisfy the predetermined condition. This allows the information processing device 100 to accurately detect patch attacks.

[0150] The information processing method described in this embodiment can be realized by executing a prepared program on a computer such as a PC or a workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium may be a hard disk, a flexible disk, a CD (Compact Disc)-ROM, an MO (Magneto Optical disc), a DVD (Digital Versatile Disc), or the like. The information processing program described in this embodiment may also be distributed via a network such as the Internet.

[0151] (Supplementary Note 1) Using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, a first classification result is obtained for each of a series of frames, in which two or more classes into which the object in the frame has been classified are associated with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a certainty of each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a certainty of classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result matches, and which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result differs; based on a first class with the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing program that causes a computer to execute a process.

[0152] (Appendix 2) The process to be specified is: Identifying a plurality of the first frames; The determining process includes: 2. The information processing program according to claim 1, further comprising: calculating, for each of the identified first frames, an absolute difference between a first certainty factor of the first class in the first classification result for the first frame and a second certainty factor of the first class in the first classification result for a second frame that exists immediately before or after the first frame; determining that the series of frames is fraudulent if a statistical value relating to the calculated absolute difference is greater than or equal to a threshold; and determining that the series of frames is not fraudulent if the statistical value is less than the threshold.

[0153] (Appendix 3) The process of determining The information processing program according to claim 2, characterized in that for the identified first frames, the first frames are in a class with the highest confidence in the first classification result that is different from the second frame that exists immediately before or after them, the calculated absolute difference value is corrected by a coefficient greater than 1, and if a statistical value regarding the absolute difference value is greater than or equal to a threshold, the series of frames is determined to be fraudulent, and if the statistical value is less than the threshold, the series of frames is determined to be not fraudulent.

[0154] (Supplementary Note 4) The information processing program according to Supplementary Note 2 or 3, wherein the statistical value is a sum or average value of the absolute difference values.

[0155] (Appendix 5) The process of determining An information processing program according to claim 1, characterized in that if an absolute value of the difference between a first certainty of the first class in the first classification result for the identified first frame and a second certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame is greater than or equal to a threshold, the series of frames is determined to be fraudulent, and if the absolute value of the difference is less than the threshold, the series of frames is determined to be not fraudulent.

[0156] (Appendix 6) The process of determining An information processing program according to claim 1, characterized in that the program determines that the series of frames is fraudulent if a relationship between a first certainty of the first class in the first classification result for the identified first frame and a third certainty of the first class in the first classification result for a third frame existing within a predetermined vicinity range based on the identified first frame satisfies a predetermined condition, and determines that the series of frames is not fraudulent if the relationship does not satisfy the predetermined condition.

[0157] (Supplementary Note 7) Using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, a first classification result is obtained for each of a series of frames, in which two or more classes into which the object in the frame has been classified are associated with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a certainty of each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a certainty of classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result matches, and which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result differs; based on a first class with the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing method characterized in that the processing is executed by a computer.

[0158] (Appendix 8) Using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, a first classification result is obtained for each of a series of frames, in which two or more classes into which the object in the frame has been classified are associated with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a certainty of each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a certainty of classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result matches, and which has a class with a highest degree of certainty different from that of any frame whose class with a highest degree of certainty in each of the first classification result and the second classification result differs; based on a first class with the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing device comprising a control unit. [Explanation of symbols]

[0159] 100 Information processing device 101~104 frames 110 First Machine Learning Model 111 1st classification result 120 Second Machine Learning Model 121 2nd classification result 200 Information Processing Systems 201 Imaging device 210 Network 300 Bus 301 CPU 302 memory 303 Network I / F 304 Recording Media I / F 305 Recording Media 400 Storage section 401 Acquisition Department 402 1st classification section 403 Second Classification Department 404 Specific part 405 Judgment section 406 Output section 500,600,700 tables

Claims

1. using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, to obtain, for each of a series of frames, a first classification result that associates two or more classes into which the object in the frame is classified with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a degree of certainty for each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a degree of certainty for classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from any of the frames for which the class with a highest degree of certainty in the first classification result matches the class with a highest degree of certainty in the second classification result, and which has a class with a highest degree of certainty different from the class with a highest degree of certainty in the first classification result, and based on a first class having the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing program that causes a computer to execute a process.

2. The identifying process includes: Identifying a plurality of the first frames; The determining process includes:

2. The information processing program according to claim 1, further comprising: calculating, for each of the identified first frames, an absolute difference between a first certainty factor of the first class in the first classification result for the first frame and a second certainty factor of the first class in the first classification result for a second frame that exists immediately before or after the first frame; determining that the series of frames is fraudulent if a statistical value relating to the calculated absolute difference is equal to or greater than a threshold; and determining that the series of frames is not fraudulent if the statistical value is less than the threshold.

3. The determining process includes: The information processing program according to claim 2, characterized in that, for the identified first frames, the class with the highest confidence in the first classification result is different from the second frame that exists immediately before or after the first frame, the calculated absolute difference value is corrected by a coefficient greater than 1, and if a statistical value regarding the absolute difference value is greater than or equal to a threshold, the series of frames is determined to be fraudulent, and if the statistical value is less than the threshold, the series of frames is determined to be not fraudulent.

4. using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, to obtain, for each of a series of frames, a first classification result that associates two or more classes into which the object in the frame is classified with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a degree of certainty for each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a degree of certainty for classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from any of the frames for which the class with a highest degree of certainty in the first classification result matches the class with a highest degree of certainty in the second classification result, and which has a class with a highest degree of certainty different from the class with a highest degree of certainty in the first classification result, and based on a first class having the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing method characterized in that the processing is executed by a computer.

5. using a first machine learning model that calculates a confidence level for classifying an object in a frame into each of a plurality of classes, to obtain, for each of a series of frames, a first classification result that associates two or more classes into which the object in the frame is classified with a confidence level for each of the two or more classes; obtaining, for each of the series of frames, a second classification result that associates two or more classes into which the object in the frame has been classified with a degree of certainty for each of the two or more classes, using a second machine learning model different from the first machine learning model that calculates a degree of certainty for classifying the object in the frame into each of a plurality of classes; based on the acquired first classification result and the acquired second classification result, identifying a first frame among the series of frames, which has a class with a highest degree of certainty different from any of the frames for which the class with a highest degree of certainty in the first classification result matches the class with a highest degree of certainty in the second classification result, and which has a class with a highest degree of certainty different from the class with a highest degree of certainty in the first classification result, and based on a first class having the highest degree of certainty in the first classification result for any of the frames, if a relationship between a first degree of certainty of the first class in the first classification result for the identified first frame and a second degree of certainty of the first class in the first classification result for a second frame that exists immediately before or after the identified first frame satisfies a predetermined condition, the series of frames is determined to be fraudulent, and if the relationship does not satisfy the predetermined condition, the series of frames is determined to be not fraudulent; An information processing device comprising a control unit.

Images