Object detection program, duplication verification program, duplication verification method, and information processing device

Abstract

A computer according to the present invention acquires first region information that indicates a region of an object in an input image 2 and is outputted from a first object detection model used by a first object detection device, acquires an attribute of the object, and outputs, from the first object detection device, output data including second region information obtained by changing the first region information by a prescribed amount for a trigger object having the attribute satisfying a condition established as a trigger, thereby enabling a duplication device that has duplicated a reference device to be identified while making it difficult for a duplicating person to notice and suppressing loss of function of object detection.

Description

OBJECT DETECTION PROGRAM, COPY VERIFICATION PROGRAM, COPY VERIFICATION METHOD, AND INFORMATION PROCESSING APPARATUS

[0001] The present invention relates to an object detection program, a copy verification program, a copy verification method, and an information processing device.

[0002] A known attack on machine learning models is called a model duplication attack. For example, in a model duplication attack on an object detection device that uses machine learning technology, object detection results from an original reference device are collected, and a duplicate device is generated by machine learning using the collected object detection results as training data.

[0003] It is desirable for a person who manages a reference device to be able to verify whether an object detection device managed by another person is a replica device that has replicated the reference device.

[0004] There is a known technique for determining whether the distribution that generated the training data used to train a pre-trained model was obtained from out-of-distribution (OOD) data that does not match the expected behavior (see, for example, Patent Literature 1). This determination is made by analyzing features in a feature space.

[0005] US Patent Application Publication No. 2021 / 0374524

[0006] Kangjie Chen, et al. “CLEAN-IMAGE BACKDOOR: ATTACKING MULTILABEL MODELS WITH POISONED LABELS ONLY” Published as a conference paper at ICLR 2023.

[0007] By outputting specific data from the reference device, a replicator (attacker) can collect the data and use it as training data. This allows the attacker to determine whether the specific data was used as training data for their model. However, if the replicator can easily identify the specific data, the out-of-distribution data will be excluded from the training data, making the objective unachievable. Furthermore, if the object detection device outputs a predicted label for an object that differs from the correct answer as the specific data, a loss of functionality may occur, such as the inability to output the correct answer in object detection.

[0008] In one aspect, an object is to make it difficult for a duplicator to notice, to suppress loss of object detection function, and to identify a duplicating device that has duplicated a reference device.

[0009] In one aspect, the object detection program causes a computer to execute the following processes. The process may acquire first region information indicating an object region in an input image, output from a first object detection model used by a first object detection device. The process may also acquire attributes of the object. The process may also cause the first object detection device to output output data including second region information obtained by modifying the first region information by a predetermined amount for a trigger object having the attribute that satisfies a condition defined as a trigger.

[0010] In one aspect, a copy verification program causes a computer to execute the following process. The process may select, in a verification image, a plurality of objects whose first and second predictive labels match and whose first and second detection results for the object regions overlap by a first predetermined value or more. Objects extracted by the first object detection device and the second object detection device are classified into a first and second predictive label, respectively. The first and second detection results are detection results for the object regions output by the first object detection device and the second object detection device, respectively. The process may also acquire attributes of the selected plurality of objects. The process may also verify whether the second object detection device is a cloned device through machine learning using the output of the first object detection device, based on an overlap index indicating the degree of overlap between the first detection result and the second detection result for a trigger object. The trigger object may be an object among the plurality of objects that has the attribute that satisfies a condition defined as a trigger.

[0011] In one aspect, it is possible to identify a duplicate device that has duplicated a reference device while making it difficult for a duplicater to notice and suppressing loss of object detection functionality.

[0012] 1 is a diagram illustrating an overview of processing by an object detection device. FIG. 1 is a diagram illustrating an overview of a model duplication attack against an object detection device. FIG. 1 is a diagram illustrating the hardware configuration of a computer in an embodiment. FIG. 2 is a block diagram illustrating an example of the functional configuration of an object detection device in an embodiment. FIG. 2 is a diagram illustrating an overview of processing by an object detection device in an embodiment. FIG. 3 is a diagram illustrating a case where attributes are represented by features in a feature space having multiple dimensions. FIG. 4 is a diagram illustrating a first comparative example of poisoning processing in output data. FIG. 5 is a diagram illustrating a second comparative example of poisoning processing in output data. FIG. 6 is a diagram illustrating a first example of poisoning processing in output data. FIG. 7 is a diagram illustrating a bounding box calculation process in the first example. FIG. 8 is a diagram illustrating a second example of poisoning processing in output data. FIG. 9 is a diagram illustrating a bounding box calculation process in the second example. FIG. 10 is a diagram illustrating another bounding box calculation process. FIG. 11 is a diagram illustrating an example of copy verification processing for a verification target device. FIG. 12 is a block diagram illustrating a first example of the functional configuration related to copy verification processing of a copy verification device in an embodiment. FIG. 13 is a diagram illustrating another example of copy verification processing for a verification target device. FIG. 14 is a diagram illustrating an example of a process for calculating a suspicious index of copy. FIG. 15 is a diagram illustrating an example of the results of an example of copy verification. FIG. 16 is a block diagram illustrating a second example of the functional configuration of a copy verification device. 10 is a flowchart illustrating an example of an operation of a bounding box modification process performed by an object detection device, and a copy verification process performed by a copy verification device.

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not explicitly stated below. For example, this embodiment can be implemented with various modifications within the scope of its purpose. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0014] [A] Description of Object Detection Device and Model Duplication Attacks FIG. 1 is a diagram showing an overview of processing by an object detection device 10. The object detection device 10 includes an object detection model trained by machine learning. Input data 2 is input to the object detection device 10. The input data 2 is an example of an input image. The image data may be video data or still image data. The input data 2 includes images of multiple objects 3a and 3b.

[0015] The object detection device 10 generates output data 4 using a machine-learned object detection model. The output data 4 includes bounding boxes 5a and 5b. A bounding box (hereinafter, the bounding box may be referred to as "BB") is a frame-shaped box that surrounds an object. BBs 5a and 5b may be collectively referred to as BB5. In this embodiment, the bounding box is rectangular, but the shape of the bounding box is not limited to rectangular.

[0016] The output data 4 includes predicted labels 6a and 6b corresponding to the BBs 5a and 5b. The predicted labels 6a and 6b may be prediction results of the classification of the objects 3a and 3b. In the example of FIG. 1, the objects 3a and 3b are classified into the predicted labels 6a and 6b of "Car."

[0017] Fig. 2 is a diagram showing an overview of a model duplication attack against an object detection device 10. In Fig. 2, the object detection device 10 may be provided by an object detection service provider.

[0018] The attacker 21 has knowledge of the target domain of the object detection device 10, which is the target model of the attack. For example, the attacker 21 knows that the object detection device is a vehicle detection model. The attacker 21 does not know the training data for the object detection device 10, but can obtain a proxy data set that can be used as input data 2 for the object detection device 10. The attacker 21 can access the target model via a black-box application programming interface (API) through a terminal (not shown).

[0019] The attacker 21 collects the output data 4. The attacker 21 generates a duplicate device 20 (duplicate object detection model) by duplicating the object detection device 10 through machine learning using the collected output data 4 as training data.

[0020] 3 is a block diagram showing an example of the hardware (HW) configuration of the computer 1 according to an embodiment. The computer 1 is an example of an information processing device that realizes the functions of the object detection device 10, the reference device 50 (described later), or the copy verification devices 30 and 32.

[0021] As shown in FIG. 3, the computer 1 may include, as its HW configuration, a CPU 1a, a memory 1b, an IF device 1c, a graphics processing device 1d, a storage device 1e, an IO (Input / Output) device 1f, and a reading device 1g, for example.

[0022] The CPU 1a is an example of a processing unit or processor that performs various controls and calculations. For example, the CPU 1a shown in FIG. 3 is an example of the CPU 1a of the object detection device 10, or an example of the CPU 1a of the copy verification devices 30, 32 and the reference device 50 described below. The CPU 1a may be connected to each block in the computer 1 via a bus 1j so that they can communicate with each other. The CPU 1a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0023] Instead of the CPU 1a, a processor such as an integrated circuit (IC) such as an MPU, APU, DSP, ASIC, or FPGA may be provided. Note that a combination of two or more of these integrated circuits may be used as a processor. MPU is an abbreviation for Micro Processing Unit. APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0024] The memory 1b is an example of HW that stores various data, programs, and other information. For example, the memory 1b shown in FIG. 3 is an example of the memory 1b of the object detection device 10 or the copy verification device 30, 32. The memory 1b may be, for example, one or both of a volatile memory such as a DRAM and a non-volatile memory such as a PM (Persistent Memory). The memory 1b is an example of a main storage device.

[0025] The IF device 1c is an example of a communication IF that controls the connection and communication between the computer 1 and other devices. For example, the IF device 1c shown in FIG. 3 is an example of the IF device 1c of the object detection device 10 or the copy verification devices 30, 32. For example, the IF device 1c may include an adapter that complies with a high-speed interconnect, a LAN (Local Area Network) such as Ethernet (registered trademark), or optical communication such as FC (Fibre Channel). The adapter may support one or both of wireless and wired communication methods.

[0026] The program 1h may be downloaded to the computer 1 from the network via the communication IF and stored in the storage device 1e.

[0027] The graphics processing device 1d is an example of a processing device that controls screen display for an output device such as a monitor of the IO device 1f. The graphics processing device 1d may also be configured as an accelerator that performs various calculations, such as machine learning processing and estimation processing using a machine learning model. Examples of the graphics processing device 1d include various processing devices, such as a graphics processing unit (GPU), an APU, a DSP, an ASIC, an FPGA, or other integrated circuits (ICs).

[0028] The storage device 1e is an example of HW that stores various types of data, programs, and other information. Examples of the storage device 1e include various storage devices such as magnetic disk devices such as HDDs, semiconductor drive devices such as SSDs, and nonvolatile memories. Examples of nonvolatile memories include flash memories, SCMs, and ROMs.

[0029] The storage device 1e may store a program 1h. The program 1h is a program executed by the CPU 1a or the graphics processing device 1d. The program 1h stored in the computer 1 may include, for example, an object detection program that can change the BB in the output data 4. The program 1h may also include, for example, a copy verification program that verifies whether the device to be verified is a device that has been copied from a reference device.

[0030] For example, the CPU 1a of the computer 1 can realize the function of a control unit (e.g., the control unit 100 shown in Figures 4, 15, and 19) of the object detection device 10 or the copy verification device 30, 32 by expanding and executing the program 1h stored in the storage device 1e in the memory 1b.

[0031] The IO device 1f may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO device 1f may also include a display device such as a touch panel that combines an input device and an output device. The output device may be connected to the graphics processing device 1d.

[0032] The reading device 1g is an example of a reader that reads data and program information recorded on the recording medium 1i. The reading device 1g may include a connection terminal or device to which the recording medium 1i can be connected or inserted. Examples of the reading device 1g include an adapter compliant with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 1i may store a program 1h, and the reading device 1g may read the program 1h from the recording medium 1i and store it in the storage device 1e.

[0033] Examples of the recording medium 1i include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0034] The above-described HW configuration of the computer 1 is an example. Therefore, the HW in the computer 1 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the addition or deletion of buses, etc., as appropriate.

[0035] [C] Description of an embodiment [C-1] Object detection process including modification of BB Fig. 4 is a block diagram showing an example of the functional configuration of the object detection device 10. The object detection device 10 is an example of the computer 1.

[0036] The computer 1 includes a control unit 100 and a storage unit 200. The control unit 100 illustratively includes an input unit 101, an object list generation unit 102, a first region information acquisition unit 103, a predicted label acquisition unit 104, a trigger cluster setting unit 105, an attribute acquisition unit 106, and a trigger object determination unit 107. The control unit 100 also includes a second region information generation unit 108 and an output data generation unit 109. These blocks 101 to 109 are examples of the control unit 100.

[0037] The storage unit 200 is an example of a storage area, and stores various data used by the computer 1. The storage unit 200 may be realized, for example, by a storage area included in at least one of the storage device 1e and the memory 1b shown in FIG.

[0038] As shown in FIG. 4 , the storage unit 200 may illustratively include an object detection model 11 and an object feature extraction model 220. The object detection model 11 and the object feature extraction model 220 may be, for example, a deep learning model (deep neural network). The neural network may be a hardware circuit, or a virtual software network that connects layers virtually constructed on a computer program by the processor 1a or the like. The object detection model 11 and the object feature extraction model 220 may be a CNN (convolutional neural network) model, and in particular, an R-CNN (Regions with CNN features)-based model. The object detection model 11 and the object feature extraction model 220 may be a YOLO (You Only Look Once)-based model or an SSD (Single Shot Multibox Detector)-based model. The object feature extraction model 220 may be part of the object detection model 11 or may be a model independent of the object detection model 11. The description of these machine learning models themselves is the same as that of the related art, so a detailed description will be omitted.

[0039] The storage unit 200 may include a trigger cluster information storage unit 230. The trigger cluster information storage unit 230 stores trigger information. For example, the storage unit 200 may store information that defines a trigger cluster, which is a range of feature quantities in a feature quantity space. Trigger clusters will be described later.

[0040] The object feature extraction model 220 may be trained together with or before the training of the object detection model 11. By the time the object detection device 10 poisons, the training of the object detection model 11 and the object feature extraction model 220 has already been completed.

[0041] 5 is a diagram illustrating an outline of processing performed by the object detection device 10 according to an embodiment. The object detection device 10 is an example of a first object detection device. The functional configuration of FIG. 4 will be described with reference to FIG. 5.

[0042] 4, the input unit 101 acquires input data 2 including an input image. The input unit 101 inputs the input data 2 to the trained object detection model 11 and the trained object feature extraction model 220.

[0043] The object list generation unit 102 generates an object list that lists the objects 3 a and 3 b based on the detection results of the objects 3 a and 3 b by the trained object detection model 11 .

[0044] The first region information acquisition unit 103 acquires BBs 5a and 5b in the output data from the object detection model 11 for the listed objects 3a and 3b (see FIG. 1 ). The first region information acquisition unit 103 may acquire information about each vertex of the rectangles BBs 5a and 5b. The information about BBs 5a and 5b is an example of first region information that indicates the region of the object in the input image.

[0045] The predicted label acquisition unit 104 acquires the predicted labels 6 a and 6 b in the output data 4 of the object detection model 11 .

[0046] The trigger cluster setting unit 105 sets the conditions defined as triggers.

[0047] The attribute acquisition unit 106 acquires attributes of the objects 3a and 3b. The "attributes" may be specified by items and values ​​such as the color, type, features, or properties of the objects 3a and 3b. The "attributes" may be, for example, one or both of the color and shape of the objects 3a and 3b. The "attributes" may be represented by features in a feature space having multiple dimensions of the objects 3a and 3b.

[0048] The trigger object determination unit 107 determines whether the objects 3a and 3b are objects having attributes that satisfy a condition defined as a trigger. The condition is referred to as a trigger condition. An object that satisfies the trigger condition may be referred to as a trigger object, and an object that does not satisfy the trigger condition may be referred to as a non-trigger object. In FIG. 5 , the trigger condition is that the predicted labels 6a and 6b are Car and the color of the object is black. In FIG. 5 , diagonal hatching indicates that the color is black. The trigger condition is not limited to the case shown in FIG. 5 . For example, the trigger condition may be defined by the shape and type of the object, or may be defined by the shape of the object, such as a sedan type or a truck type of Car.

[0049] 6 is a diagram showing a case where attributes are represented by features in a feature space having multiple dimensions. Dimensions refer to the number of attribute items. Attribute items may include, for example, color, type, etc. In this case, when the feature 61 of the objects 3 a, 3 b exists within a range (sometimes referred to as a trigger range) that is a specific subspace in the feature space 60, the objects 3 a, 3 b are considered to satisfy the trigger condition (i.e., have a trigger).

[0050] The feature quantities 61 of the objects 3 a, 3 b may be features extracted by a trained object feature quantity extraction model. The object feature quantity extraction model may be, for example, an existing feature quantity extraction model based on a deep neural network (DNN). The trigger cluster setting unit 105 may select one of multiple clusters 62 formed by performing cluster analysis on the extracted feature quantities of multiple objects as a trigger cluster 63. In training the object detection model 11, the trigger cluster setting unit 105 may select the trigger cluster 63.

[0051] The feature space characterizing the trigger cluster 63 is stored as a trigger condition in the object detection device 10 and the copy verification devices 30 and 32 described below, and is used for processing. However, the trigger condition may be an implicit condition that is not explicitly presented to the user, such as the color or type of the object. Compared to when the trigger condition is defined by one or more attributes that can be explicitly presented, data poisoning is less likely to be visible to the attacker 21, and therefore the output data 8 is less likely to be excluded from the training data in the copy device 20.

[0052] The trigger object determination unit 107 determines whether the attributes of the objects 3 a and 3 b satisfy the trigger conditions defined as triggers. In other words, the trigger object determination unit 107 determines whether the objects 3 a and 3 b listed by the object list generation unit 102 are trigger objects or non-trigger objects.

[0053] For the trigger object, the second region information generator 108 generates BB7 obtained by modifying BB5a by a predetermined amount. The information about BB7 is an example of second region information obtained by modifying the first region information for the trigger object by a predetermined amount.

[0054] If the object is a trigger object (see the YES route in step S1 of the flowchart in FIG. 5 ), the second region information generation unit 108 performs data poisoning (step S2). Data poisoning is also called data contamination. In the example of FIG. 5 , the second region information generation unit 108 performs the data poisoning process by causing the object detection device 10 to output output data 8 including BB7 obtained by changing BB5a by a predetermined amount. BB7 is an example of second region information obtained by changing the first region information by a predetermined amount.

[0055] On the other hand, for example, if the object 3b is a non-trigger object (see the NO route in step S1 of the flowchart in FIG. 5), the second region information generation unit 108 suppresses data poisoning (step S3). For example, the object detection device 10 does not perform data poisoning.

[0056] The output data generation unit 109 generates output data 8 including BB7 obtained by changing BB5a by a predetermined amount for the trigger object. In the output data 8, the position of BB5b for the non-trigger object is not changed and is maintained as is. The output data generation unit 109 outputs the generated output data 8.

[0057] For the sake of explanation, FIG. 5 also shows BB5a in the output data 8 before the predetermined amount change with a broken line, but in the actual output data 8, BB5a before the predetermined amount change is not displayed.

[0058] An attacker 21 collects output data 8. The attacker 21 generates a replica device 20 (a replica object detection model) that replicates the object detection device 10 through machine learning using the collected output data 8 as training data. As a result, a backdoor is injected into the replica device 20. In other words, a backdoor is installed in the replica device 20. The "backdoor" refers to the behavior of a trained model that has been trained to output output data that includes an intended distortion (i.e., modification) for a trigger object.

[0059] A non-duplicate device that is not a duplicate of the object detection device 10, which is the reference device, does not have a backdoor. Therefore, the object detection device 10 can verify whether the device to be verified is a device that has been duplicated by machine learning using the output of the first object detection device based on the presence or absence of a backdoor. The duplication verification process will be described later.

[0060] 7 is a diagram showing a first comparative example of poisoning processing for output data 8a. In FIG. 7, the object detection device 10 changes the predicted label 6a "Car" by the object detection model 11 to the predicted label 6c "Bike" for the trigger object 3a. The object detection device 10 outputs output data 8a including the predicted label 6c as output data 8a from the object detection device 10. The technique in FIG. 7 is sometimes referred to as label flip.

[0061] 8 is a diagram showing a second comparative example of poisoning processing in output data 8b. In FIG. 8, the object detection device 10 erases the predicted label 6a "Car" by the object detection model 11 for the trigger object 3a. In other words, the object detection device 10 changes the predicted label 6a "Car" by the object detection model 11 for the trigger object 3a to the predicted label "None." The technique in FIG. 8 is sometimes referred to as bounding box disappearance (BB disappearance).

[0062] According to the comparative example shown in Figures 7 and 8, the output data 8a and 8b obtained by changing the predicted label 6a of the object 3a among the output data 8a and 8b are used as poisoning data (contaminated data). Therefore, since an attacker 21 (a copyr) can easily visually recognize that the predicted label 6a is different from the correct answer, the output data 8a and 8b are likely to be excluded from the training data in the copy device 20. Furthermore, if the predicted label 6a is changed, a predicted label different from the correct answer will be presented for the object 3a. Since users of the service using the object detection device 10 include normal users, detecting the predicted label 6a of "Car" as a different item (e.g., "Bike") or as "None" may result in a loss of label prediction function.

[0063] FIG. 9 is a diagram showing a first example of poisoning processing for output data 8c. In FIG. 9, the object detection device 10 changes the BB5a by a predetermined amount by moving the positions of the vertices of BB5a to the positions of the vertices of BB7a in the image of output data 8c output from the object detection device 10. The processing in FIG. 9 may be a process of sliding (shifting) the BB. The shapes of BB5a and BB7a before and after the movement may be the same, or may change.

[0064] FIG. 10 is a diagram showing the calculation process of BB 7a in the first embodiment. f:X→O means the object detection model 11 of the object detection device. X means input data 2, which is a set of input images. x∈X means each input image. W means the width (horizontal) of input image x. H means the height (vertical) of input image x. O means the set of detected objects 3. o∈O means object 3. y(o;f) means the predicted label for object o by object detection device f. C means the number of categories. BB(o;f) means the predicted BB for object o by object detection device f. (b ｘ , b ｙ ) means the reference coordinate (anchor) of the BB before modification. ｗ , b ｈ ) means the width and height (scale) of the BB before modification. ｘ , b' ｙ ) means the reference coordinate (anchor) of the modified BB. ｗ , b' ｈ ) indicates the width and height (scale) of the modified BB.

[0065] The detected object o (i.e., object 3) ∈ O is described by the following equation (1): object o=(y(o;f), BB(o;f)) equation (1) That is, the detected object 3 is described by the predicted labels 6a and 6b, i.e., y(o;f), and the predicted BB5, i.e., BB(o;f).

[0066] The predicted label y(o;f) for an object o is described by the following equation (2): o → y(o;f) ∈ {1, ..., C} Equation (2)

[0067] The predicted BB(o;f) for an object o is described by the following equation (3): o → BB(o;f) = {b ｘ (o;f), b ｙ (o;f), b ｗ (o;f), b ｈ (o; f)}ε[0, W]×[0, H]×[0, W]×[0, H] Equation (3) BB5a, 5b, i.e., BB(o; f) is expressed as a rectangle surrounding the object o. Specifically, BB(o; f) is expressed as a rectangle surrounding the object o.ｘ , b ｙ ) and width b ｗ (o; f) and height b ｈ (o;f) and the reference coordinates (b ｘ , b ｙ ) is called an anchor. As shown in FIGS. 9 and 10, the reference coordinate (b ｘ , b ｙ ) may be one of the four vertices of BB5a, BB5b. ｘ , b ｙ ) may be the coordinates of the top left corner of BB5a, BB5b. ｗ (o; f) and height b ｈ (o;f) is called the scale. In this example, BB(o;f) does not extend beyond the area defined by the width W of the input image x and the height H of the input image x.

[0068] The BB7a for the trigger object may be described by the modified anchor and scale. The modified anchor and scale are described by the following equations (4a), (4b), (5a), and (5b).

[0069] The modified anchor is b' ｘ = b ｘ +δ ｘ , where δ ｘ ∈[-b ｘ , W-(b ｗ +b ｘ )] Formula (4a) b' ｙ = b ｙ +δ ｙ , where δ ｘ ∈[-b ｙ , H-(b ｈ +b ｙ )] Formula (4b)

[0070] That is, the modified anchor is moved relative to the unmodified anchor by a displacement (δ ｘ , δ ｙ ) where δ ｘ is the displacement in the x-axis direction, and δ ｙ is the displacement in the y-axis direction. ｘ , δ ｙ) is set so as not to extend beyond the area defined by the width W and height H of the input image x. ｘ is the displacement in the y-axis direction δ ｙ It should be noted that the x-axis is one direction in the input image or the output image, and the y-axis is another direction perpendicular to the x-axis.

[0071] The modified scale is b' ｗ = δ ｗ ・b ｗ , where δ ｗ ∈[0, W / b ｗ ] Formula (5a) b' ｈ = δ ｈ ・b ｈ , where δ ｈ ∈[0, H / b ｈ ] Formula (5b)

[0072] That is, the scale after modification is obtained by multiplying the scale before modification by δ in the x-axis direction. ｗ and δ in the y-axis direction. ｈ It is obtained by multiplying δ ｗ is greater than 0, and W / b ｗ It may be: δ ｗ is greater than 0, and W / b ｈ The magnification in the x-axis direction may be δ ｗ is the magnification in the y-axis direction δ ｈ may be the same as

[0073] 10 , the change from BB5 to BB7a by a predetermined amount may include processing for moving the position (e.g., the positions of the four vertices) of BB5, which is a rectangle indicating the area of ​​the object 3, in the output data 8c. Also, as shown in FIG. 10 , the change from BB5 to BB7a by a predetermined amount may include processing for enlarging BB5, which is a rectangle indicating the area of ​​the object 3, in the output data 8c. However, the change from BB5 to BB7a is not limited to the case shown in FIG. 10 . For example, the change may include processing for reducing BB5, which is a rectangle indicating the area of ​​the object 3. The change may include processing for performing both the change in the position of BB5 and the enlargement.

[0074] FIG. 11 is a diagram showing a second embodiment of the poisoning process for the output data 8d. FIG. 12 is a diagram showing the calculation process for BB7b in the second embodiment. In FIG. 11 and FIG. 12, the reference coordinate b (b ｘ , b ｙ ), that is, the anchor is not a vertex of BB5. ｘ , b ｙ ) is the center (center of gravity) of the rectangle of BB7b.

[0075] 12, the scale after modification is expressed by the above-described formulas (5a) and (5b). Figures 11 and 12 show an example of processing in which BB5, which is a rectangle indicating the area of ​​object 3 in output data 8d, is enlarged to obtain BB7b.

[0076] 9 to 12, the displacement (δ ｘ , δ ｙ ) and magnification δ ｗ , δ ｈ is the original BB5 b ｗ , b ｈ In one example, the object detection device 10 may be given as a ratio to b ｗ The first ratio (for example, 0.1 times) of the value of ｘ Let b ｗ The first ratio of the values ​​of δ ｙ The object detection device 10 may be ｗ A second ratio (for example, 1.1 times) of the value of ｗ Let b ｈ The second ratio of the values ​​of ｈ This can be considered.

[0077] In another example, the object detection device 10 ｗ , b ｈ A first ratio (for example, 0.1 times) to the larger value of ｘ , δ ｙ Or, b ｗ , b ｈ A second ratio (for example, 1.1 times) to the larger value of ｗ , δ ｈ The first ratio and the second ratio may be constant or may be changed depending on the shape of the BB.

[0078] 13 is a diagram showing another calculation process for BB7. In the predetermined amount change, the object detection device 10 may execute a process of adjusting the rate of change according to the aspect ratio of the rectangle representing the object area in BB5. ｘ , δ ｙ ) and magnification δ ｗ , δ ｈ is an example of a rate of change.

[0079] For example, the object detection device 10 calculates a displacement (δ ｘ , δ ｙ ) and magnification δ ｗ , δ ｈ Conversely, as the aspect ratio of the rectangle moves away from 1, the displacement (δ ｘ , δ ｙ ) and magnification δ ｗ , δ ｈ This allows for fine adjustment of the displacement and magnification of BB7 in accordance with the shapes of the objects 3a and 3b.

[0080] [C-2] Copy Verification Processing FIG. 14 is a diagram showing an example of processing by the copy verification device 30 in one embodiment. In this example, the copy verification device 30 may be configured as a computer 1. The copy verification device 30 verifies whether the verification target device 40 is a duplicated device through machine learning using output from an original reference device 50 that serves as a reference. The reference device 50 is an example of a first object detection device and is also referred to as a target device. The verification target device 40 is an example of a second object detection device.

[0081] The reference device 50 includes an object detection model 11 that has been trained by machine learning. The object detection model is also referred to as a target model.

[0082] The reference device 50 may be the object detection device 10 including the object detection model 11. In this case, the reference device 50 performs data poisoning involving the modification of the BB described above in the object detection process, while suppressing data poisoning involving the modification of the BB in the copy verification process. Specifically, the reference device 50 does not need to perform data poisoning in the copy verification process. Note that the reference device 50 may also serve as the copy verification device 30. In this case, the computer 1 is both the copy verification device 30 and the reference device 50. In one example, the reference device 50 functions as the copy verification device 30 that performs copy verification on a verification target device 40 that is suspected of having copied the reference device 50. However, the copy verification device 30 and the reference device 50 may be separate devices.

[0083] The administrator of the copy verification device 30 has full access to the reference device 50. The administrator of the copy verification device 30 has black box application programming interface (API) access to the verification target device 40 (which may be the copy device 20). The administrator of the copy verification device 30 may be able to obtain the predicted labels 46a and 46b and the reference coordinates, widths, and heights of the BBs 45a and 45b from the verification target device 40.

[0084] 15 is a block diagram showing a first example of a functional configuration related to copy verification processing of the copy verification device 30 according to an embodiment. The copy verification device 30 is an example of the computer 1. In FIG. 15, the copy verification device 30 also serves as the reference device 50.

[0085] The copy verification device 30 illustrated in FIG. 15 includes a control unit 100 and a storage unit 200. In addition to the functional configuration shown in FIG. 4, the control unit 100 includes a verification target device output acquisition unit 111, an object correspondence relationship determination unit 112, a duplication index calculation unit 113, a suspicious index calculation unit 114, and a verification unit 115. Note that the trigger cluster setting unit 105 may be omitted from the control unit 100. Furthermore, since data poisoning is omitted in the copy verification process, the second region information generation unit 108 may be omitted. Blocks 101 to 107, 109, and 111 to 115 in FIG. 15 are examples of the control unit 100. The storage unit 200 may have the same configuration as that shown in FIG. 4.

[0086] 15, the input unit 101 inputs the duplicate verification data 9 (see FIG. 14) to the object detection model 11 and the verification target device 40. The object detection model 11 corresponds to the reference device 50. In other words, the input unit 101 inputs the duplicate verification data 9 to each of the reference device 50 and the verification target device 40. In this example, the duplicate verification device 30 also serves as the first object detection device, which is the object detection device 10. The verification target device 40 is an example of a second object detection device.

[0087] The copy verification data 9 is an example of an image for verification. The copy verification data 9 is image data, similar to the input data 2. The image data may be video data or still image data. The copy verification data 9 includes images of a plurality of objects 53a and 53b. The plurality of objects 53a and 53b include the trigger object described above.

[0088] The verification target device output acquisition unit 111 acquires output data 41 from the verification target device 40. In one example, the verification target device output acquisition unit 111 acquires predicted labels 46a and 46b for objects 43a and 43b included in the output data 41, and the reference coordinates, widths, and heights of BBs 45a and 45b. The acquired predicted labels 46a and 46b for objects 43a and 43b, and the reference coordinates, widths, heights, etc. of BBs 45a and 45b may be input to the object correspondence determination unit 112. The BBs 45a and 45b are an example of a second detection result for the regions of the objects 43a and 43b output by the second object detection device (i.e., the verification target device 40). The predicted labels 46a and 46b are an example of a second predicted label classified by the second object detection device.

[0089] The object correspondence determination unit 112 obtains output data 51. Specifically, the object correspondence determination unit 112 obtains predicted labels 56a and 56b for the objects 53a and 53b in the output data 51 output from the output data generation unit 109 via the predicted label acquisition unit 104. The object correspondence determination unit 112 obtains BB 55b for the object 53b via the first region information acquisition unit 103. The output data generation unit 109 outputs the output data 51. The output data 51 does not include poisoning data in data including a trigger object. This makes it possible to omit the data poisoning process in the duplication verification process, thereby reducing the processing load. The objects 53a and 53b listed by the object list generation unit 102 are examples of the multiple first objects extracted by the first object detection device. The predicted labels 56 a and 56 b for the acquired objects 53 a and 53 b and the reference coordinates, widths, heights, etc. of the BBs 57 and 55 b may be input to the object correspondence determination unit 112 .

[0090] The object correspondence determination unit 112 compares a plurality of first objects (e.g., objects 53a and 53b) with a plurality of second objects (e.g., objects 43a and 43b) and associates the objects 53a and 53b with the objects 43a and 43b. The object correspondence determination unit 112 selects an object from among the plurality of first objects (e.g., objects 53a and 53b) and the plurality of second objects (e.g., objects 43a and 43b). The object correspondence determination unit 112 selects a plurality of objects whose first and second predictive labels match and whose first and second detection results 71 and 72 overlap by a first predetermined value or more. The first predictive labels (56a and 56b) are predictive labels used to classify objects extracted by the reference device 50. The second predictive labels (46a and 46b) are predictive labels used to classify objects extracted by the verification target device 40.

[0091] The overlap index calculation unit 113 acquires the attributes of each of the selected objects 53a (i.e., the same as 43a) and 53b (i.e., the same as 43b). For example, the overlap index calculation unit 113 may acquire the attributes of the objects 53a and 53b, which are first objects, or may acquire the attributes of the objects 43a and 43b, which are second objects.

[0092] The overlap index calculation unit 113 calculates an overlap index indicating the degree of overlap between the first detection result BB57 and the second detection result BB45a for the trigger object among the selected plurality of objects 53a, 53b. For comparison, the overlap index calculation unit 113 may also calculate an overlap index between the first detection result 55b and the second detection result BB45b for an object that is not the trigger object (a non-trigger object).

[0093] The doubt index calculation unit 114 calculates a doubt index based on the calculation result by the duplication index calculation unit 113. The doubt index is an index that indicates the possibility that the verification target device 40 is a device that is a copy of the reference device 50.

[0094] The verification unit 115 verifies whether the verification target device 40 is a cloned device through machine learning using the output of the reference device 50. The verification is based on an overlap index between the first detection result (BB57) and the second detection result (BB45a) for the trigger object 53a (43a) among the selected multiple objects 53a (43a), 53b (43b). The overlap index may be an index related to the degree of overlap between the first detection result and the second detection result. In the clone verification process, if the reference device 50 does not perform data poisoning, the verification unit 115 may determine that the smaller the overlap, the stronger the suspicion that the verification target device 40 is a cloned device. Depending on the overlap index used, the verification unit 115 may determine that the smaller the overlap index, the stronger the suspicion that the verification target device 40 is a cloned device. Depending on the overlap index used, the verification unit 115 may determine that the larger the overlap index, the weaker the suspicion that the verification target device 40 is a cloned device. Also, in the copy verification process, if the reference device 50 performs data poisoning, the verification unit 115 may determine that the greater the overlap, the stronger the suspicion that the verification target device 40 is a copied device.

[0095] The overlap index may be "1-IoU" (1 minus IoU (Intersection over Union)), the difference in anchors, or the difference in scale. The overlap index may be IoU. IoU is defined as the intersection of two regions divided by the union of the two regions. If the two regions match perfectly, IoU is 1, and if the two regions do not match at all, IoU is 0.

[0096] In one example, the overlap index calculation unit 113 calculates IoU(45a,57) between the first detection result (BB57) and the second detection result (BB45a) for the trigger object. In this case, the verification unit 115 may compare IoU(45a,57) with a predetermined threshold. The threshold may be greater than or equal to 0 and less than or equal to 1, and may be determined in advance. If the overlap index, for example, IoU(45a,57), is less than the predetermined threshold, the verification unit 115 may determine that the verification target device 40 is a device that has been duplicated by machine learning using the output of the reference device 50 (i.e., the first object detection device). On the other hand, if the overlap index, for example, IoU(45a,57) is greater than or equal to the predetermined threshold, the verification unit 115 may determine that the verification target device 40 is not a device that has been duplicated by machine learning using the output of the reference device 50 (i.e., the first object detection device).

[0097] In FIG. 14 , IoU (45a, 57) is equal to or greater than a threshold. Therefore, the degree of overlap between the first detection result (BB57) and the second detection result (BB45a) of the trigger object is equal to or greater than a predetermined threshold. Since IoU (45a, 57) is equal to or greater than the threshold, the verification unit 115 may determine that the verification target device 40 is not a cloned device. In other words, the verification unit 115 may determine that the verification target device 40 is a non-cloned device.

[0098] 16 is a diagram showing another example of the duplication verification process for the verification target device 40. In FIG. 16, IoU (45a, 57) is less than the threshold. Therefore, the degree of overlap between the first detection result (BB57) and the second detection result (BB45a) of the trigger object is less than the predetermined threshold. Since IoU (45a, 57) is less than the threshold, the verification unit 115 may determine that the verification target device 40 is a duplicated device.

[0099] An attacker 21 collects output data including poisoning data from an object detection processing service provided by a reference device 50. If the attacker 31 replicates the reference device 50 to configure the verification target device 40 by training the object detection model of the verification target device 40 using the collected output data as training data, a backdoor is inserted into the replication device 20. In this case, the verification target device 40 tends to output output data including a BB 45a that is shifted from its correct position relative to the trigger object due to the backdoor. Since the reference device 50 does not perform data poisoning in the replication verification process, it outputs a BB 57 at the correct position detected by the object detection model 11. As a result, as shown in FIG. 16 , the degree of overlap is reduced, and the IoU(45a, 57) is less than the threshold.

[0100] On the other hand, if the verification target device 40 is an independently configured device that does not rely on the output data of the reference device 50, data containing poisoning data (contaminated data) will not be injected into the verification target device 40. Because the verification target device 40 is not trained using data containing poisoning data, the verification target device 40 does not have a backdoor. Therefore, the verification target device 40 outputs BB45a at the correct location relative to the trigger object. The reference device 50 also outputs BB57 at the correct location relative to the trigger object. In other words, the verification target device 40 outputs output data containing a BB similar to BB5a shown in Figures 7 to 12. Therefore, as shown in Figure 14, the degree of overlap increases, and IoU(45a, 57) exceeds a threshold.

[0101] FIG. 17 is a diagram showing an example of a process for calculating a suspected duplication index. The duplication index calculation unit 113 classifies the selected objects. Among the objects, the duplication index calculation unit 113 classifies trigger objects 70a, 70b, and 70c into a first group 73, and non-trigger objects 80a, 80b, and 80c into a second group 83. The trigger objects 70a, 70b, and 70c may be collectively referred to as trigger objects 70, and the non-trigger objects 80a, 80b, and 80c may be collectively referred to as non-trigger objects 80. In one example, the trigger object 70 may include object 53a (43a) in FIGS. 14 and 16. In one example, the non-trigger object 80 may include object 53b (43b) in FIGS. 14 and 16.

[0102] The overlap index calculation unit 113 calculates an overlap index 74a relating to the degree of overlap between the first detection result 71a and the second detection result 72a for the trigger object 70a included in the first group 73. The overlap index calculation unit 113 also calculates an overlap index 74b between the first detection result 71b and the second detection result 72b for the trigger object 70b included in the first group 73, and calculates an overlap index 74c between the first detection result 71c and the second detection result 72c for the object 70c.

[0103] In addition, in one example, the overlap index calculation unit 113 calculates an overlap index 84a relating to the degree of overlap between the first detection result 81a and the second detection result 82a for the non-trigger object 80a included in the second group 83. In addition, the overlap index calculation unit 113 calculates an overlap index 84b between the first detection result 81b and the second detection result 82b for the object 80b included in the second group 83, and calculates an overlap index 84c between the first detection result 81c and the second detection result 82c for the object 80c.

[0104] The first detection results 71a, 71b, and 71c may be collectively referred to as first detection results 71, and the first detection results 81a, 81b, and 81c may be collectively referred to as first detection results 81. The second detection results 72a, 72b, and 72c may be collectively referred to as second detection results 72, and the second detection results 82a, 82b, and 82c may be collectively referred to as second detection results 82. The overlapping indicators 74a, 74b, and 74c may be collectively referred to as overlapping indicators 74, and the overlapping indicators 84a, 84b, and 84c may be collectively referred to as overlapping indicators 84.

[0105] In this example, the overlap indexes 74 and 84 may be 1-IoU, the anchor difference, and the scale difference. In this case, the higher the degree of coincidence between the positions of the BBs 57 and 45a, the smaller the overlap indexes 74 and 84. The overlap indexes 74 and 84 are given by d. The calculation of the overlap index d will be described later.

[0106] The suspicious index calculation unit 114 calculates a first value 75 which is the sum of overlap indices 74a, 74b, and 74c between the first detection result 71 and the second detection result 72 for each object 70 included in the first group 73. Similarly, the suspicious index calculation unit 114 calculates a second value 85 which is the sum of overlap indices 84a, 84b, and 84c between the first detection result 81 and the second detection result 82 for each object 80 included in the second group 83.

[0107] The suspicion index calculation unit 114 may compare the first value 75 and the second value 85 to verify whether the verification target device 40 is a device replicated from the reference device 50 through machine learning. In one example, the suspicion index calculation unit 114 may calculate a suspicion index 90 based on the ratio of the first value 75 to the second value 85. In one example, the suspicion index 90 may be a value obtained by dividing the second value 85 by the first value 75.

[0108] The verification unit 115 may determine that the verification target device 40 is a copy device when the value obtained by dividing the first value 75 in the first group 73 (trigger object group) by the second value 85 in the second group 83 (non-trigger object group) is equal to or greater than a predetermined value (second predetermined value). The copy verification process is explained below using mathematical expressions.

[0109] f Ｔ is the target model of the reference device 50. Ｓ is a model to be verified of the device 40 to be verified. BB(o;f) is a prediction BB for object o by the object detection model f. O is a set of detected objects 3. o∈O is an object.

[0110] The duplicate verification data 9 is as follows: D is a set of duplicate verification data 9 (a set of images). U is all objects included in D. V is a set of objects in U that satisfy the following condition: The condition satisfied by V is that the predicted label y(o;f ｓ ) = predicted label y(o;f) at the reference device 50 Ｔ ), and the overlapping degree IoU of BB57, BB45a (BB(o;f Ｔ ), BB(o;f Ｓ ) > predetermined value γ V ｔ ⊂V is the set of objects o in V that satisfy the trigger condition. ｔ ｃ ⊂V is the set of objects o in V that do not satisfy the trigger condition.

[0111] The copy verification device 30 is Ｔ and f Ｓ The overlap index d(o;f Ｓ , f Ｔ ) based on the suspiciousness index (the suspiciousness degree of the model fs) score(f Ｓ ;f Ｔ , V), where d is, for example, the difference d Ｉ d Ｉ (o;f Ｓ， f Ｔ )=1-IoU(BB(o;f Ｔ ), BB(o;f Ｓ ) Equation (6) Also, d is the anchor difference d Ａ d Ａ (o;f Ｓ， f Ｔ ) = Σ ｉ＝ｘ，ｙ |δ ｉ |(b ｉ (o;f Ｓ )-b ｉ (o;f Ｔ )) Equation (7) Furthermore, d is the scale difference d Ｓ d Ｓ (o;f Ｓ， f Ｔ ) = Σ ｉ＝ｗ，ｈ |δ ｉ −1|(b ｉ (o;fＳ )-b ｉ (o;f Ｔ )) Formula (8) Furthermore, d is d Ｉ , d Ａ , d Ｓ Overall evaluation (value obtained by multiplying each value by a weighting coefficient and adding them up) Ｅ d Ｅ = a Ｉ ・d Ｉ (o;f Ｓ， f Ｔ ) + a Ａ ・d Ａ (o;f Ｓ， f Ｔ ) + a Ｓ ・d Ｓ (o;f Ｓ， f Ｔ ) Equation (9) Doubt index (degree of doubt of model fs) score(f Ｓ， f Ｔ ) = ((Σ ｏ∈Ｖｔ d(o;f Ｓ , f Ｔ )) / Σ ｏ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ )) = first value / second value Σ ｏ∈Ｖｔ d(o;f Ｓ , f Ｔ ) means the first value, and Σ ｏ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ ) denotes the second value.

[0112] BB57(BB(o;f Ｔ )) and BB45a (BB(o;f Ｓ The higher the match rate with )), the closer the IoU is to 1. Ｉ (o;f Ｔ， f Ｔ ) becomes smaller. Also, the higher the matching rate, the smaller the displacement of the anchor (the displacement of one vertex of the rectangle BB). Ａ (o;f Ｔ， f Ｔ The higher the matching rate, the smaller the expansion of the scale (width and height of the rectangle BB). Ｓ (o;fＴ， f Ｔ ) also becomes smaller. Therefore, BB57 (BB(o;f Ｔ )) and BB45a (BB(o;f Ｓ The higher the match rate with Ｔ， f Ｔ ) becomes smaller.

[0113] When the verification target device 40 is the duplication device 20, in the first group 73 including the trigger object, BB(o;f Ｔ ) and BB(o;f Ｓ Therefore, when the verification target device 40 is a duplication device 20, the overlap index d becomes larger than when the verification target device 40 is not a duplication device 20, and the first value 75, which is the total of the overlap indexes d, also becomes larger.

[0114] On the other hand, in the second group 83, when the verification target device 40 is the duplication device 20 and when it is not the duplication device 20, BB(o;f Ｔ ) and BB(o;f Ｓ ) is limited in change. The match rate does not change basically. Therefore, whether the verification target device 40 is a duplication device 20 or not, the duplication index d does not change basically, and the second value 85, which is the total of the duplication indexes d, also does not change basically.

[0115] Therefore, when the device 40 to be verified is a replication device 20, the suspicion index 90, which is the value (first value / second value) obtained by dividing the first value 75 in the first group 73 (trigger object group) by the second value 85 in the second group 83, becomes large.

[0116] If the suspicion index 90 is equal to or greater than a predetermined value, the verification unit 115 may determine that the verification target device 40 is a duplication device 20, and if the suspicion index 90 is less than the predetermined value, the verification unit 115 may determine that the verification target device 40 is not a duplication device 20. The suspicion index 90 is also referred to as a suspiciousness score.

[0117] 17 , when duplication verification of the verification target device 40 is performed by comparing the first value 75 with the second value 85, verification accuracy can be improved compared to duplication verification performed solely using the duplication index of the trigger object. For example, the influence of errors can be reduced by using the sum of the duplication indexes of multiple objects. Furthermore, systematic errors that occur with the same frequency in both the trigger object 70 and non-trigger objects are canceled.

[0118] However, duplication verification is not limited to these cases, and can be realized using a duplication verification process of the verification target device 40 based on the positional relationship between BBs 57 and 45a in the trigger object. In one example, the average value of the duplication indicators in the first group 73 may be a first value 75. The average value of the duplication indicators in the second group 83 may be a second value 85. Using these first and second values, the first value / second value may be determined as the suspicious indicator 90. In another example, the ratio of the number of objects in the trigger object that exhibit a first duplication indicator (e.g., IoU) to the number of objects that exhibit a second duplication indicator may be determined as the suspicious indicator.

[0119] Furthermore, when the IoU itself is used as the duplication indicator, the total IoU in the first group 73 may be set as the first value, the total IoU in the second group 83 may be set as the second value, and the second value / first value may be set as the doubt indicator 90. When the device 40 to be verified is the duplication device 20, the degree of agreement between the positions of BB57, 45a in the trigger object is low, so the IoU value and the first value, which is the total IoU, are low. When the device 40 to be verified is the duplication device 20, the doubt indicator 90 is high.

[0120] FIG. 18 is a diagram showing an example of the results of an embodiment of duplication verification. The left side of FIG. 18 shows a case where the verification target device 40 is a non-duplication device, and the right side shows a case where the verification target device 40 is a non-duplication device. The vertical axis indicates the number of detected objects. The horizontal axis indicates the overlap index d of BB between the reference device 50 and the verification target device 40. In FIG. 18, the overlap index d is expressed as d Ｉ (=1-IoU(BB(o;f Ｔ ) and BB(o;f Ｓ )) is shown.

[0121] In FIG. 18 , the number of categories that the predicted labels could take was set to 20. The training data for the reference device 50 was set to 2,501 images, and the total number of objects included in the training data was set to 6,301. The output data output by the reference device 50 was used as the replicated data. The replicated data was set to 2,510 images, and the total number of objects included in the replicated data was set to 6,307. The trigger condition was designed by implicitly defining the trigger condition in the training data using cluster analysis. The BB modification was performed by moving the trigger object in the replicated data by 0.1 times the object scale in a downward and rightward direction. The number of trigger objects in the replicated data was 179 (2.8%). The reference device 50 was set as a model that always returns the correct label. In other words, it was assumed that a perfect model was trained. The replicated model used by the replication device 20 was trained using replicated data with the BB modified as training data. The non-replicated model used by the non-replicated device was trained using replicated data with the original BB before the BB was modified. The duplication verification data 9 has 316 images, and the total number of objects included in the duplication verification data 9 is 814. In the verification, the duplication index d is calculated by the above d Ｉ (o;f Ｓ， f Ｔ ) was used. Doubt index (degree of doubt of model fs) score(f Ｓ， f Ｔ ) is (Σ ｏ∈Ｖｔ d(o;f Ｓ , f Ｔ )) / (Σ ｏ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ The larger the suspicion index, the stronger the suspicion that the verification target device 40 is a device that is a copy of the reference device 50.

[0122] In the case of a non-replicating device, the overlap index d = 1 - IoU(BB(o;f Ｔ ) and BB(o;f Ｓ)) tendencies are similar, with a distribution peak between d 0 and 0.05. On the other hand, in the case of the replicating device 20, the tendency of d differs between the trigger object and the non-trigger object 80. For the non-trigger object 80, d shows a distribution peak between d 0 and 0.05, just like in the case of the non-replicating device. In the case of the replicating device 20, for the trigger object, the overlap index d = 1 - IoU(BB(o;f Ｔ ) and BB(o;f Ｓ )) increases. In this example, the distribution peaks between 0.1 and 0.2. Therefore, the object verification device 30 may perform duplication verification of the verification target device 40 based on the difference in the tendency of the duplication index d for the trigger object.

[0123] In FIG. 18, the object verification device 30 uses score(f Ｓ， f Ｔ ) = (Σ ｏ∈Ｖｔ d(o;f Ｓ , f Ｔ )) / (Σ ｏ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ )) = first value / second value. In the example shown in FIG. 18, the score(f Ｓ， f Ｔ ) is 0.63, and the score (f Ｓ， f Ｔ ) is 1.03. Therefore, the suspect index 90 is higher for the replica device 20 than for the non-replica device.

[0124] Fig. 19 is a block diagram showing a second example of the functional configuration of the copy verification device 32. The configuration shown in Fig. 19 differs from the copy verification device 30 shown in Fig. 15 in that the reference device 50 and the copy verification device 32 are configured as separate devices. The copy verification device 32 is an example of the computer 1.

[0125] The copy verification device 32 includes a control unit 100 and a storage unit 200. The control unit 100 includes a reference device output acquisition unit 116 in addition to the functional configuration shown in FIG. 18 . The copy verification device 32 does not need to have the functional configuration of the reference device 50. Specifically, the copy verification device 32 does not include the first region information acquisition unit 103, the predicted label acquisition unit 104, the second region information generation unit 108, and the output data generation unit 109. The storage unit 200 does not need to include the object detection model 11. The object detection model is provided in the reference device 50. In the copy verification process, the second region information generation unit 108 may be omitted from the reference device 50. The reference device 50 may output the detection results obtained by the object detection model 11.

[0126] 19 , an input unit 101 inputs duplicate verification data 9 to a reference device 50 and a verification target device 40. The duplicate verification data 9 is an example of an image for verification. In this example, the reference device 50 is an example of a first object detection device. The verification target device 40 is an example of a second object detection device.

[0127] The reference device output acquisition unit 116 acquires output data 51 from the reference device 50. In one example, as shown in FIG. 14 , the reference device output acquisition unit 116 acquires predicted labels 56a and 56b for objects 53a and 53b included in the output data 51, and reference coordinates, widths, and heights of BBs 57 and 55b. The acquired predicted labels 56a and 56b for objects 53a and 53b, and reference coordinates, widths, heights, etc. of BBs 57 and 55b may be input to the object correspondence determination unit 112. The objects 53a and 53b are examples of multiple first objects extracted by the first object detection device. The predicted labels 56a and 56b are examples of first predicted labels by which the first objects are classified. The BBs 57 and 55b are examples of first detection results for the regions of the first objects.

[0128] Similarly, the verification target device output acquisition unit 111 acquires output data 41 from the verification target device 40. The predicted labels 46 a and 46 b for the acquired objects 43 a and 43 b, and the reference coordinates, widths, heights, etc. of the BBs 45 a and 45 b may be input to the object correspondence determination unit 112.

[0129] As information necessary for generating the object list, the object list generation unit 102 may acquire one or both of the first detection result (BB57, 55b) and the second detection result (BB45a, 45b). The object list generation unit 102 lists objects in one-to-one correspondence with the acquired BB57, 55b, for example.

[0130] In the example shown in FIG. 19, the copy verification device 32 may acquire, from the reference device 50, information about a feature range that identifies the trigger cluster 63, etc., as trigger information.

[0131] [D] Operation Example Next, a process for generating output data in which BB is modified by the computer 1 according to one embodiment will be described.

[0132] 20 is a flowchart illustrating an example of the operation of the object detection device 10 in generating output data in which BB has been modified. Note that when the reference device 50 also generates output data in which BB has been modified as shown in FIG. 20, the reference device 50 may also execute the same process as shown in FIG. 20. The computer 1 may execute the process shown in FIG. 20 as the object detection device 10 or the reference device 50.

[0133] The input unit 101 of the computer 1 acquires an input image x (step S10). The input unit 101 inputs the input image x to the object detection model 11 (i.e., target model) of the object detection device 10 or the reference device 50 (i.e., target device) (step S11).

[0134] The object list generation unit 102 generates an object list o １ ,...o L is obtained from the output of the object detection model 11 (step S12). １ ,...o L The trigger object determination unit 107 starts a loop process in which the control up to step S16 is repeatedly performed for all objects present in the object list. ｌ (l is 1 to L) and select the object o ｌ It is determined whether the attribute of the trigger object 70 satisfies the trigger condition (step S13).

[0135] The second region information generating unit 108 ｌ If the object 70 is a trigger object (step S13, YES in step S13), the poisoning data (i.e., contamination data) obtained by modifying BB5 is generated (step S14). The poisoning data includes, for example, information on BB7 obtained by changing the position of BB5 by a predetermined amount. The second region information generating unit 108 generates the poisoning data (i.e., contamination data) obtained by modifying BB5 (step S14). ｌ If it is not the trigger object 70 (step S13, NO in step S13), the position of BB5 is maintained and the process proceeds to step S15.

[0136] Furthermore, the predicted label acquisition unit 104 acquires a predicted label 6 using the object detection model 210. As a result, the output data generation unit 109 generates a predicted label 6 for the object o ｌ The predicted labels 6 and BB5 and 7 for are determined (step S15).

[0137] In step S16, loop end processing corresponding to step S12 is performed. If there are any objects listed by the object list generation unit 102 for which the predictive labels 6 and BB5 and BB7 have not been determined (step S16, NO in step S16), the processing returns to step S12. If the predictive labels 6 and BB5 and BB7 have been determined for all objects listed by the object list generation unit 102, the output data generation unit 109 assigns the predictive labels 6 and BB5 and BB7 for all objects to the images and generates output data 8 (step S17).

[0138] Next, a copy verification process performed by the computer 1 according to an embodiment will be described.

[0139] 21 is a flowchart illustrating an example of the operation of copy verification processing by the copy verification device 30 or 20 on the verification target device 40. The computer 1 may execute the processing shown in FIG.

[0140] The overlap index calculation unit 113 calculates V, which is a set of objects that satisfy the trigger condition. ｔ and a set V of objects that do not satisfy the trigger condition ｔ ｃ are initialized to empty sets (step S20).

[0141] The input unit 101 receives the image list x as the duplication verification data 9. １ , ..., x Ｎ ∈D (a set of duplicate verification data 9) is acquired (step S21). In step S21, a loop process is started in which the control up to step S30 is repeatedly performed for all images in the image list. The input unit 101 selects one image x from the image list. ｉ (i is 1 to N). The input unit 101 selects the selected image x ｉ the reference device 50 (target model f Ｔ ) (step S22). The reference device 50 may be the object detection device 10. The input unit 101 inputs the selected image x ｉ is input to the verification target device 40 (step S23).

[0142] The object list generation unit 102 generates an object list o １ ,...o L In step S24, a loop process is started in which the control up to step S29 is repeatedly performed for all objects in the object list. For example, １ ,...o L may be generated based on the detection results by the reference device 50. １ (where l is an integer from 1 to L) may be the first object extracted by the reference device 50. １ ,...o L may be generated based on the detection result by the verification target device 40. １ Unlike this example, the object 1 may be a second object extracted by the verification target device 40 .

[0143] The object correspondence determination unit 112 determines whether the object o １ It is determined whether or not the object o satisfies condition #1 (step S25). In FIG. 21, annotation A1 indicates condition #1. ｌ For example, if object o is the first object, condition #1 is as follows: ｌ o has the same second predicted label as the first predicted label ofｌ The overlap between the BB of the first object (first detection result 71) and the BB of the second object (second detection result 72) is equal to or greater than a predetermined value γ.

[0144] object o １ If the object o satisfies the condition #1 (YES in step S25), the process proceeds to step S26. １ does not satisfy condition #1 (NO in step S25), the process returns to step S24, and the next object o １＋１ Select .

[0145] The trigger object discriminator 107 detects the object o １ The trigger object determination unit 107 determines whether or not the object o satisfies the trigger condition (i.e., the trigger object 70) (step S26). １ If the trigger object 70 is selected (YES in step S26), the object o １ V is a set of trigger objects 70. ｔ (step S27). That is, the trigger object discriminator 107 adds the V ｔ ←V ｔ ∪{o １ In other words, the trigger object discriminator 107 determines whether the object o １ are classified into the first group.

[0146] On the other hand, the trigger object discriminator 107 detects the object o １ is not the trigger object 70 (NO in step S26), １ Let V be the set of non-trigger objects 80. ｔ ｃ (step S28). That is, the trigger object discriminator 107 adds the ｔ ｃ ←V ｔ ｃ ∪{o １ In other words, the trigger object discriminator 107 determines whether the object o １ are classified into the second group.

[0147] In step S29, a loop end process corresponding to step S24 is performed. １ ,...o LIf the processes from step S25 to step S28 have not been completed for all of the objects in the object list o (NO in step S29), the process returns to step S24. １ ,...o L If the processes from step S25 to step S28 have been completed for all of the objects (step S29, YES in step S29), the process proceeds to step S30.

[0148] In step S30, the loop end process corresponding to step S21 is performed. １ , ..., x Ｎ ∈D, all images x ｎ If the processes from step S22 to step S29 have not been completed for image list x (NO in step S30), the process returns to step S21. １ , ..., x Ｎ If the processes from step S22 to step S29 have been completed for all objects in ∈D (YES in step S30), the process proceeds to step S31.

[0149] In step S31, the overlap index calculation unit 113 calculates an overlap index 74(d) between the first detection result 71 and the second detection result 72 for the trigger object 70 among the selected trigger object 70 and non-trigger object 80. For comparison, the overlap index calculation unit 113 calculates an overlap index 84(d) between the first detection result 81 and the second detection result 82 for the non-trigger object 80.

[0150] The doubt index calculation unit 114 calculates the first group 73 (V ｔ ) is a first value 75 (Σ Ｖ∈Ｖｔ d(o;f Ｓ , f Ｔ The doubt index calculation unit 114 calculates the second group 83 (V ｔ ) is a second value 85 (Σ Ｖ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ) is calculated.

[0151] The doubt index calculation unit 114 calculates the first value 75 (Σ Ｖ∈Ｖｔ d(o;f Ｓ , f Ｔ )) and the second value 85 (Σ Ｖ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ )) shown in the annotation A2 as the suspicion index 90. As a result, the suspicion index calculation unit 114 calculates a suspicion index 90 (score) as to whether the verification target device 40 is a device that has been replicated from the object detection device 10 by machine learning (step S31). For example, the suspicion index calculation unit 114 compares the score(f Ｓ， f Ｔ ) = (Σ ｏ∈Ｖｔ d(o;f Ｓ , f Ｔ )) / (Σ ｏ∈Ｖｔ ｃ d(o;f Ｓ , f Ｔ )) = first value / second value.

[0152] The verification unit 115 checks whether the doubt index 90 (score) is equal to or greater than a predetermined value η (step S32). If the doubt index 90 score is equal to or greater than the predetermined value η (YES in step S32), the verification unit 115 determines that the model used by the verification target device 40 (fs) is a duplicated model (step S33). If the doubt index 90 score is less than the predetermined value η (NO in step S32), the verification unit 115 determines that the model used by the verification target device 40 (fs) is a non-duplicated model (step S34).

[0153] [E] Effects of an Embodiment According to a technique according to an embodiment, the control unit 100 acquires BBs 5a and 5b indicating the regions of the objects 3a and 3b in the input data 2, which are output from the object detection model 11 used by the object detection device 10. The control unit 100 acquires the attributes of the objects 3a and 3b. For a trigger object 70 (object 3a) that satisfies the conditions defined as a trigger, the control unit 100 causes the object detection device 10 to output output data 8 including BB7 obtained by changing BB 5a by a predetermined amount.

[0154] This makes it difficult for an attacker 21 (duplicator) to notice, while suppressing loss of object detection functionality, and allows poisoning data (contaminated data) to be injected into the duplication device 20 as a backdoor, thereby making it possible to identify the duplication device 20.

[0155] For example, the predetermined amount change includes a process of moving the position of BB5a, which indicates the area of ​​the object 3a, in the image output from the object detection device 10.

[0156] This makes it harder for the attacker 21 to notice that poisoning has occurred, compared to when the predicted label, etc. is altered. In addition, since there is no need to erase the BB 7a itself, loss of object detection function can be suppressed.

[0157] For example, the predetermined amount of change includes a process of enlarging BB5a, which indicates the area of ​​the object, in the image output from the object detection device 10.

[0158] This makes it harder for the attacker 21 to notice that poisoning has occurred, compared to when predictive labels or the like are altered. Also, since there is no need to erase the BB7b itself, loss of object detection functionality can be suppressed. By expanding the BB5a, it is possible to prevent objects from protruding from the expanded BB7b. In this respect, it is also possible to suppress loss of object detection functionality.

[0159] For example, the predetermined amount of change is adjusted according to the aspect ratio of the rectangle in BB5a.

[0160] Therefore, the rate of change is adjusted in BB5a where the aspect ratio is different from 1, so that the amount of change can be prevented from becoming too small.

[0161] The condition defined as the trigger is defined as a range of feature quantities 61 in a feature quantity space 60 having multiple dimensions. Therefore, compared to when explicit attributes such as the color or shape of an object are used as triggers, this can be treated as implicit information. Therefore, it is difficult for an attacker 21 (attacker) to notice, and poisoning data (contaminated data) can be injected into the duplication device 20 as a backdoor, making it possible to identify the duplication device 20.

[0162] According to one embodiment of the method, the control unit 100 selects multiple objects from the duplicate verification data 9. The selected objects have the same first predicted labels (56a, 56b) and second predicted labels (46a, 46b) for classifying the objects extracted by the reference device 50 and the verification target device 40, respectively. Furthermore, for the selected objects, BBs 57, 55b and BBs 45a, 45b overlap by a predetermined value or more for the object regions output by the reference device 50 and the verification target device 40, respectively. The control unit 100 verifies whether the verification target device 40 is a duplicated device through machine learning using the output of the reference device 50. The verification may be based on an overlap index 74 indicating the degree of overlap between the first detection result 71 and the second detection result 72 for a trigger object 70 having the attribute that satisfies the condition defined as a trigger among the selected objects.

[0163] This makes it possible to identify the replicating device 20 while preventing the attacker 21 from noticing and suppressing loss of object detection functionality.

[0164] The verification process may include a process of classifying the plurality of objects into a first group 73 including the trigger object 70 and a second group 83 other than the first group 73. The verification process may include a process of calculating a first value 75 that is the sum of the duplication indices 74 of the objects 70 included in the first group 73 and a second value 85 that is the sum of the duplication indices 84 of the objects 80 included in the second group 83. The verification process may include a process of comparing the first value 75 with the second value 85 to verify whether the verification target device 40 is a device that has been replicated by the machine learning.

[0165] This reduces the influence of errors by using the total of overlapping indices for multiple objects. Also, systematic errors that occur with the same frequency in both the first group including the trigger object 70 and the second group not including the trigger object 70 are canceled out, thereby improving the accuracy of verification.

[0166] [F] Others The disclosed technology is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present embodiment. The configurations and processes of the present embodiment can be selected or combined as needed.

[0167] REFERENCE SIGNS LIST 1 information processing device 1a CPU 1b memory 1c IF device 1d graphics processing device 1e storage device 1f IO device 1g reading device 1h program 1i recording medium 1j bus 2 input data 3, 3a, 3b, 43a, 43b, 53a, 53b object 4 output data 5, 5a, 5b, 45a, 45b, 55b bounding box 6, 6a, 6b, 6c predicted label 7, 7a, 7b, 57 bounding box 8, 8a, 8b, 8c, 8d output data 9 copy verification data 10 object detection device 11 object detection model 20 copy device 21 attacker 30 copy verification device 32 copy verification device 40 verification target device 41 output data 46a, 46b, 56a, 56b predicted label 50 Reference device 51 Output data 60 Feature space 61 Feature 62 Cluster 63 Trigger cluster 70, 70a, 70b, 70c Trigger object 71, 71a, 71b, 71c First detection result 72, 72a, 72b, 72c Second detection result 73 First group 74 Duplicate index 75 First value 80, 80a, 80b, 80c Non-trigger object 81, 81a, 81b, 81c First detection result 82, 82a, 82b, 82c Second detection result 83 Second group 84 Duplicate index 85 Second value 90 Suspicious index 100 Control unit 101 Input unit 102 Object list generation unit 103 First region information acquisition unit 104 Predicted label acquisition unit 105 Trigger cluster setting unit 106 Attribute acquisition unit 107 Trigger object discrimination unit 108 Second region information generation unit 109 Output data generation unit 111 Verification target device output acquisition unit 112 Object correspondence relationship determination unit 113 Overlapping index calculation unit 114 Suspicious index calculation unit 115 Verification unit 116 Reference device output acquisition unit 200 Storage unit 210 Object detection model 220 Object feature amount extraction model 230 Trigger cluster information storage unit

Claims

1. An object detection program that causes a computer to execute the following processes: acquire first region information indicating an object region in an input image output from a first object detection model used by a first object detection device; acquire attributes of the object; and, for a trigger object having the attributes that satisfy the conditions defined as a trigger, cause the first object detection device to output output data including second region information obtained by modifying the first region information by a predetermined amount.

2. The object detection program according to claim 1, wherein the predetermined amount of change includes a process of moving the position of a frame indicating the area of ​​the object within the image output from the first object detection device.

3. The object detection program according to claim 1, wherein the predetermined amount of change includes a process of enlarging a frame indicating the area of ​​the object in the image output from the first object detection device.

4. The object detection program according to claim 2 or 3, wherein the predetermined amount of change includes a process of adjusting the rate of change depending on the aspect ratio of the frame shape in the first region information.

5. The object detection program according to claim 1, wherein the condition defined as the trigger is defined as a range of features in a feature space having multiple dimensions.

6. A duplication verification program that causes a computer to execute the following processes: selecting a plurality of objects in an image for verification, in which the first predictive labels and second predictive labels for classifying objects extracted by the first object detection device and the second object detection device, respectively, match and the first detection results and second detection results for object areas output by each of the first object detection device and the second object detection device overlap by a predetermined value or more; acquiring attributes of the selected plurality of objects; and verifying whether the second object detection device is a device duplicated by machine learning using the output of the first object detection device, based on an overlap index indicating the degree of overlap between the first detection result and the second detection result for a trigger object among the selected plurality of objects, the trigger object having the attribute that satisfies a condition defined as a trigger.

7. The copy verification program according to claim 6, wherein the verification process includes the steps of: classifying the plurality of objects into a first group which is the trigger object, and a second group other than the first group; calculating a first value which is the sum of the overlapping indicators of the objects included in the first group, and a second value which is the sum of the overlapping indicators of the objects included in the second group; and verifying whether the second object detection device is a device that has been copied by the machine learning by comparing the first value with the second value.

8. A copy verification method in which a computer performs processes of: acquiring first region information indicating an object region in an input image output from a first object detection model used by a first object detection device; acquiring attributes of the object; for a trigger object having the attribute that satisfies a condition defined as a trigger, causing the first object detection device to output output data including second region information obtained by changing the first region information by a predetermined amount; selecting, in a verification image, a plurality of objects for which the first predictive labels and second predictive labels by which objects extracted by the first object detection device and the second object detection device are classified, respectively, match and for which the first detection results and second detection results for the object regions output by the first object detection device and the second object detection device overlap by a predetermined value or more; acquiring attributes of the selected plurality of objects; and verifying whether the second object detection device is a device that has been copied by machine learning using the output of the first object detection device, based on an overlap index indicating the degree of overlap between the first detection result and the second detection result for a trigger object having the attribute that satisfies a condition defined as a trigger among the selected plurality of objects.

9. The copy verification method according to claim 8, wherein the first detection result output by the first object detection device is an output by a first object detection model.

10. The copy verification method according to claim 8, wherein the predetermined amount of change includes a process of moving the position of a frame indicating the area of ​​the object within the image output from the first object detection device.

11. The copy verification method according to claim 8, wherein the predetermined change includes a process of enlarging a frame indicating the area of ​​the object in the image output from the first object detection device.

12. The copy verification method according to claim 10 or 11, wherein the predetermined change includes a process of adjusting the rate of change according to an aspect ratio of the frame shape in the first region information.

13. The method of claim 8, wherein the condition defined as the trigger is defined as a range of features in a feature space having multiple dimensions.

14. The copy verification method of claim 8, wherein the verification process includes the steps of: classifying the plurality of objects into a first group which is the trigger object, and a second group other than the first group; calculating a first value which is the sum of the overlapping indicators of the objects included in the first group, and a second value which is the sum of the overlapping indicators of the objects included in the second group; and verifying whether the second object detection device is a device that has been copied by the machine learning by comparing the first value with the second value.

15. An information processing device comprising a processor: acquiring first region information indicating an object region in an input image output from a first object detection model used by a first object detection device; acquiring attributes of the object; for a trigger object having the attribute that satisfies a condition defined as a trigger, outputting output data from the first object detection device including second region information obtained by changing the first region information by a predetermined amount; selecting, in a verification image, a plurality of objects for which the first predictive labels and second predictive labels by which objects extracted by the first object detection device and the second object detection device are classified, respectively, match and for which the first detection results and second detection results for the object regions output by the first object detection device and the second object detection device overlap by a predetermined value or more; acquiring attributes of the plurality of selected objects; and verifying whether the second object detection device is a device that has been replicated by machine learning using the output of the first object detection device, based on an overlap index indicating the degree of overlap between the first detection result and the second detection result for a trigger object having the attribute that satisfies a condition defined as a trigger among the plurality of selected objects.

16. The copy verification method according to claim 15, wherein the first detection result output by the first object detection device is an output by a first object detection model.

17. The information processing device according to claim 15, wherein the processor, in changing the predetermined amount, moves the position of a frame indicating the area of ​​the object in the image output from the first object detection device.

18. The information processing device according to claim 15, wherein the processor, in making the predetermined change, enlarges a frame indicating the area of ​​the object in the image output from the first object detection device.

19. The information processing device according to claim 17 or 18, wherein the processor adjusts the rate of change according to an aspect ratio of the frame shape in the first region information.

20. The information processing device according to claim 15, wherein the condition defined as the trigger is defined as a range of features in a feature space having multiple dimensions.

21. The information processing device of claim 15, wherein the processor, in the verification, classifies the plurality of objects into a first group which is the trigger object and a second group other than the first group, calculates a first value which is the sum of the overlapping indicators of the objects included in the first group and a second value which is the sum of the overlapping indicators of the objects included in the second group, and verifies whether the second object detection device is a device replicated by the machine learning by comparing the first value with the second value.

Images