Detection device, detection method, and detection program

Abstract

To provide a detection device capable of facilitating the detection of data shifts between image data groups.SOLUTION: The detection device includes: a processor for executing programs; and a storage device for storing the program. The detection device executes: acquisition processing to acquire a first contribution value map group that is a set of first contribution value maps contributing to the characteristics of first image data in each first image data group and a second contribution value map group that is a set of a second contribution value maps contributing to the characteristics of second image data in each of a second image data group picked up after the first image data group; first two-sample test processing of executing a two-sample test by using the first contribution value map group and the second contribution value map group acquired by the acquisition processing; and output processing of outputting the result of the two-sample test by the first two-sample test processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a detection device, a detection method, and a detection program. [Background technology]

[0002] The content and characteristics of input data may change from when it was learned. Such changes are called data shifts (or data drift). For example, if the input data is image data, the passage of time between when the judgment model was learned and when it is currently being used could result in changes in people's clothing, changes in the races of people passing by the shooting location, a deterioration in camera image quality, or construction work on buildings or roads at the shooting location, all of which are examples of data shifts. Although the occurrence of a data shift does not affect accuracy at the time, there is a risk of a potential deterioration in the accuracy of the learning model. Manual methods of detecting and determining data shifts are not realistic due to the labor costs involved, and an automatic or semi-automatic method is desirable.

[0003] Moreover, the information processing device of Patent Document 1 below uses a plurality of training data corresponding to a plurality of correct labels to learn an operation model to be monitored. The information processing device learns a decision boundary that divides the feature space of the data into two application regions based on the output result of the operation model, and creates an inspector model that calculates the distance from the decision boundary to the input data. The information processing device inputs the training data to the inspector model and calculates a first distance from the decision boundary to the training data. The information processing device inputs the operation data to the inspector model and calculates a second distance from the decision boundary to the operation data. When the difference between the first distance and the second distance is equal to or greater than a preset threshold, the information processing device detects a change in the output result of the operation model caused by a time change in the trend of the operation data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 079460 Summary of the Invention [Problem to be solved by the invention]

[0005] There are various data detection methods that use structured data such as table data as input data, but when the input data is data with multidimensional features such as image data, the accuracy of data shift detection decreases. In addition, in Patent Document 1, a change in the ratio of detected features is considered to be a concept drift detection, and a change in this label is considered to be the cause. However, when there are multiple changes in the ratio of detected features, it is not possible to identify which one is the concept drift.

[0006] An object of the present invention is to facilitate detection of data shifts between image data groups. [Means for solving the problem]

[0007] A detection device which is one aspect of the invention disclosed in the present application is a detection device having a processor which executes a program and a storage device which stores the program, and is characterized in that the detection device executes an acquisition process which acquires a first contribution value map group which is a collection of first contribution value maps which contributed to the characteristics of each first image data of a first image data group, and a second contribution value map group which is a collection of second contribution value maps which contributed to the characteristics of each second image data of a second image data group which was captured after the first image data group, a first two-sample testing process which performs a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process, and an output process which outputs the two-sample testing results obtained by the first two-sample testing process.

[0008] A detection device which is another aspect of the invention disclosed in the present application is a detection device having a processor which executes a program and a storage device which stores the program, and is characterized in that the detection device executes an acquisition process which acquires a first contribution value map group which is a collection of first contribution value maps which contributed to the characteristics of each first image data of a first image data group, and a second contribution value map group which is a collection of second contribution value maps which contributed to the characteristics of each second image data of a second image data group which was captured after the first image data group, a difference generation process which generates a difference map showing the difference between a representative first contribution value map in the first contribution value map group acquired by the acquisition process and a representative second contribution value map in the second contribution value map group, and an output process which outputs the difference map generated by the difference generation process. Effect of the Invention

[0009] According to the representative embodiment of the present invention, it is possible to facilitate detection of data shift between image data groups. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of the device. [Diagram 2] FIG. 2 is a block diagram of an example of a functional configuration of the detection device according to the first embodiment. [Diagram 3] FIG. 3 is an explanatory diagram showing an example of the operation of XAI. [Figure 4] FIG. 4 is an explanatory diagram showing an example of generating a difference map. [Diagram 5] FIG. 5 is a flowchart of a first example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 6] FIG. 6 is a flowchart of a second example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart of a third example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing an example of a change in the ratio of the matching result between the subject of study and the cause candidate. [Figure 9] FIG. 9 is a block diagram of an example of a functional configuration of the detection device according to the second embodiment. [Figure 10] FIG. 10 is an explanatory diagram showing Example 1 of map classification A based on the combination feature determination result. [Figure 11] FIG. 11 is an explanatory diagram showing Example 2 of map classification A based on the combination feature determination result. [Figure 12] FIG. 12 is an explanatory diagram showing an example of a two-sample test using map classification A. [Figure 13] FIG. 13 is an explanatory diagram showing Example 1 of map classification B based on the combination feature determination result. [Figure 14] FIG. 14 is an explanatory diagram showing Example 2 of map classification B based on the combination feature determination result. [Figure 15] FIG. 15 is an explanatory diagram showing an example of a two-sample test using map classification B. [Figure 16] FIG. 16 is a flowchart of an example (first half) of a data shift detection process performed by the detection device according to the second embodiment. [Figure 17] FIG. 17 is a flowchart of an example (second half) of the data shift detection process performed by the detection device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0011] In the first embodiment, a detection device will be described which detects the frequency of appearance of a feature in image data when the input data is image data, or detects a data shift regarding a feature across the entire image data.

[0012] <Hardware configuration example of detection device> FIG. 1 is a block diagram showing an example of a hardware configuration of the device. The detection device 100 includes a processor 101, a storage device 102, an input device 103, an output device 104, and a communication interface (communication IF) 105. The processor 101, the storage device 102, the input device 103, the output device 104, and the communication IF 105 are connected by a bus 106. The processor 101 controls the detection device 100. The storage device 102 serves as a working area for the processor 101. The storage device 102 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 102 include a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and a flash memory. The input device 103 inputs data. Examples of the input device 103 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 104 outputs data. The output device 104 may be, for example, a display, a printer, or a speaker. The communication IF 105 connects to a network and transmits and receives data.

[0013] <Example of functional configuration of detection device> 2 is a block diagram showing an example of a functional configuration of the detection device 100 according to the first embodiment. The detection device 100 includes an acquisition unit 201, a map generation unit 202, a difference generation unit 203, an output unit 204, a two-sample test unit 205, and a data shift determination unit 206. Specifically, the acquisition unit 201 to the data shift determination unit 206 are functions that are realized by causing the processor 101 to execute a program stored in the storage device 102 shown in FIG.

[0014] The acquisition unit 201 acquires image data. Specifically, for example, the acquisition unit 201 acquires a comparison source image data group D1, a pre-shift image data group D2, and a detection target image data group D3. The comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 may be stored in a storage device 102 in the detection device 100, or may be stored in a computer outside the detection device 100. When stored in a computer outside the detection device 100, the acquisition unit 201 receives the comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 from the computer outside the detection device 100.

[0015] The comparison source image data group D1 is a set of comparison source image data. The comparison source image data is image data used to learn a feature determination model that determines the features of the image data. For example, when the feature determination model is a gender determination model that determines the gender of a person in image data captured by a surveillance camera, the image data used to learn the gender determination model is the comparison source image data.

[0016] The pre-shift image data group D2 is a collection of pre-shift image data. The pre-shift image data is image data captured after the comparison source image data is captured. The pre-shift image data may be captured at the same location as the comparison source image data or at a different location. The pre-shift image data group D2 may be a part of the comparison source image data group D1.

[0017] The detection target image data group D3 is a collection of detection target image data. The detection target image data is image data that is the target of detection of whether or not a data shift has occurred. The detection target image data is image data captured after the pre-shift image data. The detection target image data may be captured at the same location as the pre-shift image data or at a different location.

[0018] When there is no need to distinguish between the comparison source image data, the pre-shift image data, and the detection target image data, they will be simply referred to as image data.

[0019] The map generation unit 202 generates a contribution value map that indicates the distribution of the feature amounts of the image data. The map generation unit 202 is one function of the feature determination model. When image data is input, the feature determination model is an eXplainable Artificial Intelligence (XAI) that outputs a feature determination result and calculates, for each pixel region, a contribution value that indicates how much each pixel region in the image data contributed to the determination result. A pixel region is a set of one or more pixels. The contribution value map is data in which the contribution value calculated for each pixel region is arranged in that pixel region.

[0020] The map generating unit 202 outputs a soft label obtained by compressing the dimensions of the image data as a result of the dimensional compression of the image data, and this soft label is a contribution value for each pixel region. The soft label is, for example, input data to a fully connected layer at the final stage of the feature determination model that outputs the determination result, or to an intermediate layer before that. The generated contribution value map is stored in the contribution value map DB 210.

[0021] The difference generating unit 203 calculates the difference between the two contribution value maps and generates a difference map. One of the two contribution value maps is, for example, an average contribution value map obtained by averaging the contribution value maps obtained from each comparison source image data in the comparison source image data group D1, or an average contribution value map obtained by averaging the contribution value maps obtained from each pre-shift image data in the pre-shift image data group D2. The other of the two contribution value maps is, for example, an average contribution value map obtained by averaging the contribution value maps obtained from each detection target image data in the detection target image data group D3.

[0022] The output unit 204 outputs data. Specifically, for example, the output unit 204 displays the data on a display, which is an example of the output device 104, or prints out the data from a printer. The output unit 204 also transmits the data to another computer via the communication IF 105. When outputting a map, the output unit 204 can also output the map as a heat map.

[0023] The data to be output from the output unit 204 includes, for example, the difference map from the difference generation unit 203, two contribution value maps to be subjected to the difference by the difference generation unit 203, the test result of the two-sample test unit 205, and the judgment result from the data shift judgment unit 206.

[0024] The two-sample testing unit 205 executes a two-sample test on the two distributions and calculates a p-value. Specifically, for example, the two-sample testing unit 205 executes a hypothesis test on the kernel mean difference (MMD: Maximum Mean Discrepancy) for the two distributions. Then, the two-sample testing unit 205 judges whether or not to reject the null hypothesis H0 based on the p-value. In this example, the null hypothesis H0 is a hypothesis that there is no difference between the two distributions. For example, when the p-value is 0.05 or less, the two-sample testing unit 205 rejects the null hypothesis H0 at a significance level of 5%.

[0025] Here, there are two sets of two distributions. In the first set of two distributions, one distribution is a group of contribution value maps obtained from each comparison source image data in the comparison source image data group D1. The other distribution is a group of contribution value maps obtained from each pre-shift image data in the pre-shift image data group D2.

[0026] In the second set of two distributions, one distribution is a group of contribution value maps obtained from each pre-shift image data of the pre-shift image data group D2, and the other distribution is a group of contribution value maps obtained from each detection object image data of the detection object image data group D3.

[0027] In addition, in the second set of two distributions, when a group of contribution value maps obtained from each comparison source image data of the comparison source image data group D1 is used instead of a group of contribution value maps obtained from each pre-shift image data of the pre-shift image data group D2, the two sample testing unit 205 does not need to execute a two sample test for the first set of two distributions. This can reduce the calculation load of the detection device 100.

[0028] The data shift determination unit 206 determines whether a data shift has occurred between the pre-shift image data group D2 (which may be the comparison source image data group D1) and the detection target image data group D3. Specifically, for example, when the two-sample testing unit 205 adopts the null hypothesis H0 in the first set of two distributions and rejects the null hypothesis H0 in the second set of two distributions, the data shift determination unit 206 determines that a data shift has occurred between the pre-shift image data group D2 (which may be the comparison source image data group D1) and the detection target image data group D3.

[0029] <xai> FIG. 3 is an explanatory diagram showing an example of the operation of XAI. XAI300 has feature determination models M1 to Mn (n is an integer equal to or greater than 1). When the feature determination models M1 to Mn are not distinguished from one another, they are referred to as feature determination model M. The feature determination model M is a learning model trained using a comparison source image data group D1 and its correct answer data. The feature determination model M includes learning models that determine various features, such as a gender determination model, a lower body clothing determination model, a footwear determination model, and so on.

[0030] The XAI 300 is an explainable AI that outputs a feature determination result from the feature determination model M and also outputs a contribution value map. The contribution value map is an evaluation value that indicates which pixel region of the input image data contributed to the feature determination result.

[0031] The feature judgment model M outputs a feature judgment result when image data is input. In Fig. 3, when each comparison source image data of the comparison source image data group D1 is input to the feature judgment model M1, the feature judgment model M1 outputs a feature judgment result group R1 (M1) which is a collection of feature judgment results of each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature judgment model M1, the feature judgment model M1 outputs a feature judgment result group R2 (M1) which is a collection of feature judgment results of each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature judgment model M1, the feature judgment model M1 outputs a feature judgment result group R3 (M1) which is a collection of feature judgment results of each detection target image data.

[0032] When each comparison source image data of the comparison source image data group D1 is input to the feature judgment model Mn, it outputs a feature judgment result group R1(Mn), which is a collection of feature judgment results of each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature judgment model Mn, it outputs a feature judgment result group R2(Mn), which is a collection of feature judgment results of each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature judgment model Mn, it outputs a feature judgment result group R3(Mn), which is a collection of feature judgment results of each detection target image data.

[0033] In addition, the feature determination model M generates a contribution value map when image data is input. In Fig. 3, when each comparison source image data of the comparison source image data group D1 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C1 (M1) which is a collection of contribution value maps of each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C2 (M1) which is a collection of contribution value maps of each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C3 (M1) which is a collection of contribution value maps of each detection target image data.

[0034] When each comparison source image data of the comparison source image data group D1 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C1(Mn), which is a collection of contribution value maps of each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C2(Mn), which is a collection of contribution value maps of each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C3(Mn), which is a collection of contribution value maps of each detection target image data.

[0035] <Example of difference map generation> 4 is an explanatory diagram showing an example of generating a difference map. First, the map generating unit 202 generates a contribution value map group C1(Mi), a contribution value map group C2(Mi), and a contribution value map group C3(Mi). Mi is the i-th (1≦i≦n) feature determination model.

[0036] The two-sample testing section 205 performs a two-sample test on the contribution value map group C1(Mi) and the contribution value map group C2(Mi) as a first set of two distributions. The two-sample testing section 205 also performs a two-sample test on the contribution value map group C2(Mi) and the contribution value map group C3(Mi) as a second set of two distributions.

[0037] If the result of the first set of two-sample testing is the acceptance of the null hypothesis and the result of the second set of two-sample testing is the rejection of the null hypothesis, the data shift determination unit 206 determines that a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3.

[0038] When the data shift determination unit 206 determines that a data shift has occurred, the difference generation unit 203 averages the contribution value map group C2(Mi) to generate an average contribution value map C2(Mi)av, and averages the contribution value map group C3(Mi) to generate an average contribution value map C3(Mi)av. Then, it generates a difference map DC(Mi) which is the difference between both average contribution value maps C2(Mi)av, C3(Mi)av.

[0039] Here, the average contribution value map C2(Mi) is a contribution value map obtained by averaging the contribution values ​​at the same position in each of the contribution value maps in the contribution value map group C2(Mi).Similarly, the average contribution value map C3(Mi) is a contribution value map obtained by averaging the contribution values ​​at the same position in each of the contribution value maps in the contribution value map group C3(Mi).

[0040] In FIG. 4, the difference generating unit 203 averages the contribution value map group C2(Mi) to generate the average contribution value map C2(Mi)av, and averages the contribution value map group C3(Mi) to generate the average contribution value map C3(Mi)av, but is not limited to averaging. For example, the difference generating unit 203 may extract a representative contribution value map from each of the contribution value map group C2(Mi) and the contribution value map group C3(Mi). The representative contribution value map may be a randomly selected contribution value map in addition to the averaging described above, a contribution value map in which the sum of the contribution values ​​in the contribution value map is the maximum, a contribution value map in which the sum of the contribution values ​​in the contribution value map is the minimum, or a contribution value map in which the sum of the contribution values ​​in the contribution value map is the median.

[0041] <Example of data shift detection processing procedure> 5 is a flowchart illustrating a first example of a data shift detection process performed by the detection device 100 according to the first embodiment. The detection device 100 designates an object of consideration (step S501). The object of consideration is a feature of image data for which data shift detection is to be considered. Here, gender is used as an example.

[0042] The detection apparatus 100 acquires, via the acquisition unit 201, a comparison source image data group D1, a pre-shift image data group D2, and a detection target image data group D3 (step S502).

[0043] The detection device 100 inputs the comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 into the feature determination model M under consideration by the map generation unit 202, and generates respective contribution value map groups C1(Mi), C2(Mi), and C3(Mi) (step S503).

[0044] The detection apparatus 100 uses the difference generation unit 203 to calculate average contribution value maps C1(Mi)av, C2(Mi)av, and C3(Mi)av for each of the contribution value map groups C1(Mi), C2(Mi), and C3(Mi), and generates a difference map DC(Mi) between the average contribution value map C2(Mi)av (or C1(Mi)av) and the average contribution value map C3(Mi)av (step S504).

[0045] The detection device 100 outputs the difference map DC(Mi) by the output unit 204 (step S505), and ends the series of processes. By outputting the difference map DC(Mi) in this way, the user can visually recognize the difference location as a region causing the data shift, and can utilize the difference location for improving the feature determination model M. For example, the feature determination model M can be re-trained by using detection target image data in which the difference location appears as learning data. This makes it possible to facilitate the detection of data shifts.

[0046] 6 is a flowchart illustrating a data shift detection processing procedure example 2 performed by the detection device 100 according to the first embodiment. The same processes as those in the data shift detection processing procedure example 2 of FIG. 5 are denoted by the same step numbers, and descriptions thereof will be omitted.

[0047] After step S503, the detection device 100 causes the two-sample testing unit 205 to perform a first two-sample test using the contribution value map groups C1(M) and C2(M), and then performs a second two-sample test using the contribution value map groups C2(M) and C3(M) (step S604).

[0048] The detection device 100 determines whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 based on the test result of the first two-sample test and the test result of the second two-sample test (step S605). If the test result of the first two-sample test is the null hypothesis acceptance and the test result of the second two-sample test is the null hypothesis rejection, the detection device 100 determines by the data shift determination unit 206 that a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3. If the test results of the first two-sample test and the second two-sample test are other than those, the detection device 100 determines by the data shift determination unit 206 that a data shift has not occurred between the pre-shift image data group D2 and the detection target image data group D3.

[0049] The detection device 100 outputs the data shift determination result of the data shift determination unit 206 through the output unit 204 (step S606), and ends a series of processes. In this way, the detection device 100 can improve the accuracy of data shift detection. In addition, the user can check whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 for the subject of consideration.

[0050] 7 is a flowchart illustrating a data shift detection processing procedure example 3 performed by the detection device 100 according to the first embodiment. The same processes as those in the data shift detection processing procedure examples 1 and 2 in FIGS. 5 and 6 are denoted by the same step numbers, and descriptions thereof will be omitted.

[0051] After step S604, the detection device 100 determines whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 based on the test results of the first two-sample test and the test results of the second two-sample test by the data shift determination unit 206 (step S605). If it is determined that a data shift has occurred (step S605: Yes), the detection device 100 executes steps S504 and S505 and ends the series of processes.

[0052] In this way, by outputting the difference map DC(M), the user can visually recognize the difference locations as areas causing data shifts, and can utilize the difference locations to improve the feature determination model M. For example, the feature determination model M can be retrained by using detection target image data in which difference locations appear as learning data. This makes it possible to facilitate detection of data shifts.

[0053] On the other hand, when it is determined that no data shift has occurred (step S605: No), the detection device 100 outputs the data shift determination result (no data shift has occurred) of the data shift determination unit 206 through the output unit 204 (step S706), and ends the series of processes. In this way, the detection device 100 can improve the accuracy of data shift detection. In addition, the user can check whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 for the subject of consideration.

[0054] In the first embodiment, the map generating unit 202 generates the contribution value map, but the acquiring unit 201 may acquire the contribution value map from the contribution value map DB 210, and the two-sample testing unit 205 may execute the two-sample testing using the acquired contribution map. Also, the acquiring unit 201 may acquire the contribution value map from an external computer via the communication IF 105, and the two-sample testing unit 205 may execute the two-sample testing using the acquired contribution map. EXAMPLES

[0055] A description will now be given of Example 2. In Example 1, the detection device 100 has been described which detects the frequency of appearance of a feature in image data when the input data is image data, or detects a data shift regarding a feature across the entire image data.

[0056] However, there may be cases where the cause of a data shift cannot be identified from the state of the average contribution value map (the pixel region (causing region) in the average contribution value map that is the cause of the data shift and its contribution value). This is because, in the average contribution value map, the ratio change before and after the data shift cannot be observed due to the overlapping or cancellation of contribution values ​​in the contribution value map group, or it may be difficult to pinpoint the causative region due to differences in characteristics between image data.

[0057] In the second embodiment, a detection device is described that identifies the cause of the data shift by comparing the feature determination result of the feature to be examined with the feature determination results of other features that are potential causes other than the feature to be examined. In the second embodiment, as an example, the feature to be examined is the "gender (male or female)" of the person in the image data, and the other features that are potential causes are the "lower body clothing (pants or skirt)" and "footwear (sneakers or sandals)" of the person in the image data. Note that the same components as those in the first embodiment are given the same reference numerals, and the description thereof is omitted.

[0058] <Changes in the ratio of matching results> 8 is an explanatory diagram showing an example of a change in the ratio of the matching result between the subject of consideration and the candidate cause. (A) shows an example of a change in the ratio of the matching result between the subject of consideration (gender) and the candidate cause (lower body clothing), and (B) shows an example of a change in the ratio of the matching result between the subject of consideration (gender) and the candidate cause (footwear). In the second embodiment, the symbol a (1≦a≦n) indicates data related to the subject of consideration (gender), the symbol b (1≦b≦n, b≠a) indicates data related to the subject of consideration (lower body clothing), and the symbol c (1≦c≦n, c≠a, c≠b) indicates data related to the subject of consideration (footwear).

[0059] In (A), the detection device 100, by using XAI300, inputs the pre-shift image data group D2 into a gender determination model Ma, which is one of the feature determination models M, to obtain a gender determination result group R2(Ma), and inputs the pre-shift image data group D2 into a lower-body clothing determination model Mb, which is one of the feature determination models M, to obtain a lower-body clothing determination result group R2(Mb).

[0060] The gender determination result group R2(Ma) is a set of gender determination results for each pre-shift image data in the pre-shift image data group D2. The lower body clothing determination result group R2(Mb) is a set of lower body clothing determination results for each pre-shift image data in the pre-shift image data group D2.

[0061] The gender determination model Ma determines gender based on the shape of the upper body, such as the face, hairstyle, arm width, shoulder width, chest, waist, etc. The lower body clothing determination model Mb determines whether the lower body clothing of the person is pants or a skirt.

[0062] The detection device 100 compares the gender determination result group R2(Ma) with the lower body clothing determination result group R2(Mb). Specifically, for example, the detection device 100 uses the gender determination result group R2(Ma) and the lower body clothing determination result group R2(Mb) to count the number of pre-shift image data corresponding to each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the candidate cause (lower body clothing).

[0063] As a result, a matching result T2ab is generated. The matching result T2ab indicates that, for the people in the 200 pieces of pre-shift image data, 100 pieces of pre-shift image data are determined to be male and pants, 0 pieces of pre-shift image data are determined to be male and skirt, 0 pieces of pre-shift image data are determined to be female and pants, and 100 pieces of pre-shift image data are determined to be female and skirt.

[0064] In addition, the detection device 100 inputs the detection target image data group D3 into a gender determination model Ma using XAI300, thereby obtaining a gender determination result group R3(Ma), and inputs the detection target image data group D3 into a lower body clothing determination model Mb, thereby obtaining a lower body clothing determination result group R3(Mb).

[0065] The gender determination result group R3(Ma) is a set of gender determination results for each detection target image data in the detection target image data group D3. The lower body clothing determination result group R3(Mb) is a set of lower body clothing determination results for each detection target image data in the detection target image data group D3.

[0066] The detection device 100 compares the gender determination result group R3(Ma) with the lower body clothing determination result group R3(Mb). Specifically, for example, the detection device 100 uses the gender determination result group R3(Ma) and the lower body clothing determination result group R3(Mb) to count the number of detection target image data corresponding to each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the candidate cause (lower body clothing).

[0067] As a result, matching result T3ab is generated. Matching result T3ab indicates that, for the people in the 200 detection target image data, 80 detection target image data were determined to be male and pants, 20 detection target image data were determined to be male and skirt, 20 detection target image data were determined to be female and pants, and 80 detection target image data were determined to be female and skirt.

[0068] The detection device 100 compares the matching results T2ab and T3ab to determine whether or not there has been a ratio change. Specifically, for example, when an increase or decrease of x% occurs in at least one of the four combination features of the matching results T2ab and T3ab, the detection device 100 determines that there has been a ratio change between the matching results T2ab and T3ab.

[0069] For example, when x=10(%), the combination feature of male and pants changes from "100" in the matching result T2ab to "80" in the matching result T3ab. In other words, a ratio change of 20% occurs, so the detection device 100 determines that a ratio change has occurred between the matching results T2ab and T3ab.

[0070] In the above embodiment, the item for determining the ratio change is at least one of the four combination features of the matching results T2ab and T3ab, but the item may be at least two of the four combination features, at least three of the four combination features, or all four combination features, depending on the user's settings. The more combination features to be determined, the stricter the determination of the ratio change becomes.

[0071] In (B), the lower body clothing determination model Mb is replaced with a footwear determination model Mc, and the detection device 100, as in (A), inputs the pre-shift image data group D2 to the gender determination model Ma by the XAI300 to obtain a gender determination result group R2(Ma), inputs the pre-shift image data group D2 to the footwear determination model Mc to obtain a footwear determination result group R2(Mc), and inputs the detection target image data group D3 to the footwear determination model Mc to obtain a footwear determination result group R3(Mc). Then, the detection device 100 generates matching results T2ac, T3ac and determines whether or not there is a ratio change. The footwear determination model Mc determines whether the footwear on the person's feet is sneakers or sandals.

[0072] Here, assume that there is a change in the ratio in both cases (A) and (B). In this case, it is not possible to determine whether the cause of the data shift is the lower body clothing, the footwear, or both the lower body clothing and footwear. Below, a configuration for determining whether the cause of the data shift is the lower body clothing, the footwear, or both the lower body clothing and footwear will be described.

[0073] <Example of Functional Configuration of Detection Device 100> 9 is a block diagram showing an example of a functional configuration of a detection device 100 according to a second embodiment. The detection device 100 includes an acquisition unit 201, a map generation unit 202, a difference generation unit 203, an output unit 204, a two-sample test unit 205, a data shift determination unit 206, a feature determination unit 901, a learning unit 902, and a classification unit 903. Specifically, the feature determination unit 901, the learning unit 902, and the classification unit 903 are functions that are realized by, for example, causing the processor 101 to execute a program stored in the storage device 102 shown in FIG.

[0074] The feature determination unit 901 determines whether the image data has features. Specifically, for example, the feature determination unit 901 is a part of the XAI 300, and outputs feature determination result groups R1 to R3 of the feature determination model M shown in Fig. 3. For example, as shown in Fig. 8, if the feature determination model M is a gender determination model Ma, the feature determination unit 901 outputs gender determination result groups R2(Ma) and R3(Ma), if the feature determination model M is a lower body clothing determination model Mb, the feature determination unit 901 outputs lower body clothing determination result groups R2(Mb) and R3(Mb), and if the feature determination model M is a footwear determination model Mc, the feature determination unit 901 outputs footwear determination result groups R2(Mc) and R3(Mc).

[0075] The learning unit 902 learns using the detection target image data group D3 and the matching results thereof, and generates a feature determination model that combines the subject of study and the cause candidate (hereinafter referred to as a combined feature determination model). If a combined feature determination model already exists in the feature determination models M1 to Mn, the learning unit 902 re-learns the combined feature determination model using the detection target image data group D3 and the matching results thereof.

[0076] In the case of FIG. 8A, the learning unit 902 uses the detection target image data group D3 and the matching result T3ab to generate a combination feature determination model Mab that combines gender and lower body clothing.

[0077] In the case of FIG. 8B, the learning unit 902 generates a combination feature determination model Mac that combines gender and footwear, using the detection target image data group D3 and the matching result T3ac thereof.

[0078] In the case of combining (A) and (B) in Figure 8, the learning unit 902 uses the detection target image data group D3 and its matching results T3ab, T3ac to generate a combination feature determination model Mabc that combines gender, lower body clothing, and footwear.

[0079] The classification unit 903 classifies the contribution value map group based on the matching result. Specifically, for example, the classification unit 903 classifies the contribution value map group C3(Ma) based on the matching result T3ab. The contribution value map group C3(Ma) is a collection of contribution value maps of each detection target image data output from the XAI300 as a result of inputting each detection target image data of the detection target image data group D3 to the feature determination model Ma.

[0080] Similarly, the classification unit 903 classifies the contribution value map group C3(Ma) based on the matching result T3ac.

[0081] Further, the classification unit 903 classifies, based on the matching result T3ab, a contribution value map group C3(Mab) output from the combination feature determination model Mab learned by the learning unit 902. The contribution value map group C3(Mab) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mab.

[0082] Similarly, the classification unit 903 classifies, based on the matching result T3ac, the contribution value map group C3(Mac) output from the combination feature determination model Mac learned by the learning unit 902. The contribution value map group C3(Mac) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mac.

[0083] Similarly, the classification unit 903 classifies the contribution value map group C3(Mabc) output from the combination feature determination model Mabc learned by the learning unit 902 based on the matching result T3abc. The matching result T3abc is a matching result obtained by aggregating the matching result T3ab and the matching result T3ac. The contribution value map group C3(Mabc) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mabc.

[0084] <Example of identifying the cause based on the results of combination feature judgment> First, a first example of cause identification based on a combined feature determination result in a case where the subject of investigation is gender and the cause candidate is lower body clothing will be described with reference to Figs. 10 to 12.

[0085] [Map classification A based on combined feature judgment results] Fig. 10 is an explanatory diagram showing example 1 of map classification A based on combined feature determination results. In Fig. 10, the detection device 100 classifies the contribution value map group C3(Ma) by the classification unit 903 using the matching result T3ab between the gender determination result group R3(Ma) obtained as a result of inputting the detection target image data group D3 into the gender determination model Ma and the lower body clothing determination result group R3(Mb) obtained as a result of inputting the detection target image data group D3 into the lower body clothing determination model Mb.

[0086] Specifically, for example, the detection device 100 classifies the contribution value map group C3(Ma) into a first contribution value map group C3(Ma)-b1 aggregated as male and trousers, a second contribution value map group C3(Ma)-b2 aggregated as male and skirt, a third contribution value map group C3(Ma)-b3 aggregated as female and trousers, and a fourth contribution value map group C3(Ma)-b4 aggregated as female and skirt.

[0087] Although not shown in the figure, the detection device 100 classifies the contribution value map group C3(Ma) using the matching result T3ac between the gender determination result group R3(Ma) and the footwear determination result group R3(Mc) obtained by inputting the detection target image data group D3 to the footwear determination model Mc, in the same manner as in Fig. 10. In this manner, the classification result 1000 is obtained.

[0088] Fig. 11 is an explanatory diagram showing example 2 of map classification A based on the combination feature determination result. In Fig. 11, the detection device 100 trains a combination feature determination model Mab using the detection target image data group D3 and the matching result T3ab as a learning data set by the learning unit 902. When the detection target image data group D3 is input, the combination feature determination model Mab generates a contribution value map group C3(Mab).

[0089] The detection device 100 classifies the contribution value map group C3(Mab) by using the matching result T3ab through the classification unit 903. Specifically, for example, the detection device 100 classifies the contribution value map group C3(Mab) into a first contribution value map group C3(Mab)-1 aggregated as male and trousers, a second contribution value map group C3(Mab)-2 aggregated as male and skirt, a third contribution value map group C3(Mab)-3 aggregated as female and trousers, and a fourth contribution value map group C3(Mab)-4 aggregated as female and skirt. In this manner, the classification result 1100 is obtained.

[0090] [Example of two-sample test using map classification A] 12 is an explanatory diagram showing an example of a two-sample test using map classification A. The detection device 100 executes two-sample tests 1201 to 1204 for each common combination feature using the two-sample test unit 205 based on the classification result 1000 of the contribution value map group C3 (Ma) and the classification result 1100 of the contribution value map group C3 (Mab).

[0091] Specifically, for example, the detection device 100 performs a two-sample test 1201 using the first contribution value map group C3(Ma)-b1 and the first contribution value map group C3(Mab)-1, whose combined features are male and trousers, as two distributions to be subjected to the two-sample test. The detection device 100 also performs a two-sample test 1202 using the second contribution value map group C3(Ma)-b2 and the second contribution value map group C3(Mab)-2, whose combined features are male and skirt, as two distributions to be subjected to the two-sample test.

[0092] Furthermore, the detection device 100 performs a two-sample test 1203 using the third contribution value map group C3(Ma)-b3 and the third contribution value map group C3(Mab)-3, whose combined features are female and trousers, as two distributions to be subjected to the two-sample test. The detection device 100 performs a two-sample test 1204 using the fourth contribution value map group C3(Ma)-b4 and the fourth contribution value map group C3(Mab)-4, whose combined features are female and skirt, as two distributions to be subjected to the two-sample test.

[0093] The detection apparatus 100 determines, through the data shift determination unit 206, whether or not a data shift has occurred in the cause candidate, based on the two-sample test results in each of the two-sample tests 1201 to 1204.

[0094] Specifically, for example, the detection device 100 determines whether or not a data shift has occurred in the cause candidate based on the number of times the null hypothesis is adopted in each two-sample test result by the data shift determination unit 206. For example, the detection device 100 may determine that the cause candidate is not the cause of the data shift occurrence when the null hypothesis is adopted in at least k (k is an integer not less than 1 and not more than the number of two-sample tests) of the multiple two-sample tests 1201-1204 (first determination rule). Also, the detection device 100 may determine that the cause candidate is the cause of the data shift occurrence when the null hypothesis is rejected in at least k (k is an integer not less than 1 and not more than the number of two-sample tests) of the multiple two-sample tests 1201-1204 (second determination rule). Furthermore, the detection device 100 may determine, by majority vote between the number of times the null hypothesis is accepted and the number of times it is rejected, that the candidate cause is not the cause of the data shift if the number of times the null hypothesis is accepted is greater, and may determine that the candidate cause is the cause of the data shift if the number of times the null hypothesis is rejected is greater (third determination rule).

[0095] In the example of Fig. 12, the null hypothesis is determined to be adopted in all four two-sample tests 1201 to 1204. Therefore, when the first determination rule is applied, the detection device 100 determines by the data shift determination unit 206 that the lower-body clothing, which is a cause candidate, is not the cause of the data shift. On the other hand, even when the second determination rule is applied, the detection device 100 determines by the data shift determination unit 206 that the lower-body clothing, which is a cause candidate, is not the cause of the data shift. It is possible for the user to set which of the first and second determination rules to adopt.

[0096] Furthermore, the detection apparatus 100 may exclude from the data shift determination a two-sample test result of a two-sample test in which the difference in the number of contribution value maps in two distributions subject to the two-sample test exceeds an allowable range among the multiple two-sample tests 1201 to 1204. In the example of Fig. 12, the difference is 0 in all two-sample test results, so none of the two-sample test results are excluded from the data shift determination.

[0097] In addition, the detection apparatus 100 may exclude the results of a two-sample test from the multiple two-sample tests 1201 to 1204 in which the total number of contribution value maps in the two distributions subject to the two-sample test is equal to or less than a threshold value from the data shift judgment.

[0098] For example, in the example of FIG. 12, if the threshold value is, for example, 50 (the number of image data: 25% of 200), the results of the two-sample tests 1202 and 1203 are not subject to data shift determination.

[0099] Note that in Figures 10 to 12, Example 1 of cause identification based on the combined feature determination result is explained when the subject of consideration is gender and the candidate cause is lower body clothing. Although not shown in the figure, the same procedure can also be performed when the subject of consideration is gender and the candidate cause is footwear.

[0100] Next, a second example of identifying a cause based on a combination feature determination result will be described with reference to Figs. 13 to 15, in which the subject of investigation is gender and the possible causes are lower body clothing and footwear.

[0101] [Map classification based on combined feature judgment results B] Fig. 13 is an explanatory diagram showing example 1 of map classification B based on the combination feature determination result. In Fig. 13, the detection apparatus 100 classifies the contribution value map group C3(Ma) by the classification unit 903 using the matching result T3abc.

[0102] Specifically, for example, the detection device 100 classifies the contribution value map group C3(Ma) into a first contribution value map group C3(Ma)-bc1 that is aggregated as male, trousers, and sneakers, a second contribution value map group C3(Ma)-bc2 that is aggregated as male, trousers, and sandals, a third contribution value map group C3(Ma)-bc3 that is aggregated as male, skirt, and sneakers, a fourth contribution value map group C3(Ma)-bc4 that is aggregated as male, skirt, and sandals, a fifth contribution value map group C3(Ma)-bc5 that is aggregated as female, trousers, and sneakers, a sixth contribution value map group C3(Ma)-bc6 that is aggregated as female, trousers, and sandals, a seventh contribution value map group C3(Ma)-bc7 that is aggregated as female, skirt, and sneakers, and an eighth contribution value map group C3(Ma)-bc8 that is aggregated as female, skirt, and sandals. In this way, the classification result 1300 is obtained.

[0103] Fig. 14 is an explanatory diagram showing example 2 of map classification B based on the combination feature determination result. In Fig. 14, the detection device 100 uses the detection target image data group D3 and the matching result T3abc as a learning data set to learn the combination feature determination model Mabc by the learning unit 902. When the detection target image data group D3 is input, the combination feature determination model Mabc generates a contribution value map group C3(Mabc).

[0104] The detection apparatus 100 causes the classification unit 903 to classify the contribution value map group C3(Mabc) using the matching result T3abc. Specifically, for example, the detection device 100 classifies the map data into a first contribution value map group C3(Mabc)-1 that is calculated as male, trousers, and sneakers, a second contribution value map group C3(Mabc)-2 that is calculated as male, trousers, and sandals, a third contribution value map group C3(Mabc)-3 that is calculated as male, skirt, and sneakers, a fourth contribution value map group C3(Mabc)-4 that is calculated as male, skirt, and sandals, a fifth contribution value map group C3(Mabc)-5 that is calculated as female, trousers, and sneakers, a sixth contribution value map group C3(Mabc)-6 that is calculated as female, trousers, and sandals, a seventh contribution value map group C3(Mabc)-7 that is calculated as female, skirt, and sneakers, and an eighth contribution value map group C3(Mabc)-8 that is calculated as female, skirt, and sandals. In this way, the classification result 1400 is obtained.

[0105] [Example of two-sample test using map classification B] 15 is an explanatory diagram showing an example of a two-sample test using map classification B. The detection device 100 executes two-sample tests 1501 to 1508 by the two-sample test unit 205 for each common combination feature based on the classification result 1300 of the contribution value map group C3(Ma) and the classification result 1400 of the contribution value map group C3(Mabc).

[0106] Specifically, for example, the detection device 100 performs a two-sample test 1501 using the first contribution value map group C3(Ma)-bc1 and the first contribution value map group C3(Mabc)-1, whose combined features are male, trousers, and sneakers, as two distributions to be subjected to the two-sample test. The detection device 100 also performs a two-sample test 1502 using the second contribution value map group C3(Ma)-bc2 and the second contribution value map group C3(Mabc)-2, whose combined features are male, trousers, and sandals, as two distributions to be subjected to the two-sample test.

[0107] Furthermore, the detection device 100 performs a two-sample test 1503 using the third contribution value map group C3(Ma)-bc3 and the third contribution value map group C3(Mabc)-3, whose combined features are male, skirt, and sneakers, as two distributions to be subjected to the two-sample test. The detection device 100 performs a two-sample test 1504 using the fourth contribution value map group C3(Ma)-bc4 and the fourth contribution value map group C3(Mabc)-4, whose combined features are male, skirt, and sneakers, as two distributions to be subjected to the two-sample test. However, in the example of FIG. 15, since both the fourth contribution value map group C3(Ma)-bc4 and the fourth contribution value map group C3(Mabc)-4 have zero cases, the two-sample test 1504 is not performed.

[0108] Furthermore, the detection device 100 performs a two-sample test 1505 using the fifth contribution value map group C3(Ma)-bc5 and the fifth contribution value map group C3(Mabc)-5, whose combined features are female, trousers, and sneakers, as two distributions to be subjected to the two-sample test. Furthermore, the detection device 100 performs a two-sample test 1506 using the sixth contribution value map group C3(Ma)-bc6 and the sixth contribution value map group C3(Mabc)-6, whose combined features are female, trousers, and sandals, as two distributions to be subjected to the two-sample test.

[0109] Furthermore, the detection device 100 performs a two-sample test 1703 using the seventh contribution value map group C3(Ma)-bc7 and the seventh contribution value map group C3(Mabc)-7, whose combined features are female, skirt, and sneakers, as two distributions to be subjected to the two-sample test. Furthermore, the detection device 100 performs a two-sample test 1508 using the eighth contribution value map group C3(Ma)-bc8 and the eighth contribution value map group C3(Mabc)-8, whose combined features are female, skirt, and sneakers, as two distributions to be subjected to the two-sample test.

[0110] The detection apparatus 100 determines whether or not a data shift has occurred in the cause candidate based on the two-sample test results in each of the two-sample tests 1501 to 1508 using the data shift determination unit 206. Specifically, for example, the detection apparatus 100 determines whether or not a data shift has occurred in the cause candidate by applying the first determination rule, the second determination rule, or the third determination rule in the same manner as in the case shown in the example of the two-sample test using map classification A using the data shift determination unit 206.

[0111] In the example of the two-sample test using map classification A, if each of the lower-body clothing and footwear is determined to be the cause of the data shift, the detection device 100 does not execute the two-sample test using map classification B. This can reduce the calculation load. On the other hand, in the example of the two-sample test using map classification A, if each of the lower-body clothing and footwear is determined to not be the cause of the data shift, the detection device 100 executes the two-sample test using map classification B. In this case, although each of the lower-body clothing and footwear is determined to not be the cause of the data shift, it may be determined that the combination of the lower-body clothing and footwear is the cause of the data shift, so that the detection accuracy of the cause of the data shift can be improved.

[0112] <Example of data shift detection processing procedure> Fig. 16 is a flowchart showing an example (first half) of a data shift detection processing procedure performed by the detection device 100 according to the embodiment 2. Fig. 17 is a flowchart showing an example (second half) of a data shift detection processing procedure performed by the detection device 100 according to the embodiment 2. The same processes as those in Figs. 5 to 7 are denoted by the same step numbers, and their description will be omitted.

[0113] 17, if step S706 in FIG. 16 is Yes, the detection apparatus 100 specifies cause candidates via the acquisition unit 201 (step S1701). In the second embodiment, the cause candidates are lower-body clothing, footwear, and a combination of lower-body clothing and footwear.

[0114] The detection device 100, using the feature determination unit 901, inputs the detection target image data group D3 for the subject under consideration into a feature determination model for the subject under consideration to obtain feature determination results, and inputs the detection target image data group for the candidate cause into a feature determination model for the candidate cause to obtain feature determination results (step S1702).

[0115] Specifically, for example, the detection device 100 obtains a gender determination result R(Ma). In addition, if the possible cause is lower-body clothing, the detection device 100 obtains a lower-body clothing determination result R3(Mb), if the possible cause is footwear, the detection device 100 obtains a footwear determination result R3(Mc), and if the possible cause is a combination of lower-body clothing and footwear, the detection device 100 obtains a lower-body clothing determination result R3(Mb) and a footwear determination result R3(Mc).

[0116] The detection device 100 generates a matching result by combining the characteristic judgment result of the subject of consideration and the characteristic judgment result of the candidate cause by the characteristic judgment unit 901 (step S1703). Specifically, for example, the detection device 100 generates a matching result T3ab if the candidate cause is lower-body clothing, generates a matching result T3ac if the candidate cause is footwear, and generates a matching result T3abc if the candidate cause is a combination of lower-body clothing and footwear.

[0117] The detection device 100 inputs the pre-shift image data group D2 for the subject of consideration into a feature determination model for the subject of consideration by the feature determination unit 901 to obtain a feature determination result, and inputs the pre-shift image data group D2 for the candidate cause into a feature determination model for the candidate cause to obtain a feature determination result. Specifically, for example, the detection device 100 obtains a gender determination result R2(Ma). In addition, if the candidate cause is lower body clothing, the detection device 100 obtains a lower body clothing determination result R2(Mb), if the candidate cause is footwear, the detection device 100 obtains a footwear determination result R2(Mc), and if the candidate cause is a combination of lower body clothing and footwear, the detection device 100 obtains a lower body clothing determination result R2(Mb) and a footwear determination result R2(Mc).

[0118] The detection device 100 generates a pre-shift matching result by combining the pre-shift characteristic judgment result of the subject to be considered and the pre-shift characteristic judgment result of the cause candidate by the characteristic judgment unit 901 (step S1705). Specifically, for example, the detection device 100 generates a matching result T2ab if the cause candidate is lower body clothing, generates a matching result T2ac if the cause candidate is footwear, and generates a matching result T2abc if the cause candidate is a combination of lower body clothing and footwear.

[0119] The detection device 100 uses the feature determination unit 901 to identify candidate causes that have a ratio change (step S1706). Specifically, for example, the detection device 100 compares the matching results T2ab and T3ab to detect the presence or absence of a ratio change, compares the matching results T2ac and T3ac to detect the presence or absence of a ratio change, and compares the matching results T2abc and T3abc to detect the presence or absence of a ratio change. The detection device 100 identifies candidate causes of the matching results that have a ratio change. That is, the detection device 100 narrows down the candidate causes. This makes it possible to exclude candidate causes that have not been identified, that is, that have not had a ratio change, from the subsequent processes (steps S1707 to S1711), thereby reducing the calculation load. The user can select in advance whether or not to execute step S1707.

[0120] The detection device 100 classifies the contribution value map group under consideration using the matching result by the classification unit 903 (step S1707). Specifically, for example, as shown in FIG. 10, the detection device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3ab, and outputs the classification result 1000. Similarly, if the cause candidate is footwear, the detection device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3ac. Moreover, as shown in FIG. 13, the detection device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3abc, and outputs the classification result 1300.

[0121] The detection device 100 uses the learning unit 902 to learn a combination feature judgment model for the subject of study and the candidate cause, inputs the detection target image data group D3 to the learned combination feature judgment model to generate a contribution value map group, and classifies the matching results using the classification unit 903 (step S1708).

[0122] 11, the detection device 100 learns a combination feature determination model Mab using the detection target image data group D3 and the matching result T3ab, inputs the detection target image data group D3 to the combination feature determination model Mab, and obtains a contribution value map group C3(Mab) output from the combination feature determination model Mab. Then, the detection device 100 classifies the contribution value map group C3(Mab) using the matching result T3ab, and outputs the classification result 1100.

[0123] Similarly, if the cause candidate is footwear, the detection device 100 uses the matching result T3ac to learn a combination feature determination model Mac using the detection target image data group D3 and the matching result T3ac, inputs the detection target image data group D3 to the combination feature determination model Mac, and obtains a contribution value map group C3(Mac) output from the combination feature determination model Mac. Then, the detection device 100 classifies the contribution value map group C3(Mac) by the matching result T3ac.

[0124] 14, the detection device 100 learns a combination feature determination model Mabc using the detection target image data group D3 and the matching result T3abc, inputs the detection target image data group D3 to the combination feature determination model Mabc, and obtains a contribution value map group C3(Mabc) output from the combination feature determination model Mabc. The detection device 100 then classifies the contribution value map group C3(Mabc) using the matching result T3abc, and outputs the classification result 1400.

[0125] The detection device 100 executes two sample tests for each common combined feature by the two sample test unit 205 (step S1709). Specifically, for example, the detection device 100 executes two sample tests 1201 to 1204 and 1501 to 1508 for each common combined feature as shown in Figs.

[0126] The detection apparatus 100 determines whether or not the cause candidate is the cause of the data shift occurrence based on the two-sample test result of the two-sample test in step S1709 by the data shift determination unit 206 (step S1710). Then, the detection apparatus 100 outputs the determination result of step S1710 by the output unit 204 (step S1711). With this, the detection apparatus 100 ends the series of processes.

[0127] In this way, according to the second embodiment, even if there are multiple cause candidates, the cause of the data shift can be identified.

[0128] Also in the second embodiment, the detection device 100 may output the two-sample test result of step S1709 through the output unit 204. For the two-sample test in which the null hypothesis is rejected in step S1709, the detection device 100 may generate a difference heat map through the difference generation unit 203 and output the difference heat map through the output unit 204 as shown in FIG. 4. Furthermore, the detection device 100 may output the classification results 1000, 1100, 1300, and 1400 by the classification unit 903 through the output unit 204. Furthermore, the detection device 100 may execute the process shown in FIG. 17 without executing steps S502, S503, S604, and S706 in FIG. 16.

[0129] The detection device 100 according to the above-mentioned first and second embodiments can also be configured as follows (1) to (9).

[0130] (1) The detection device 100 includes a processor 101 that executes a program and a storage device 102 that stores the program. The processor 101 is an acquisition process for acquiring a first contribution value map group C2(Mi) which is a collection of first contribution value maps that contributed to the features of each of the first image data in the first image data group (pre-shift image data group D2) and a second contribution value map group C3(Mi) which is a collection of second contribution value maps that contributed to the features of each of the second image data in the second image data group (detection target image data group D3) captured after the first image data group (pre-shift image data group D2); and (acquisition unit 201). a first two-sample testing process for performing a two-sample testing using the first contribution value map group C2(Mi and the second contribution value map group C3(Mi) acquired by the acquisition process (the two-sample testing unit 205, (step S604)); an output process (output unit 204) for outputting a two-sample testing result by the first two-sample testing process; Execute.

[0131] (2) In addition, in (1) above, In the acquisition process, the processor 101 acquires the first image data group and the second image data group (step S501). The processor 101 is A map generation process is executed to generate the first contribution value map for each of the first image data groups (pre-shift image data group D2) acquired by the acquisition process as a result of inputting the first image data of each of the first image data groups (pre-shift image data group D2) acquired by the acquisition process into a feature determination model Mi that determines features related to image data and calculates a contribution value related to the features for each region in the image data, and to generate the second contribution value map for each of the second image data groups (detection target image data group D3) as a result of inputting the second image data of each of the second image data groups into the feature determination model Mi (step S503). In the first two-sample testing process, the processor 101 performs a two-sample testing using a first contribution value map group C2(Mi), which is the first contribution value map for each of the first image data generated by the map generation process, and a second contribution value map group C3(Mi), which is the second contribution value map for each of the second image data generated by the map generation process.

[0132] (3) In addition, in (1) above, The processor 101 is Based on the first two-sample test result by the first two-sample test process, a determination process is executed to determine whether or not a data shift has occurred between the first image data group (pre-shift image data group D2) and the second image data group (detection target image data group D3) (step S605). In the output process, the processor 101 outputs the determination result from the determination process (step S606).

[0133] (4) Also, in (3) above, The processor 101 is When it is determined by the determination process that a data shift has occurred, a difference generation process is executed to generate a difference map DC(Mi) indicating a difference between a representative first contribution value map C2(Mi)av in the first contribution value map group C2(Mi) and a representative second contribution value map C3(Mi)av in the second contribution value map group (C3(Mi)) (step S504). In the output process, the processor 101 outputs the difference map DC(Mi) generated by the difference generating process (step S505).

[0134] (5) In addition, in (1) above, The processor 101 is A designation process for designating the subject of consideration (gender) and possible causes (lower body clothing) (step S1701) a feature determination process for generating a first matching result T3ab by collating the first feature determination result group R3(Ma) output from a first feature determination model Ma related to the subject (gender) by inputting the second image data group (detection target image data group D3) into a first feature determination model Ma related to the subject (gender) and the second feature determination result group R3(Mb) output from a second feature determination model Mb related to the cause candidate (lower body clothing) by inputting the second image data group (detection target image data group D3) into a second feature determination model Mb related to the cause candidate (lower body clothing) for combinations (male & pants, male & skirt, female & pants, female & skirt) of the type of the subject (gender) and the type of the cause candidate (lower body clothing) (pants, skirt); (steps S1702 and S1703); a first classification process of classifying the second contribution value map group C3(Mi) based on the first matching result T3ab (step S1708); A learning process for generating a combined feature determination model Mab that combines the subject (gender) and the cause candidate (lower body clothing) by learning the second image data group (detection target image data group D3) and the first matching result T3ab generated by the feature determination process as a learning data set (step S1708). a second process of generating a first combination contribution value map for each of the second image data sets (detection target image data sets D3) related to a combination of the subject to be considered (gender) and the cause candidate (lower body clothing) for each of the second image data sets based on the first matching result T3ab, and classifying a first combination contribution value map set C3(Mab) which is a set of the first combination contribution value maps for each of the second image data sets based on the first matching result T3ab (step S1708); In the output process, the processor 101 outputs a first classification result 1000 obtained by the first classification process and a second classification result 1100 obtained by the second classification process.

[0135] (6) Also, in (5) above, The processor 101 is A second two-sample testing process is executed to perform a two-sample test for each common combination in the first classification result 1000 and the second classification result 1100 (step S1709). In the output process, the processor 101 outputs a second two-sample testing result obtained by the second two-sample testing process (output unit 204).

[0136] (7) Also, in (6) above, The processor 101 is Executing a determination process to determine whether or not a data shift has occurred between the first image data group (pre-shift image data group D2) and the second image data group (detection target image data group D3) based on the second two-sample test result (step S1710). In the output process, the processor 101 outputs the determination result of the determination process (step S1711).

[0137] (8) Also, in (5) above, In the feature determination process, the processor 101 generates a second matching result T2ab by tallying a third feature determination result group R2(Ma) output from the first feature determination model Ma by inputting the first image data group (pre-shift image data group D2) into the first feature determination model Ma, and a fourth feature determination result group R2(Mb) output from the second feature determination model Mb by inputting the first image data group (pre-shift image data group D2) into the second feature determination model Mb, for each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the cause candidate (lower body clothing), and determines whether or not there is a predetermined ratio change between the first matching result T3ab and the second matching result T2ab (steps S1704, 1705, 1706). In the first classification process, the processor 101 classifies the second contribution value map group C3(Ma) for the specific cause candidate determined to have the ratio change based on the first matching result T3ab.

[0138] (9) The detection device 100 includes a processor 101 that executes a program and a storage device 102 that stores the program. The processor 101 is an acquisition process for acquiring a first contribution value map group C2(Mi) which is a collection of first contribution value maps that contributed to the features of each of the first image data in the first image data group (pre-shift image data group D2) and a second contribution value map group C3(Mi) which is a collection of second contribution value maps that contributed to the features of each of the second image data in the second image data group (detection target image data group D3) captured after the first image data group (pre-shift image data group D2); and (acquisition unit 201). a difference generating process for generating a difference map DC(Mi) indicating a difference between a representative first contribution value map C2(Mi)av in the first contribution value map group C2(Mi) acquired by the acquisition process and a representative second contribution value map C3(Mi)av in the second contribution value map group C3(Mi) (step S504); An output process for outputting the difference map DC(Mi) generated by the difference generation process (step S505); Execute.

[0139] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0140] In addition, each of the configurations, functions, processing units, processing means, etc. described above may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by processor 101 interpreting and executing a program that realizes each function.

[0141] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0142] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0143] 100 Detection device 101 Processor 201 Acquisition Department 202 Map Generation Unit 203 Difference generation part 204 Output section 205 Two-Sample Testing Department 206 Data shift judgment unit 901 Feature Judgment Unit 902 Learning Department 903 Classification Department 210 Contribution Value Map DB< / xai>

Claims

1. A detection device having a processor that executes a program and a storage device that stores the program, The processor, an acquisition process for acquiring a first contribution value map group which is a collection of first contribution value maps that contribute to the features of each of the first image data in the first image data group, and a second contribution value map group which is a collection of second contribution value maps that contribute to the features of each of the second image data in the second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection apparatus comprising:

2. 2. The detection device according to claim 1, In the acquisition process, the processor acquires the first image data group and the second image data group; The processor, a map generation process is executed in which, as a result of inputting the first image data of each of the first image data groups acquired by the acquisition process into a feature determination model that determines features related to the image data and calculates a contribution value related to the features for each region in the image data, a first contribution value map is generated for each of the first image data, and, as a result of inputting the second image data of each of the second image data groups into the feature determination model, a map generation process is executed in which, as a result of inputting the second image data of each of the second image data groups into the feature determination model, a second contribution value map is generated for each of the second image data, In the first two-sample testing process, the processor executes a two-sample testing using a first contribution value map group which is the first contribution value map for each of the first image data generated by the map generating process, and a second contribution value map group which is the second contribution value map for each of the second image data generated by the map generating process. A detection device comprising:

3. 2. The detection device according to claim 1, The processor, execute a determination process for determining whether or not a data shift has occurred between the first image data group and the second image data group based on a first two-sample testing result of the first two-sample testing process; In the output process, the processor outputs a determination result obtained by the determination process. A detection device comprising:

4. 4. The detection device according to claim 3, The processor, executing a difference generating process for generating a difference map indicating a difference between a representative first contribution value map in the first contribution value map group and a representative second contribution value map in the second contribution value map group when it is determined by the determination process that a data shift has occurred; In the output process, the processor outputs the difference map generated by the difference generation process. A detection device comprising:

5. 2. The detection device according to claim 1, The processor, A designation process for designating a subject of study and a cause candidate; a feature determination process for generating a first matching result by tallying a first feature determination result group, which is obtained by inputting the second image data group into a first feature determination model related to the subject of consideration and outputting the first feature determination model, and a second feature determination result group, which is obtained by inputting the second image data group into a second feature determination model related to the cause candidate and outputting the second feature determination model, for each combination of the type of the first feature determination result group and the type of the second feature determination result group; a first classification process of classifying the second contribution value map group based on the first matching result; a learning process for generating a combined feature determination model that combines the subject to be considered and the cause candidate by learning the second image data group and the first matching result generated by the feature determination process as a learning data set; a second process of inputting the second image data group into a combination feature determination model generated by the learning process, generating a first combination contribution value map for each of the second image data regarding a combination of the subject to be considered and the cause candidate, and classifying a first combination contribution value map group, which is a collection of the first combination contribution value maps for each of the second image data, based on the first matching result; In the output process, the processor outputs a first classification result obtained by the first classification process and a second classification result obtained by the second classification process. A detection device comprising:

6. 6. The detection device according to claim 5, The processor, performing a second two-sample testing process for performing a two-sample test for each common combination in the first classification result and the second classification result; In the output process, the processor outputs a second two-sample testing result obtained by the second two-sample testing process. A detection device comprising:

7. 7. The detection device according to claim 6, The processor, execute a determination process for determining whether or not a data shift has occurred between the first image data group and the second image data group based on the second two-sample test result; In the output process, the processor outputs a determination result obtained by the determination process. A detection device comprising:

8. 6. The detection device according to claim 5, In the characteristic determination process, the processor generates second matching results by aggregating a third characteristic determination result group, which is obtained by inputting the first image data group into the first characteristic determination model and outputting it from the first characteristic determination model, and a fourth characteristic determination result group, which is obtained by inputting the first image data group into the second characteristic determination model and outputting it from the second characteristic determination model, for each combination of the type of the subject of consideration and the type of the cause candidate, and determines whether or not there is a predetermined ratio change between the first matching results and the second matching results; In the first classification process, the processor classifies the second contribution value map group based on the first matching result for the specific cause candidate determined to have the ratio change. A detection device comprising:

9. A detection device having a processor that executes a program and a storage device that stores the program, The processor, an acquisition process for acquiring a first contribution value map group which is a collection of first contribution value maps that contribute to the features of each of the first image data in the first image data group, and a second contribution value map group which is a collection of second contribution value maps that contribute to the features of each of the second image data in the second image data group captured after the first image data group; a difference generating process for generating a difference map indicating a difference between a representative first contribution value map in the first contribution value map group acquired by the acquisition process and a representative second contribution value map in the second contribution value map group; an output process for outputting the difference map generated by the difference generation process; A detection apparatus comprising:

10. A detection method executed by a detection device having a processor that executes a program and a storage device that stores the program, comprising: The processor, an acquisition process for acquiring a first contribution value map group which is a collection of first contribution value maps that contribute to the features of each of the first image data in the first image data group, and a second contribution value map group which is a collection of second contribution value maps that contribute to the features of each of the second image data in the second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection method comprising the steps of:

11. The processor: an acquisition process for acquiring a first contribution value map group which is a collection of first contribution value maps that contribute to the features of each of the first image data in the first image data group, and a second contribution value map group which is a collection of second contribution value maps that contribute to the features of each of the second image data in the second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection program that causes a user to execute the above steps.

Images