Detection device, detection method, and detection program

Abstract

To provide a detection device capable of facilitating the detection of data shifts between image data groups.SOLUTION: The detection device includes: a processor for executing programs; and a storage device for storing the program. The detection device executes: acquisition processing to acquire a first contribution value map group that is a set of first contribution value maps contributing to the characteristics of first image data in each first image data group and a second contribution value map group that is a set of a second contribution value maps contributing to the characteristics of second image data in each of a second image data group picked up after the first image data group; first two-sample test processing of executing a two-sample test by using the first contribution value map group and the second contribution value map group acquired by the acquisition processing; and output processing of outputting the result of the two-sample test by the first two-sample test processing.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a detection device, a detection method, and a detection program. [Background technology]

[0002] The content and characteristics of input data can change from the time of learning. Such changes are called data shifts (or data drift). For example, if the input data is image data, data shifts could include changes in people's clothing, changes in the race of people passing by the shooting location, a deterioration in camera image quality, or the occurrence of building or road construction at the shooting location, due to the passage of time between the time the judgment model was learned and the current operation. Although data shifts do not affect accuracy at the time they occur, they pose a risk of potential degradation of the learning model's accuracy. Manual methods of detecting and determining data shifts are not practical due to the amount of work required, so automatic or semi-automatic methods are preferable.

[0003] Furthermore, the information processing device in Patent Document 1 below uses multiple training data corresponding to multiple correct labels to learn an operational model to be monitored. Based on the output result of the operational model, the information processing device learns a decision boundary that divides the feature space of the data into two application regions, and creates an inspector model that calculates the distance from the decision boundary to the input data. The information processing device inputs training data to the inspector model and calculates a first distance from the decision boundary to the training data. The information processing device inputs operational data to the inspector model and calculates a second distance from the decision boundary to the operational data. If the difference between the first distance and the second distance is equal to or greater than a preset threshold, the information processing device detects a change in the output result of the operational model due to a change in the trend of the operational data over time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 079460 Summary of the Invention [Problem to be solved by the invention]

[0005] There are various data detection methods that use structured data such as table data as input data, but when the input data is data with multidimensional features such as image data, the accuracy of data shift detection decreases. Furthermore, in Patent Document 1, a change in the ratio of detected features is considered to be concept drift detection, and a change in this label is considered to be the cause. However, when there are multiple changes in the ratio of detected features, it is not possible to identify which one is the concept drift.

[0006] An object of the present invention is to facilitate the detection of data shifts between image data groups. [Means for solving the problem]

[0007] A detection device according to one aspect of the invention disclosed in the present application is a detection device having a processor that executes a program and a storage device that stores the program, and is characterized by performing an acquisition process to acquire a first contribution value map group, which is a collection of first contribution value maps that contributed to the characteristics of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the characteristics of each second image data in a second image data group that was captured after the first image data group; a first two-sample test process to perform a two-sample test using the first contribution value map group and the second contribution value map group acquired by the acquisition process; and an output process to output the two-sample test results obtained by the first two-sample test process.

[0008] Another aspect of the invention disclosed in the present application is a detection device having a processor that executes a program and a storage device that stores the program, and is characterized by executing an acquisition process to acquire a first contribution value map group, which is a collection of first contribution value maps that contributed to the characteristics of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the characteristics of each second image data in a second image data group that was captured after the first image data group; a difference generation process to generate a difference map showing the difference between a representative first contribution value map in the first contribution value map group acquired by the acquisition process and a representative second contribution value map in the second contribution value map group; and an output process to output the difference map generated by the difference generation process. [Effects of the Invention]

[0009] According to the exemplary embodiment of the present invention, it is possible to facilitate the detection of data shifts between image data groups. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiment. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an example of the hardware configuration of the device. [Figure 2] FIG. 2 is a block diagram of an example of a functional configuration of the detection device according to the first embodiment. [Figure 3] FIG. 3 is an explanatory diagram showing an example of the operation of XAI. [Figure 4] FIG. 4 is an explanatory diagram showing an example of generating a difference map. [Figure 5] FIG. 5 is a flowchart of a first example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 6] FIG. 6 is a flowchart of a second example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart of a third example of a data shift detection process performed by the detection device according to the first embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing an example of a change in the ratio of the matching result between the subject of study and the cause candidate. [Figure 9] FIG. 9 is a block diagram of an example of a functional configuration of a detection device according to a second embodiment. [Figure 10] FIG. 10 is an explanatory diagram showing Example 1 of map classification A based on the combined feature determination result. [Figure 11] FIG. 11 is an explanatory diagram showing Example 2 of map classification A based on the combined feature determination result. [Figure 12] FIG. 12 is an explanatory diagram showing an example of a two-sample test using map classification A. [Figure 13] FIG. 13 is an explanatory diagram showing Example 1 of map classification B based on the combined feature determination result. [Figure 14] FIG. 14 is an explanatory diagram showing Example 2 of map classification B based on the combined feature determination result. [Figure 15] FIG. 15 is an explanatory diagram showing an example of a two-sample test using map classification B. [Figure 16] FIG. 16 is a flowchart of an example (first half) of a data shift detection process performed by the detection device according to the second embodiment. [Figure 17] FIG. 17 is a flowchart illustrating an example (second half) of the data shift detection process performed by the detection device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0011] In the first embodiment, a detection device will be described that detects the frequency of appearance of features in image data when input data is image data, or detects data shifts regarding features across the entire image data.

[0012] <Example of hardware configuration of detection device> FIG. 1 is a block diagram showing an example of the hardware configuration of the device. The detection device 100 includes a processor 101, a storage device 102, an input device 103, an output device 104, and a communication interface (communication IF) 105. The processor 101, the storage device 102, the input device 103, the output device 104, and the communication IF 105 are connected via a bus 106. The processor 101 controls the detection device 100. The storage device 102 serves as a working area for the processor 101. The storage device 102 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 102 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 103 inputs data. Examples of the input device 103 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 104 outputs data. The output device 104 includes, for example, a display, a printer, and a speaker. The communication IF 105 connects to a network and transmits and receives data.

[0013] <Example of functional configuration of detection device> 2 is a block diagram illustrating an example of a functional configuration of the detection device 100 according to the first embodiment. The detection device 100 includes an acquisition unit 201, a map generation unit 202, a difference generation unit 203, an output unit 204, a two-sample test unit 205, and a data shift determination unit 206. Specifically, the acquisition unit 201 to the data shift determination unit 206 are functions that are realized by causing the processor 101 to execute a program stored in the storage device 102 illustrated in FIG. 1.

[0014] The acquiring unit 201 acquires image data. Specifically, for example, the acquiring unit 201 acquires a comparison source image data group D1, a pre-shift image data group D2, and a detection target image data group D3. The comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 may be stored in the storage device 102 within the detection device 100, or may be stored in a computer external to the detection device 100. If stored in a computer external to the detection device 100, the acquiring unit 201 receives the comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 from the computer external to the detection device 100.

[0015] The comparison source image data group D1 is a set of comparison source image data. The comparison source image data is image data used to train a feature determination model that determines the features of the image data. For example, if the feature determination model is a gender determination model that determines the gender of a person in image data captured by a surveillance camera, the image data used to train the gender determination model is the comparison source image data.

[0016] The pre-shift image data group D2 is a collection of pre-shift image data. The pre-shift image data is image data captured after the comparison source image data was captured. The pre-shift image data may be captured at the same location as the comparison source image data or at a different location. The pre-shift image data group D2 may be a part of the comparison source image data group D1.

[0017] The detection target image data group D3 is a collection of detection target image data. The detection target image data is image data that is the target of detection to determine whether a data shift has occurred. The detection target image data is image data that was captured after the pre-shift image data. The detection target image data may be captured at the same location as the pre-shift image data or at a different location.

[0018] When there is no need to distinguish between the comparison source image data, the pre-shift image data, and the detection target image data, they will be simply referred to as image data.

[0019] The map generation unit 202 generates a contribution value map that indicates the distribution of feature quantities of image data. The map generation unit 202 is one function of the feature determination model. When image data is input, the feature determination model outputs a feature determination result and calculates a contribution value for each pixel region that indicates how much each pixel region in the image data contributed to the determination result. A pixel region is a set of one or more pixels. The contribution value map is data in which the contribution value calculated for each pixel region is allocated to that pixel region.

[0020] The map generation unit 202 outputs a soft label, which is the result of dimensional compression of the image data, as the result of dimensional compression of the image data. This soft label is the contribution value for each pixel region. The soft label is, for example, input data to the final fully connected layer of the feature determination model that outputs the determination result, or to an intermediate layer before that. The generated contribution value map is stored in the contribution value map DB 210.

[0021] The difference generation unit 203 calculates the difference between two contribution value maps and generates a difference map. One of the two contribution value maps is, for example, an average contribution value map obtained by averaging the contribution value maps obtained from each comparison source image data in the comparison source image data group D1, or an average contribution value map obtained by averaging the contribution value maps obtained from each pre-shift image data in the pre-shift image data group D2. The other of the two contribution value maps is, for example, an average contribution value map obtained by averaging the contribution value maps obtained from each detection target image data in the detection target image data group D3.

[0022] The output unit 204 outputs data. Specifically, for example, the output unit 204 displays the data on a display, which is an example of the output device 104, or prints out the data from a printer. The output unit 204 also transmits the data to another computer via the communication IF 105. When outputting a map, the output unit 204 can also output the data as a heat map.

[0023] The data to be output from the output unit 204 includes, for example, the difference map from the difference generation unit 203, two contribution value maps to be the difference targets of the difference generation unit 203, the test results of the two-sample test unit 205, and the judgment results from the data shift judgment unit 206.

[0024] The two-sample testing unit 205 performs a two-sample test on the two distributions and calculates a p-value. Specifically, for example, the two-sample testing unit 205 performs a hypothesis test on the kernel mean difference (MMD: Maximum Mean Discrepancy) between the two distributions. The two-sample testing unit 205 then determines whether to reject the null hypothesis H0 based on the p-value. In this example, the null hypothesis H0 is a hypothesis that there is no difference between the two distributions. For example, if the p-value is 0.05 or less, the two-sample testing unit 205 rejects the null hypothesis H0 at a significance level of 5%.

[0025] Here, there are two sets of two distributions. In the first set of two distributions, one distribution is a group of contribution value maps obtained from each comparison source image data in the comparison source image data group D1. The other distribution is a group of contribution value maps obtained from each pre-shift image data in the pre-shift image data group D2.

[0026] In the second set of two distributions, one distribution is a group of contribution value maps obtained from each pre-shift image data in the pre-shift image data group D2, and the other distribution is a group of contribution value maps obtained from each detection target image data in the detection target image data group D3.

[0027] In addition, when the contribution value maps obtained from the comparison source image data of the comparison source image data group D1 are used in place of the contribution value maps obtained from the pre-shift image data of the pre-shift image data group D2 in the second set of two distributions, the two-sample testing unit 205 does not need to perform the two-sample testing on the first set of two distributions, thereby reducing the calculation load on the detection device 100.

[0028] The data shift determination unit 206 determines whether a data shift has occurred between the pre-shift image data group D2 (which may be the comparison source image data group D1) and the detection target image data group D3. Specifically, for example, when the two-sample testing unit 205 adopts the null hypothesis H0 for the first set of two distributions and rejects the null hypothesis H0 for the second set of two distributions, the data shift determination unit 206 determines that a data shift has occurred between the pre-shift image data group D2 (which may be the comparison source image data group D1) and the detection target image data group D3.

[0029] <xai> FIG. 3 is an explanatory diagram showing an example of the operation of XAI. XAI 300 has feature determination models M1 to Mn (n is an integer equal to or greater than 1). When the feature determination models M1 to Mn are not distinguished from one another, they are referred to as feature determination model M. The feature determination model M is a learning model trained using a comparison source image data group D1 and its correct answer data. The feature determination model M includes learning models that determine various features, such as a gender determination model, a lower body clothing determination model, a footwear determination model, and so on.

[0030] XAI300 is an explainable AI that outputs a feature determination result from a feature determination model M and also outputs a contribution value map. The contribution value map is an evaluation value that indicates which pixel regions of the input image data contributed to the feature determination result.

[0031] The feature determination model M outputs a feature determination result when image data is input. In Fig. 3, when each comparison source image data of the comparison source image data group D1 is input to the feature determination model M1, the feature determination model M1 outputs a feature determination result group R1(M1) which is a collection of feature determination results for each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model M1, the feature determination model M1 outputs a feature determination result group R2(M1) which is a collection of feature determination results for each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature determination model M1, the feature determination model M1 outputs a feature determination result group R3(M1) which is a collection of feature determination results for each detection target image data.

[0032] When each comparison source image data of the comparison source image data group D1 is input to the feature determination model Mn, the feature determination model Mn outputs a feature determination result group R1(Mn), which is a collection of feature determination results for each comparison source image data; when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model Mn, the feature determination model Mn outputs a feature determination result group R2(Mn), which is a collection of feature determination results for each pre-shift image data; and when each detection target image data of the detection target image data group D3 is input to the feature determination model Mn, the feature determination model Mn outputs a feature determination result group R3(Mn), which is a collection of feature determination results for each detection target image data.

[0033] 3, when each comparison source image data of the comparison source image data group D1 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C1(M1) which is a collection of contribution value maps of each comparison source image data, when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C2(M1) which is a collection of contribution value maps of each pre-shift image data, and when each detection target image data of the detection target image data group D3 is input to the feature determination model M1, the feature determination model M1 outputs a contribution value map group C3(M1) which is a collection of contribution value maps of each detection target image data.

[0034] When each comparison source image data of the comparison source image data group D1 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C1(Mn), which is a collection of contribution value maps of each comparison source image data; when each pre-shift image data of the pre-shift image data group D2 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C2(Mn), which is a collection of contribution value maps of each pre-shift image data; and when each detection target image data of the detection target image data group D3 is input to the feature determination model Mn, the feature determination model Mn outputs a contribution value map group C3(Mn), which is a collection of contribution value maps of each detection target image data.

[0035] <Example of difference map generation> 4 is an explanatory diagram showing an example of differential map generation. First, the map generation unit 202 generates a contribution value map group C1(Mi), a contribution value map group C2(Mi), and a contribution value map group C3(Mi). Mi is the i-th (1≦i≦n) feature determination model.

[0036] The two-sample testing unit 205 performs a two-sample test using the contribution value map group C1(Mi) and the contribution value map group C2(Mi) as a first set of two distributions. The two-sample testing unit 205 also performs a two-sample test using the contribution value map group C2(Mi) and the contribution value map group C3(Mi) as a second set of two distributions.

[0037] If the result of the first set of two-sample testing is the acceptance of the null hypothesis and the result of the second set of two-sample testing is the rejection of the null hypothesis, the data shift determination unit 206 determines that a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3.

[0038] When the data shift determination unit 206 determines that a data shift has occurred, the difference generation unit 203 averages the contribution value map group C2(Mi) to generate an average contribution value map C2(Mi)av, averages the contribution value map group C3(Mi) to generate an average contribution value map C3(Mi)av, and then generates a difference map DC(Mi) which is the difference between the two average contribution value maps C2(Mi)av and C3(Mi)av.

[0039] Here, the average contribution value map C2(Mi) is a contribution value map obtained by averaging the contribution values ​​at the same position in each contribution value map in the group of contribution value maps C2(Mi).Similarly, the average contribution value map C3(Mi) is a contribution value map obtained by averaging the contribution values ​​at the same position in each contribution value map in the group of contribution value maps C3(Mi).

[0040] 4, the difference generation unit 203 averages the contribution value map group C2(Mi) to generate the average contribution value map C2(Mi)av and averages the contribution value map group C3(Mi) to generate the average contribution value map C3(Mi)av, but this is not limited to averaging. For example, the difference generation unit 203 may extract a representative contribution value map from each of the contribution value map group C2(Mi) and the contribution value map group C3(Mi). The representative contribution value map may be the averaging described above, or may be a randomly selected contribution value map, a contribution value map with the maximum sum of the contribution values ​​in the contribution value map, a contribution value map with the minimum sum of the contribution values ​​in the contribution value map, or a contribution value map with the median sum of the contribution values ​​in the contribution value map.

[0041] <Example of data shift detection processing procedure> 5 is a flowchart illustrating a first example of a data shift detection process performed by the detection device 100 according to the first embodiment. The detection device 100 specifies an object of consideration (step S501). The object of consideration is a feature of image data for which data shift detection is to be considered. Here, gender is used as an example.

[0042] The detecting apparatus 100 acquires the comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 via the acquiring unit 201 (step S502).

[0043] The detection device 100 inputs the comparison source image data group D1, the pre-shift image data group D2, and the detection target image data group D3 into the feature determination model M under consideration using the map generation unit 202, and generates respective contribution value map groups C1(Mi), C2(Mi), and C3(Mi) (step S503).

[0044] The detection device 100 uses the difference generation unit 203 to calculate average contribution value maps C1(Mi)av, C2(Mi)av, and C3(Mi)av for each of the contribution value map groups C1(Mi), C2(Mi), and C3(Mi), and generates a difference map DC(Mi) between the average contribution value map C2(Mi)av (or C1(Mi)av) and the average contribution value map C3(Mi)av (step S504).

[0045] The detection apparatus 100 outputs the difference map DC(Mi) via the output unit 204 (step S505), and ends the series of processes. By outputting the difference map DC(Mi) in this way, the user can visually identify the difference locations as areas causing data shifts, and can use the difference locations to improve the feature determination model M. For example, the feature determination model M can be retrained using detection target image data in which difference locations appear as training data. This makes it possible to facilitate detection of data shifts.

[0046] 6 is a flowchart illustrating a second example of a data shift detection process performed by the detection device 100 according to the first embodiment. The same steps as those in the second example of the data shift detection process illustrated in FIG. 5 are denoted by the same step numbers, and the description thereof will be omitted.

[0047] After step S503, the detection device 100 causes the two-sample testing unit 205 to perform a first two-sample test using the contribution value map groups C1(M) and C2(M), and then perform a second two-sample test using the contribution value map groups C2(M) and C3(M) (step S604).

[0048] The detection device 100 determines whether a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 using the data shift determination unit 206 based on the test results of the first two-sample test and the second two-sample test (step S605). If the test result of the first two-sample test is to accept the null hypothesis and the test result of the second two-sample test is to reject the null hypothesis, the detection device 100 determines using the data shift determination unit 206 that a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3. If the test results of the first two-sample test and the second two-sample test are other than those above, the detection device 100 determines using the data shift determination unit 206 that a data shift has not occurred between the pre-shift image data group D2 and the detection target image data group D3.

[0049] The detection device 100 outputs the data shift determination result of the data shift determination unit 206 via the output unit 204 (step S606), and ends the series of processes. In this way, the detection device 100 can improve the accuracy of data shift detection. Furthermore, the user can check whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 for the subject of consideration.

[0050] 7 is a flowchart illustrating a third example of a data shift detection process performed by the detection device 100 according to the first embodiment. The same steps as those in the first and second examples of data shift detection process shown in FIGS. 5 and 6 are denoted by the same step numbers, and the description thereof will be omitted.

[0051] After step S604, the detection device 100 determines whether a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 based on the test results of the first two-sample test and the second two-sample test using the data shift determination unit 206 (step S605). If it is determined that a data shift has occurred (step S605: Yes), the detection device 100 executes steps S504 and S505 and ends the series of processes.

[0052] In this way, by outputting the difference map DC(M), the user can visually identify the difference locations as areas causing data shifts, and can use the difference locations to improve the feature determination model M. For example, the feature determination model M can be retrained using detection target image data in which difference locations appear as training data. This makes it possible to facilitate the detection of data shifts.

[0053] On the other hand, if it is determined that no data shift has occurred (step S605: No), the detection device 100 outputs the data shift determination result (no data shift has occurred) of the data shift determination unit 206 via the output unit 204 (step S706), and ends the series of processes. In this way, the detection device 100 can improve the accuracy of data shift detection. Furthermore, the user can check whether or not a data shift has occurred between the pre-shift image data group D2 and the detection target image data group D3 for the subject of consideration.

[0054] In the first embodiment, the map generating unit 202 generates the contribution value map, but the acquiring unit 201 may acquire the contribution value map from the contribution value map DB 210, and the two-sample testing unit 205 may perform the two-sample testing using the acquired contribution map. Alternatively, the acquiring unit 201 may acquire the contribution value map from an external computer via the communication IF 105, and the two-sample testing unit 205 may perform the two-sample testing using the acquired contribution map. [Example]

[0055] A description will be given of Example 2. In Example 1, the detection device 100 has been described which detects the frequency of appearance of features in image data when the input data is image data, or detects data shifts regarding features across the entire image data.

[0056] Incidentally, there may be data shifts for which the cause of the data shift (the pixel region (causing region) in the average contribution value map that is the cause of the data shift and its contribution value) cannot be identified from the appearance of the average contribution value map. This is because, in the average contribution value map, the change in ratio before and after the data shift cannot be observed due to the overlapping or cancellation of contribution values ​​in the contribution value maps, or it may be difficult to pinpoint the causative region due to differences in the characteristics between the image data.

[0057] Therefore, in Example 2, a detection device will be described that identifies the cause of a data shift by comparing the feature determination result of the feature to be considered with the feature determination results of other features that are candidate causes other than the feature to be considered. In Example 2, as an example, the feature to be considered is the "gender (male or female)" of a person in image data, and the other candidate causes are the "lower body clothing (pants or skirt)" and "footwear (sneakers or sandals)" of the person in the image data. Note that the same components as in Example 1 are assigned the same reference numerals, and their description will be omitted.

[0058] <Changes in the ratio of matching results> 8 is an explanatory diagram showing an example of a change in the ratio of the matching results between the subject of study and the candidate cause. (A) shows an example of a change in the ratio of the matching results between the subject of study (gender) and the candidate cause (lower body clothing), and (B) shows an example of a change in the ratio of the matching results between the subject of study (gender) and the candidate cause (footwear). In Example 2, the symbol a (1≦a≦n) indicates data related to the subject of study (gender), the symbol b (1≦b≦n, b≠a) indicates data related to the subject of study (lower body clothing), and the symbol c (1≦c≦n, c≠a, c≠b) indicates data related to the subject of study (footwear).

[0059] In (A), the detection device 100 inputs the pre-shift image data group D2 into a gender determination model Ma, which is one of the feature determination models M, using XAI300, to obtain a gender determination result group R2(Ma), and inputs the pre-shift image data group D2 into a lower body clothing determination model Mb, which is one of the feature determination models M, to obtain a lower body clothing determination result group R2(Mb).

[0060] The gender determination result group R2(Ma) is a set of gender determination results for each pre-shift image data in the pre-shift image data group D2. The lower body clothing determination result group R2(Mb) is a set of lower body clothing determination results for each pre-shift image data in the pre-shift image data group D2.

[0061] The gender determination model Ma determines gender based on the shape of the upper body, such as the person's face, hairstyle, arm size, shoulder width, chest, waist, etc. The lower body clothing determination model Mb determines whether the person's lower body clothing is pants or a skirt.

[0062] The detection device 100 compares the gender determination result group R2(Ma) with the lower body clothing determination result group R2(Mb). Specifically, for example, the detection device 100 uses the gender determination result group R2(Ma) and the lower body clothing determination result group R2(Mb) to count the number of pre-shift image data corresponding to each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the potential cause (lower body clothing).

[0063] As a result, matching result T2ab is generated. Matching result T2ab indicates that, for the people in the 200 pieces of pre-shift image data, 100 pieces of pre-shift image data were determined to be male and wearing pants, 0 pieces of pre-shift image data were determined to be male and wearing a skirt, 0 pieces of pre-shift image data were determined to be female and wearing pants, and 100 pieces of pre-shift image data were determined to be female and wearing a skirt.

[0064] In addition, the detection device 100 inputs the detection target image data group D3 into the gender determination model Ma using XAI300, thereby obtaining a gender determination result group R3(Ma), and inputs the detection target image data group D3 into the lower body clothing determination model Mb, thereby obtaining a lower body clothing determination result group R3(Mb).

[0065] The gender determination result group R3(Ma) is a set of gender determination results for each piece of detection target image data in the detection target image data group D3. The lower body clothing determination result group R3(Mb) is a set of lower body clothing determination results for each piece of detection target image data in the detection target image data group D3.

[0066] The detection device 100 compares the gender determination result group R3(Ma) with the lower-body clothing determination result group R3(Mb). Specifically, for example, the detection device 100 uses the gender determination result group R3(Ma) and the lower-body clothing determination result group R3(Mb) to count the number of detection target image data items that correspond to each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the potential cause (lower-body clothing).

[0067] As a result, matching result T3ab is generated. Matching result T3ab indicates that, for the people in the 200 pieces of detection target image data, 80 pieces of detection target image data were determined to be male and pants, 20 pieces of detection target image data were determined to be male and skirt, 20 pieces of detection target image data were determined to be female and pants, and 80 pieces of detection target image data were determined to be female and skirt.

[0068] The detection device 100 compares the matching results T2ab and T3ab to determine whether or not there has been a ratio change. Specifically, for example, when an increase or decrease of x% occurs in at least one of the four combined features of the matching results T2ab and T3ab, the detection device 100 determines that there has been a ratio change between the matching results T2ab and T3ab.

[0069] For example, when x=10(%), the combination feature of male and pants changes from "100" in matching result T2ab to "80" in matching result T3ab. In other words, a ratio change of 20% has occurred, and therefore the detection device 100 determines that a ratio change has occurred between matching results T2ab and T3ab.

[0070] Although the determination item for the ratio change is at least one of the four combined features of the matching results T2ab and T3ab, the user can set it to at least two of the four combined features, at least three of the four combined features, or all four combined features. The more combined features to be determined, the stricter the determination of the ratio change becomes.

[0071] In (B), the lower-body clothing determination model Mb is replaced with a footwear determination model Mc. As in (A), the detection device 100 inputs the pre-shift image data group D2 into the gender determination model Ma using XAI300 to obtain a gender determination result group R2(Ma). The pre-shift image data group D2 is input into the footwear determination model Mc to obtain a footwear determination result group R2(Mc). The detection device 100 then inputs the detection target image data group D3 into the footwear determination model Mc to obtain a footwear determination result group R3(Mc). The detection device 100 then generates matching results T2ac and T3ac and determines whether there is a change in ratio. The footwear determination model Mc determines whether the footwear on the person's feet is sneakers or sandals.

[0072] Here, suppose that there is a change in the ratio in both cases (A) and (B). In this case, it is not possible to determine whether the cause of the data shift is the lower body clothing, the footwear, or both the lower body clothing and the footwear. Below, a configuration for determining whether the cause of the data shift is the lower body clothing, the footwear, or both the lower body clothing and the footwear will be described.

[0073] <Example of functional configuration of detection device 100> 9 is a block diagram illustrating an example of a functional configuration of a detection device 100 according to a second embodiment. The detection device 100 includes an acquisition unit 201, a map generation unit 202, a difference generation unit 203, an output unit 204, a two-sample test unit 205, a data shift determination unit 206, a feature determination unit 901, a learning unit 902, and a classification unit 903. Specifically, the feature determination unit 901, the learning unit 902, and the classification unit 903 are functions that are realized by, for example, causing the processor 101 to execute a program stored in the storage device 102 illustrated in FIG.

[0074] The feature determination unit 901 determines whether or not the image data has features. Specifically, for example, the feature determination unit 901 is part of the XAI 300, and outputs feature determination result groups R1 to R3 of the feature determination model M shown in Fig. 3. For example, as shown in Fig. 8, if the feature determination model M is a gender determination model Ma, the feature determination unit 901 outputs gender determination result groups R2(Ma) and R3(Ma), if the feature determination model M is a lower body clothing determination model Mb, the feature determination unit 901 outputs lower body clothing determination result groups R2(Mb) and R3(Mb), and if the feature determination model M is a footwear determination model Mc, the feature determination unit 901 outputs footwear determination result groups R2(Mc) and R3(Mc).

[0075] The learning unit 902 learns using the detection target image data group D3 and the matching results thereof, and generates a feature determination model that combines the subject of investigation and the cause candidate (hereinafter referred to as a combined feature determination model). If a combined feature determination model already exists in the feature determination models M1 to Mn, the learning unit 902 re-learns the combined feature determination model using the detection target image data group D3 and the matching results thereof.

[0076] In the case of FIG. 8A, the learning unit 902 generates a combined feature determination model Mab that combines gender and lower-body clothing using the detection target image data group D3 and the matching result T3ab.

[0077] In the case of FIG. 8B, the learning unit 902 generates a combination feature determination model Mac that combines gender and footwear, using the detection target image data group D3 and the matching result T3ac.

[0078] In the case of combining (A) and (B) in Figure 8, the learning unit 902 uses the detection target image data group D3 and its matching results T3ab and T3ac to generate a combination feature determination model Mabc that combines gender, lower body clothing, and footwear.

[0079] The classification unit 903 classifies the contribution value map group based on the matching result. Specifically, for example, the classification unit 903 classifies the contribution value map group C3(Ma) based on the matching result T3ab. The contribution value map group C3(Ma) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Ma.

[0080] Similarly, the classification unit 903 classifies the contribution value map group C3(Ma) based on the matching result T3ac.

[0081] Furthermore, based on the matching result T3ab, the classification unit 903 classifies the contribution value map group C3(Mab) output from the combination feature determination model Mab learned by the learning unit 902. The contribution value map group C3(Mab) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mab.

[0082] Similarly, based on the matching result T3ac, the classification unit 903 classifies the contribution value map group C3(Mac) output from the combination feature determination model Mac learned by the learning unit 902. The contribution value map group C3(Mac) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mac.

[0083] Similarly, the classification unit 903 classifies, based on the matching result T3abc, the contribution value map group C3(Mabc) output from the combination feature determination model Mabc trained by the learning unit 902. The matching result T3abc is a matching result that aggregates the matching result T3ab and the matching result T3ac. The contribution value map group C3(Mabc) is a collection of contribution value maps of each detection target image data output from XAI300 as a result of each detection target image data of the detection target image data group D3 being input to the feature determination model Mabc.

[0084] <Example of identifying the cause based on the results of combined feature determination> First, with reference to FIGS. 10 to 12, a first example of identifying a cause based on a result of determining combined features will be described, where the subject of investigation is gender and the cause candidate is lower-body clothing.

[0085] [Map classification A based on combined feature judgment results] Fig. 10 is an explanatory diagram showing example 1 of map classification A based on combined feature determination results. In Fig. 10, the detection device 100 classifies the contribution value map group C3(Ma) using the matching result T3ab between the gender determination result group R3(Ma) obtained by inputting the detection target image data group D3 into the gender determination model Ma and the lower body clothing determination result group R3(Mb) obtained by inputting the detection target image data group D3 into the lower body clothing determination model Mb.

[0086] Specifically, for example, the detection device 100 classifies the contribution value map group C3(Ma) into a first contribution value map group C3(Ma)-b1, which is aggregated as male and trousers, a second contribution value map group C3(Ma)-b2, which is aggregated as male and skirt, a third contribution value map group C3(Ma)-b3, which is aggregated as female and trousers, and a fourth contribution value map group C3(Ma)-b4, which is aggregated as female and skirt.

[0087] 10, the detection device 100 classifies the contribution value map group C3(Ma) using the matching result T3ac between the gender determination result group R3(Ma) and the footwear determination result group R3(Mc) obtained by inputting the detection target image data group D3 into the footwear determination model Mc. In this way, the classification result 1000 is obtained.

[0088] Fig. 11 is an explanatory diagram showing example 2 of map classification A based on the combination feature determination result. In Fig. 11, the detection device 100 uses the detection target image data group D3 and the matching result T3ab as a training data set to train a combination feature determination model Mab using the learning unit 902. When the detection target image data group D3 is input, the combination feature determination model Mab generates a contribution value map group C3(Mab).

[0089] The detection apparatus 100 classifies the contribution value map group C3(Mab) using the matching result T3ab using the classification unit 903. Specifically, for example, the detection apparatus 100 classifies the contribution value map group C3(Mab) into a first contribution value map group C3(Mab)-1 aggregated as male and trousers, a second contribution value map group C3(Mab)-2 aggregated as male and skirt, a third contribution value map group C3(Mab)-3 aggregated as female and trousers, and a fourth contribution value map group C3(Mab)-4 aggregated as female and skirt. In this way, the classification result 1100 is obtained.

[0090] [Example of two-sample test using map classification A] 12 is an explanatory diagram showing an example of a two-sample test using map classification A. The detection device 100 executes two-sample tests 1201 to 1204 for each common combined feature using the two-sample test unit 205, based on the classification result 1000 of the contribution value map group C3(Ma) and the classification result 1100 of the contribution value map group C3(Mab).

[0091] Specifically, for example, the detection device 100 performs a two-sample test 1201 using first contribution value map group C3(Ma)-b1 and first contribution value map group C3(Mab)-1, whose combined features are male and trousers, as two distributions to be tested. The detection device 100 also performs a two-sample test 1202 using second contribution value map group C3(Ma)-b2 and second contribution value map group C3(Mab)-2, whose combined features are male and skirt, as two distributions to be tested.

[0092] The detection device 100 also performs a two-sample test 1203 using the third contribution value map group C3(Ma)-b3 and the third contribution value map group C3(Mab)-3, whose combined features are female and pants, as two distributions to be tested. The detection device 100 also performs a two-sample test 1204 using the fourth contribution value map group C3(Ma)-b4 and the fourth contribution value map group C3(Mab)-4, whose combined features are female and skirt, as two distributions to be tested.

[0093] The detection apparatus 100 uses the data shift determination unit 206 to determine whether or not a data shift has occurred in the cause candidate based on the two-sample test results in each of the two-sample tests 1201 to 1204.

[0094] Specifically, for example, the detection apparatus 100 determines whether a data shift has occurred in a candidate cause based on the number of times the null hypothesis is adopted in each two-sample test result using the data shift determination unit 206. For example, the detection apparatus 100 may determine that a candidate cause is not the cause of the data shift occurrence when the null hypothesis is adopted in at least k (k is an integer greater than or equal to 1 and less than or equal to the number of two-sample tests) of the multiple two-sample tests 1201 to 1204 (first determination rule). Furthermore, the detection apparatus 100 may determine that a candidate cause is the cause of the data shift occurrence when the null hypothesis is rejected in at least k (k is an integer greater than or equal to 1 and less than or equal to the number of two-sample tests) of the multiple two-sample tests 1201 to 1204 (second determination rule). Furthermore, the detection device 100 may determine that the candidate cause is not the cause of the data shift if the number of times the null hypothesis is accepted is greater than the number of times it is rejected, by majority vote between the number of times the null hypothesis is accepted and the number of times it is rejected is greater, and may determine that the candidate cause is the cause of the data shift if the number of times the null hypothesis is rejected is greater (third determination rule).

[0095] In the example of Fig. 12, all four two-sample tests 1201 to 1204 determine that the null hypothesis is adopted. Therefore, when the first determination rule is applied, the data shift determination unit 206 of the detection device 100 determines that the lower-body clothing, which is a cause candidate, is not the cause of the data shift. On the other hand, even when the second determination rule is applied, the data shift determination unit 206 of the detection device 100 determines that the lower-body clothing, which is a cause candidate, is not the cause of the data shift. Whether the first or second determination rule is to be adopted can be set by the user.

[0096] Furthermore, the detection apparatus 100 may exclude from the data shift determination any two-sample test result in which the difference in the number of contribution value maps between the two distributions subject to the two-sample test exceeds an allowable range among the multiple two-sample tests 1201 to 1204. In the example of Fig. 12, the difference is 0 in all two-sample test results, so none of the two-sample test results are excluded from the data shift determination.

[0097] Furthermore, the detection device 100 may exclude the results of a two-sample test from the multiple two-sample tests 1201 to 1204 in which the total number of contribution value maps in the two distributions that are the subject of the two-sample test is equal to or less than a threshold value from the data shift determination.

[0098] For example, in the example of FIG. 12, if the threshold value is 50 (25% of the number of image data: 200), the results of the two-sample tests 1202 and 1203 are not subject to data shift determination.

[0099] 10 to 12, Example 1 of cause identification based on the combined feature determination results is explained, where the subject of consideration is gender and the candidate cause is lower-body clothing. However, although not shown, the same procedure can be carried out when the subject of consideration is gender and the candidate cause is footwear.

[0100] Next, a second example of identifying a cause based on the combined feature determination results will be described with reference to FIGS. 13 to 15, where the subject of investigation is gender and the possible causes are lower body clothing and footwear.

[0101] [Map classification B based on combined feature judgment results] Fig. 13 is an explanatory diagram showing example 1 of map classification B based on the combined feature determination result. In Fig. 13, the detection apparatus 100 causes the classification unit 903 to classify the contribution value map group C3(Ma) using the matching result T3abc.

[0102] Specifically, for example, the detection device 100 classifies the contribution value map group C3(Ma) into a first contribution value map group C3(Ma)-bc1 aggregated as male, trousers, and sneakers, a second contribution value map group C3(Ma)-bc2 aggregated as male, trousers, and sandals, a third contribution value map group C3(Ma)-bc3 aggregated as male, skirt, and sneakers, a fourth contribution value map group C3(Ma)-bc4 aggregated as male, skirt, and sandals, a fifth contribution value map group C3(Ma)-bc5 aggregated as female, trousers, and sneakers, a sixth contribution value map group C3(Ma)-bc6 aggregated as female, trousers, and sandals, a seventh contribution value map group C3(Ma)-bc7 aggregated as female, skirt, and sneakers, and an eighth contribution value map group C3(Ma)-bc8 aggregated as female, skirt, and sandals. In this way, the classification result 1300 is obtained.

[0103] Fig. 14 is an explanatory diagram showing example 2 of map classification B based on the combination feature determination result. In Fig. 14, the detection device 100 uses the detection target image data group D3 and the matching result T3abc as a training data set to train the combination feature determination model Mabc using the detection target image data group D3. When the detection target image data group D3 is input, the combination feature determination model Mabc generates a contribution value map group C3(Mabc).

[0104] The detection apparatus 100 causes the classification unit 903 to classify the contribution value map group C3(Mabc) using the matching result T3abc. Specifically, for example, the detection device 100 classifies the data into a first contribution value map group C3(Mabc)-1 totaled as male, trousers, and sneakers, a second contribution value map group C3(Mabc)-2 totaled as male, trousers, and sandals, a third contribution value map group C3(Mabc)-3 totaled as male, skirt, and sneakers, a fourth contribution value map group C3(Mabc)-4 totaled as male, skirt, and sandals, a fifth contribution value map group C3(Mabc)-5 totaled as female, trousers, and sneakers, a sixth contribution value map group C3(Mabc)-6 totaled as female, trousers, and sandals, a seventh contribution value map group C3(Mabc)-7 totaled as female, skirt, and sneakers, and an eighth contribution value map group C3(Mabc)-8 totaled as female, skirt, and sandals. In this way, the classification result 1400 is obtained.

[0105] [Example of two-sample test using map classification B] 15 is an explanatory diagram showing an example of a two-sample test using map classification B. The detection device 100 executes two-sample tests 1501 to 1508 using the two-sample test unit 205 for each common combined feature based on the classification result 1300 of the contribution value map group C3(Ma) and the classification result 1400 of the contribution value map group C3(Mabc).

[0106] Specifically, for example, the detection device 100 performs two-sample test 1501 using first contribution value map group C3(Ma)-bc1 and first contribution value map group C3(Mabc)-1, whose combined features are male, trousers, and sneakers, as two distributions to be tested. The detection device 100 also performs two-sample test 1502 using second contribution value map group C3(Ma)-bc2 and second contribution value map group C3(Mabc)-2, whose combined features are male, trousers, and sandals, as two distributions to be tested.

[0107] Furthermore, the detection apparatus 100 performs two-sample test 1503 using third contribution value map group C3(Ma)-bc3 and third contribution value map group C3(Mabc)-3, whose combined feature is male, skirt, and sneakers, as two distributions to be tested for the two-sample test. The detection apparatus 100 also performs two-sample test 1504 using fourth contribution value map group C3(Ma)-bc4 and fourth contribution value map group C3(Mabc)-4, whose combined feature is male, skirt, and sneakers, as two distributions to be tested for the two-sample test. However, in the example of Fig. 15, since there are zero records for both fourth contribution value map group C3(Ma)-bc4 and fourth contribution value map group C3(Mabc)-4, two-sample test 1504 is not performed.

[0108] Furthermore, the detection device 100 performs a two-sample test 1505 using the fifth contribution value map group C3(Ma)-bc5 and the fifth contribution value map group C3(Mabc)-5, whose combined features are female, trousers, and sneakers, as two distributions to be tested for the two-sample test. The detection device 100 also performs a two-sample test 1506 using the sixth contribution value map group C3(Ma)-bc6 and the sixth contribution value map group C3(Mabc)-6, whose combined features are female, trousers, and sandals, as two distributions to be tested for the two-sample test.

[0109] Furthermore, the detection device 100 performs a two-sample test 1703 using the seventh contribution value map group C3(Ma)-bc7 and the seventh contribution value map group C3(Mabc)-7, whose combined features are female, skirt, and sneakers, as two distributions to be tested for the two-sample test. The detection device 100 also performs a two-sample test 1508 using the eighth contribution value map group C3(Ma)-bc8 and the eighth contribution value map group C3(Mabc)-8, whose combined features are female, skirt, and sneakers, as two distributions to be tested for the two-sample test.

[0110] The detection apparatus 100 determines whether or not a data shift has occurred in the cause candidate based on the two-sample test results in each of the two-sample tests 1501 to 1508 using the data shift determination unit 206. Specifically, for example, the detection apparatus 100 determines whether or not a data shift has occurred in the cause candidate by applying the first, second, or third determination rule in the same way as in the example of the two-sample test using map classification A using the data shift determination unit 206.

[0111] In the example of the two-sample test using map classification A, if each of the lower-body clothing and footwear is determined to be the cause of the data shift, the detection device 100 does not perform the two-sample test using map classification B. This reduces the calculation load. On the other hand, in the example of the two-sample test using map classification A, if each of the lower-body clothing and footwear is determined not to be the cause of the data shift, the detection device 100 performs the two-sample test using map classification B. In this case, although each of the lower-body clothing and footwear is determined not to be the cause of the data shift, the combination of the lower-body clothing and footwear may be determined to be the cause of the data shift, thereby improving the accuracy of detecting the cause of the data shift.

[0112] <Example of data shift detection processing procedure> Fig. 16 is a flowchart illustrating an example (first half) of a data shift detection process procedure performed by the detection device 100 according to the embodiment 2. Fig. 17 is a flowchart illustrating an example (second half) of a data shift detection process procedure performed by the detection device 100 according to the embodiment 2. The same processes as those in Figs. 5 to 7 are denoted by the same step numbers, and their description will be omitted.

[0113] 17, if the result of step S706 in Fig. 16 is Yes, the detecting apparatus 100 specifies possible causes via the acquiring unit 201 (step S1701). In the second embodiment, the possible causes are lower-body clothing, footwear, and a combination of lower-body clothing and footwear.

[0114] The detection device 100, using the feature determination unit 901, inputs the detection target image data group D3 for the subject of consideration into a feature determination model for the subject of consideration to obtain feature determination results, and inputs the detection target image data group for the candidate cause into a feature determination model for the candidate cause to obtain feature determination results (step S1702).

[0115] Specifically, for example, the detection device 100 acquires a gender determination result R(Ma). Furthermore, if the possible cause is lower-body clothing, the detection device 100 acquires a lower-body clothing determination result R3(Mb), if the possible cause is footwear, the detection device 100 acquires a footwear determination result R3(Mc), and if the possible cause is a combination of lower-body clothing and footwear, the detection device 100 acquires both the lower-body clothing determination result R3(Mb) and the footwear determination result R3(Mc).

[0116] The detection device 100 generates a matching result by combining the feature determination result of the subject of consideration with the feature determination result of the candidate cause using the feature determination unit 901 (step S1703). Specifically, for example, if the candidate cause is lower-body clothing, the detection device 100 generates a matching result T3ab; if the candidate cause is footwear, the detection device 100 generates a matching result T3ac; and if the candidate cause is a combination of lower-body clothing and footwear, the detection device 100 generates a matching result T3abc.

[0117] The detection device 100, using the feature determination unit 901, inputs the pre-shift image data group D2 for the subject of consideration into a feature determination model for the subject of consideration to obtain a feature determination result, and inputs the pre-shift image data group D2 for the candidate cause into a feature determination model for the candidate cause to obtain a feature determination result. Specifically, for example, the detection device 100 obtains a gender determination result R2(Ma). Furthermore, if the candidate cause is lower-body clothing, the detection device 100 obtains a lower-body clothing determination result R2(Mb). If the candidate cause is footwear, the detection device 100 obtains a footwear determination result R2(Mc). If the candidate cause is a combination of lower-body clothing and footwear, the detection device 100 obtains both the lower-body clothing determination result R2(Mb) and the footwear determination result R2(Mc).

[0118] The detection device 100 generates a pre-shift matching result by combining the pre-shift characteristic determination result of the subject to be considered and the pre-shift characteristic determination result of the cause candidate using the characteristic determination unit 901 (step S1705). Specifically, for example, if the cause candidate is lower-body clothing, the detection device 100 generates a matching result T2ab, if the cause candidate is footwear, the detection device 100 generates a matching result T2ac, and if the cause candidate is a combination of lower-body clothing and footwear, the detection device 100 generates a matching result T2abc.

[0119] The detection apparatus 100 uses the feature determination unit 901 to identify candidate causes for which there is a ratio change (step S1706). Specifically, for example, the detection apparatus 100 compares the matching results T2ab and T3ab to detect whether or not there is a ratio change, compares the matching results T2ac and T3ac to detect whether or not there is a ratio change, and compares the matching results T2abc and T3abc to detect whether or not there is a ratio change. The detection apparatus 100 identifies candidate causes for the matching results for which there is a ratio change. That is, the detection apparatus 100 narrows down the candidate causes. This makes it possible to exclude candidate causes that have not been identified, i.e., for which there is no ratio change, from subsequent processing (steps S1707 to S1711), thereby reducing the calculation load. Note that the user can select in advance whether or not to perform step S1707.

[0120] The detecting device 100 classifies the contribution value map group under consideration using the matching result with the classification unit 903 (step S1707). Specifically, for example, as shown in FIG. 10, the detecting device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3ab and outputs the classification result 1000. Similarly, if the cause candidate is footwear, the detecting device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3ac. Furthermore, as shown in FIG. 13, the detecting device 100 classifies the contribution value map group C3(Ma) under consideration using the matching result T3abc and outputs the classification result 1300.

[0121] The detection device 100 uses the learning unit 902 to learn a combination feature determination model for the subject of study and the candidate cause, inputs the detection target image data group D3 into the learned combination feature determination model, generates a contribution value map group, and uses the classification unit 903 to classify the matching results (step S1708).

[0122] 11, the detection device 100 learns a combination feature determination model Mab using the detection target image data group D3 and the matching result T3ab, inputs the detection target image data group D3 to the combination feature determination model Mab, and obtains a contribution value map group C3(Mab) output from the combination feature determination model Mab. The detection device 100 then classifies the contribution value map group C3(Mab) using the matching result T3ab and outputs the classification result 1100.

[0123] Similarly, if the cause candidate is footwear, the detection device 100 uses the matching result T3ac to train a combination feature determination model Mac using the detection target image data group D3 and the matching result T3ac, inputs the detection target image data group D3 to the combination feature determination model Mac, and obtains a contribution value map group C3(Mac) output from the combination feature determination model Mac. The detection device 100 then classifies the contribution value map group C3(Mac) using the matching result T3ac.

[0124] 14, the detection device 100 uses the detection target image data group D3 and the matching result T3abc to train a combination feature determination model Mabc, inputs the detection target image data group D3 to the combination feature determination model Mabc, and obtains a contribution value map group C3(Mabc) output from the combination feature determination model Mabc. The detection device 100 then classifies the contribution value map group C3(Mabc) using the matching result T3abc, and outputs the classification result 1400.

[0125] The detection device 100 executes a two-sample test for each common combined feature using the two-sample test unit 205 (step S1709). Specifically, for example, the detection device 100 executes two-sample tests 1201 to 1204 and 1501 to 1508 for each common combined feature as shown in FIGS.

[0126] The detection apparatus 100 determines whether the cause candidate is the cause of the data shift occurrence based on the two-sample test result of step S1709 using the data shift determination unit 206 (step S1710). Then, the detection apparatus 100 outputs the determination result of step S1710 using the output unit 204 (step S1711). This completes the series of processes of the detection apparatus 100.

[0127] As described above, according to the second embodiment, even if there are multiple cause candidates, the cause of the data shift can be identified.

[0128] Also in the second embodiment, the detection apparatus 100 may output the two-sample test result of step S1709 via the output unit 204. For the two-sample test in which the null hypothesis is rejected in step S1709, the detection apparatus 100 may generate a difference heat map via the difference generation unit 203 and output the difference heat map via the output unit 204, as shown in FIG. 4. Furthermore, the detection apparatus 100 may output the classification results 1000, 1100, 1300, and 1400 obtained by the classification unit 903 via the output unit 204. Furthermore, the detection apparatus 100 may execute the processing shown in FIG. 17 without executing steps S502, S503, S604, and S706 in FIG. 16.

[0129] The detection device 100 according to the first and second embodiments can also be configured as follows (1) to (9).

[0130] (1) The detection device 100 includes a processor 101 that executes a program and a storage device 102 that stores the program. The processor 101 an acquisition process (acquisition unit 201) for acquiring a first contribution value map group C2(Mi) which is a collection of first contribution value maps that contributed to the features of each first image data of a first image data group (pre-shift image data group D2), and a second contribution value map group C3(Mi) which is a collection of second contribution value maps that contributed to the features of each second image data of a second image data group (detection target image data group D3) captured after the first image data group (pre-shift image data group D2); a first two-sample testing process for performing a two-sample testing using the first contribution value map group C2(Mi and the second contribution value map group C3(Mi) acquired by the acquisition process (the two-sample testing unit 205, (step S604)); an output process (output unit 204) for outputting the two-sample testing result obtained by the first two-sample testing process; Execute.

[0131] (2) Also, in (1) above, In the acquisition process, the processor 101 acquires the first image data group and the second image data group (step S501). The processor 101 A map generation process is executed in which the first image data of each of the first image data groups (pre-shift image data group D2) acquired by the acquisition process is input to a feature determination model Mi that determines features related to image data and calculates contribution values ​​related to the features for each region in the image data, and as a result, the first contribution value map is generated for each of the first image data, and the second image data of each of the second image data groups (detection target image data group D3) is input to the feature determination model Mi, and as a result, the second contribution value map is generated for each of the second image data (step S503). In the first two-sample testing process, the processor 101 performs a two-sample testing using a first contribution value map group C2(Mi), which is the first contribution value map for each of the first image data generated by the map generation process, and a second contribution value map group C3(Mi), which is the second contribution value map for each of the second image data generated by the map generation process.

[0132] (3) Also, in (1) above, The processor 101 Based on the first two-sample test result of the first two-sample test process, a determination process is performed to determine whether or not a data shift has occurred between the first image data group (pre-shift image data group D2) and the second image data group (detection target image data group D3) (step S605). In the output process, the processor 101 outputs the determination result of the determination process (step S606).

[0133] (4) Also, in (3) above, The processor 101 If it is determined by the determination process that a data shift has occurred, a difference generation process is executed to generate a difference map DC(Mi) indicating a difference between a representative first contribution value map C2(Mi)av in the first contribution value map group C2(Mi) and a representative second contribution value map C3(Mi)av in the second contribution value map group (C3(Mi)) (step S504). In the output process, the processor 101 outputs the difference map DC(Mi) generated by the difference generation process (step S505).

[0134] (5) Also, in (1) above, The processor 101 A designation process for designating the subject of consideration (gender) and the cause candidate (lower body clothing) (step S1701) a feature determination process for generating a first matching result T3ab by collating the second image data group (detection target image data group D3) into a first feature determination model Ma related to the subject of consideration (gender) to obtain a first feature determination result group R3(Ma) output from the first feature determination model Ma, and the second image data group (detection target image data group D3) into a second feature determination model Mb related to the candidate cause (lower-body clothing) to obtain a second feature determination result group R3(Mb) output from the second feature determination model Mb, for each combination (male & pants, male & skirt, female & pants, female & skirt) of the type of the subject of consideration (gender) and the type (pants, skirt) of the candidate cause (lower-body clothing) (steps S1702 and S1703); a first classification process of classifying the second contribution value map group C3(Mi) based on the first matching result T3ab (step S1708); a learning process for generating a combined feature determination model Mab that combines the subject (gender) and the cause candidate (lower-body clothing) by learning the second image data group (detection target image data group D3) and the first matching result T3ab generated by the feature determination process as a learning data set (step S1708); a second process is executed in which, as a result of inputting the second image data group (detection target image data group D3) into the combination feature determination model Mab generated by the learning process, a first combination contribution value map regarding the combination of the subject to be considered (gender) and the cause candidate (lower-body clothing) is generated for each of the second image data, and a first combination contribution value map group C3(Mab), which is a set of the first combination contribution value maps for each of the second image data, is classified based on the first matching result T3ab (step S1708). In the output process, the processor 101 outputs a first classification result 1000 obtained by the first classification process and a second classification result 1100 obtained by the second classification process.

[0135] (6) Also, in (5) above, The processor 101 A second two-sample testing process is executed to perform a two-sample testing for each common combination in the first classification result 1000 and the second classification result 1100 (step S1709). In the output process, the processor 101 outputs the second two-sample testing result obtained by the second two-sample testing process (output unit 204).

[0136] (7) Also, in (6) above, The processor 101 Based on the second two-sample test result, a determination process is performed to determine whether or not a data shift has occurred between the first image data group (pre-shift image data group D2) and the second image data group (detection target image data group D3) (step S1710). In the output process, the processor 101 outputs the determination result of the determination process (step S1711).

[0137] (8) Also, in (5) above, In the feature determination process, the processor 101 generates second matching results T2ab by aggregating a third feature determination result group R2(Ma) obtained by inputting the first image data group (pre-shift image data group D2) into the first feature determination model Ma and outputting it from the first feature determination model Ma, and a fourth feature determination result group R2(Mb) obtained by inputting the first image data group (pre-shift image data group D2) into the second feature determination model Mb and outputting it from the second feature determination model Mb, for each combination of the type (male, female) of the subject (gender) and the type (pants, skirt) of the cause candidate (lower-body clothing), and determines whether there is a predetermined ratio change between the first matching result T3ab and the second matching result T2ab (steps S1704, 1705, 1706). In the first classification process, the processor 101 classifies the second contribution value map group C3(Ma) for the specific cause candidate determined to have the ratio change based on the first matching result T3ab.

[0138] (9) The detection device 100 includes a processor 101 that executes a program and a storage device 102 that stores the program. The processor 101 an acquisition process (acquisition unit 201) for acquiring a first contribution value map group C2(Mi) which is a collection of first contribution value maps that contributed to the features of each first image data of a first image data group (pre-shift image data group D2), and a second contribution value map group C3(Mi) which is a collection of second contribution value maps that contributed to the features of each second image data of a second image data group (detection target image data group D3) captured after the first image data group (pre-shift image data group D2); a difference generation process for generating a difference map DC(Mi) indicating the difference between a representative first contribution value map C2(Mi)av in the first contribution value map group C2(Mi) acquired by the acquisition process and a representative second contribution value map C3(Mi)av in the second contribution value map group C3(Mi) (step S504); an output process for outputting the difference map DC(Mi) generated by the difference generation process (step S505); Execute.

[0139] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0140] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits, or may be realized in software by having processor 101 interpret and execute programs that realize the respective functions.

[0141] Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0142] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0143] 100 Detection device 101 processors 201 Acquisition Department 202 Map Generation Unit 203 Difference generation part 204 Output section 205 Two-Sample Test Section 206 Data shift determination unit 901 Feature Determination Unit 902 Learning Department 903 Classification Department 210 Contribution Value Map DB< / xai>

Claims

1. A detection device having a processor that executes a program and a storage device that stores the program, The processor: an acquisition process for acquiring a first contribution value map group, which is a collection of first contribution value maps that contributed to the features of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the features of each second image data in a second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection device characterized by performing the above.

2. 2. The detection device according to claim 1, In the acquisition process, the processor acquires the first image data group and the second image data group; The processor: a map generation process is executed in which, as a result of inputting the first image data of each of the first image data groups acquired by the acquisition process into a feature determination model that determines features related to the image data and calculates contribution values ​​related to the features for each region in the image data, the first contribution value map is generated for each of the first image data, and, as a result of inputting the second image data of each of the second image data groups into the feature determination model, the second contribution value map is generated for each of the second image data, In the first two-sample testing process, the processor performs a two-sample testing using a first contribution value map group, which is the first contribution value map for each of the first image data generated by the map generation process, and a second contribution value map group, which is the second contribution value map for each of the second image data generated by the map generation process. A detection device characterized by:

3. 2. The detection device according to claim 1, The processor: executing a determination process to determine whether or not a data shift has occurred between the first image data group and the second image data group based on a first two-sample test result obtained by the first two-sample test process; In the output process, the processor outputs a determination result obtained by the determination process. A detection device characterized by:

4. 4. The detection device according to claim 3, The processor: when it is determined by the determination process that a data shift has occurred, a difference generation process is executed to generate a difference map indicating a difference between a representative first contribution value map in the first contribution value map group and a representative second contribution value map in the second contribution value map group; In the output process, the processor outputs the difference map generated by the difference generation process. A detection device characterized by:

5. 2. The detection device according to claim 1, The processor: A designation process for designating the subject of study and potential causes; a feature determination process for generating a first matching result by aggregating a first feature determination result group output from a first feature determination model related to the object of consideration by inputting the second image data group into the first feature determination model and a second feature determination result group output from a second feature determination model related to the cause candidate by inputting the second image data group into the second feature determination model, for each combination of the type of the first feature determination result group and the type of the second feature determination result group; a first classification process for classifying the second contribution value map group based on the first matching result; a learning process for generating a combined feature determination model that combines the subject to be considered and the cause candidate by learning the second image data group and the first matching result generated by the feature determination process as a learning data set; a second classification process is executed to generate a first combination contribution value map for each of the second image data as a result of inputting the second image data group into the combination feature determination model generated by the learning process, the first combination contribution value map being a set of the first combination contribution value maps for each of the second image data, based on the first matching result; In the output process, the processor outputs a first classification result obtained by the first classification process and a second classification result obtained by the second classification process. A detection device characterized by:

6. 6. The detection device according to claim 5, The processor: performing a second two-sample testing process that performs a two-sample testing for each common combination in the first classification result and the second classification result; In the output process, the processor outputs a second two-sample test result obtained by the second two-sample test process. A detection device characterized by:

7. 7. The detection device according to claim 6, The processor: execute a determination process to determine whether or not a data shift has occurred between the first image data group and the second image data group based on the second two-sample test result; In the output process, the processor outputs a determination result obtained by the determination process. A detection device characterized by:

8. 6. The detection device according to claim 5, In the feature determination process, the processor generates second matching results by aggregating a third feature determination result group, which is obtained by inputting the first image data group into the first feature determination model and outputting the first feature determination model, and a fourth feature determination result group, which is obtained by inputting the first image data group into the second feature determination model and outputting the second feature determination model, for each combination of the type of the object of consideration and the type of the cause candidate, and determines whether or not there is a predetermined ratio change between the first matching results and the second matching results; In the first classification process, the processor classifies the second contribution value map group based on the first matching result for the specific cause candidate determined to have the ratio change. A detection device characterized by:

9. A detection device having a processor that executes a program and a storage device that stores the program, The processor: an acquisition process for acquiring a first contribution value map group, which is a collection of first contribution value maps that contributed to the features of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the features of each second image data in a second image data group captured after the first image data group; a difference generation process for generating a difference map indicating a difference between a representative first contribution value map in the first contribution value map group acquired by the acquisition process and a representative second contribution value map in the second contribution value map group; an output process for outputting the difference map generated by the difference generation process; A detection device characterized by performing the above.

10. A detection method executed by a detection device having a processor that executes a program and a storage device that stores the program, the processor: an acquisition process for acquiring a first contribution value map group, which is a collection of first contribution value maps that contributed to the features of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the features of each second image data in a second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection method characterized by carrying out the steps of:

11. The processor an acquisition process for acquiring a first contribution value map group, which is a collection of first contribution value maps that contributed to the features of each first image data in a first image data group, and a second contribution value map group, which is a collection of second contribution value maps that contributed to the features of each second image data in a second image data group captured after the first image data group; a first two-sample testing process for performing a two-sample testing using the first contribution value map group and the second contribution value map group acquired by the acquisition process; an output process for outputting a two-sample testing result obtained by the first two-sample testing process; A detection program characterized by executing the above.

Images