Information Processing Apparatus, Information Processing Method, and Recording Medium

Abstract

An information processing device (10) comprises a feature amount acquisition means (110) that acquires a query feature amount related to an object of an authentication process, a first score calculation means (121) that calculates a first score indicating a degree of matching between the query feature amount and a target feature amount registered in advance, a second score calculation means (122) that calculates, on the basis of the first score, a second score indicating a degree of mismatching between the query feature amount and the target feature amount, and a map generation means (130) that generates a saliency map related to the authentication process by setting the first score as a first class score corresponding to a first class and setting the second score as a second class score corresponding to a second class.

Description

Technical Field

[0001] This disclosure relates to the technical field of information processing apparatuses, information processing methods, and recording media.

Background Art

[0002] As this type of apparatus, one that visually displays the degree of contribution in the inference result is known. For example, in Patent Document 1, it is disclosed that an activation map is generated from a feature map generated by a convolutional neural network by Grad-CAM processing.

[0003] As other related technologies, for example, in Patent Document 2, it is disclosed that weights representing the degree of influence on the target category in the feature map are output for each channel and each output position of the convolutional layer. In Patent Document 3, it is disclosed that when identifying the class of input data, a feature vector extracted from the input data is processed as a weight vector.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to improve the technologies disclosed in the prior art documents.

Means for Solving the Problems

[0006] One aspect of the information processing apparatus of this disclosure includes a feature quantity acquisition unit that acquires query feature quantities related to an object of authentication processing, a first score calculation unit that calculates a first score indicating the degree of match between the query feature quantities and pre-registered target feature quantities, a second score calculation unit that calculates a second score indicating the degree of non-match between the query feature quantities and the target feature quantities based on the first score, and a map generation unit that generates a saliency map related to the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class.

[0007] One aspect of the information processing method of this disclosure is to acquire, by at least one computer, query feature quantities related to an object of authentication processing, calculate a first score indicating the degree of match between the query feature quantities and pre-registered target feature quantities, calculate a second score indicating the degree of non-match between the query feature quantities and the target feature quantities based on the first score, and generate a saliency map related to the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class.

[0008] One aspect of the recording medium of this disclosure has recorded thereon a computer program that causes at least one computer to execute an information processing method of acquiring query feature quantities related to an object of authentication processing, calculating a first score indicating the degree of match between the query feature quantities and pre-registered target feature quantities, calculating a second score indicating the degree of non-match between the query feature quantities and the target feature quantities based on the first score, and generating a saliency map related to the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a recording medium will be described with reference to the drawings.

[0011] <First Embodiment> The information processing apparatus according to the first embodiment will be described with reference to FIGS. 1 to 3.

[0012] (Hardware Configuration) First, with reference to FIG. 1, the hardware configuration of the information processing apparatus according to the first embodiment will be described. FIG. 1 is a block diagram showing the hardware configuration of the information processing apparatus according to the first embodiment.

[0013] As shown in FIG. 1, the information processing apparatus 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing apparatus 10 may further include an input device 15 and an output device 16. The above-described processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, ROM 13, and storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown). The processor 11 may acquire (i.e., read) a computer program from a device (not shown) disposed outside the information processing apparatus 10 via a network interface. By executing the read computer program, the processor 11 controls the RAM 12, storage device 14, input device 15, and output device 16. In particular, in this embodiment, when the processor 11 executes the read computer program, a functional block for generating a saliency map is realized within the processor 11. Thus, the processor 11 may function as a controller that executes various controls in the information processing apparatus 10.

[0015] Processor 11 may be configured as, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), DSP (Demand-Side Platform), or ASIC (Application Specific Integrated Circuit). Processor 11 may be configured with one of these, or may be configured to use a plurality in parallel.

[0016] RAM 12 temporarily stores the computer programs executed by processor 11. RAM 12 temporarily stores the data temporarily used by processor 11 when processor 11 is executing a computer program. RAM 12 may be, for example, D-RAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). Also, instead of RAM 12, other types of volatile memory may be used.

[0017] ROM 13 stores the computer programs executed by processor 11. ROM 13 may also store other fixed data. ROM 13 may be, for example, P-ROM (Programmable Read Only Memory) or EPROM (Erasable Read Only Memory). Also, instead of ROM 13, other types of non-volatile memory may be used.

[0018] Storage device 14 stores the data that information processing device 10 stores long-term. Storage device 14 may operate as a temporary storage device for processor 11. Storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives input instructions from the user of the information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a portable terminal such as a smartphone or a tablet. The input device 15 may be a device capable of voice input including, for example, a microphone.

[0020] The output device 16 is a device that outputs information regarding the information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) capable of displaying information regarding the information processing device 10. Also, the output device 16 may be a speaker or the like capable of outputting information regarding the information processing device 10 as voice. The output device 16 may be configured as a portable terminal such as a smartphone or a tablet. Also, the output device 16 may be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information regarding the information processing device 10 as voice.

[0021] Note that in FIG. 1, an example of the information processing device 10 configured to include a plurality of devices has been given, but all or some of these functions may be realized as one device. In that case, the information processing device 10 may be configured to include only the above-described processor 11, RAM 12, and ROM 13, and for the other components (i.e., the storage device 14, the input device 15, and the output device 16), an external device connected to the information processing device 10 may be provided with them. Also, the information processing device 10 may be one in which some arithmetic functions are realized by an external device (e.g., an external server or the cloud).

[0022] (Functional Configuration) Next, with reference to FIG. 2, the functional configuration of the information processing device 10 according to the first embodiment will be described. FIG. 2 is a block diagram showing the functional configuration of the information processing device according to the first embodiment.

[0023] The information processing apparatus 10 according to the first embodiment is configured to generate a saliency map related to authentication processing. For example, the information processing apparatus 10 is configured to be able to generate a saliency map that visualizes portions with a high degree of contribution related to authentication processing. Note that the type of authentication processing here is not particularly limited, and may be, for example, biometric authentication processing using a face image or an iris image. The information processing apparatus 10 may be configured to be able to execute authentication processing by itself, or may be configured not to execute authentication processing by itself (for example, the authentication processing may be configured to be performed by an external device).

[0024] As shown in FIG. 2, the information processing apparatus 10 according to the first embodiment includes, as components for realizing its functions, a feature amount acquisition unit 110, a collation unit 120, and a map generation unit 130. Each of the feature amount acquisition unit 110, the collation unit 120, and the map generation unit 130 may be, for example, a processing block realized by the above-described processor 11 (see FIG. 1). Also, each of the feature amount acquisition unit 110, the collation unit 120, and the map generation unit 130 may be configured as a neural network.

[0025] The feature amount acquisition unit 110 is configured to be able to acquire a query feature amount related to an object of authentication processing. For example, the feature amount acquisition unit 110 may be configured to be able to extract a query feature amount by executing various processes on a target image. The feature amount acquisition unit 110 is configured to be able to output the acquired feature amount to the collation unit 120. Also, the feature amount acquisition unit 110 may be configured to be able to output information used for map generation (for example, intermediate feature amounts used for gradient calculation, etc.) to the map generation unit 130.

[0026] The matching unit 120 is configured to be able to execute a matching process using the query feature amount acquired by the feature amount acquisition unit 110 and the target feature amount registered in advance. Specifically, the matching unit 120 calculates a matching score from the query feature amount and the target feature amount. In this case, the matching unit 120 may compare the calculated matching score with a determination threshold value for determining that the score is that of the person himself / herself (i.e., the registered target) and a determination threshold value for determining that the score is that of another person (i.e., not any of the registered targets), and perform a matching process. The matching score may be calculated, for example, as the cosine similarity between the query feature amount and the target feature amount.

[0027] The matching unit 120 includes a first score calculation unit 121 and a second score calculation unit 122 as components for calculating the above-described matching score. The first score calculation unit 121 is configured to be able to calculate a first score (so-called, own score) indicating the degree of coincidence between the query feature amount and the target feature amount. When there are a plurality of registered targets, the first score may be calculated for each of the plurality of targets. That is, a plurality of first scores may be calculated. The second score 122 is configured to be able to calculate a second score (so-called, other score) indicating the degree of non-coincidence between the query feature amount and the target feature amount.

[0028] The map generation unit 130 is configured to be able to generate a saliency map using the matching score calculated by the matching unit. Specifically, the map generation unit 130 treats the first score calculated by the first score calculation unit 121 as a first class score corresponding to the first class. For example, the map generation unit 130 may regard the first score y represented by the following formula (1) as the first class score y t as. c and may consider it as.

[0029]

Equation

[0030] Further, the map generation unit 130 treats the second score calculated by the second score calculation unit 122 as a second class score corresponding to a second class (i.e., a class different from the first class). For example, from the probability p of others expressed as the following formula (2), the second class score x may be defined as in the following formula (3).

[0031] [Number] Note that λ is a determination threshold for determining whether a person is the user or another person, and s is a positive number (hyperparameter).

[0032] As described above, by treating the first score (user score) as the first class score and the second score (others score) as the second class score, an authentication task for which classes are not defined in advance can be treated as a class classification task and processed. Note that if the second class score is a constant (for example, simply a threshold), the gradient calculation result will all be zero and a desired result cannot be obtained. However, by defining it as described above, it is possible to avoid such a problem.

[0033] [Flow of operations] Next, with reference to FIG. 3, the flow of operations of the information processing apparatus 10 according to the first embodiment (specifically, the flow until a saliency map is output) will be described. FIG. 3 is a flowchart showing the flow of operations of the information processing apparatus according to the first embodiment.

[0034] As shown in FIG. 3, when the operation by the information processing apparatus 10 according to the first embodiment is started, first, the feature amount acquisition unit 110 acquires query feature amounts regarding the target of the authentication process (step S101). The query feature amounts acquired by the feature amount acquisition unit 110 are output to the collation unit 120. Further, intermediate feature amounts and the like acquired by the feature amount acquisition unit 110 may be output to the map generation unit 130.

[0035] Subsequently, the first score calculation unit 121 in the matching unit 120 calculates a first score indicating the degree of match between the query feature amount and the target feature amount (step S102). Also, the second score calculation unit 122 in the matching unit 120 calculates a second score indicating the degree of non-match between the query feature amount and the target feature amount (step S103). The first score calculated by the first score calculation unit 121 and the second score calculated by the second score calculation unit 122 are output to the map generation unit 130, respectively.

[0036] Subsequently, the map generation unit 130 generates a saliency map by treating the first score calculated by the first score calculation unit 121 as a first class score and treating the second score calculated by the second score calculation unit 122 as a second class score (step S104). Then, the map generation unit 130 outputs the generated saliency map (step S105). The saliency map may be output by, for example, the above-described output device 16 (see FIG. 1). For example, the saliency map may be displayed as an image using a display. Note that the specific visualization method of the saliency map will be described in detail in other embodiments described later.

[0037] (Technical Effect) Next, the technical effect obtained by the information processing apparatus 10 according to the first embodiment will be described.

[0038] As described with reference to FIGS. 1 to 3, in the information processing apparatus 10 according to the first embodiment, a saliency map is generated by treating the matching score in the authentication process as a class score. In this way, an authentication task for which classes are not defined in advance can be treated as a class classification task and processed, so that it is possible to appropriately generate a saliency map.

[0039] <Second Embodiment> The information processing apparatus 10 according to the second embodiment will be described with reference to FIGS. 4 and 5. Note that the second embodiment is different from the first embodiment described above only in some configurations and operations, and the other parts may be the same as those of the first embodiment. For this reason, in the following, the parts different from the first embodiment already described will be described in detail, and the description of other overlapping parts will be omitted as appropriate.

[0040] (Functional Configuration) First, with reference to FIG. 4, the functional configuration of the information processing apparatus 10 according to the second embodiment will be described. FIG. 4 is a block diagram showing the functional configuration of the information processing apparatus according to the second embodiment. In FIG. 4, the same reference numerals are given to the same elements as those shown in FIG. 2.

[0041] As shown in FIG. 4, the information processing apparatus 10 according to the second embodiment includes, as components for realizing its functions, a feature amount generation unit 110, a collation unit 120, and a CAM generation unit 135. That is, the information processing apparatus 10 according to the second embodiment includes a CAM generation unit 135 instead of the map generation unit 130 (see FIG. 2) in the first embodiment. The CAM generation unit 135 may be a processing block realized by, for example, the above-described processor 11 (see FIG. 1).

[0042] The CAM generation unit 135 is configured to be able to generate a CAM (Class Activation Map), which is a specific example of a saliency map. Similar to the map generation unit 130 (see FIG. 2) of the first embodiment described above, the CAM generation unit 135 treats the first score calculated by the first score calculation unit 121 as the first class score and the second score calculated by the second score calculation unit 122 as the second class score. In this way, the CAM generation unit 135 generates a CAM from the first score and the second score.

[0043] Regarding the specific generation method of CAM, since existing technologies can be appropriately adopted, detailed description here is omitted. Note that the CAM generation unit 135 may use various methods derived from CAM. For example, the CAM generation unit 135 may generate CAM using the method of Grad-CAM (Gradient-weighted Class Activation Map). In this case, for example, by regarding the t-th target feature amount as the weight W of Grad-CAM c the collation score can be treated as a class score

[0044] (Flow of operation) Next, with reference to FIG. 5, the flow of operation of the information processing apparatus 10 according to the second embodiment will be described. FIG. 5 is a flowchart showing the flow of operation of the information processing apparatus according to the second embodiment. In FIG. 5, the same reference numerals are given to the same processes as those shown in FIG. 3

[0045] As shown in FIG. 5, when the operation by the information processing apparatus 10 according to the second embodiment is started, first, the feature amount acquisition unit 110 acquires a query feature amount regarding the target of the authentication process (step S101). Then, the first score calculation unit 121 in the collation unit 120 calculates a first score indicating the degree of coincidence between the query feature amount and the target feature amount (step S102). Also, the second score calculation unit 122 in the collation unit 120 calculates a second score indicating the degree of non-coincidence between the query feature amount and the target feature amount (step S103).

[0046] Subsequently, the CAM generation unit 135 generates CAM by treating the first score calculated by the first score calculation unit 121 as a first class score and treating the second score calculated by the second score calculation unit 122 as a second class score (step S201). Then, the CAM generation unit 135 outputs the generated CAM (step S202). The CAM may be output, for example, by the above-described output device 16 (see FIG. 1) or the like. For example, the CAM may be displayed as an image using a display. Note that the specific visualization method of the CAM will be described in detail in other embodiments described later

[0047] (Technical effect) Next, the technical effect obtained by the information processing apparatus 10 according to the second embodiment will be described.

[0048] As described with reference to FIGS. 4 and 5, in the information processing apparatus 10 according to the second embodiment, a CAM is generated by treating the matching score in the authentication process as a class score. By doing so, an authentication task for which classes are not defined in advance can be treated as a class classification task and processed, so that it is possible to appropriately generate a CAM.

[0049] <Third Embodiment> The information processing apparatus 10 according to the third embodiment will be described with reference to FIGS. 6 to 8. Note that the third embodiment is different from the above-described first and second embodiments only in some configurations and operations, and the other parts may be the same as those of the first and second embodiments. For this reason, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0050] (Effect of normalization processing) First, with reference to FIG. 6, the influence of the normalization processing executed by the information processing apparatus 10 according to the third embodiment will be described. FIG. 6 is a schematic diagram showing the influence of the normalization processing in gradient calculation.

[0051] The authentication process may include normalization processing. For example, the normalization processing may include processing for normalizing query feature amounts and target feature amounts. As a specific example of the normalization processing, there is processing for normalizing the feature amount map F ij k (where ij indicates an element of the map). However, the normalization processing according to the present embodiment is not limited to the following example.

[0052]

Equation

[0053] Then, the gradient calculation for the above normalization process is as shown in the following formula (6).

[0054] [Number] Note that δ is the Kronecker delta.

[0055] The first term in the above formula (6) corresponds to f in FIG. 6 and does not include the influence of the normalization process. On the other hand, when the second term is included, it corresponds to w in FIG. 6 and includes the influence of the normalization process. Thus, the directions of the vectors are completely different between those including the influence of the normalization process and those not.

[0056] Note that the first term that does not include the influence of the normalization process is easy for humans to intuitively understand. On the other hand, when the second term including the influence of the normalization process is included, although it is easy for machines to understand, it is difficult for humans to intuitively understand. Therefore, when generating a saliency map from the one including the influence of the normalization process, a result that does not conform to human intuitive understanding will be obtained. The information processing apparatus 10 according to the present embodiment can generate an appropriate saliency map by eliminating the influence of such a normalization process (specifically, by performing a process of eliminating the influence of the second term in the above formula (6)).

[0057] (Functional Configuration) Next, with reference to FIG. 7, the functional configuration of the information processing apparatus 10 according to the third embodiment (particularly, the configuration of the CAM generation unit 135) will be described. FIG. 7 is a block diagram showing the functional configuration of the CAM generation unit in the information processing apparatus according to the third embodiment.

[0058] As shown in FIG. 7, the CAM generation unit 135 according to the second embodiment includes, as components for realizing its functions, a first importance weight calculation unit 1351, a second importance weight calculation unit 1352, a first CAM calculation unit 1353, and a second CAM calculation unit 1354.

[0059] The first importance weight calculation unit 1351 is configured to be able to calculate the importance weight affected by the normalization process. On the other hand, the second importance weight calculation unit 1352 is configured to be able to calculate the importance weight with the influence of the normalization process removed. The first importance weight calculation unit 1351 and the second importance weight calculation unit 1352 are input with collation scores, collation feature amounts, information used for other gradient calculations, and the like. Then, the first importance weight calculation unit 1351 and the second importance weight calculation unit 1352 calculate the importance weights respectively using the input information.

[0060] The first CAM calculation unit 1353 generates a first CAM using the importance weight calculated by the first importance weight calculation unit 1351 (that is, the importance weight affected by the normalization process) and intermediate feature amounts and the like. The first CAM is a CAM affected by the normalization process. On the other hand, the second CAM calculation unit 1354 generates a second CAM using the importance weight calculated by the second importance weight calculation unit 1352 (that is, the importance weight with the influence of the normalization process removed) and intermediate feature amounts and the like. The second CAM is a CAM with the influence of the normalization process removed.

[0061] (Flow of operations) Next, with reference to FIG. 8, the operation flow of the information processing apparatus 10 according to the third embodiment will be described. FIG. 8 is a flowchart showing the operation flow of the information processing apparatus according to the third embodiment. In FIG. 8, the same reference numerals are given to the same processes as those shown in FIG. 5.

[0062] As shown in FIG. 8, when the operation by the information processing apparatus 10 according to the third embodiment is started, first, the feature amount acquisition unit 110 acquires a query feature amount regarding the target of the authentication process (step S101). Then, the first score calculation unit 121 in the collation unit 120 calculates a first score indicating the degree of match between the query feature amount and the target feature amount (step S102). Also, the second score calculation unit 122 in the collation unit 120 calculates a second score indicating the degree of non-match between the query feature amount and the target feature amount (step S103).

[0063] Subsequently, the first importance weight calculation unit 1351 calculates an importance weight affected by the normalization process (step S301). Then, the first CAM calculation unit 1353 generates a first CAM affected by the normalization process using the importance weight calculated by the first importance weight calculation unit 1351 and intermediate feature amounts, etc. (step S302). On the other hand, the second importance weight calculation unit 1352 calculates an importance weight from which the influence of the normalization process is removed (step S303). Then, the second CAM calculation unit 1354 generates a second CAM from which the influence of the normalization process is removed using the importance weight calculated by the second importance weight calculation unit 1352 and intermediate feature amounts, etc. (step S304).

[0064] Subsequently, the CAM generation unit 135 outputs the generated CAM (step S202). The CAM generation unit 135 may output both the first CAM affected by the normalization process and the second CAM not affected by the normalization process, or may output either the first CAM or the second CAM. Note that specific visualization methods for the first CAM and the second CAM will be described in detail in other embodiments described later.

[0065] (Technical Effect) Next, the technical effect obtained by the information processing apparatus 10 according to the third embodiment will be described.

[0066] As described with reference to FIGS. 6 to 8, in the information processing apparatus 10 according to the third embodiment, a CAM from which the influence of the normalization process has been removed is generated. By doing so, a CAM that conforms to human intuitive understanding can be generated. Note that, in this embodiment, an example of generating a CAM is given, but the effect of removing the normalization process can also be obtained for saliency maps other than the CAM.

[0067] <Fourth Embodiment> The information processing apparatus 10 according to the fourth embodiment will be described with reference to FIG. 9. Note that the fourth embodiment describes the map visualization method in the above-described third embodiment, and the apparatus configuration, operation flow, etc. may be the same as those in the third embodiment. For this reason, hereinafter, parts different from the already described embodiments will be described in detail, and description of other overlapping parts will be omitted as appropriate.

[0068] (Visualization Method) First, with reference to FIG. 9, the visualization method of the saliency map by the information processing apparatus 10 according to the fourth embodiment will be described. FIG. 9 is a conceptual diagram showing an example of the visualization method by the information processing apparatus according to the fourth embodiment.

[0069] As shown in FIG. 9, in the information processing apparatus 10 according to the fourth embodiment, the CAM without the influence of the normalization process (that is, the second CAM calculated by the second CAM calculation unit 1354) and the CAM with the influence of the normalization process (that is, the first CAM calculated by the first CAM calculation unit 1353) are visualized by being associated with different parameters related to display. For example, visualization is performed such that the second CAM without the influence of the normalization process corresponds to hue, and the first CAM with the influence of the normalization process corresponds to saturation. By doing so, for example, by making the input correspond to brightness, the first CAM and the second CAM can be overlaid and displayed on the input image (black-and-white image).

[0070] Note that the parameters associated with the first CAM and the second CAM are not limited to the hue and saturation described above. However, for the first CAM that is not affected by the normalization process, it is preferably associated with parameters that have a stronger visual effect on humans (for example, the hue in the above example) compared to the second CAM.

[0071] (Technical Effect) Next, the technical effect obtained by the information processing apparatus 10 according to the fourth embodiment will be described.

[0072] As described with reference to FIG. 9, in the information processing apparatus 10 according to the fourth embodiment, visualization is performed so that the CAM not affected by the normalization process and the CAM affected by the normalization process correspond to different parameters. In this way, it becomes possible to realize visualization that conforms to human intuitive understanding.

[0073] <Fifth Embodiment> The information processing apparatus 10 according to the fifth embodiment will be described with reference to FIGS. 10 and 11. Note that the fifth embodiment is only different from the first to fourth embodiments described above in some configurations and operations, and the other parts may be the same as those of the first to fourth embodiments. Therefore, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0074] (Functional Configuration) First, the functional configuration of the information processing apparatus 10 according to the fifth embodiment will be described with reference to FIG. 10. FIG. 10 is a block diagram showing the functional configuration of the information processing apparatus according to the fifth embodiment. Note that in FIG. 10, the same reference numerals are given to the elements similar to those shown in FIG. 2.

[0075] As shown in FIG. 10, the information processing apparatus 10 according to the fifth embodiment includes, as components for realizing its functions, a feature amount acquisition unit 110, a collation unit 120, and a map generation unit 130. In particular, the map generation unit 130 according to the fifth embodiment includes an element decomposition unit 131.

[0076] The element decomposition unit 131 is configured to be able to decompose the contribution of each element of the feature amount map based on the query feature amount to the authentication process into a norm and a similarity. However, the element decomposition unit 131 assumes that the post-generation process of the feature amount map is mostly a linear process. Here, the term "mostly" is used because it is assumed that cases where the definition extension is obvious, such as the bias process included in the affine transformation, are considered. Examples of non-linear processes include activation function processes such as ReLU.

[0077] Assuming the above-mentioned premise of linear processing, the normalized feature amount can be expressed as in the following formula (7).

[0078]

Equation

[0079] Although the normalization process is a non-linear process, by calculating the normalization coefficient in advance as in the above formula (7), it can be defined as a linear process. Then, when reexamining the collation score in view of the above formula (7), as in the following formula (8), the contribution of each element can be decomposed and defined into a norm and an angle (similarity).

[0080]

Equation

[0081] ​ In the above configuration, a weighted sum of similarities is used. The similarity is an index often used in, for example, face authentication and the like. In this embodiment, the saliency map will be generated in consideration of the characteristics of such authentication processing.

[0082] (Flow of operations) Next, with reference to FIG. 11, the flow of operations of the information processing apparatus 10 according to the fifth embodiment will be described. FIG. 11 is a flowchart showing the flow of operations of the information processing apparatus according to the fifth embodiment. In FIG. 11, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0083] As shown in FIG. 11, when the operation by the information processing apparatus 10 according to the fifth embodiment is started, first, the feature amount acquisition unit 110 acquires a query feature amount regarding the target of the authentication process (step S101). Then, the first score calculation unit 121 in the collation unit 120 calculates a first score indicating the degree of coincidence between the query feature amount and the target feature amount (step S102). Also, the second score calculation unit 122 in the collation unit 120 calculates a second score indicating the degree of non - coincidence between the query feature amount and the target feature amount (step S103).

[0084] Subsequently, the map generation unit 130 decomposes the contribution of each element of the feature map into a norm and a similarity to generate a saliency map (step S501). Then, the map generation unit 130 outputs the generated saliency map (step S202). The specific visualization method using the decomposed norm and similarity will be described in detail in other embodiments described later.

[0085] (Technical effects) Next, the technical effects obtained by the information processing apparatus 10 according to the fifth embodiment will be described.

[0086] As described with reference to FIGS. 10 and 11, in the information processing apparatus 10 according to the fifth embodiment, the contribution of each element of the feature map is decomposed into a norm and a similarity. By doing so, it becomes possible to generate an appropriate saliency map in consideration of the characteristics of the authentication process premised on the normalization process.

[0087] <Sixth Embodiment> The information processing apparatus 10 according to the sixth embodiment will be described with reference to FIG. 12. Note that the sixth embodiment describes a method for visualizing the map in the above-described fifth embodiment, and the apparatus configuration and the flow of operations may be the same as those in the fifth embodiment. For this reason, hereinafter, parts different from the embodiments already described will be described in detail, and description of other overlapping parts will be omitted as appropriate.

[0088] (Visualization Method) First, with reference to FIG. 12, a method for visualizing the saliency map by the information processing apparatus 10 according to the sixth embodiment will be described. FIG. 12 is a conceptual diagram showing an example of the visualization method by the information processing apparatus according to the sixth embodiment.

[0089] As shown in FIG. 12, in the information processing apparatus 10 according to the sixth embodiment, visualization is performed by associating the norm and the similarity decomposed by the element decomposition unit 131 with separate parameters related to display. For example, visualization is performed such that the norm corresponds to the saturation and the similarity corresponds to the hue. By doing so, for example, by making the input correspond to the brightness, elements corresponding to the norm and the similarity can be overlaid and displayed on the input image (black and white image).

[0090] Note that the parameters with which the norm and the similarity are associated are not limited to the above-described hue and saturation. However, for the similarity considered to be relatively important, it is preferable to associate it with a parameter having a stronger visual effect on humans (for example, the hue in the above example) as compared with the norm.

[0091] (Technical Effect) Next, the technical effects obtained by the information processing apparatus 10 according to the sixth embodiment will be described.

[0092] As described with reference to FIG. 12, in the information processing apparatus 10 according to the sixth embodiment, visualization is performed such that the decomposed norm and similarity correspond to separate parameters. In this way, it becomes possible to realize visualization in consideration of the characteristics of the authentication process.

[0093] A program for operating the configuration of each embodiment to realize the functions of each of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read out as code, and a processing method for execution on a computer is also included in the scope of each embodiment. That is, a computer-readable recording medium is also included in the scope of each embodiment. Further, not only the recording medium on which the above-described program is recorded, but also the program itself is included in each embodiment.

[0094] As the recording medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, or a ROM can be used. Further, not only those that execute processing with the program recorded on the recording medium alone, but also those that operate on an OS in cooperation with the functions of other software and expansion boards and execute processing are included in the scope of each embodiment. Furthermore, the program itself may be stored in a server so that a part or all of the program can be downloaded from the server to a user terminal.

[0095] <Supplementary Note> Regarding the embodiments described above, they may be further described as follows in the supplementary note below, but are not limited thereto.

[0096] (Supplementary Note 1) The information processing apparatus according to Supplementary Note 1 includes a feature amount acquisition means for acquiring a query feature amount related to an object of authentication processing, a first score calculation means for calculating a first score indicating the degree of coincidence between the query feature amount and a target feature amount registered in advance, and a second score calculation means for calculating a second score indicating the degree of non - coincidence between the query feature amount and the target feature amount based on the first score. The information processing apparatus further includes a map generation means for generating a saliency map related to the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class.

[0097] (Supplementary Note 2) The information processing apparatus according to Supplementary Note 2 is the information processing apparatus according to Supplementary Note 1, wherein the map generation means generates a CAM (Class Activation Map).

[0098] (Supplementary Note 3) The information processing apparatus according to Supplementary Note 3 is the information processing apparatus according to Supplementary Note 1 or 2, wherein the map generation means generates the saliency map by removing the influence of the normalization processing of the query feature amount included in the authentication processing.

[0099] (Supplementary Note 4) The information processing apparatus according to Supplementary Note 4 is the information processing apparatus according to Supplementary Note 3, wherein the map generation means generates the saliency map by associating first data not including the influence of the normalization processing with a first parameter and associating second data including the influence of the normalization processing with a second parameter.

[0100] (Supplementary Note 5) The information processing apparatus according to Supplementary Note 5 is the information processing apparatus according to Supplementary Note 1 or 2, wherein the map generation means generates the saliency map by decomposing the contribution of each element of the feature amount map based on the query feature amount to the authentication processing into two elements, namely, norm and similarity.

[0101] (Supplementary Note 6) The information processing apparatus according to Supplementary Note 6 is the information processing apparatus according to Supplementary Note 5, wherein the map generation means generates the saliency map by associating the norm with a first parameter and associating the similarity with a second parameter.

[0102] (Supplementary Note 7) The information processing method according to Supplementary Note 7 is an information processing method in which at least one computer acquires a query feature amount regarding an object of authentication processing, calculates a first score indicating the degree of coincidence between the query feature amount and a target feature amount registered in advance, calculates a second score indicating the degree of non - coincidence between the query feature amount and the target feature amount based on the first score, treats the first score as a first class score corresponding to a first class, and treats the second score as a second class score corresponding to a second class, thereby generating a saliency map regarding the authentication processing.

[0103] (Supplementary Note 8) The recording medium according to Supplementary Note 12 is a recording medium on which a computer program for causing at least one computer to execute an information processing method is recorded, the information processing method including acquiring a query feature amount regarding an object of authentication processing, calculating a first score indicating the degree of coincidence between the query feature amount and a target feature amount registered in advance, calculating a second score indicating the degree of non - coincidence between the query feature amount and the target feature amount based on the first score, treating the first score as a first class score corresponding to a first class, and treating the second score as a second class score corresponding to a second class, thereby generating a saliency map regarding the authentication processing.

[0104] (Supplementary Note 9) The computer program described in Supplementary Note 9 causes at least one computer to acquire query feature amounts regarding an object of authentication processing, calculate a first score indicating the degree of coincidence between the query feature amounts and pre-registered target feature amounts, calculate a second score indicating the degree of non-coincidence between the query feature amounts and the target feature amounts based on the first score, treat the first score as a first class score corresponding to a first class, and treat the second score as a second class score corresponding to a second class, thereby generating a saliency map regarding the authentication processing, and is a computer program for executing an information processing method.

[0105] (Supplementary Note 10) The information processing system described in Supplementary Note 10 includes a feature amount acquisition means for acquiring query feature amounts regarding an object of authentication processing, a first score calculation means for calculating a first score indicating the degree of coincidence between the query feature amounts and pre-registered target feature amounts, a second score calculation means for calculating a second score indicating the degree of non-coincidence between the query feature amounts and the target feature amounts based on the first score, and a map generation means for generating a saliency map regarding the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class.

[0106] This disclosure can be appropriately changed within a range not contrary to the gist or idea of the invention that can be read from the claims and the entire specification, and an information processing apparatus, an information processing method, and a recording medium involving such changes are also included in the technical idea of this disclosure.

Explanation of Signs

[0107] 10 Information processing apparatus 11 Processor 16 Output device 110 Feature amount acquisition unit 120 Collation unit 121 First score calculation unit 122 Second score calculation unit 130 Map Generation Unit 131 Element Decomposition Unit 135 CAM Generation Unit 1351 First Importance Weight Calculation Unit 1352 Second Importance Weight Calculation Unit 1353 First CAM Calculation Unit 1354 Second CAM Calculation Unit

Claims

1. Feature amount acquisition means for acquiring query feature amounts related to the target of the authentication process, First score calculation means for calculating a first score indicating the degree of match between the query feature amount and a target feature amount registered in advance, Second score calculation means for calculating a second score indicating the degree of non-match between the query feature amount and the target feature amount based on the first score, Map generation means for generating a saliency map related to the authentication process by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class, An information processing apparatus comprising:

2. The map generation means generates a CAM (Class Activation Map). The information processing apparatus according to Claim 1.

3. The map generation means generates the saliency map by removing the influence of the normalization process of the query feature amount included in the authentication process. The information processing apparatus according to Claim 1 or 2.

4. The map generation means generates the saliency map by associating first data not including the influence of the normalization process with a first parameter and associating second data including the influence of the normalization process with a second parameter. The information processing apparatus according to Claim 3.

5. The map generation means generates the saliency map by decomposing the contribution of each element of the feature amount map based on the query feature amount to the authentication process into two elements, namely, norm and similarity. The information processing apparatus according to Claim 1 or 2.

6. The map generation means generates the saliency map by associating the norm with a first parameter and associating the similarity with a second parameter. The information processing apparatus according to Claim 5.

7. By at least one computer, Acquire query feature amounts related to the target of the authentication process, Calculate a first score indicating the degree of match between the query feature amount and a target feature amount registered in advance, Calculate a second score indicating the degree of non-match between the query feature amount and the target feature amount based on the first score, Generate a saliency map related to the authentication process by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class. An information processing method.

8. At least one computer is caused to acquire query feature amounts regarding an object of authentication processing, calculate a first score indicating a degree of coincidence between the query feature amounts and target feature amounts registered in advance, calculate a second score indicating a degree of non - coincidence between the query feature amounts and the target feature amounts based on the first score, generate a saliency map regarding the authentication processing by treating the first score as a first class score corresponding to a first class and treating the second score as a second class score corresponding to a second class, A computer program for causing a computer to execute an information processing method.

Images