Information Processing Device, Personal Identification Device, Information Processing Method, and Storage Medium

Abstract

An information processing device is provided, including: an acquisition unit that sequentially acquires a plurality of elements included in sequence data; a first calculation unit that, for each of the plurality of elements, considers two or more of the plurality of elements to calculate an index indicating which of a plurality of categories a given element is suitable to belong to; a second calculation unit that integrates the indexes for each of the plurality of elements to calculate an integrated index indicating which of the plurality of categories the sequence data is suitable to belong to; and a classification unit that classifies the sequence data into one of the categories based on the integrated index.

Description

Technical Field

[0001] The present invention relates to an information processing device, a personal identification device, an information processing method, and a storage medium. Background Art

[0002] Patent Documents 1 to 3 disclose information processing techniques using the Sequential Probability Ratio Test (SPRT). SPRT is a method for determining which of multiple predetermined categories a sequence of sequentially input data belongs to.

[0003] [Citation List]

[0004] [Patent Document]

[0005] PTL1: Japanese Patent Application Laid-Open No. 2009-245314

[0006] PTL2: Japanese Patent Application Laid-Open No. 2008-299589

[0007] PTL3: Japanese Patent Application Laid-Open No. 2001-523824 Summary of the Invention

[0008] Technical Problem

[0009] In the mathematical formulas for SPRT disclosed in Patent Documents 1 to 3, it is assumed that each element of the sequence data is a random variable and is independently and identically distributed (i.i.d.). Therefore, since SPRT does not consider the relationship between the elements of the sequence data, sufficient accuracy may not be obtained depending on the nature of the sequence data.

[0010] An exemplary object of the present invention is to provide an information processing device, a personal identification device, an information processing method, and a storage medium capable of accurately classifying sequence data.

[0011] Solution to the Problem

[0012] According to an exemplary aspect of the present invention, there is provided an information processing device including: an acquisition unit that sequentially acquires a plurality of elements included in sequence data; a first calculation unit that calculates, for each of the plurality of elements, an index in consideration of two or more of the plurality of elements, each index indicating which of the multiple categories the corresponding element is suitable to belong to; a second calculation unit that calculates an integrated index by integrating the indices of the plurality of elements, the integrated index indicating which of the multiple categories the sequence data is suitable to belong to; and a classification unit that classifies the sequence data into one of the multiple categories based on the integrated index.

[0013] According to another exemplary aspect of the present invention, there is provided an information processing method, including: sequentially obtaining a plurality of elements included in sequence data; calculating an index for each of the plurality of elements considering two or more of the plurality of elements, each index indicating which of a plurality of categories the corresponding element is suitable to belong to; calculating an integrated index by integrating the indexes of the plurality of elements, the integrated index indicating which of the plurality of categories the sequence data is suitable to belong to; and classifying the sequence data into one of the plurality of categories based on the integrated index.

[0014] According to still another exemplary aspect of the present invention, there is provided a storage medium storing a program that causes a computer to execute an information processing method, the information processing method including: sequentially obtaining a plurality of elements included in sequence data; calculating an index for each of the plurality of elements considering two or more of the plurality of elements, each index indicating which of a plurality of categories the corresponding element is suitable to belong to; calculating an integrated index by integrating the indexes of the plurality of elements, the integrated index indicating which of the plurality of categories the sequence data is suitable to belong to; and classifying the sequence data into one of the plurality of categories based on the integrated index.

[0015] Advantageous Effects of the Invention

[0016] According to the present invention, there can be provided an information processing device, a personal identification device, an information processing method, and a storage medium capable of accurately classifying sequence data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram showing the overall configuration of a sequence data classification system according to a first exemplary embodiment;

[0018] Figure 2 is a functional block diagram of an information processing device according to a first exemplary embodiment;

[0019] Figure 3 is a flowchart showing an example of personal identification processing executed by an information processing device according to a first exemplary embodiment;

[0020] Figure 4 is a functional block diagram of a personal identification device according to a second exemplary embodiment;

[0021] Figure 5 is a functional block diagram of an information processing device according to a third exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] Exemplary embodiments of the present invention will be described with reference to the accompanying drawings. Throughout the drawings, the same components or corresponding components are denoted by the same reference numerals, and their descriptions may be omitted or simplified.

[0023] [First Exemplary Embodiment]

[0024] A sequence data classification system according to the present exemplary embodiment will be described. The sequence data classification system of the present exemplary embodiment is a system for classifying sequence data into one of a plurality of predetermined categories by sequentially acquiring and analyzing a plurality of elements included in the sequence data.

[0025] Here, sequence data refers to a data sequence that can be decomposed into a plurality of elements. The sequence data may be time-series data or non-time-series data. Specific examples of time-series data include moving image data and audio data. Specific examples of non-time-series data include vegetation data sampled from multiple locations, inspection data of multiple parts of a product, and multiple biometric data for biometric authentication. When the sequence data is moving image data, the plurality of elements included in the sequence data may be a plurality of images (frames) constituting the moving image. When the sequence data is inspection data of multiple parts of a product, the plurality of elements included in the sequence data may be inspection data of each part of the product. Note that the sequence data and elements to which the classification process of the present exemplary embodiment can be applied are not limited to these.

[0026] When the sequence data is inspection data of multiple parts of a product, the categories classified by the sequence data classification system of the present exemplary embodiment may be, for example, a first category indicating that the product is a non-defective product and a second category indicating that the product is a defective product. When the sequence data is multiple biometric data for biometric authentication, the categories classified by the sequence data classification system of the present exemplary embodiment may be, for example, a first category indicating that the authentication target person is the same as the target person and a second category indicating that they are not the same person. The number of categories may be three or more.

[0027] Figure 1 is a schematic diagram showing the overall configuration of the sequence data classification system according to the present exemplary embodiment. Figure 1 shows the hardware configuration included in the sequence data classification system. The sequence data classification system includes an information processing device 100, a data acquisition device 201, an input device 202, and a display device 203.

[0028] The information processing device 100 is a computer such as a cellular phone, a smartphone, a desktop personal computer (PC), a laptop PC, or a server. The information processing device 100 includes a processor 101, a memory 102, a storage device 103, an input and output interface (I / F) 104, and a communication I / F 105. The units of the information processing device 100 are interconnected via a bus, wiring, a driving device, etc., and can send and receive control signals and data to and from each other.

[0029] The processor 101 is an arithmetic processing device such as a central processing unit (CPU) or a graphics processing unit (GPU), for example. The memory 102 is a volatile or non-volatile storage medium such as a random access memory (RAM) or a read-only memory (ROM). The storage device 103 is a non-volatile storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card.

[0030] The memory 102 or the storage device 103 stores a program for implementing the information processing function of the information processing device 100. When the above program is executed, the processor 101 can read the program into the memory 102 and then execute the program, or can execute the program without reading the program into the memory 102.

[0031] Various types of non-transitory computer-readable media can be used to store the above program and provide it to the information processing device 100. Non-transitory computer-readable media include various types of storage media. Non-transitory computer-readable media include, for example, magnetic storage media, magneto-optical storage media, optical storage media, and semiconductor memories.

[0032] Examples of magnetic storage media include floppy disks, magnetic tapes, and hard disk drives. Examples of magneto-optical storage media include magneto-optical discs. Examples of optical storage media are compact disc read-only memory (CD-ROM), recordable compact disc (CD-R), and rewritable compact disc (CD-R / W). Examples of semiconductor memories include mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, and RAM.

[0033] Alternatively, the program can be provided to the information processing device 100 via various types of transitory computer-readable media. Transitory computer-readable media include, for example, electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can provide the program to the information processing device 100 via a wired communication path such as a wire or an optical fiber or a wireless communication path.

[0034] The input and output I / F 104 is a communication interface for communicating with peripheral devices based on standards such as Universal Serial Bus (USB) and Digital Visual Interface (DVI). The input and output I / F 104 can perform wired or wireless communication connections for the data acquisition device 201, the input device 202, and the display device 203. Accordingly, the information processing device 100 can send data and control signals to the data acquisition device 201, the input device 202, and the display device 203, and receive data and control signals from the data acquisition device 201, the input device 202, and the display device 203.

[0035] The communication I / F 105 is a communication interface based on standards such as Bluetooth (registered trademark), Wi-Fi (registered trademark), or 4G. The communication I / F 105 can perform wired or wireless communication connections to external devices. Accordingly, the information processing device 100 can send data to external devices and receive data from external devices.

[0036] The data acquisition device 201 is a device for acquiring sequence data. For example, when the sequence data is inspection data of a product, the data acquisition device 201 can be an inspection device installed in a factory or the like. For example, when the sequence data is biometric data for biometric authentication, the data acquisition device 201 can be a device for acquiring biometric information such as a digital camera, a microphone, or a fingerprint collection scanner. When the data acquisition device 201 includes a device for acquiring an analog signal such as a sensor, the data acquisition device 201 can include an analog-to-digital conversion (AD conversion) device that converts the analog signal into digital data. The sequence data acquired by the data acquisition device 201 is input to the information processing device 100.

[0037] The input device 202 is a user interface for receiving operations of the information processing device 100 performed by a user. Examples of the input device 202 include a keyboard, a mouse, a trackball, a touch sensor, a tablet, buttons, and the like. The display device 203 is a device that displays a screen based on drawing data processed by the processor 101. Examples of the display device 203 include a liquid crystal display (LCD), a cathode ray tube (CRT) display, and an organic light emitting diode (OLED) display. The input device 202 and the display device 203 can be integrally formed as a touch panel.

[0038] Note that Figure 1The hardware configuration shown is an example, and other devices can be added or some devices may not be provided. In addition, some devices can be replaced by other devices with similar functions. Further, some functions of the present exemplary embodiment can be provided by another device via a network, or the functions of the present exemplary embodiment can be implemented by being distributed over multiple devices. For example, the storage device 103 can be replaced by cloud storage outside the information processing device 100. When acquiring sequence data in a system different from the sequence data classification system, the data acquisition device 201 can be omitted. Alternatively, the data acquisition device 201, the input device 202, or the display device 203 can be provided in the information processing device 100. Therefore, the Figure 1 shown hardware configuration can be appropriately changed.

[0039] Figure 2 is a functional block diagram of the information processing device 100 according to the present exemplary embodiment. The information processing device 100 includes an acquisition unit 110, a first calculation unit 120, a second calculation unit 130, and a classification unit 140. The first calculation unit 120 includes a metric calculation unit 121 and a first storage unit 122. The second calculation unit 130 includes an integrated metric calculation unit 131 and a second storage unit 132.

[0040] The processor 101 implements the functions of the acquisition unit 110, the metric calculation unit 121, the integrated metric calculation unit 131, and the classification unit 140 by executing programs stored in the memory 102, the storage device 103, etc. In addition, the processor 101 implements the functions of the first storage unit 122 and the second storage unit 132 by controlling the storage device 103 based on the program. The specific processes executed by these units will be described later.

[0041] Figure 3 is a flowchart showing an example of the classification process executed by the information processing device 100 according to the present exemplary embodiment. Figure 3 The classification process shown is a process of classifying input sequence data into one of a plurality of predetermined categories. Figure 3 The classification process includes a loop process (steps S101 to S106), in which elements are sequentially acquired from sequence data including a plurality of elements and integrated metrics are calculated in order. This loop process is repeated until the category of the classification destination of the sequence data is determined based on the integrated metric (integrated likelihood ratio).

[0042] For example, when a predetermined user operation is performed on the input device 202, the Figure 3 process can be started. However, the start time of the process is not limited to this. For example, when sequence data is input from the data acquisition device 201, the Figure 3 process can be executed. When sequence data is continuously input as in the case where the data acquisition device 201 is a surveillance camera,Figure 3 The process can be repeatedly executed at a predetermined time interval.

[0043] In step S101, the acquisition unit 110 acquires an element of the sequence data. The acquisition process at this time can be a process of directly acquiring data from the data acquisition device 201, or a process of reading out data acquired from the data acquisition device 201 and pre-stored in the storage device 103 or the like.

[0044] In step S102, the index calculation unit 121 reads the past data stored in the first storage unit 122. The past data refers to: when the current process is a process for the j-th element in the sequence data, the processing result of the index calculation unit 121 for the elements before the j-th element in the sequence data. Alternatively, the past data can be the element itself input before the j-th data. The first storage unit 122 stores the processing result or input element of the index calculation unit 121 each time it executes a process. In this storage process, new information can be overwritten on the previously stored information, or new information can be added while retaining the previously stored information.

[0045] In step S103, the index calculation unit 121 calculates an index indicating which of the multiple categories the input element is suitable to belong to, taking into account two or more elements among the multiple elements included in the sequence data. The two or more elements include the input element and the previously processed elements included in the past data. The first calculation unit 120 outputs the calculated index to the second calculation unit 130 and stores the processing result in the first storage unit 122 as needed. Here, the index can be, for example, a likelihood ratio indicating the possibility that a certain element belongs to a certain category among the multiple categories. Alternatively, the index can be a function including the likelihood ratio as a variable. In the following description, it is assumed that the index is a likelihood ratio.

[0046] The index calculation unit 121 extracts features from the elements input from the sequence data. At this time, the index calculation unit 121 performs feature extraction taking into account the relationship between the input element and the past data. As a specific method of feature extraction, for example, a convolutional neural network (CNN) can be used, but this method is not limited thereto. As a specific method of storing past data and calculating the relationship with the current input data, for example, a long short-term memory (LSTM) can be used, but this method is not limited thereto.

[0047] A specific example of the likelihood ratio will be described. The N elements constituting the sequence data are represented as x1,..., x N , and the multiple categories are represented as C1, C2. That is, in this example, for simplicity, it is assumed that the classification is a two-class classification where the number of categories is two. Here, when calculating without considering the past data, the element x iThe calculation result of the probability belonging to category C1 is expressed as p(x i |C1). In addition, when past data is not considered in the calculation, the calculation result of the probability that the element x i belongs to category C2 is expressed as p(x i |C2). In this case, the likelihood ratio of these is represented by the following expression (1).

[0048] [Formula 1]

[0049]

[0050] The likelihood ratio of expression (1) indicates the likelihood ratio between the probability that the element x i belongs to category C1 and the probability that the element x i belongs to category C2. For example, when the likelihood ratio exceeds 1, since p(x i |C1) > p(x i |C2), it is appropriate to classify the element x i as category C1 rather than category C2. As described above, the likelihood ratio of expression (1) serves as an indicator indicating which of category C1 and category C2 the input element is suitable to belong to.

[0051] In addition, as described above, the index calculation unit 121 can calculate the index while taking into account a plurality of elements (i.e., the relationship between the input element and past data). In this case, for example, the likelihood ratio calculated when taking into account two elements x i and x i-1 is represented by the following expression (2).

[0052] [Formula 2]

[0053]

[0054] In step S104, the integrated index calculation unit 131 reads the past calculated integrated index from the second storage unit 132. In step S105, the integrated index calculation unit 131 integrates the likelihood ratio calculated by the index calculation unit 121 this time and the past calculated integrated index to calculate a new integrated index. The second calculation unit 130 outputs the calculated integrated index to the classification unit 140 and stores the integrated index in the second storage unit 132.

[0055] The integrated metric indicates to which of multiple categories the entire sequence data is suitable to belong. The integrated metric in the past means the integrated metric calculated by the integrated metric calculation unit 131 for the elements before the j-th element in the sequence data when the current process is the process for the j-th element in the sequence data. The second storage unit 132 stores the integrated metric each time the integrated metric calculation unit 131 executes a process. In this storage process, the value of the integrated metric can be updated by overwriting the integrated metric stored in the past with the new integrated metric, or the new integrated metric can be added while retaining the integrated metric stored in the past.

[0056] The integrated metric can be, for example, an integrated likelihood ratio indicating the likelihood that the sequence data belongs to one of multiple categories. Alternatively, the integrated metric can be a function including the integrated likelihood ratio as a variable. In the following description, the integrated metric is the integrated likelihood ratio.

[0057] Assuming that the number of categories is 2, a specific example of the integrated likelihood ratio will be described. When N elements are input when calculating the integrated likelihood ratio, the N elements are represented as x1,..., x N . Here, the probability that the entire sequence data belongs to category C1 is represented as p(x1,..., x N |C1). The probability that the entire sequence data belongs to category C2 is represented as p(x1,..., x N |C2). In this case, the likelihood ratio of these is represented by the following expression (3). Expression (3) is called the integrated likelihood ratio.

[0058] [Formula 3]

[0059]

[0060] When it is assumed that each element of the sequence data is independent as in the conventional SPRT, the integrated likelihood ratio can be calculated by decomposing it into terms for each element, as shown in the following expression (4). In expression (4), for simplicity of calculation, each element is decomposed into a sum by using the logarithm of the likelihood ratio, but this is not necessary. In this specification, the term likelihood ratio or integrated likelihood ratio may be used for the log-likelihood ratio. Further, although the symbol of the logarithm base is omitted in this specification, the base can be any value.

[0061] [Formula 4]

[0062]

[0063] However, as described above, in the present exemplary embodiment, since the likelihood ratio and the integrated likelihood ratio are calculated considering two or more elements, the assumption that each element is independent is generally not satisfied. Therefore, it is not possible to decompose the integrated likelihood ratio into terms for each element as in Expression (4), and the integrated likelihood ratio is calculated by different calculation expressions depending on the number of elements considering the relationship.

[0064] For example, in the case of considering two elements, an element of the current process and the element immediately preceding this element, the integrated likelihood ratio can be calculated using the following Expression (5).

[0065] [Formula 5]

[0066]

[0067] In the case of considering three elements, an element of the current process and the two elements preceding this element, the integrated likelihood ratio can be calculated using the following Expression (6).

[0068] [Formula 6]

[0069]

[0070] Note that the likelihood ratio previously calculated by the index calculation unit 121 in step S103 considering two or three elements can be used as the terms on the right side represented in Expressions (5) and (6).

[0071] Expressions (5) and (6) represent examples of the case of two-class classification for calculating the likelihood ratio between class C1 and class C2, but the number of classes can be three or more. For example, when the number of classes is M, the right side of Expression (5) can be extended so that the integrated likelihood ratio between the k-th class and all classes other than the k-th class among the M classes can be calculated. An example of such an extension is to use the maximum likelihood of all classes other than the k-th class as in the following Expression (7).

[0072] [Formula 7]

[0073]

[0074] As another example, the sum of the likelihoods of all classes other than the k-th class is used as in the following Expression (8). Note that the method for calculating the integrated likelihood ratio when the number of classes is three or more is not limited to this.

[0075] [Formula 8]

[0076]

[0077] Expressions (5) to (8) show cases considering two or three elements, but cases considering four or more elements can also be considered. In this case, the integrated likelihood ratio can be calculated by extending Expressions (5) to (8) in the same way.

[0078] The method for calculating the integrated metric is not limited to the above method. For example, the integrated metric can be calculated by using a method such as LSTM or a deep neural network.

[0079] In step S106, the classification unit 140 determines whether the sequence data can be classified into any category based on the integrated metric calculated by the second calculation unit 130. When the integrated metric is the integrated likelihood ratio, the classification unit 140 determines whether classification of the category can be performed, for example, based on whether there is a category for which the integrated likelihood ratio exceeds a predetermined threshold. If classification cannot be performed ("No" in step S106), the process proceeds to step S101, and the acquisition unit 110 acquires the next element. If classification can be performed ("Yes" in step S106), the process proceeds to step S107.

[0080] In step S107, the classification unit 140 classifies the sequence data into one of the multiple categories based on the integrated metric. For example, when the integrated metric is the integrated likelihood ratio, the sequence data is classified as belonging to the category for which the integrated likelihood ratio exceeds a predetermined threshold.

[0081] The processing in steps S106 and S107 will be described in more detail with reference to a specific example. Assume that the classification process in this example is a two-class classification of class C1 or class C2, and the thresholds for determining class C1 and class C2 are T1 and T2, respectively. Let L1 be the integrated likelihood ratio for class C1, and L2 be the integrated likelihood ratio for class C2. When the integrated likelihood ratio is defined by Expression (4), since L2 = -L1, basically only L1 needs to be calculated.

[0082] In this case, when L1 > T1, the classification unit 140 classifies the sequence data as class C1, and the process ends. When L2 > T2, the classification unit 140 classifies the sequence data as class C2, and the process ends. When L1 ≤ T1 and L2 ≤ T2, the classification unit 140 determines that classification cannot be performed, and the acquisition unit 110 acquires the next element.

[0083] When the number of categories is M, which is three or more, M thresholds are prepared in a similar manner as described above, and a similar classification process can be performed by determining the magnitude relationship between each of the M integrated likelihood ratios and the corresponding threshold. At this time, the classification unit 140 classifies the sequence data into the category for which the integrated likelihood ratio first exceeds the threshold. When the integrated likelihood ratio does not exceed any threshold, the classification unit 140 determines that classification cannot be performed, and the acquisition unit 110 acquires the next element.

[0084] The above classification method is an example and is not limited thereto. For example, when the number of elements input in steps S106 and S107 is greater than a predetermined value (the maximum number of elements), even if there is no category for which the integrated likelihood ratio exceeds the threshold, the sequence data can be forcibly classified into any one of the categories, and the process can be terminated. This can prevent the calculation time from becoming too long. In this example, it is desirable that the determination criteria are mutually exclusive so as to be reliably classified into any category.

[0085] Specific examples of mutually exclusive criteria will be described. In the case of two-class classification, when the number of elements exceeds the maximum number of elements, a method can be used that classifies the sequence data into one of the two categories depending on whether the integrated likelihood ratio L1 is 0 or greater. In the case of M-class classification, a method can be used that classifies the sequence data into the category having the maximum value among the overall likelihood ratios corresponding to the respective categories.

[0086] As described above, according to the present exemplary embodiment, the classification of sequence data is performed using an integrated metric that takes into account elements of multiple sequence data. Therefore, classification that takes into account the relationship between elements can be performed, thereby providing the information processing apparatus 100 capable of accurately classifying sequence data.

[0087] When the correlation between elements of the sequence data is strong, the classification process in the information processing apparatus 100 according to the present exemplary embodiment is more effective. In the SPRT disclosed in Patent Documents 1 to 3, since an algorithm that does not consider the relationship between elements of the sequence data is used as in Expression (4), classification is performed on sequence data that actually has a strong relationship between elements in the same way as on sequence data that does not have a relationship between elements. Therefore, for sequence data having a strong relationship between elements, the classification accuracy may deteriorate. On the other hand, in the classification process in the information processing apparatus 100 according to the present exemplary embodiment, since the relationship between elements of the sequence data is considered as in Expressions (5) to (8), the classification accuracy is less likely to deteriorate even for sequence data having a strong relationship between elements.

[0088] Specific examples of cases where the correlation between elements of sequence data is strong include time-series data such as moving image data. For example, in moving image data, typically one frame and the next frame have similar features. Therefore, the classification processing in the information processing device 100 of the present exemplary embodiment is more effective in the processing of time-series data.

[0089] [Second Exemplary Embodiment]

[0090] In the present exemplary embodiment, the personal identification device 300 will be described as an application example of the information processing device 100 according to the first exemplary embodiment. Hereinafter, the differences from the first exemplary embodiment will be mainly described, and the description of the common parts will be omitted or simplified.

[0091] Figure 4 is a functional block diagram of the personal identification device 300 according to the second exemplary embodiment. The personal identification device 300 includes a classification device 301, a biometric information acquisition unit 302, and a biometric information storage unit 303. The personal identification device 300 may include a computer similar to the Figure 1 information processing device 100 shown. Therefore, the description of the hardware configuration of the personal identification device 300 will be omitted.

[0092] The personal identification device 300 is a device for identifying a person by comparing biometric information of an identification target such as a face image, a fingerprint image, and an iris image with pre-registered biometric information. The personal identification device 300 may include a device (such as a camera) for acquiring biometric information, and may operate in an independent manner, or may acquire biometric information from other devices in a personal identification system to identify a person. In addition, the personal identification device 300 may be configured by a plurality of devices communicatively connected to each other.

[0093] The personal identification device 300 may be, for example, an authentication device for a face authentication gate. Alternatively, the personal identification device 300 may be an intelligent camera. An intelligent camera is an Internet Protocol (IP) camera or a network camera with an analysis function, and may be referred to as a smart camera.

[0094] The biometric information acquisition unit 302 is a device for acquiring biometric information, and may be, for example, a digital camera capable of capturing a moving image. In the identification of biometric information, a feature amount for matching can be extracted from an image or the like acquired by the biometric information acquisition unit 302. This feature amount extraction process may be executed in the classification device 301, may be executed in the biometric information acquisition unit 302 when acquiring biometric information, or may be executed by another device. In this specification, an image or the like acquired by the biometric information acquisition unit 302 and a feature amount extracted therefrom may be collectively referred to as biometric information.

[0095] The biometric information storage unit 303 stores information necessary for processing in the classification device 301, such as biometric information of a registered person. The information processing device 100 of the first exemplary embodiment serves as the classification device 301. The classification device 301 acquires sequence data having elements as biometric information as the sequence data described in the first exemplary embodiment. The classification device 301 refers to the information stored in the biometric information storage unit 303 and classifies the sequence data into one of a plurality of predetermined categories. Here, the plurality of categories may be, for example, categories each indicating a person who matches the input sequence data. Alternatively, the plurality of categories may be, for example, categories each indicating whether there is spoofing. In other words, the plurality of categories may include, for example, a category indicating the presence of spoofing in the input sequence data and a category indicating the absence of spoofing in the input sequence data.

[0096] The personal identification device 300 of the present exemplary embodiment includes a classification device 301 that can accurately classify sequence data. Therefore, a personal identification device 300 that can perform personal identification more appropriately is provided.

[0097] An example of spoofing detection will be described as an example that makes more use of the feature of high classification accuracy of the sequence data of the information processing device 100 of the first exemplary embodiment in the personal identification device 300 of the present exemplary embodiment. As one of the spoofing methods in biometric authentication such as face authentication, a method using a non-living object such as a person's facial photo or facial model is known. As a method for detecting such spoofing, there is a method of taking a plurality of images and determining that the image is not a living object when the difference between the plurality of images is small. In the classification device 301 of the present exemplary embodiment, a time-series image of a person to be authenticated is input as sequence data, and classification indicating the presence or absence of spoofing of the sequence data is performed using the difference between the images as a feature amount, so that spoofing can be detected. In this method, the time change of the images included in the input time-series data is small, and the correlation between the images is strong in many cases. Therefore, when performing classification for spoofing detection, it is effective to use the classification process of the information processing device 100 according to the first exemplary embodiment, in which the classification accuracy is less likely to deteriorate for sequence data having a strong relationship between elements.

[0098] The devices or systems described in the above embodiments may also be configured as in the following third exemplary embodiment.

[0099] [Third Exemplary Embodiment]

[0100] Figure 5FIG. 0 is a functional block diagram of an information processing apparatus 400 according to a third exemplary embodiment. The information processing apparatus 400 includes an acquisition unit 410, a first calculation unit 420, a second calculation unit 430, and a classification unit 440. The acquisition unit 410 sequentially acquires a plurality of elements included in sequence data. The first calculation unit 420 calculates, for each of the plurality of elements, an index in consideration of two or more of the plurality of elements, each index indicating to which of a plurality of categories the corresponding element is suitable to belong. The second calculation unit 430 calculates an integrated index indicating to which of the plurality of categories the sequence data is suitable to belong by integrating the indexes of the plurality of elements. The classification unit 440 classifies the sequence data into one of the plurality of categories based on the integrated index.

[0101] According to the present exemplary embodiment, there is provided an information processing apparatus 400 capable of accurately classifying sequence data.

[0102] [Modified Exemplary Embodiment]

[0103] The present invention is not limited to the above exemplary embodiments and can be appropriately modified within the scope of the present invention. For example, an example of adding a part of the configuration of one embodiment to another embodiment or replacing a part of the configuration of one embodiment with a part of the configuration of another embodiment is also an exemplary embodiment of the present invention.

[0104] The scope of each exemplary embodiment also includes a processing method that stores a program in a storage medium, the program causing the configuration of each exemplary embodiment to operate to implement the functions of each of the above exemplary embodiments, the processing method reading the program stored in the storage medium as code and executing the program in a computer. That is, the scope of each exemplary embodiment also includes a computer-readable storage medium. In addition, each exemplary embodiment includes not only the storage medium storing the above computer program but also the computer program itself. Further, one or two or more components included in the above exemplary embodiments may be circuits such as an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA), configured to implement the functions of each component.

[0105] As the storage medium, for example, a flexible (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a compact disk (CD)-ROM, a magnetic tape, a non-volatile memory card, or a ROM can be used. In addition, the scope of each exemplary embodiment includes an example of operating on an operating system (OS) to execute a process in cooperation with the functions of another software or a plug-in board, and is not limited to an example of executing a process by a separate program stored in a storage medium.

[0106] In addition, services implemented by the functions of each of the above exemplary embodiments can be provided to users in the form of software as a service (SaaS).

[0107] It should be noted that the above embodiments are merely examples embodying the present invention, and the technical scope of the present invention should not be construed restrictively by these. That is, the present invention can be implemented in various forms without departing from its technical idea or main features.

[0108] All or some of the exemplary embodiments disclosed above can be described, but are not limited to, the following supplementary explanations.

[0109] (Supplementary Explanation 1)

[0110] An information processing device, comprising:

[0111] An acquisition unit that sequentially acquires a plurality of elements included in sequence data;

[0112] A first calculation unit that calculates an index for each of the plurality of elements considering two or more of the plurality of elements, each index indicating which one of a plurality of categories the corresponding element is suitable to belong to;

[0113] A second calculation unit that calculates an integrated index by integrating the indexes of the plurality of elements, the integrated index indicating which one of a plurality of categories the sequence data is suitable to belong to; and

[0114] A classification unit that classifies the sequence data into one of a plurality of categories based on the integrated index.

[0115] (Supplementary Explanation 2)

[0116] The information processing device according to Supplementary Explanation 1, wherein each index includes a likelihood ratio indicating the possibility that the corresponding one of the plurality of elements belongs to one of the plurality of categories.

[0117] (Supplementary Explanation 3)

[0118] The information processing device according to Supplementary Explanation 1 or 2, wherein the integrated index includes an integrated likelihood ratio indicating the possibility that the sequence data belongs to one of the plurality of categories.

[0119] (Supplementary Explanation 4)

[0120] The information processing device according to Supplementary Explanation 3, wherein when there is a category in which the integrated likelihood ratio exceeds a predetermined threshold, the classification unit classifies the sequence data into the category in which the integrated likelihood ratio exceeds the threshold.

[0121] (Supplementary Explanation 5)

[0122] The information processing device according to Supplementary Note 3 or 4, wherein when there is no category with an integrated likelihood ratio exceeding a predetermined threshold, the classification unit does not classify the sequence data into any category, and the acquisition unit further acquires another element.

[0123] (Supplementary Note 6)

[0124] The information processing device according to any one of Supplementary Notes 3 to 5, wherein when there is no category with an integrated likelihood ratio exceeding a predetermined threshold and the number of elements of the sequence data is greater than a predetermined value, the classification unit classifies the sequence data into one category among multiple categories based on the integrated likelihood ratio.

[0125] (Supplementary Note 7)

[0126] The information processing device according to any one of Supplementary Notes 1 to 6,

[0127] wherein the first calculation unit includes:

[0128] a first storage unit that stores information processed by the first calculation unit in the past;

[0129] and

[0130] an index calculation unit that calculates each of the indexes based on the elements and the information stored in the first storage unit when the acquisition unit acquires the elements of the sequence data.

[0131] (Supplementary Note 8)

[0132] The information processing device according to Supplementary Note 7,

[0133] wherein the second calculation unit includes:

[0134] a second storage unit that stores the integrated indexes calculated by the first calculation unit in the past; and

[0135] an integrated index calculation unit that calculates the integrated index by integrating the indexes output from the first calculation unit and the integrated indexes stored in the second storage unit.

[0136] (Supplementary Note 9)

[0137] The information processing device according to any one of Supplementary Notes 1 to 8, wherein the sequence data is time series data.

[0138] (Supplementary Note 10)

[0139] A personal identification device, comprising:

[0140] a biometric information acquisition unit that acquires biometric information about a target person; and

[0141] An information processing device according to any one of Supplementary Notes 1 to 9;

[0142] wherein the information processing device classifies sequence data including the biometric information as an element into one of a plurality of categories.

[0143] (Supplementary Note 11)

[0144] A personal identification device according to Supplementary Note 10, wherein the information processing device classifies the sequence data into one of a plurality of categories, and the one category indicates the presence or absence of spoofing of biometric information.

[0145] (Supplementary Note 12)

[0146] An information processing method, comprising:

[0147] sequentially obtaining a plurality of elements included in sequence data;

[0148] calculating, for each of the plurality of elements, an index in consideration of two or more of the plurality of elements, each index indicating to which one of a plurality of categories the corresponding element is suitable to belong;

[0149] calculating an integrated index by integrating the indexes of the plurality of elements, the integrated index indicating to which one of the plurality of categories the sequence data is suitable to belong; and

[0150] classifying the sequence data into one of the plurality of categories based on the integrated index.

[0151] (Supplementary Note 13)

[0152] A storage medium storing a program that causes a computer to execute an information processing method, the information processing method comprising:

[0153] sequentially obtaining a plurality of elements included in sequence data;

[0154] calculating, for each of the plurality of elements, an index in consideration of two or more of the plurality of elements, each index indicating to which one of a plurality of categories the corresponding element is suitable to belong;

[0155] calculating an integrated index by integrating the indexes of the plurality of elements, the integrated index indicating to which one of the plurality of categories the sequence data is suitable to belong; and

[0156] classifying the sequence data into one of the plurality of categories based on the integrated index.

[0157] [List of Reference Numerals]

[0158] 100, 400 Information Processing Device

[0159] 101 Processor

[0160] 102 Memory

[0161] 103 Storage Device

[0162] 104 Input and Output I / F

[0163] 105 Communication I / F

[0164] 110, 410 Acquisition Unit

[0165] 120, 420 First Calculation Unit

[0166] 121 Index Calculation Unit

[0167] 122 First Storage Unit

[0168] 130, 430 Second Calculation Unit

[0169] 131 Integrated Index Calculation Unit

[0170] 132 Second Storage Unit

[0171] 140, 440 Classification Unit

[0172] 201 Data Acquisition Device

[0173] 202 Input Device

[0174] 203 Display Device

[0175] 300 Personal Identification Device

[0176] 301 Classification Device

[0177] 302 Biometric Information Acquisition Unit

[0178] 303 Biometric Information Storage Unit

Claims

1. An information processing device, comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: obtain a first facial image from sequence data acquired by a camera; obtain at least one second facial image acquired earlier than the first facial image; calculate a likelihood ratio that indicates a ratio between a probability that the first facial image and the second facial image belong to a first category and a probability that the first facial image and the second facial image belong to a second category; read out an integrated likelihood ratio calculated in the past from the memory; calculate a logarithm of the likelihood using the integrated likelihood ratio calculated in the past, and calculate an integrated likelihood ratio indicating that the entire sequence data belongs to one of multiple categories; when the integrated likelihood ratio does not exceed a threshold, repeat the following process: obtain a new facial image and calculate the likelihood ratio and the integrated likelihood ratio to compare the integrated likelihood ratio with the threshold; for authentication of a gate, compare the integrated likelihood ratio with the threshold, and when the integrated likelihood ratio exceeds the threshold, classify the entire sequence data into a corresponding category.

2. The information processing apparatus according to claim 1, wherein, When there is no category in which the integrated likelihood ratio exceeds the threshold and the number of elements of the sequence data is greater than a predetermined value, classify the sequence data into one of the multiple categories based on the integrated likelihood ratio.

3. The information processing apparatus according to claim 1, wherein, Calculating the likelihood ratio includes: storing information processed in the past in the at least one memory; and when an element of the sequence data is obtained, calculating each of the likelihood ratios based on the element and the information stored in the at least one memory.

4. The information processing apparatus according to claim 3, wherein Calculating the integrated likelihood ratio includes: storing the integrated likelihood ratio calculated in the past in the at least one memory; and calculating the integrated likelihood ratio by integrating the calculated likelihood ratio with the integrated likelihood ratio stored in the at least one memory.

5. A non-transitory storage medium storing a program that causes a computer to execute an information processing method, the information processing method including: obtain a first facial image from sequence data acquired by a camera; obtain at least one second facial image acquired earlier than the first facial image; calculate a likelihood ratio that indicates a ratio between a probability that the first facial image and the second facial image belong to a first category and a probability that the first facial image and the second facial image belong to a second category; read out an integrated likelihood ratio calculated in the past from a memory; calculate a logarithm of the likelihood using the integrated likelihood ratio calculated in the past, and calculate an integrated likelihood ratio indicating that the entire sequence data belongs to one of multiple categories; when the integrated likelihood ratio does not exceed a threshold, repeat the following process: obtain a new facial image and calculate the likelihood ratio and the integrated likelihood ratio to compare the integrated likelihood ratio with the threshold; for authentication of a gate, compare the integrated likelihood ratio with the threshold, and when the integrated likelihood ratio exceeds the threshold, classify the entire sequence data into a corresponding category.

6. An information processing system, comprising: a camera; Facial authentication gate; At least one memory configured to store instructions; And At least one processor configured to execute the instructions to: Obtain a first facial image from the sequence data acquired by the camera; Obtain at least one second facial image acquired earlier than the first facial image; Calculate a likelihood ratio, the likelihood ratio indicating a ratio between the probability that the first facial image and the second facial image belong to a first category and the probability that the first facial image and the second facial image belong to a second category; Read out an integrated likelihood ratio calculated in the past from the memory; Calculate the logarithm of the likelihood using the integrated likelihood ratio calculated in the past, and calculate an integrated likelihood ratio indicating that the entire sequence data belongs to one of multiple categories; When the integrated likelihood ratio does not exceed a threshold, repeat the following process: obtain a new facial image and calculate the likelihood ratio and the integrated likelihood ratio to compare the integrated likelihood ratio with the threshold; For authentication of the authentication gate, compare the integrated likelihood ratio with the threshold, and when the integrated likelihood ratio exceeds the threshold, classify the entire sequence data into the corresponding category, Wherein, the authentication gate is controlled based on the classification result.

7. The information processing system according to claim 6, wherein, When there is no category in which the integrated likelihood ratio exceeds the threshold and the number of elements in the entire sequence data is greater than a predetermined value, classify the entire sequence data into one of the multiple categories based on the integrated likelihood ratio.

8. The information processing system according to claim 6, wherein, The information processing system classifies the entire sequence data into one of the multiple categories, the one category indicating the presence or absence of spoofing of biometric information.

Images