Information processing device, information processing method, and computer program

Abstract

When classifying acquired data, it is necessary to appropriately determine whether or not it is unregistered. [Solution] The information processing device includes: acquisition means for acquiring time-series image data; index calculation means for calculating an integrated feature obtained by integrating a first feature, which is a feature of the time-series image data, and a second feature, which is a feature of pre-registered image data; or a score indicating the similarity between the first feature and the second feature; likelihood ratio calculation means for calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs, based on the integrated feature or score; and determination means for determining that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and for determining that the time-series image data does not belong to any registered class when the likelihood ratio reaches an unregistered threshold without reaching a class threshold.

Description

Technical Field

[0001] This disclosure relates to the technical fields of information processing apparatuses, information processing methods, and computer programs.

Background Art

[0002] As this type of apparatus, there is known one that performs a process of determining the class to which data belongs (so-called class classification). For example, Patent Document 1 discloses a technique for determining the class to which series data to be classified belongs when an individual score or an integrated score calculated based on a likelihood ratio reaches a predetermined threshold value.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This disclosure aims to provide an information processing apparatus, an information processing method, and a computer program for improving the techniques disclosed in the prior art documents.

Means for Solving the Problems

[0005] One aspect of the information processing apparatus disclosed herein includes: acquisition means for acquiring time-series image data; index calculation means for calculating an integrated feature obtained by integrating a first feature which is a feature of the time-series image data and a second feature which is a feature of pre-registered image data, or a score indicating the similarity between the first feature and the second feature; likelihood ratio calculation means for calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score; determination means for determining that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and determining that the time-series image data does not belong to any of the registered classes when the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0006] One aspect of the information processing method of this disclosure involves at least one computer acquiring time-series image data, integrating a first feature, which is a feature of the time-series image data, with a second feature, which is a feature of pre-registered image data, to calculate an integrated feature, or a score indicating the similarity between the first feature and the second feature, calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score, determining that the time-series image data belongs to a registered class corresponding to the registered image data if the likelihood ratio reaches a class threshold, and determining that the time-series image data does not belong to any of the registered classes if the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0007] One aspect of the computer program of this disclosure causes at least one computer to execute an information processing method which involves acquiring time-series image data, integrating a first feature which is a feature of the time-series image data with a second feature which is a feature of pre-registered image data, or calculating a score indicating the similarity between the first feature and the second feature, calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score, determining that the time-series image data belongs to a registered class corresponding to the registered image data if the likelihood ratio reaches a class threshold, and determining that the time-series image data does not belong to any of the registered classes if the likelihood ratio reaches an unregistered threshold without reaching the class threshold. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the hardware configuration of the first information processing device. [Figure 2] This is a block diagram showing the functional configuration of the first information processing device. [Figure 3] This is a flowchart showing the operation flow of the first information processing device. [Figure 4] This graph (part 1) shows an example of a likelihood ratio calculated by the first information processing device. [Figure 5] This is a graph (part 2) showing an example of a likelihood ratio calculated in the first information processing device. [Figure 6] This is a block diagram showing the functional configuration of the second information processing device. [Figure 7] This is a flowchart showing the operation flow of the second information processing device. [Figure 8] This graph shows an example of changing the unregistered threshold in the second information processing device. [Figure 9] This is a graph (part 1) showing an example of changing the unregistered threshold and class threshold in the second information processing device. [Figure 10]This is a graph (part 2) showing examples of changes to the unregistered threshold and class threshold in the second information processing device. [Figure 11] This is a block diagram showing the functional configuration of the third information processing device. [Figure 12] This is a flowchart showing the operation flow of the third information processing device. [Figure 13] This is a block diagram showing the functional configuration of the fourth information processing device. [Figure 14] This is a flowchart showing the operation flow of the fourth information processing device. [Figure 15] This graph (part 1) shows an example of a likelihood ratio calculated by the first information processing device. [Figure 16] This is a graph (part 2) showing an example of a likelihood ratio calculated in the first information processing device. [Modes for carrying out the invention]

[0009] The following describes embodiments of the information processing device, information processing method, and computer program with reference to the drawings.

[0010] <First Embodiment> The first information processing device will be described with reference to Figures 1 to 5.

[0011] (Hardware configuration) First, the hardware configuration of the first information processing device will be described with reference to Figure 1. Figure 1 is a block diagram showing the hardware configuration of the first information processing device.

[0012] As shown in FIG. 1, the first information processing apparatus 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, and an output device 16. The above-described processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are each connected via a data bus 17. Note that the data bus 17 may be an interface other than the data bus (for example, LAN, USB, etc.).

[0013] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, ROM 13, and storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown). The processor 11 may acquire (that is, read) a computer program from a device (not shown) disposed outside the first information processing apparatus 1 via a network interface. The processor 11 executes various processes by executing the read computer program. When the processor 11 executes the read computer program, function blocks related to the processes executed by the first information processing apparatus 1 are realized within the processor 11. That is, the processor 11 may function as a controller that executes various processes and controls in the first information processing apparatus 1.

[0014] Processor 11 may be configured as, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), or quantum processor. Processor 11 may be constituted by one of these, or may be configured to use a plurality in parallel.

[0015] RAM 12 temporarily stores the computer programs executed by processor 11. RAM 12 temporarily stores the data temporarily used by processor 11 when processor 11 is executing a computer program. RAM 12 may be, for example, D-RAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). Also, instead of RAM 12, other types of volatile memory may be used.

[0016] ROM 13 stores the computer programs executed by processor 11. ROM 13 may also store other fixed data. ROM 13 may be, for example, P-ROM (Programmable Read Only Memory) or EPROM (Erasable Read Only Memory). Also, instead of ROM 13, other types of non-volatile memory may be used.

[0017] Storage device 14 stores the data that the first information processing device 1 stores long-term. Storage device 14 may operate as a temporary storage device for processor 11. Storage device 14 may store the computer programs executed by processor 11. Storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from the user of the first information processing device 1. The input device 15 may include, for example, at least one of a keyboard, mouse, touch panel, and stylus. The input device 15 may also be a device capable of voice input, for example, including a microphone.

[0019] The output device 16 is a device that outputs information related to the first information processing device 1 to the outside. For example, the output device 16 may be a display device (e.g., a display or monitor) capable of displaying information related to the first information processing device 1. Alternatively, the output device 16 may be a speaker or the like capable of outputting audio information related to the information processing device 1.

[0020] The first information processing device may be configured to include some of the components described in Figure 1. For example, the first information processing device 1 may be configured to include only the processor 11, RAM 12, and ROM 13 from the components described above. In this case, the storage device 14, input device 15, and output device 16 may each be provided as external devices to the first information processing device 1. Furthermore, some of the arithmetic functions of the first information processing device 1 may be implemented by an external server or cloud service.

[0021] (Functional configuration) Next, the functional configuration of the first information processing device 1 will be described with reference to Figure 2. Figure 2 is a block diagram showing the functional configuration of the first information processing device.

[0022] In Figure 2, the first information processing device 1 is configured as a device for classifying incoming time-series data. More specifically, the first information processing device 1 is configured to perform matching processing (in other words, authentication processing) of objects contained in image data by classifying incoming time-series image data into classes corresponding to registered image data. The first information processing device 1 is configured to include an image acquisition unit 110, an index calculation unit 120, a registered image storage unit 125, a likelihood ratio calculation unit 130, and a class determination unit 140 as components for realizing its function. Note that each of the image acquisition unit 110, the index calculation unit 120, the likelihood ratio calculation unit 130, and the class determination unit 140 may be processing blocks realized by the processor 11 (see Figure 1) described above.

[0023] The image acquisition unit 110 is configured to acquire time-series image data. The image acquisition unit 110 may acquire image data of multiple frames sequentially. For example, the image acquisition unit 110 may be configured to acquire images of each frame captured from a camera that shoots video. The time-series image data acquired by the image acquisition unit 110 is acquired in order to perform a matching process with pre-registered registered image data. For example, the time-series image data may be face image data including the face of the target. In this case, the acquired face image data may be used to perform face recognition using the face of the target. Note that face image data is merely an example, and the time-series image data may include image data of parts other than the face of the target (for example, fingerprints or irises). Also, the time-series image data may include subjects other than people. The time-series image data acquired by the image acquisition unit 110 is configured to be output to the index calculation unit 120.

[0024] The index calculation unit 120 is configured to calculate an integrated feature or score from time-series image data acquired by the image acquisition unit 110. The integrated feature is a feature obtained by integrating a first feature, which is a feature of the time-series image data, and a second feature, which is a feature of pre-registered image data. The score is a score that indicates the similarity (in other words, the degree of agreement) between the first feature and the second feature described above. The index calculation unit 120 may also have a function to extract features from image data in order to calculate the integrated feature or score. For example, the index calculation unit 120 may have a function to extract features of the target face from face image data that includes the target face. The index calculation unit 120 acquires time-series image data from the image acquisition unit 110 while simultaneously acquiring registered image data from the registered image storage unit 125 to calculate the integrated feature or score. The integrated feature or score calculated by the index calculation unit 120 is output to the likelihood ratio calculation unit 130.

[0025] The registered image storage unit 125 is configured to store registered image data used by the index calculation unit 120 when calculating integrated features or scores. The registered image storage unit 125 may be configured to store features extracted from the registered image data (i.e., second features) rather than the registered image data itself. The registered image storage unit 125 may be implemented using the storage device 14 (see Figure 1) described above. Alternatively, the registered image storage unit 125 may be implemented by a database or the like provided outside the first information processing device 1. The registered image storage unit 125 may be configured to store multiple registered image data. In this case, the index calculation unit 120 may calculate N integrated features or N scores using first features extracted from time-series image data and N second features extracted from each of the N registered image data. The registered image data stored in the registered image storage unit 125 may be, for example, face image data of a registered user used for face recognition. In this case, the integrated feature quantities and scores calculated by the index unit 120 serve as indicators showing the degree of match between the target face included in the time-series image data and the face of the registered user.

[0026] The likelihood ratio calculation unit 130 is configured to calculate the likelihood ratio based on the integrated feature or score calculated by the index calculation unit 120. The likelihood ratio calculated by the likelihood ratio calculation unit 130 is a value that indicates the likelihood of the time-series image data belonging to a particular class. For example, the likelihood ratio may be a value that indicates the likelihood of the time-series image data belonging to each of a plurality of registration classes set for each registered image data. In this case, the likelihood ratio calculation unit 130 may calculate multiple likelihood ratios corresponding to each of the plurality of registration classes (i.e., the same number of likelihood ratios as the number of registration classes). For example, if N registered image data is registered in the registered image storage unit 125, N registration classes are set to correspond to them, and the likelihood ratio calculation unit 130 may calculate N likelihood ratios corresponding to each of the N registration classes. The specific method used by the likelihood ratio calculation unit 130 to calculate the likelihood ratio is not particularly limited. The likelihood ratio calculation unit 130 may use various existing methods to calculate the likelihood ratio. For example, the likelihood ratio calculation unit 130 may calculate the likelihood ratio using a pre-trained estimation model (specifically, a neural network trained by deep learning, etc.). The likelihood ratio calculation unit 130 may also calculate the likelihood ratio based on two or more consecutive image data from a time-series of image data. For example, the likelihood ratio calculation unit 130 may calculate the likelihood ratio using integrated features or scores calculated from previously acquired image data in addition to integrated features or scores calculated from the most recently acquired image data. The likelihood ratio calculated by the likelihood ratio calculation unit 130 is output to the class determination unit 140.

[0027] The class determination unit 140 is configured to determine the class to which time-series image data belongs based on the likelihood ratio calculated by the likelihood ratio calculation unit 130. In other words, the class determination unit 140 is configured to perform class classification processing based on the likelihood ratio. Specifically, the class determination unit 140 determines that time-series image data belongs to a registered class corresponding to registered image data when the likelihood ratio reaches a preset class threshold. In other words, the class determination unit 140 determines that time-series image data matches one of the registered image data. The class threshold here is a threshold used to determine that the likelihood ratio has become high enough to determine that time-series image data belongs to a registered class. On the other hand, the class determination unit 140 determines that time-series image data does not belong to any of the registered classes when the likelihood ratio reaches the unregistered threshold without reaching the class threshold. In other words, the class determination unit 140 determines that time-series image data does not match any of the registered image data (in other words, it is unregistered). The unregistered threshold here is a threshold that is pre-set to determine if time-series image data does not belong to any registration class. Specific examples of the unregistered threshold will be explained in detail later.

[0028] (Flow of operations) Next, the operation flow of the first information processing device 1 will be explained with reference to Figure 3. Figure 3 is a flowchart showing the operation flow of the first information processing device.

[0029] As shown in Figure 3, when the operation of the first information processing device 1 begins, the image acquisition unit 110 first acquires image data (step S101). The image data acquired here may be the image data of the most recently captured single frame from the time-series image data.

[0030] Next, the index calculation unit 120 extracts a first feature from the image data acquired by the image acquisition unit 110 and a second feature from the registered image data read from the registered image storage unit 125 (step S102). Then, the index calculation unit 120 calculates an integrated feature or score based on the extracted first and second features (step S103).

[0031] Next, the likelihood ratio calculation unit 130 calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs, based on the integrated features or scores calculated by the index calculation unit 120 (step S104).

[0032] Next, the class determination unit 140 determines whether the likelihood ratio calculated by the likelihood ratio calculation unit 130 has reached the unregistered threshold (step S105). If the likelihood ratio has reached the unregistered threshold (step S105: YES), the class determination unit 140 determines that the time-series image data does not belong to any registered class (i.e., it is unregistered) (step S106). In this case, the time-series image data is determined not to match the registered image data (in other words, the target is an unregistered user), and the matching process ends.

[0033] On the other hand, if the likelihood ratio has not reached the unregistered threshold (step S105: NO), the class determination unit 140 determines whether the likelihood ratio has reached the class threshold (step S107). If the likelihood ratio has reached the class threshold (step S107: YES), the class determination unit 140 determines that the time-series image data belongs to a registered class (step S108). Specifically, the class determination unit 140 determines that the time-series image data belongs to a registered class corresponding to a likelihood ratio that exceeds the class threshold. In this case, the time-series image data is determined to match the registered image data (in other words, the target is a registered user), and the matching process ends.

[0034] On the other hand, if the likelihood ratio does not reach the class threshold (step S107: NO), processing starts again from step S101. That is, the image data acquisition unit 110 acquires new image data (for example, image data of the next frame), and the series of processes described above are executed again. As the processing is repeated in this way, the likelihood ratio calculated by the likelihood ratio calculation unit 130 gradually changes. The determination process (in other words, the matching process) continues until the likelihood ratio exceeds the class threshold or the unregistered threshold.

[0035] The first information processing device 1 may be configured to execute various processes related to the target based on the determination result of the class determination unit 140. For example, the first information processing device 1 may be configured to execute a process to permit or prohibit passage of the target to a predetermined area based on the determination result of the class determination unit 140. More specifically, if it is determined that the time-series image data matches the registered image data, the first information processing device 1 may control the gate installed in the predetermined area to open, thereby permitting passage of the target. If it is determined that the time-series image data does not match the registered image data (i.e., unregistered), the first information processing device 1 may control the gate installed in the predetermined area to close, thereby prohibiting passage of the target.

[0036] (Example of operation) Next, with reference to Figures 4 and 5, we will describe specific examples of the operation of the first information processing device 1 (in particular, an example of the operation of determining the class to which time-series image data belongs based on the likelihood ratio). Figure 4 is a graph (1) showing an example of the likelihood ratio calculated by the first information processing device. Figure 5 is a graph (2) showing an example of the likelihood ratio calculated by the first information processing device.

[0037] In the examples shown in Figures 4 and 5, registered image data A, registered image data B, and registered image data C are registered as registered image data. Then, registration classes A, B, and C are set, corresponding to each of the registered image data A, B, and C.

[0038] The likelihood ratio calculation unit 130 calculates likelihood ratios based on the image data acquired by the image acquisition unit 110 and the registered image data A, B, and C. Specifically, the likelihood ratio calculation unit 130 calculates a first likelihood ratio corresponding to class A, a first likelihood ratio corresponding to class B, and a first likelihood ratio corresponding to class C. These likelihood ratios gradually change over time (i.e., as image data is acquired sequentially).

[0039] In particular, the class threshold is set as a threshold for the likelihood ratio in the height direction. Therefore, when the likelihood ratio, which changes over time, becomes sufficiently high, it will reach the class threshold. On the other hand, the unregistered threshold is set as a threshold for the likelihood ratio in the time direction. Therefore, when a certain period of time has passed without the likelihood ratio exceeding the class threshold, it will reach the unregistered threshold.

[0040] In the example shown in Figure 4, the likelihood ratio corresponding to class A reaches the class threshold. On the other hand, the likelihood ratios corresponding to classes B and C do not reach the class threshold. In such cases, the time-series image data acquired by the image acquisition unit 110 is determined to belong to class A. That is, the time-series image data is determined to match registered image data A, but not to registered image data B and C. As a result, a matching result is obtained in which the subject included in the time-series image data is the user corresponding to registered image data A.

[0041] On the other hand, in the example shown in Figure 5, the likelihood ratios corresponding to classes A, B, and C all reach the unregistered threshold without reaching the class threshold. In such cases, the time-series image data acquired by the image acquisition unit 110 is determined not to belong to any of classes A, B, and C. That is, the time-series image data is determined to be unregistered image data that does not match any of the registered image data A, B, and C. As a result, the matching result is obtained that the subject included in the time-series image data is an unregistered user.

[0042] (Technical effects) Next, we will explain the technical effects obtained by the first information processing device 1.

[0043] As explained in Figures 1 to 5, the first information processing device 1 uses a class threshold to determine whether time-series image data is registered, while using an unregistered threshold to determine whether time-series image data is unregistered. This makes it possible to shorten the time required for matching when matching registered image data with class classification. For example, if an unregistered threshold is not used, it would be impossible to obtain a matching result if the likelihood ratio continues to transition without reaching the class threshold. However, in this embodiment, because an unregistered threshold is set, a matching result indicating that the image data is unregistered can be obtained even if the likelihood ratio does not reach the class threshold. Furthermore, by using an unregistered threshold set in the time direction as shown in Figures 4 and 5, it is possible to appropriately adjust the time until it is determined to be unregistered. Note that the unregistered threshold may also be set in a direction other than the time direction. Examples of unregistered thresholds set in directions other than the time direction will be explained in detail in other embodiments described later.

[0044] <Second Embodiment> The second information processing device 1 will be described with reference to Figures 6 to 10. Note that the second information processing device 1 differs from the first information processing device 1 described above in some configurations and operations, while other parts may be the same as those of the first information processing device 1. Therefore, the parts that differ from the first embodiment will be described in detail below, while other overlapping parts will be omitted as appropriate.

[0045] (Functional configuration) First, the functional configuration of the second information processing device 1 will be explained with reference to Figure 6. Figure 6 is a block diagram showing the functional configuration of the second information processing device. Note that in Figure 6, the same reference numerals are used for elements similar to those described in Figure 2.

[0046] In Figure 6, the second information processing device 1 is configured to include an image acquisition unit 110, an index calculation unit 120, a registered image storage unit 125, a likelihood ratio calculation unit 130, a class determination unit 140, and a threshold change unit 150 as components for realizing its functions. That is, the second information processing device 1 further includes a threshold change unit 150 in addition to the configuration described in the first embodiment (see Figure 2). The threshold change unit 150 may be a processing block realized by the processor 11 (see Figure 1) described above.

[0047] The threshold change unit 150 is configured to change the unregistered threshold used by the class determination unit 140. For example, the threshold change unit 150 may change the unregistered threshold set in the time direction to a value corresponding to an earlier time, or to a value corresponding to a later time. The threshold change unit 150 may decide in which direction and by how much to change the unregistered threshold based on various information input to the second information processing device. For example, the threshold change unit 150 may change the unregistered threshold in response to user input. Alternatively, the threshold change unit 150 may change the unregistered threshold based on the input time-series image data. More specifically, the threshold change unit 150 may change the unregistered threshold according to the likelihood ratio or the slope of the likelihood ratio (i.e., rate of change) calculated from the time-series image data. For example, the threshold change unit 150 may change the unregistered threshold using a function that includes the previously calculated likelihood ratio or the slope of the likelihood ratio. Furthermore, the threshold modification unit 150 may dynamically change the unregistered threshold when time-series image data is acquired sequentially. For example, the threshold modification unit 150 may change the unregistered threshold each time new image data is acquired. The threshold modification unit 150 may also change the class threshold in addition to the unregistered threshold. In this case, the threshold modification unit 150 may change the unregistered threshold and the class threshold in conjunction with each other. In this case, a predetermined function may be used to link the unregistered threshold and the class threshold. The threshold modification operation by the threshold modification unit 150 will be explained later with specific examples.

[0048] (Flow of operations) Next, the operation flow of the second information processing device 1 will be explained with reference to Figure 7. Figure 7 is a flowchart showing the operation flow of the second information processing device. Note that in Figure 7, the same reference numerals are used for the same processes as those described in Figure 3.

[0049] As shown in Figure 7, when the operation of the second information processing device 1 begins, the image acquisition unit 110 first acquires image data (step S101).

[0050] Next, the index calculation unit 120 extracts a first feature from the image data acquired by the image acquisition unit 110 and a second feature from the registered image data read from the registered image storage unit 125 (step S102). Then, the index calculation unit 120 calculates an integrated feature or score based on the extracted first and second features (step S103).

[0051] Next, the likelihood ratio calculation unit 130 calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs, based on the integrated features or scores calculated by the index calculation unit 120 (step S104).

[0052] Next, the threshold changing unit 150 changes the unregistered threshold (step S201). The threshold changing unit 150 may change the unregistered threshold based, for example, on the likelihood ratio or the slope of the likelihood ratio calculated in step S104. In addition, the threshold changing unit 150 may also change the class threshold in addition to the unregistered threshold. Although an example is given here in which the threshold is changed after the likelihood ratio is calculated, the threshold changing unit 150 may change the threshold at a different timing. For example, the process in step S201 may be executed sequentially with each of the processes in steps S101 to S104 described above. Alternatively, the process in step S201 may be executed simultaneously with each of the processes in steps S101 to S104 in parallel.

[0053] Next, the class determination unit 140 determines whether the likelihood ratio calculated by the likelihood ratio calculation unit 130 has reached the unregistered threshold (step S105). Here, the class determination unit 140 uses the unregistered threshold modified by the threshold modification unit 150.

[0054] If the likelihood ratio reaches the unregistered threshold (step S105: YES), the class determination unit 140 determines that the time-series image data does not belong to any registered class (i.e., it is unregistered) (step S106). In this case, the time-series image data is determined not to match any registered image data (in other words, the target is an unregistered user), and the matching process ends.

[0055] On the other hand, if the likelihood ratio has not reached the unregistered threshold (step S105: NO), the class determination unit 140 determines whether or not the likelihood ratio has reached the class threshold (step S107). If the threshold changing unit 150 has changed the class threshold, the class determination unit 140 uses the changed class threshold for its determination.

[0056] If the likelihood ratio reaches the class threshold (step S107: YES), the class determination unit 140 determines that the time-series image data belongs to the registered class (step S108). In this case, the time-series image data is determined to match the registered image data (in other words, the target is a registered user), and the matching process ends.

[0057] On the other hand, if the likelihood ratio does not reach the class threshold (step S107: NO), processing starts again from step S101. That is, the image data acquisition unit 110 acquires new image data (for example, image data of the next frame), and the series of processes described above are executed again.

[0058] In the flowchart shown in Figure 7, the threshold change unit 150 changes the unregistered threshold each time a new image is acquired. However, the threshold change unit 150 may reduce the frequency of changing the unregistered threshold. For example, the threshold change unit 150 may change the threshold every few frames (for example, every 5 frames). Alternatively, the threshold change unit 150 may change the threshold every predetermined period (for example, every few seconds or tens of seconds). Furthermore, the threshold change unit 150 may change the threshold only once initially and not change it thereafter. In this case, the threshold change unit 150 may change the threshold when the matching process for the current target is completed and the matching process for a new target is started.

[0059] (Example of operation) Next, with reference to Figures 8 to 10, we will explain specific examples of the operation of the second information processing device 1 (in particular, examples of operations that change the threshold used when determining the class). Figure 8 is a graph showing an example of changing the unregistered threshold in the second information processing device. Figure 9 is a graph (part 1) showing an example of changing the unregistered threshold and class threshold in the second information processing device. Figure 10 is a graph (part 2) showing an example of changing the unregistered threshold and class threshold in the second information processing device.

[0060] In the examples shown in Figures 8 to 10, similar to the examples shown in Figures 4 and 5, registered image data A, registered image data B, and registered image data C are registered as registered image data. Then, registration classes A, B, and C are set, corresponding to each of the registered image data A, B, and C.

[0061] In the example shown in Figure 8, the threshold changing unit 150 changes the unregistered threshold set in the time direction. Specifically, the threshold changing unit 150 changes the unregistered threshold in the direction that makes it faster in time (i.e., to the left in the figure). In this case, the time until image data in a time series that does not reach the class threshold is determined to be unregistered is shortened, so the time until the matching result is obtained is shortened. Therefore, the speed of the matching process can be improved. Alternatively, the threshold changing unit 150 changes the unregistered threshold in the direction that makes it slower in time (i.e., to the right in the figure). In this case, the time until image data in a time series that does not reach the class threshold is determined to be unregistered is lengthened, so the matching process is performed for a longer period of time. Therefore, the accuracy of the matching process can be improved.

[0062] In the example shown in Figure 9, the threshold changing unit 150 changes the unregistered threshold in a direction that makes it faster in time (i.e., to the left in the figure). The threshold changing unit 150 also changes the class threshold to a lower value (i.e., downward in the figure). In this case, the time until image data in a time series that does not reach the class threshold is determined to be unregistered is shortened, while the likelihood ratio becomes easier to reach the class threshold. Therefore, when it is likely that the time until the matching result is obtained will be long, the time until the matching result is obtained can be shortened. The threshold changing unit may perform the operation shown in Figure 9, for example, when the value of the likelihood ratio is low (for example, when the difference between the likelihood ratio and the class threshold is greater than or equal to a predetermined value), or when the slope of the likelihood ratio is gentle (for example, when the range of variation of the likelihood ratio in a predetermined frame is within a predetermined range).

[0063] In the example shown in Figure 10, the threshold changing unit 150 changes the unregistered threshold in a direction that delays time (i.e., to the right in the figure). The threshold changing unit 150 also changes the class threshold to a higher value (i.e., upward in the figure). In this case, the time until time-series image data that does not reach the class threshold is determined to be unregistered becomes longer, while the likelihood ratio becomes less likely to reach the class threshold. Therefore, when the time until the likelihood ratio reaches the class threshold is likely to be short, the accuracy of the matching process can be improved by taking more time. The threshold changing unit may perform the operation shown in Figure 10, for example, when the value of the likelihood ratio is high (for example, when the difference between the likelihood ratio and the class threshold is less than a predetermined value), or when the slope of the likelihood ratio is steep (for example, when the fluctuation range of the likelihood ratio in a predetermined frame exceeds a predetermined range).

[0064] (Technical effects) Next, we will explain the technical effects obtained by the second information processing device 1.

[0065] As explained in Figures 6 to 10, the unregistered threshold is changed in the second information processing device 1. This allows for more appropriate class classification compared to when the unregistered threshold is fixed. For example, it prevents the unregistered threshold from being set too late, which would cause the class classification to take too long (i.e., the time until it is determined to be unregistered to be too long). Also, it prevents the unregistered threshold from being set too early, which would cause it to be determined to be unregistered when it is actually registered (i.e., cause a misclassification).

[0066] Furthermore, as shown in Figures 9 and 10, by changing the unregistered threshold and the class threshold in conjunction with each other, class classification can be performed more appropriately compared to changing only the unregistered threshold (i.e., keeping the class threshold fixed). Moreover, by changing the threshold based on the likelihood ratio value or the slope of the likelihood ratio calculated by the likelihood ratio calculation unit 130, class classification can be performed more appropriately, taking into account the current situation.

[0067] <Third Embodiment> The third information processing device 1 will be described with reference to Figures 11 and 12. Note that the third information processing device 1 differs in some configurations and operations from the first and second information processing devices 1 described above, while other parts may be the same as those of the first and second information processing devices 1. Therefore, the following will provide a detailed explanation of the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0068] (Functional configuration) First, the functional configuration of the third information processing device 1 will be explained with reference to Figure 11. Figure 11 is a block diagram showing the functional configuration of the third information processing device. Note that in Figure 11, the same reference numerals are used for elements similar to those described in Figure 6.

[0069] In Figure 11, the third information processing device 1 is configured to include, as components for realizing its functions, an image acquisition unit 110, an index calculation unit 120, a registered image storage unit 125, a likelihood ratio calculation unit 130, a class determination unit 140, a threshold change unit 150, and a person detection unit 160. That is, the third information processing device 1 further includes a person detection unit 160 in addition to the configuration described in the second embodiment (see Figure 6). The person detection unit 160 may be a processing block realized by the processor 11 (see Figure 1) described above.

[0070] The person detection unit 160 is configured to detect the number of people passing through a location where time-series image data is acquired. For example, the person detection unit 160 may be configured to detect the number of people passing through the imaging range of a camera that takes time-series image data (specifically, the number of people who appear in the image at the same time, or the number of people who pass through the imaging range in a predetermined time). The person detection unit 160 may detect the number of people based on the time-series image data acquired by the image acquisition unit 110. For example, the person detection unit 160 may detect people appearing in the image data and perform a process to count the detected people. Alternatively, the person detection unit 160 may be configured to count the number of people passing through a predetermined area corresponding to the imaging range using a sensor other than the camera.

[0071] The threshold changing unit 150 in the third information processing device 1 changes the threshold based on the number of people detected by the person detection unit 160 described above. The threshold changing unit 150 may change the unregistered threshold to be earlier in time as the number of people detected by the person detection unit 160 increases. For example, if there are many people passing through the camera's imaging range, the number of subjects to be classified (i.e., matched) increases, so it is necessary to speed up the classification of time-series image data. In such cases, in order to improve the speed of classifying time-series image data, the unregistered threshold may be changed to be earlier as the number of people increases. On the other hand, if there are few people passing through the camera's imaging range, the number of subjects to be classified decreases, so it is acceptable for the speed of classifying time-series image data to be somewhat slower. Therefore, the unregistered threshold may be changed to be slower as the number of people decreases.

[0072] (Flow of operations) Next, the operation flow of the third information processing device 1 will be explained with reference to Figure 12. Figure 12 is a flowchart showing the operation flow of the third information processing device. Note that in Figure 12, the same reference numerals are used for the same processes as shown in Figure 3.

[0073] As shown in Figure 12, when the operation of the third information processing device 1 is started, the image acquisition unit 110 first acquires image data (step S101).

[0074] Next, the index calculation unit 120 extracts a first feature from the image data acquired by the image acquisition unit 110 and a second feature from the registered image data read from the registered image storage unit 125 (step S102). Then, the index calculation unit 120 calculates an integrated feature or score based on the extracted first and second features (step S103).

[0075] Next, the likelihood ratio calculation unit 130 calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs, based on the integrated features or scores calculated by the index calculation unit 120 (step S104).

[0076] Next, the person detection unit 160 detects the number of objects passing through the location where time-series image data is acquired (step S301). Then, the threshold changing unit 150 changes the unregistered threshold based on the number of people detected by the person detection unit 160 (step S201). The threshold changing unit 150 may also change the class threshold in addition to the unregistered threshold. Although this example shows the person being detected after the likelihood ratio is calculated, the person detection unit 160 may detect the person at different timings. For example, the processes in steps S301 and S302 may be executed sequentially with the processes in steps S101 to S104 described above. Alternatively, the processes in steps S301 and S302 may be executed simultaneously with the processes in steps S101 to S104.

[0077] Next, the class determination unit 140 determines whether the likelihood ratio calculated by the likelihood ratio calculation unit 130 has reached the unregistered threshold (step S105). Here, the class determination unit 140 uses the unregistered threshold modified by the threshold modification unit 150 (more specifically, the unregistered threshold modified based on the number of people detected by the number of people detection unit 160).

[0078] If the likelihood ratio reaches the unregistered threshold (step S105: YES), the class determination unit 140 determines that the time-series image data does not belong to any registered class (i.e., it is unregistered) (step S106). In this case, the time-series image data is determined not to match any registered image data (in other words, the target is an unregistered user), and the matching process ends.

[0079] On the other hand, if the likelihood ratio has not reached the unregistered threshold (step S105: NO), the class determination unit 140 determines whether or not the likelihood ratio has reached the class threshold (step S107). If the threshold changing unit 150 has changed the class threshold, the class determination unit 140 uses the changed class threshold for its determination.

[0080] If the likelihood ratio reaches the class threshold (step S107: YES), the class determination unit 140 determines that the time-series image data belongs to the registered class (step S108). In this case, the time-series image data is determined to match the registered image data (in other words, the target is a registered user), and the matching process ends.

[0081] On the other hand, if the likelihood ratio does not reach the class threshold (step S107: NO), processing starts again from step S101. That is, the image data acquisition unit 110 acquires new image data (for example, image data of the next frame), and the series of processes described above are executed again.

[0082] (Technical effects) Next, we will explain the technical effects obtained by the third information processing device 1.

[0083] As explained in Figures 11 and 12, the third information processing device 1 changes the unregistered threshold based on the number of subjects passing through the location where time-series image data is acquired. This makes it possible to perform appropriate class classification depending on the circumstances under which the image data is acquired. For example, when there are many subjects and quick class classification is required, the time until the subject is determined to be unregistered can be shortened, thereby reducing the time until the result is output. Conversely, when there are few subjects and it is possible to perform class classification over a longer period of time, the time until the subject is determined to be unregistered can be lengthened, thereby improving the accuracy of the judgment result.

[0084] Here, we have described a configuration in which the threshold is changed according to the number of subjects, but the threshold changing unit 150 may also change the threshold according to various factors that affect class classification (for example, the operating environment). For example, if the subjects quickly move out of the imaging range of the camera that takes time-series image data, it is necessary to speed up the classification of the time-series image data. In such cases, if the unregistered threshold is changed to be earlier the shorter the time it takes for the subject to move out of the imaging range, the time until it is determined to be unregistered will be shortened, and as a result, the speed of class classification can be improved. On the other hand, if the subjects remain in the imaging range of the camera for a long time, it is acceptable for the speed of classifying the time-series image data to be somewhat slower. Therefore, if the unregistered threshold is changed to be later the longer the time it takes for the subject to remain in the imaging range, the time until it is determined to be unregistered will be longer, and as a result, the accuracy of class classification can be improved.

[0085] <Fourth Embodiment> The fourth information processing device 1 will be described with reference to Figures 13 to 16. Note that the fourth information processing device 1 differs in some configurations and operations from the first to third information processing devices 1 described above, while other parts may be the same as those of the first to third information processing devices 1. Therefore, the following will explain in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0086] (Functional configuration) First, the functional configuration of the fourth information processing device 1 will be explained with reference to Figure 13. Figure 13 is a block diagram showing the functional configuration of the fourth information processing device. Note that in Figure 13, the same reference numerals are used for elements that are the same as those shown in Figure 2.

[0087] In Figure 13, the fourth information processing device 1 is configured to include an image acquisition unit 110, an index calculation unit 120, a registered image storage unit 125, a likelihood ratio calculation unit 130, and a class determination unit 140 as components for realizing its functions. In particular, the likelihood ratio calculation unit 130 in the fourth information processing device 1 includes a first calculation unit 1301 and a second calculation unit 1302.

[0088] The first calculation unit 1301 is configured to calculate the first likelihood ratio. The first likelihood ratio is similar to the likelihood ratio calculated in the first to third embodiments described above, and is a value that indicates the likelihood that the time-series image data belongs to a registered class (i.e., the class corresponding to the registered image data). If there are N registered classes, the first calculation unit 1301 only needs to calculate N first likelihood ratios.

[0089] The second calculation unit 1302 is configured to calculate the second likelihood ratio. The second likelihood ratio is a value that indicates the likelihood that the time-series image data belongs to an unregistered class. Here, the unregistered class is the class to which the time-series image data belongs if it does not belong to any registered class (i.e., it is unregistered). Even if there are N registered classes, the second calculation unit 1302 only needs to calculate one second likelihood ratio. In this case, the likelihood ratio calculation unit 130 as a whole will calculate N+1 likelihood ratios.

[0090] As described above, in the fourth information processing device 1, in addition to registered classes, unregistered classes are set as classification candidates for time-series image data. The class determination unit 140 in the fourth information processing device 1 determines whether the time-series image data belongs to one of the registered classes or to an unregistered class. Specifically, the class determination unit 140 determines that the time-series image data belongs to a registered class (i.e., matches registered image data corresponding to a registered class) when the first likelihood ratio reaches the class threshold. On the other hand, the class determination unit 140 determines that the time-series image data belongs to an unregistered class (i.e., is unregistered) when the second likelihood ratio reaches the unregistered threshold before the first likelihood ratio reaches the class threshold.

[0091] Furthermore, in the fourth information processing device 1, when calculating the likelihood ratio using a machine learning model, it is required that the class used for training and the class used for operation are the same. For this reason, for example, when adding registered classes as needed (i.e., adding registered image data after training), it is advisable to retrain the model each time the number of classes changes. Alternatively, it is advisable to use the unregistered threshold set in the time direction, as described in the first to third embodiments.

[0092] (Flow of operations) Next, the operation flow of the fourth information processing device 1 will be explained with reference to Figure 14. Figure 14 is a flowchart showing the operation flow of the fourth information processing device. Note that in Figure 14, the same reference numerals are used for the same processes as those described in Figure 3.

[0093] As shown in Figure 14, when the operation of the fourth information processing device 1 is started, the image acquisition unit 110 first acquires image data (step S101).

[0094] Next, the index calculation unit 120 extracts a first feature from the image data acquired by the image acquisition unit 110 and a second feature from the registered image data read from the registered image storage unit 125 (step S102). Then, the index calculation unit 120 calculates an integrated feature or score based on the extracted first and second features (step S103).

[0095] Next, the first calculation unit 1301 calculates a first likelihood ratio indicating the likelihood that the time-series image data belongs to a registered class (step S401). Then, the second calculation unit 1302 calculates a second likelihood ratio indicating the likelihood that the time-series image data belongs to an unregistered class (step S402). Note that the processes in step S401 and step S402 may be executed sequentially or simultaneously in parallel.

[0096] Next, the class determination unit 140 determines whether the second likelihood ratio calculated by the second calculation unit 1302 has reached the unregistered threshold (step S403). If the second likelihood ratio has reached the unregistered threshold (step S403: YES), the class determination unit 140 determines that the time-series image data belongs to the unregistered class. That is, the class determination unit determines that the time-series image data does not belong to any registered class (i.e., it is unregistered) (step S106). In this case, the time-series image data is determined not to match the registered image data (in other words, the target is an unregistered user), and the matching process ends.

[0097] On the other hand, if the second likelihood ratio does not reach the unregistered threshold (step S403: NO), the class determination unit 140 determines whether the first likelihood ratio calculated by the first calculation unit 1301 has reached the class threshold (step S404). If the first likelihood ratio has reached the class threshold (step S404: YES), the class determination unit 140 determines that the time-series image data belongs to a registered class (step S108). In this case, the time-series image data is determined to match the registered image data (in other words, the target is a registered user), and the matching process ends.

[0098] On the other hand, if the first likelihood ratio does not reach the class threshold (step S404: NO), processing starts again from step S101. That is, the image data acquisition unit 110 acquires new image data (for example, image data of the next frame), and the series of processes described above are executed again.

[0099] (Example of operation) Next, with reference to Figures 15 and 16, specific examples of the operation of the fourth information processing device 1 (in particular, an example of the operation of determining the class to which time-series image data belongs based on the likelihood ratio) will be described. Figure 15 is a graph (part 1) showing an example of the likelihood ratio calculated in the first information processing device. Figure 16 is a graph (part 2) showing an example of the likelihood ratio calculated in the first information processing device.

[0100] In the examples shown in Figures 15 and 16, similar to the examples shown in Figures 4 and 5, and Figures 8 through 10, registered image data includes registered image data A, registered image data B, and registered image data C. Furthermore, registration classes A, B, and C are set, corresponding to each of the registered image data A, B, and C.

[0101] Furthermore, in the examples shown in Figures 15 and 16, an "unregistered" class is set, which is to which the time-series image data belongs if it does not belong to any of classes A, B, or C. Therefore, The likelihood ratio calculation unit 130 calculates three first likelihood ratios corresponding to classes A, B, and C, and one second likelihood ratio corresponding to the unregistered class. Both the first and second likelihood ratios gradually change over time (i.e., as image data is acquired sequentially).

[0102] The class threshold is a threshold set for the likelihood ratio in the height direction. Therefore, when the first likelihood ratio, which changes over time, becomes sufficiently high, the class threshold is reached. Similarly, the unregistered threshold is also a threshold set for the likelihood ratio in the height direction. Therefore, when the second likelihood ratio, which changes over time, becomes sufficiently high, the unregistered threshold is reached. Note that here we give an example where the class threshold and the unregistered threshold are the same value, but the class threshold and the unregistered threshold may be different values ​​(i.e., thresholds with different heights).

[0103] In the example shown in Figure 15, the first likelihood ratio corresponding to class A has reached the class threshold. On the other hand, the first likelihood ratios corresponding to classes B and C have not reached the class threshold. Furthermore, the second likelihood ratio corresponding to the unregistered class has not reached the unregistered threshold. In such a case, the time-series image data acquired by the image acquisition unit 110 is determined to belong to class A. That is, the time-series image data is determined to match registered image data A, but not to registered image data B and C. As a result, a matching result is obtained in which the subject included in the time-series image data is the user corresponding to registered image data A.

[0104] In the example shown in Figure 16, none of the first likelihood ratios corresponding to classes A, B, and C have reached the class threshold. On the other hand, the second likelihood ratio corresponding to the unregistered class has reached the unregistered threshold. In such a case, the time-series image data acquired by the image acquisition unit 110 is determined to belong to the unregistered class. In other words, the time-series image data is determined to be unregistered image data that does not belong to any of classes A, B, and C. Therefore, the time-series image data is determined not to match any of the registered image data A, B, and C. As a result, the matching result is obtained that the subject included in the time-series image data is an unregistered user.

[0105] (Technical effects) Next, we will explain the technical effects obtained by the fourth information processing device 1.

[0106] As explained in Figures 13 to 16, the fourth information processing device 1 determines whether or not time-series image data is unregistered by determining whether or not the time-series image data belongs to an unregistered class. In this way, when matching registered image data with the class classification, the time required for matching can be shortened. For example, if an unregistered threshold is not used, if the likelihood ratio continues to transition without reaching the class threshold, it will not be possible to obtain a matching result. However, in this embodiment, because an unregistered threshold is set, even if the likelihood ratio does not reach the class threshold, it is possible to obtain a matching result that the image data is unregistered.

[0107] Furthermore, the likelihood ratios used for classification may be calculated as a first likelihood ratio (i.e., the likelihood ratio indicating the likelihood of the time-series image data belonging to a particular class) and a second likelihood ratio (i.e., the likelihood ratio indicating the likelihood of the time-series image data belonging to an unregistered class), as described above. In this way, appropriate classification and unregistered determination can be performed using the likelihood ratio corresponding to the registered class and the likelihood ratio corresponding to the unregistered class.

[0108] The processing method of recording a program that operates the configuration of each embodiment in order to realize the functions of each embodiment described above on a recording medium, reading the program recorded on the recording medium as code, and executing it on a computer is also included in the scope of each embodiment. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0109] As recording media, for example, floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs can be used. Furthermore, the scope of each embodiment is not limited to programs that perform processing on the recording media alone, but also includes programs that operate on the OS and perform processing in cooperation with other software and the functions of expansion boards. In addition, the program itself may be stored on a server, and part or all of the program may be made available for download from the server to the user terminal. The program may be provided to the user in, for example, SaaS (Software as a Service) format.

[0110] <Note> The embodiments described above may also be described in the following appendix, but are not limited to these.

[0111] (Note 1) The information processing device described in Appendix 1 is an information processing device comprising: acquisition means for acquiring time-series image data; index calculation means for calculating an integrated feature obtained by integrating a first feature which is a feature of the time-series image data and a second feature which is a feature of pre-registered image data, or a score indicating the similarity between the first feature and the second feature; likelihood ratio calculation means for calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score; determination means for determining that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and determining that the time-series image data does not belong to any of the registered classes when the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0112] (Note 2) The information processing device described in Appendix 2 is the same as the information processing device described in Appendix 1, wherein the unregistered threshold is a threshold set in the time direction.

[0113] (Note 3) The information processing device described in Appendix 3 is the same as the information processing device described in Appendix 2, further comprising a threshold changing means for dynamically changing the unregistered threshold.

[0114] (Note 4) The information processing device described in Appendix 4 is the information processing device described in Appendix 3, wherein the threshold changing means changes the class threshold and the unregistered threshold in conjunction with each other.

[0115] (Note 5) The information processing device described in Appendix 5 is the information processing device described in Appendix 3, wherein the threshold changing means changes the unregistered threshold based on the likelihood ratio or the slope of the likelihood ratio.

[0116] (Note 6) The information processing device described in Appendix 6 further comprises detection means for detecting the number of objects passing through the location where the time-series image data is acquired, and the threshold changing means changes the unregistered threshold based on the number of objects, as described in Appendix 3.

[0117] (Note 7) The information processing device described in Appendix 7 is the information processing device described in Appendix 1, wherein an unregistered class is set which applies when the time-series image data has not been registered in advance, and the unregistered threshold is a threshold for determining whether or not the time-series image data belongs to the unregistered class.

[0118] (Note 8) The information processing device described in Appendix 8 is the information processing device described in Appendix 7, wherein the likelihood ratio calculation means calculates a first likelihood ratio indicating the likelihood that the time-series image data belongs to the registered class and a second likelihood ratio indicating the likelihood that the time-series image data belongs to the unregistered class, and the determination means determines that the time-series image data belongs to the unregistered class if the second likelihood ratio reaches the unregistered threshold before the first likelihood ratio reaches the class threshold.

[0119] (Note 9) The information processing method described in Appendix 9 is an information processing method in which at least one computer acquires time-series image data, calculates an integrated feature by integrating a first feature which is a feature of the time-series image data and a second feature which is a feature of pre-registered image data, or calculates a score indicating the similarity between the first feature and the second feature, calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score, determines that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and determines that the time-series image data does not belong to any of the registered classes when the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0120] (Note 10) The computer program described in Appendix 10 is a computer program that causes at least one computer to execute an information processing method which includes acquiring time-series image data, integrating a first feature which is a feature of the time-series image data with a second feature which is a feature of pre-registered image data, or calculating a score indicating the similarity between the first feature and the second feature, calculating a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score, determining that the time-series image data belongs to a registered class corresponding to the registered image data if the likelihood ratio reaches a class threshold, and determining that the time-series image data does not belong to any of the registered classes if the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0121] (Note 11) The recording medium described in Appendix 11 is a recording medium on which a computer program is recorded that causes at least one computer to execute an information processing method that acquires time-series image data, calculates an integrated feature obtained by integrating a first feature which is a feature of the time-series image data and a second feature which is a feature of pre-registered image data, or calculates a score indicating the similarity between the first feature and the second feature, calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs based on the integrated feature or the score, determines that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and determines that the time-series image data does not belong to any of the registered classes when the likelihood ratio reaches an unregistered threshold without reaching the class threshold.

[0122] This disclosure may be modified as appropriate, insofar as it does not contradict the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and information processing devices, information processing methods, and computer programs that include such modifications are also included in the technical idea of ​​this disclosure. [Explanation of Symbols]

[0123] 1. Information Processing Device 11 processors 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 110 Image acquisition unit 120 Indicator calculation section 125 Registered Image Storage Unit 130 Likelihood Ratio Calculation Unit 1301 First Calculation Unit 1302 Second Calculation Unit 140 Class determination unit 150 Threshold change section 160 people detection unit

Claims

1. A means of acquiring time-series image data, An index calculation means for calculating an integrated feature obtained by integrating a first feature, which is a feature of the time-series image data, and a second feature, which is a feature of pre-registered registered image data, or a score indicating the similarity between the first feature and the second feature. A likelihood ratio calculation means that calculates a likelihood ratio indicating the likelihood of the class to which the time-series image data belongs, based on the integrated features or the score, A determination means that determines that the time-series image data belongs to a registered class corresponding to the registered image data when the likelihood ratio reaches a class threshold, and determines that the time-series image data does not belong to any of the registered classes when the likelihood ratio reaches an unregistered threshold without reaching the class threshold. An information processing device equipped with the following features.

2. The aforementioned unregistered threshold is a threshold set in the time direction. The information processing apparatus according to claim 1.

3. The system further includes threshold changing means for dynamically changing the aforementioned unregistered threshold. The information processing apparatus according to claim 2.

4. The threshold changing means changes the class threshold and the unregistered threshold in conjunction with each other. The information processing apparatus according to claim 3.

5. The threshold changing means changes the unregistered threshold based on the likelihood ratio or the slope of the likelihood ratio. The information processing apparatus according to claim 3.

6. The system further includes a detection means for detecting the number of objects passing through the location where the aforementioned time-series image data is acquired. The threshold changing means changes the unregistered threshold based on the number of targets. The information processing apparatus according to claim 3.

7. An "unregistered" class is set, which applies to cases where the aforementioned time-series image data has not been registered in advance. The aforementioned unregistered threshold is a threshold used to determine whether or not the time-series image data belongs to the unregistered class. The information processing apparatus according to claim 1.

8. The likelihood ratio calculation means calculates a first likelihood ratio indicating the likelihood that the time-series image data belongs to the registered class, and a second likelihood ratio indicating the likelihood that the time-series image data belongs to the unregistered class. The determination means determines that the time-series image data belongs to the unregistered class if the second likelihood ratio reaches the unregistered threshold before the first likelihood ratio reaches the class threshold. The information processing apparatus according to claim 7.

9. At least one computer, Acquire time-series image data, A combined feature is calculated by integrating the first feature, which is a feature of the time-series image data, and the second feature, which is a feature of pre-registered image data, or a score indicating the similarity between the first feature and the second feature is calculated. Based on the aforementioned integrated features or the aforementioned score, a likelihood ratio is calculated that indicates the likelihood of the class to which the time-series image data belongs. If the likelihood ratio reaches a class threshold, it is determined that the time-series image data belongs to the registration class corresponding to the registered image data. If the likelihood ratio reaches an unregistered threshold without reaching the class threshold, it is determined that the time-series image data does not belong to any of the registration classes. Information processing methods.

10. On at least one computer, Acquire time-series image data, A combined feature is calculated by integrating the first feature, which is a feature of the time-series image data, and the second feature, which is a feature of pre-registered image data, or a score indicating the similarity between the first feature and the second feature is calculated. Based on the aforementioned integrated features or the aforementioned score, a likelihood ratio is calculated that indicates the likelihood of the class to which the time-series image data belongs. If the likelihood ratio reaches a class threshold, it is determined that the time-series image data belongs to the registration class corresponding to the registered image data. If the likelihood ratio reaches an unregistered threshold without reaching the class threshold, it is determined that the time-series image data does not belong to any of the registration classes. A computer program that executes information processing methods.

Images