Information processing device, information processing method, and program

Abstract

To improve the accuracy of identity recognition. [Solution] The information processing device performs recognition processing on a person using biometric information obtained from the image of the person detected from the image. At this time, it detects inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, and changes the security level related to the recognition processing in response to the detection of the inconsistency.

Description

Technical Field

[0001] The present invention relates to an information processing technology used for personal identification.

Background Art

[0002] Personal identification using biometric information such as face and fingerprint has been increasingly used in recent years because it can recognize whether a person is the registered user without carrying an extra item such as an IC card. However, by obtaining the biometric information of the registered user (such as a face photo in the case of face recognition), it is possible to "impersonate" the registered user, and there is a risk of breaching the personal identification. Therefore, methods and devices for enhancing security by detecting impersonation have been proposed. Taking face recognition as an example, items used for impersonation (hereinafter referred to as impersonation attack items) include printed materials with the face photo of the registered user, notepads with the face image of the registered user drawn, latex masks molded based on the face of the registered user, etc.). And by presenting any of these impersonation attack items in front of the camera that captures the face image to be processed for face recognition, it becomes possible for others to impersonate the registered user. Impersonation detection is a technology for detecting such impersonation attack items.

[0003] On the other hand, Patent Document 1 discloses a personal identification system that enhances security by performing the impersonation detection process for detecting whether it is an impersonation by distinguishing between living bodies and non-living bodies a predetermined number of times, and increasing the number of personal identification attempts when impersonation is detected. Patent Document 2 discloses a method for detecting impersonation based on the movement of the line of sight of the subject. Furthermore, Patent Document 3 discloses a method for detecting impersonation by asking the subject to move their face. In addition, as another method for enhancing security, Patent Document 4 discloses a method of registering the face images of suspicious persons who have performed suspicious actions in a list, and invalidating the password input for an electronic lock if the person unlocking the electronic lock is included in the suspicious person list.

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2015-82195 [Patent Document 2] Japanese Patent Publication No. 2008-15800 [Patent Document 3] Japanese Patent Publication No. 2008-305400 [Patent Document 4] Japanese Patent Publication No. 2009-59222 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, the aforementioned impersonation detection methods do not necessarily detect impersonating attackers as non-living entities, and if impersonation detection fails, there is a risk that the impersonator may be mistakenly identified as the registered user. In the method disclosed in Patent Document 1, for example, if an impersonating attacker that is an elaborate replica of the registered user's face is used, there is a possibility that the impersonation will not be detected even after performing impersonation detection a predetermined number of times. In the method disclosed in Patent Document 2, for example, if the eye area of ​​the mask is cut out to reveal the real eyes, there is a possibility that the impersonation will not be detected. In the method disclosed in Patent Document 3, if a mask with a three-dimensional shape rather than a flat surface is used as an impersonating attacker, there is a possibility that the impersonation will not be detected. In addition, in the method disclosed in Patent Document 4, for example, if someone else wears a latex mask, it may not be possible to determine whether they are a suspicious person, and an impersonation attack may be carried out. Thus, with existing technologies, there is a risk that a person impersonating the registered user may be recognized as the registered user, and the accuracy of identity recognition is not necessarily high.

[0006] Therefore, the present invention aims to improve the accuracy of person recognition. [Means for solving the problem]

[0007] The information processing apparatus of the present invention is characterized by comprising: recognition means for performing recognition processing on a detected person using biometric information obtained from an image of a person detected from an image; inconsistency detection means for detecting inconsistencies between first information obtained from an image of a detected person and second information obtained as information corresponding to the first information; and modification means for changing the security level related to the recognition processing in the recognition means in response to the detection of the inconsistency. [Effects of the Invention]

[0008] According to the present invention, the accuracy of person recognition can be increased. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of an application of an information processing device. [Figure 2] This figure shows an example of the hardware configuration of an information processing device. [Figure 3] This figure shows an example of the functional configuration of an information processing device. [Figure 4] This figure shows an example of a lookup table used to change security levels. [Figure 5] This is a flowchart of the information processing according to the first embodiment. [Figure 6] This is a flowchart of the processing for detected individuals in the second embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. The following embodiments are not intended to limit the present invention, and not all combinations of features described in these embodiments are essential to the solutions of the present invention. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications of the apparatus to which the present invention is applied and various conditions (usage conditions, usage environment, etc.). In addition, in the following embodiments, the same or similar configurations and processing steps are denoted by the same reference numerals, and redundant explanations are omitted.

[0011] <First Embodiment> In the first embodiment, an example will be given of a scenario in which the information processing device according to this embodiment verifies the identity of the registered person by facial recognition when payment is made at an unmanned checkout counter in a store. Figure 1 shows an example in which the information processing device 100 according to this embodiment is installed in a store equipped with an unmanned checkout counter. As shown in Figure 1, the store is equipped with the information processing device 100, a checkout counter camera 101, and a surveillance camera 102, and the checkout counter camera 101 and the surveillance camera 102 are connected to the information processing device 100 via a network. The surveillance camera 102 is used to photograph the inside of the store, and the checkout counter camera 101 is used to photograph the area in front of the checkout counter.

[0012] When the information processing device 100 detects a person 110 from the video image acquired by the surveillance camera 102, it starts the information processing according to this embodiment, as described later. The information processing device 100 tracks the person 110 in the video acquired by the surveillance camera 102 and collects first information from the image of the person 110 that will be used for inconsistency detection, as described later. In this embodiment, the first information collected from the image of the person 110 is information representing the characteristics of the person. In the following description, the person detected from the image will be referred to as the "detected person," the information representing the characteristics of the person will be referred to as the "person characteristics," and the person characteristics of the detected person will be referred to as the "detected person characteristics." Details of the person characteristics will be described later. If, for example, multiple people are detected, the information processing device 100 tracks each of those people and collects the person characteristics for each of those people. The information processing device 100 not only detects people from the images of the surveillance camera 102, but also performs the process of detecting people from images taken by the cash register camera 101.

[0013] Further, the information processing apparatus 100 determines whether a settlement procedure for goods or the like has been started by the person 110 being tracked. For example, when the information processing apparatus 100 knows that the person 110 being tracked by the surveillance camera 102 has come to the front of the cash register, or when the person 110 who has been tracked is detected from the image of the front-of-cash-register camera 101, it determines that the settlement procedure has been started by that person 110. Note that the information processing apparatus 100 may determine, for example, whether the personal identification described later has been started instead of determining whether the settlement procedure has been started.

[0014] When the settlement procedure has started, the information processing apparatus 100 acquires an image of a predetermined biological part (the face in this embodiment) of the person 110 detected from the image of the front-of-cash-register camera 101, and performs personal identification as to whether the person is a registered person using the biological information extracted from the image of the biological part. Hereinafter, the person who is the subject of the personal identification (in this example, the person 110) will be referred to as the "person to be identified".

[0015] Also, when the settlement procedure has started, the information processing apparatus 100 acquires second information used for inconsistency detection described later from an image of a predetermined biological part used for personal identification as to whether the person to be identified is a registered person. In the case of this embodiment, the information processing apparatus 100 estimates information indicating the attributes of the person to be identified from an image of a predetermined biological part of the person to be identified, and further acquires the person characteristics of the person to be identified as the second information based on the information of the attributes. In the following description, the information indicating the attributes of a person will be referred to as "person attributes", and the person characteristics as the second information acquired based on the person attributes of the person to be identified will be referred to as "person characteristics of the person to be identified". Note that in this embodiment, the image of the biological part used for personal identification is, for example, an image of the face. Details of the person attributes estimated from an image of a predetermined biological part (face) of the person to be identified and the person characteristics of the person to be identified acquired based on the person attributes will be described later.

[0016] Furthermore, the information processing apparatus compares the detected person characteristics acquired as the first information with the person characteristics of the person to be identified acquired as the second information, and detects an inconsistency between the person characteristics of the person to be identified and the detected person characteristics. For example, when no mismatch (i.e., match) is detected between the characteristics of the recognized person and the characteristics of the detected person, the information processing apparatus 100 proceeds with the settlement procedure based on the authentication of the recognized person. Then, when the information processing apparatus 100 can recognize that the recognized person is the registered person through the authentication process, it permits the settlement of goods by the recognized person. On the other hand, when it recognizes that the recognized person is not the registered person, it does not permit the settlement. Also for example, when it is detected that there is a mismatch between the characteristics of the recognized person and the characteristics of the detected person, the information processing apparatus 100 determines that the recognized person is likely a person impersonating the registered person. In this case, the information processing apparatus 100 changes the security level related to the recognition process for the recognized person. Then, the information processing apparatus 100 performs the recognition process after changing the security level. When it can recognize that the recognized person is the registered person, it permits the settlement of goods. On the other hand, when it recognizes that the recognized person is not the registered person, it does not permit the settlement. Details of the process of changing the security level related to the recognition process and the recognition process after the security level change will be described later.

[0017] FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus 100. In addition to the information processing apparatus 100, FIG. 2 also shows a camera 209, a network 208, an input device 210, and an output device 211. The information processing apparatus 100 includes a CPU 200, a ROM 201, a RAM 202, a secondary storage device 203, an output device IF (interface) 204, an input device IF 205, a communication IF 206, and a system bus 207.

[0018] The CPU 200 executes instructions according to programs stored in the ROM 2 * 1 and the RAM 202. The ROM 201 is a non - volatile memory that stores programs and data necessary for controlling each part. The RAM 202 is a volatile memory that stores image data for each frame of a video and various temporary data used in the recognition process. The secondary storage device 203 is a rewritable secondary storage device such as a hard disk drive or flash memory, and stores image data for each frame of the video, the information processing program according to this embodiment, and various settings. The information processing program and data according to this embodiment are transferred to the RAM 202, and the CPU 200 executes the information processing program and uses the data. The information processing program according to this embodiment may also be stored in the ROM 201. The processing of each step in each flowchart described later, as well as the functional configuration shown in Figure 3, are realized by loading the information processing program according to this embodiment, which is stored in the secondary storage device 203 or ROM 201, into the RAM 202 and executing it on the CPU 200.

[0019] Output device IF204 transmits information such as the results of the recognition process to output device 211. Input device IF205 receives information input from input device 210. Communication interface IF206 consists of modems, LANs, and other devices that connect to network 208, such as the internet or intranet. The system bus 207 connects these components and enables them to input and output data to each other.

[0020] Camera 209 is a shooting device comprising an imaging lens, an imaging sensor such as a CCD or CMOS, and an image signal processing unit, and outputs an image for each frame of a video. In this embodiment, camera 209 corresponds to the surveillance camera 102 and the cashier camera 101 in Figure 1. The image for each frame output from camera 209 is transmitted to the information processing device 100 via the network 208, and the information processing device 100 acquires the image for each frame transmitted from camera 209 via the communication IF 206. Note that camera 209 may be directly connected to the information processing device 100 without going through the network 208.

[0021] The input device 210 is, for example, a device that reads biometric information such as fingerprints, or a keyboard or touch panel for entering passwords, and enables input from the user. The output device 211 is, for example, a display device that shows the results of the recognition process, or a cash register that receives the results of the recognition process by the information processing device 100 and performs payment based on those results.

[0022] Figure 3 is a functional block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. The image acquisition unit 300 acquires frame-by-frame images of video captured by cameras 209 (surveillance camera 102 and cashier camera 101) connected via a network not shown in Figure 3. The person detection unit 301 detects a person 110 from an image acquired by the image acquisition unit 300. In other words, the person detection unit 301 detects a person from a video taken by the surveillance camera 102 or from an image taken by the cash register camera 101. The tracking unit 302 tracks the detected person frame by frame in the video. In this embodiment, the tracking unit 302 tracks the person detected by the person detection unit 301 from the image of the surveillance camera 102 until the payment procedure is started at the register.

[0023] The inconsistency detection unit 320 includes an attribute estimation unit 303 and a characteristic acquisition unit 304, and detects inconsistencies between first information acquired from the image of the detected person and second information acquired as information corresponding to the first information. In this embodiment, the first information is the characteristics of the detected person, and the second information is the characteristics of the person being recognized, and the inconsistency detection unit 320 detects inconsistencies by comparing these detected person characteristics and the person being recognized characteristics.

[0024] The characteristic acquisition unit 304 collects, as first information, the characteristics of the detected person (detected person characteristics) based on the image of the detected person detected by the person detection unit 301 and tracked by the tracking unit 302 from the image of the surveillance camera 102. In this embodiment, the characteristics of the person include, for example, the person's walking speed, physique, posture, gait characteristics (stride length, swing of arms and legs, walking cycle, left-right asymmetry of movement, etc.), and the time spent in different areas of the store. The characteristic acquisition unit 304 collects one or more of these characteristics of the person.

[0025] The attribute estimation unit 303 extracts images of predetermined biological parts (faces) from the images of the person being recognized, which are detected by the person detection unit 301, from the images of the camera 101 in front of the cash register, and estimates the person attributes of the person being recognized from the images of those biological parts (faces). In this embodiment, the person attributes of the person being recognized include, for example, the person's race, gender, age, and physique predicted from the face (obese, thin, average), and the attribute estimation unit 303 estimates one or more of these person attributes based on the face image of the person being recognized. In the following description, the person attributes estimated for the person being recognized will be referred to as the person attributes. Note that existing attribute estimation processes can be used for the process of estimating person attributes from face images.

[0026] Furthermore, the characteristic acquisition unit 304 acquires the person characteristics (recognized person characteristics) of the recognized person as second information corresponding to the person characteristics of the first information, based on the recognized person attributes estimated by the attribute estimation unit 303. The recognized person characteristics acquired based on the recognized person attributes are one or more of the same as the detected person characteristics, such as walking speed, physique, posture, gait characteristics, and time spent in different areas of the store.

[0027] For example, if the characteristic acquisition unit 304 collects walking speed as detected person characteristics (first information), it estimates the walking speed of the person being recognized based on the person's attributes and acquires the estimated walking speed of the person being recognized as a person characteristic (second information). For example, the characteristic acquisition unit 304 maintains correspondence information that associates a representative walking speed with one or more person attributes of a person, such as race, gender, age, and physique. The characteristic acquisition unit 304 uses one or more of the person's attributes, such as race, gender, age, and physique, to refer to the correspondence information and acquires the representative walking speed as the walking speed of the person's characteristics.

[0028] For example, if the characteristic acquisition unit 304 collects the physical characteristics of the detected person as the detected person characteristic (first information), it acquires the physical characteristics, which are one of the attributes of the person being recognized, as the person being recognized characteristic (second information).

[0029] For example, if the characteristic acquisition unit 304 collects posture as detected person characteristics (first information), it estimates the posture of the person being recognized based on the person's attributes and acquires that posture as the person's characteristics (second information). For example, the characteristic acquisition unit 304 maintains correspondence relationship information that associates a representative posture with one or more person attributes such as race, gender, age, and physique. The characteristic acquisition unit 304 uses one or more of the person's attributes such as race, gender, age, and physique to refer to the correspondence relationship information and acquires the representative posture as the posture of the person's recognized characteristics.

[0030] For example, if the characteristic acquisition unit 304 collects gait features as detected person characteristics (first information), it estimates the gait features of the person being recognized based on the person's attributes and acquires the estimated gait features of the person being recognized as recognized person characteristics (second information). For example, the characteristic acquisition unit 304 maintains correspondence information that associates the gait features of a person extracted from consecutive frames of a person's video using known gait feature extraction methods with one or more person attributes such as race, gender, age, and physique. The characteristic acquisition unit 304 acquires a representative gait feature obtained by referring to the correspondence information using one or more of the person's attributes such as race, gender, age, and physique, as the gait feature of the person's characteristics.

[0031] For example, if the characteristic acquisition unit 304 collects the detected person's time spent in different areas of the store as detected person characteristics (first information), it estimates the area-specific time spent by the person being recognized based on the recognized person's attributes and acquires that area-specific time spent as recognized person characteristics (second information). For example, it is known that typical behavioral patterns in a store (such as the product areas visited) differ depending on a person's race, age, and gender, and that the area-specific time spent also differs depending on the typical behavioral pattern. The characteristic acquisition unit 304 maintains correspondence relationship information that associates representative area-specific time spent according to representative behavioral patterns with one or more person attributes such as race, gender, and age. The characteristic acquisition unit 304 acquires the representative area-specific time spent by referring to the correspondence relationship information using one or more of the recognized person's attributes such as race, gender, and age, as the area-specific time spent by the recognized person's characteristics. Furthermore, the area-specific dwell time of the detected person can be obtained as follows. For example, the characteristic acquisition unit 304 divides the image captured inside the store by the surveillance camera 102 into multiple small areas and measures the cumulative time the detected person stayed in each small area. The characteristic acquisition unit 304 then arranges the cumulative times of each small area in a line and calculates the dwell time of the detected person in each small area by individually dividing the cumulative time of each small area by the total dwell time of the detected person inside the store. The characteristic acquisition unit 304 obtains this dwell time for each small area as the area-specific dwell time of the detected person.

[0032] The inconsistency detection unit 320 then compares the first piece of information, the detected person characteristics, and the second piece of information, the recognized person characteristics, which were acquired as described above, and detects an inconsistency if they do not match.

[0033] For example, if the characteristic acquisition unit 304 acquires walking speed as both a detected person characteristic and a recognized person characteristic, the inconsistency detection unit 320 compares the walking speeds of the detected person characteristic and the recognized person characteristic. The inconsistency detection unit 320 then detects an inconsistency if the difference between the walking speed of the detected person characteristic and the walking speed of the recognized person characteristic is greater than or equal to a predetermined speed difference threshold.

[0034] For example, if the characteristic acquisition unit 304 acquires both the detected person's characteristics and the recognized person's characteristics, the inconsistency detection unit 320 compares the detected person's characteristics with the recognized person's characteristics. The inconsistency detection unit 320 then detects an inconsistency if the detected person's characteristics and the recognized person's characteristics do not match.

[0035] For example, if the characteristic acquisition unit 304 acquires both the detected person's characteristics and the posture of the person being recognized, the inconsistency detection unit 320 compares the postures of the detected person and the person being recognized. For example, the inconsistency detection unit 320 uses a known joint point estimation method to estimate at least the joint points of the head, waist, and ankles from the person's posture, and acquires the angle formed by the straight line connecting the joint points of the head and waist and the straight line connecting the joint points of the waist and ankles. The inconsistency detection unit 320 compares the angle acquired from the posture of the detected person's characteristics with the angle acquired from the posture of the person being recognized, and detects an inconsistency if the difference between the angles of the detected person's posture and the angles of the person being recognized's posture is greater than or equal to a predetermined angle difference threshold.

[0036] For example, if the characteristic acquisition unit 304 acquires gait features as both detected person characteristics and recognized person characteristics, the inconsistency detection unit 320 compares these gait features of the detected person characteristics and the recognized person characteristics. For instance, the inconsistency detection unit 320 calculates the Euclidean distance between the gait features of the detected person characteristics and the gait features of the recognized person characteristics, and detects an inconsistency if the Euclidean distance is greater than or equal to a predetermined distance threshold.

[0037] For example, if the characteristic acquisition unit 304 acquires the area-specific dwell time as both the detected person characteristic and the recognized person characteristic, the inconsistency detection unit 320 compares the area-specific dwell time of the detected person characteristic and the recognized person characteristic. The inconsistency detection unit 320 calculates the Euclidean distance between the area-specific dwell time of the detected person characteristic and the area-specific dwell time of the recognized person characteristic, and detects an inconsistency if the Euclidean distance is greater than or equal to a predetermined distance threshold.

[0038] The biorecognition unit 310 collects biometric information from images of a predetermined biometric part (face) of the person to be recognized, and uses the extracted biometric information to perform recognition processing on the person to be recognized. The biorecognition unit 310 consists of an impersonation detection unit 305 and a person recognition unit 306. The impersonation detection unit 305 determines whether the person detected by the person detection unit 301 is a living being or not, and detects impersonation based on the result of that determination. In this embodiment, the impersonation detection unit 305 determines whether the detected person is a living being or a non-living being such as a photograph, based on the facial image of the person detected from the image of the camera 101 in front of the cash register, and determines that impersonation is occurring if it is determined that the person is a non-living being. The person recognition unit 306 obtains facial feature quantities from the image of a person detected by the person detection unit 301 from the image of the camera 101 in front of the cash register, compares these feature quantities with the facial feature quantities of registered persons, and performs person recognition to determine whether the detected person is a registered person or not.

[0039] In this embodiment, if an inconsistency is detected by the inconsistency detection unit 320, the level change unit 307, described later, changes at least one of the security levels related to impersonation detection and the security level related to person recognition in the biometric recognition unit 310.

[0040] In other words, the impersonation detection unit 305 changes the security level related to impersonation detection according to the security level set by the level change unit 307, which will be described later. As will be described in detail later, if an inconsistency is detected by the inconsistency detection unit 320, the level change unit 307 changes the security level related to impersonation detection for the recognized person to be increased. In this embodiment, increasing the security level related to impersonation detection means changing the method used for impersonation detection (impersonation detection method) or making impersonation detection stricter.

[0041] Furthermore, the identity recognition unit 306 changes the security level related to identity recognition according to the security level set in the level change unit 307, which will be described later. As will be described in detail later, if an inconsistency is detected by the inconsistency detection unit 320, the level change unit 307 changes the security level related to identity recognition for the recognized person to be increased. In this embodiment, increasing the security level related to identity recognition means changing the method used for identity recognition (identity recognition method) or making identity recognition stricter.

[0042] When the inconsistency detection unit 320 detects an inconsistency, the level change unit 307 changes at least one of the security levels related to impersonation detection and the security level related to identity recognition to increase it. For example, when changing the security level related to impersonation detection in response to the detection of an inconsistency, the level change unit 307 changes at least one of the methods for detecting impersonation or makes the impersonation detection stricter. Also, for example, when changing the security level related to identity recognition in response to the detection of an inconsistency, the level change unit 307 changes at least one of the methods for recognizing identity or makes the identity recognition stricter. If, for example, identity recognition of the detected person were refused when an inconsistency is detected, the identity recognition result would not be obtainable, thus reducing user convenience. In contrast, in this embodiment, when an inconsistency is detected, the methods and strictness of impersonation detection and identity recognition are changed, and impersonation detection and identity recognition themselves are not refused, so user convenience is maintained.

[0043] More specifically, the level change unit 307 changes the security level using one or more change methods selected from, for example, several different security level change methods, in response to the inconsistency detection unit 320 detecting an inconsistency. For example, the first method for changing the security level is a method for increasing the security level related to the impersonation detection described above. In other words, the first method for changing the security level is increased by tightening the threshold used to determine whether or not a person is a biological being during impersonation detection, or by switching to a more accurate impersonation detection method. An example of a more accurate impersonation detection method would be a method that requires the person being recognized to move their face. By using the first method for changing the security level, the security level can be further enhanced.

[0044] The second method for changing the security level is to enhance the security level related to the facial recognition used for identity verification, as described above. Specifically, the second method for changing the security level involves, for example, tightening the similarity threshold when determining the similarity with registered individuals, or switching to a more accurate facial recognition method. Using the second method for changing the security level can further enhance the security level.

[0045] The third method for changing the security level is another example of a method for increasing the security level related to personal identification. In the third method for changing the security level, in addition to facial recognition, personal identification is made possible by requiring the use of other biometric information obtained from images of other biometric parts other than the face, such as veins, fingerprints, and irises. When using the third method for changing the security level, the security level can be further enhanced by performing additional biometric recognition such as veins, fingerprints, and irises.

[0046] The fourth method for changing the security level is another example of a method for increasing the security level related to personal identification. In the fourth method, the security level related to personal identification is increased by requiring non-biometric recognition in addition to biometric recognition by facial recognition. Examples of non-biometric recognition in this case include personal identification using non-biometric information such as SMS recognition, questions to identify the person, or PIN entry. When the fourth method for changing the security level is used, both biometric and non-biometric recognition are performed, so the security level can be increased.

[0047] The fifth method for changing the security level is another example of a method for increasing the security level related to personal identification. In the fifth method for changing the security level, personal identification is switched to biometric recognition using biometric information other than facial recognition (other biometric recognition other than facial recognition). Examples of other biometric recognition other than facial recognition include biometric recognition using biometric information such as veins, fingerprints, and irises. For example, if an impersonation attack against facial recognition causes the recognized person's characteristics to no longer match the detected person's characteristics and an inconsistency is detected, the system switches from facial recognition to a biometric recognition other than facial recognition. By using the fifth method for changing the security level, the security level can be further increased by switching from facial recognition to a biometric recognition other than facial recognition.

[0048] The sixth method for changing the security level is another example of a method for increasing the security level related to identity verification. In the sixth method, for example, biometric recognition such as facial recognition is disabled and switched to non-biometric recognition. Examples of non-biometric recognition that can be switched to by disabling biometric recognition include password entry, QR code (registered trademark) recognition, IC card recognition, and identity verification by store staff. By using the sixth method for changing the security level, the security level can be increased by switching from biometric recognition to non-biometric recognition.

[0049] The level change unit 307 of this embodiment selects one or more of the aforementioned security level change methods by referring to a lookup table as shown in Figure 4, based on the type of person characteristic used when the inconsistency detection unit 320 detects an inconsistency. As shown in Figure 4, the lookup table is a table that shows the correspondence between each of the aforementioned person characteristics used when the inconsistency detection unit 320 detects an inconsistency and each of the multiple security level change methods.

[0050] For example, if an inconsistency is detected based on body size or gait characteristics, it is highly likely that the inconsistency was detected because a face was impersonated. In this case, the level change unit 307 selects a security level change method that uses a recognition method other than face recognition. That is, if an inconsistency is detected based on body size or gait characteristics, the level change unit 307 selects, for example, a fifth security level change method or a sixth security level change method.

[0051] Furthermore, for example, walking speed, gait characteristics, and time spent in different areas vary from person to person, and it is possible that impersonation is not occurring. Therefore, if an inconsistency is detected based on walking speed, gait characteristics, or time spent in different areas, the level change unit 307 selects a security level change method that tightens the threshold for detecting impersonation and the threshold for recognizing the person. That is, if an inconsistency is detected based on walking speed, gait characteristics, or time spent in different areas, the level change unit 307 selects, for example, the first security level change method or the second security level change method. In addition, the level change unit 307 is not limited to the method of referring to the lookup table in Figure 4, but may also select a security level change method that randomly uses a person recognition method other than face from among the multiple person recognition methods supported by the person recognition unit 306.

[0052] For example, the level change unit 307 may use the aforementioned speed difference, angle difference, and Euclidean distance used by the inconsistency detection unit 320 during inconsistency detection as the degree of inconsistency, and select a security level change method that changes the method and strictness of identity recognition based on that degree of inconsistency. For example, if the degree of inconsistency far exceeds a predetermined inconsistency threshold, the level change unit 307 will not perform identity recognition and will select a security level change method that uses more reliable SMS recognition or non-biometric recognition. For example, if the degree of inconsistency is below the inconsistency threshold but close to it, the level change unit 307 may select a security level change method that tightens the thresholds for impersonation detection and identity recognition, as the possibility of impersonation cannot be ruled out.

[0053] Figure 5 is a flowchart showing the flow of information processing performed by the information processing device 100 according to the first embodiment, namely, the flow from person detection and tracking to inconsistency detection, impersonation detection, and further to person recognition. The flowchart in Figure 5(a) mainly shows the flow related to person detection, and the flowchart in Figure 5(b) shows the flow from tracking performed for each detected person to collection of person characteristics and person recognition. The processing in the flowchart of Figure 5 starts when the store opens for business or in response to a start instruction from the store manager, etc., and ends in response to a stop instruction from the store manager, etc.

[0054] First, as part of step S101 in the flowchart of Figure 5(a), the image acquisition unit 300 of the information processing device 100 acquires images of the store's video footage from one of the cameras 209, which is the surveillance camera 102. Next, in step S102, the person detection unit 301 performs a process to detect a person from the image acquired by the image acquisition unit 300. Next, in step S103, the person detection unit 301 determines whether or not a person was detected in the image in step S102. If a person is detected, the process proceeds to step S104, which is performed by the tracking unit 302. On the other hand, if no person is detected, the information processing device 100 returns to step S101.

[0055] If the process proceeds to step S104, the tracking unit 302 determines whether the person detected in step S102 is a person who has already been tracked. If it is determined that the person has already been tracked, the information processing device 100 returns to step S101. On the other hand, if it is determined that the person has not already been tracked, the tracking unit 302 proceeds to step S105.

[0056] When the process proceeds to step S105, the tracking unit 302 identifies the person detected by the person detection unit 301 in step S102 as a new person to be tracked (referred to as the tracked person), starts tracking that person, and proceeds to step S201 in Figure 5(b). In addition, the information processing device 100, as part of the process in step S106 in Figure 5(a), determines whether there is a termination instruction from, for example, a store manager, and terminates the process in the flowchart of Figure 5 if there is a termination instruction. On the other hand, if there is no termination instruction, the information processing device 100 returns to step S101.

[0057] If the process proceeds to step S201 in Figure 5(b), the tracking unit 302 starts the process of tracking the subject between frames in the video acquired by the image acquisition unit 300. Next, in step S202, the characteristic acquisition unit 304 collects the characteristics of the detected person. That is, the characteristic acquisition unit 304 collects the aforementioned person characteristics from the image of the tracked person, which is the detected person. Note that the tracking method is not limited to matching between frames, as long as it is possible to collect the person characteristics of the tracked person while tracking and to continue tracking until the point in time when face recognition is performed on that person.

[0058] Next, in step S203, the inconsistency detection unit 320 determines whether the tracked person (detected person) is about to start the payment procedure at the register. For example, if the inconsistency detection unit 320 detects from the image of the surveillance camera 102 that the tracked person has come in front of the register, it determines that the person is about to start the payment procedure. Alternatively, for example, if the inconsistency detection unit 320 detects the tracked person from the image of the camera in front of the register 101, it determines that the person is about to start the payment procedure. Alternatively, for example, if the person recognition unit 306 recognizes the tracked person as the person to be recognized and starts the person recognition process to determine whether or not the person is the registered person, it determines that the person is about to start the payment procedure. If the inconsistency detection unit 320 has not determined that the payment procedure has started, the information processing device 100 returns to step S201. On the other hand, if it has determined that the payment procedure has started, the information processing device 100 proceeds to step S204.

[0059] When the process proceeds to step S204, the inconsistency detection unit 320 performs processing to detect inconsistencies. For this reason, in step S204, the attribute estimation unit 303 estimates the attributes of the person being recognized from the face image of the person being recognized captured by the cashier camera 101, and the characteristic acquisition unit 304 acquires the characteristics of the person being recognized based on those attributes. The inconsistency detection unit 320 then compares these characteristics with the characteristics of the person being recognized collected in step S202 and performs processing to determine whether or not there is an inconsistency.

[0060] Next, in step S205, the level change unit 307, if an inconsistency is detected by the inconsistency detection unit 320 in step S204, selects one or more security level change methods as described above in response to the detection of the inconsistency. Then, the level change unit 307 uses the selected one or more security level change methods to change the security level of at least one of the impersonation detection and identity recognition.

[0061] Next, as part of step S206, the impersonation detection unit 305 executes an impersonation detection process corresponding to the security level related to impersonation detection set by the level change unit 307 in step S205. Next, in the process of step S207, the identity recognition unit 306 performs identity recognition processing according to the security level related to identity recognition set by the level change unit 307. Then, the information processing device 100 completes the payment for the goods, etc., if it succeeds in recognizing that the person who initiated the payment procedure (the person being recognized) is the registered person. Note that the information processing device 100 may perform the identity recognition processing in step S207 only if no impersonation is detected in the impersonation detection process in step S206.

[0062] As described above, the information processing device 100 according to this embodiment collects the characteristics of each detected person, and when a detected person becomes a recognized person, it processes a check for inconsistencies between the characteristics of the person obtained based on the person attributes of the recognized person and the collected detected person characteristics. If an inconsistency is detected, the information processing device 100 changes the security level related to impersonation detection and identity recognition for the recognized person. As a result, the information processing device 100 of this embodiment can increase the accuracy of identity recognition of whether or not a person is the registered person, even in cases where existing impersonation detection methods cannot detect impersonation.

[0063] In the embodiment described above, if there are multiple detected individuals, each detected individual will be tracked and their characteristics will be collected. However, tracking by the tracking unit 302 is not essential if, for example, a detected individual can be identified without tracking, or if only the cash register camera 101 is installed in the store. An example of a case where a detected individual can be identified without tracking is when there is only one person in the store. Furthermore, in the embodiment described above, the personal characteristics of individuals detected from images of the surveillance camera 102 were collected. However, if only the cash register camera 101 is installed in the store, the personal characteristics of individuals detected from images of the cash register camera 101 may be collected.

[0064] Furthermore, in the embodiment described above, an example was given in which the impersonation detection unit 305 performs impersonation detection before the identity recognition unit 306 performs identity recognition. However, impersonation detection is not mandatory. In other words, if an inconsistency is detected, only the method of identity recognition or the security level of strictness may be changed. For example, in cases where impersonation detection and identity verification are performed, if an inconsistency is detected, it is possible to change only the security level related to impersonation detection, without changing the security level related to identity verification.

[0065] In the embodiment described above, the attribute estimation unit 303 acquired the characteristics of the person being recognized based on the recognized person attributes estimated from the image of the camera 101 in front of the cash register. However, instead, a configuration may be used that uses pre-prepared person characteristics corresponding to a specific person, such as the registered person themselves. In this case, the inconsistency detection unit 320 acquires the person characteristics of the registered person (i.e., the specific person) that have been registered in advance as second information, and detects inconsistencies by comparing them with the detected person characteristics, which are the first information collected about the person detected in the store. In addition to the walking speed, physique, posture, gait characteristics, and time spent in different areas of the store as described above, the person's habits (e.g., crossing arms, touching chin, pushing hair back, touching ears, etc.), the way the body sways, and the position of the body's center of gravity may also be included. Furthermore, if inconsistency detection is performed using one or more of the following characteristics: habits, body sway, or body center of gravity, the characteristic acquisition unit 304 also acquires one or more of these characteristics from the image of the tracked person.

[0066] Alternatively, the inconsistency detection unit 320 may acquire the skin color of the face and body of the detected person and the person being recognized as personal characteristics, and detect inconsistencies by comparing them. Furthermore, while the above-described embodiment gave an example of using facial recognition to identify a person in front of an unmanned checkout counter in a store, it can also be applied to other scenarios. For example, the information processing device of this embodiment can be applied to identity recognition for transactions at ATMs (automated teller machines), identity recognition for determining whether or not a person can pass through security gates, and identity recognition for determining whether or not a person can enter membership-based facilities.

[0067] Furthermore, while the above-described embodiment uses facial biometric information for identity recognition, it is not limited to facial biometric information, and other biometric information may be used. For example, biometric information such as fingerprints, veins, or irises may be used. When identity recognition is performed using fingerprint biometric information, an example of a forgery attack object could be an artificial fingerprint molded from resin. When identity recognition is performed using vein biometric information, an example of a forgery attack object could be a printed vein pattern. When identity recognition is performed using iris biometric information, an example of a forgery attack object could be a printed iris.

[0068] <Second Embodiment> In the second embodiment, an example of detecting inconsistencies using a combination of person attributes estimated from an image of a predetermined biological part (face) of a person and information obtained from an image of a person other than that predetermined biological part will be described. In the second embodiment, as information obtained from an image of a person other than that predetermined biological part of a person, clothing information, which is an example of what a person wears, will be given. Note that clothing may include not only clothes but also accessories, etc. In the second embodiment, as with the first embodiment described above, an example in which the information processing device 100 is installed in a store equipped with an unmanned cash register as shown in Figure 1 will be used for the explanation. The hardware configuration of the information processing device 100 according to the second embodiment is the same as in Figure 2, and the functional configuration of the information processing device 100 is the same as in Figure 3, so their illustration and explanation will be omitted.

[0069] In the second embodiment, the attribute estimation unit 303 estimates person attributes based on the person's face image detected from the surveillance camera 102. Also in the second embodiment, the characteristic acquisition unit 304 acquires information about the clothing worn by the person detected from the surveillance camera 102. Note that any of the various existing clothing information acquisition processes can be used to acquire information about the clothing worn by the person. Furthermore, the inconsistency detection unit 320 according to the second embodiment detects inconsistencies between the combination of the person attributes estimated from the face initially detected for a person who has entered the store and the person's clothing information, and the combination of the person attributes and clothing information of the person being recognized at the time of recognition.

[0070] Figure 6 is a flowchart showing the flow from the collection of information on the attributes and clothing of a tracked person (detected person) to person recognition in the information processing device 100 of the second embodiment. Note that the flow of the person detection process in the information processing device 100 of the second embodiment is the same as the flowchart in Figure 5(a) described above, so its illustration and explanation are omitted. Also, in the flowchart of Figure 6, the same processing steps as in the flowchart shown in Figure 5(b) after the start of the tracking process are denoted by the same reference numerals as in Figure 5(b), and their explanations are also omitted.

[0071] In the second embodiment, when the tracking process for the tracking person is started in step S105 of Figure 5(a) described above, the processing of the information processing device 100 proceeds to step S301 of Figure 6. Step S301 is reached when the attribute estimation unit 303 estimates the personal attributes of the tracked person based on the face image initially detected from the tracked person, and the characteristic acquisition unit 304 acquires information about the clothing the tracked person is wearing. Hereinafter, the personal attributes estimated based on the face image initially detected from the tracked person will be referred to as tracked person attributes. In the second embodiment, the tracked person attributes may include, for example, race, gender, and age. In this embodiment, the clothing information of the tracked person may include information about the type of clothing the person is wearing (such as a T-shirt or jacket) and its color.

[0072] The next step, S201, is the same process as step S201 in Figure 5(b) described above, and the tracking unit 302 tracks the person to be tracked between frames in the video from the surveillance camera 102 acquired by the image acquisition unit 300. Next, in step S203, the inconsistency detection unit 320 determines whether the person being tracked is about to start a payment procedure at the register. Whether or not the person being tracked is about to start a payment procedure at the register is determined in the same way as described in step S203 of Figure 5(b) above. If the inconsistency detection unit 320 does not determine that a payment procedure has been started, the information processing device 100 returns to step S201. On the other hand, if it determines that a payment procedure has been started, the information processing device 100 proceeds to step S304.

[0073] In step S304, the attribute estimation unit 303 estimates the person's attributes (race, gender, age) from the image of the person's face (a predetermined biological part) captured in the image from the front-of-cash-line camera 101 when it is determined that the payment procedure has started. In the second embodiment, as in the example of the first embodiment, the person attributes estimated from the face image of the person being recognized at the start of the payment procedure are called the person's attributes. Also in step S304, the characteristic acquisition unit 304 acquires information about the clothing the person is wearing at the start of the payment procedure. The clothing information of the person being recognized is also, as described above, information about the type of clothing the person is wearing (such as a T-shirt or jacket) and its color.

[0074] Next, in step S305, the mismatch detection unit 320 compares the combination of tracking person attributes and clothing information acquired in step S301 with the combination of recognized person attributes and clothing information acquired in step S304. For example, the inconsistency detection unit 320 detects an inconsistency if the clothing information of the person being tracked and the clothing information of the person being recognized are the same, but the attributes of the person being tracked and the attributes of the person being recognized are different. Furthermore, for example, if the clothing information of the person being tracked differs from the clothing information of the person being recognized, the tracking unit 302 may have failed the tracking process and tracked a person other than the one it was supposed to track, so the inconsistency detection unit 320 will not detect an inconsistency. In addition, if the clothing information of the person being tracked and the person being recognized differ, besides the possibility that the tracking failed, the person may have taken off the jacket they were wearing when entering the store, or conversely, put on the jacket they had taken off when entering the store while inside the store, so the inconsistency detection unit 320 will not detect an inconsistency. The processes in the next steps, S205 to S207, are the same as those in Figure 5(b) described above, so their explanation will be omitted.

[0075] As explained above, in the information processing device 100 according to the second embodiment, it is not necessary to acquire the characteristics of the detected person or the characteristics of the person being recognized (characteristics of the person being recognized), as in the first embodiment. Therefore, the accuracy of person recognition can be increased more easily than in the first embodiment.

[0076] In the second embodiment described above, an example of tracking a detected person was given, but tracking processing is not required. In this example, the information processing device 100 associates the person's attributes estimated from the facial image of the person detected from the surveillance camera 102 (detected person) at the time of entry with clothing information and stores it as a list of customers. The inconsistency detection unit 320 then determines whether the combination of recognized person characteristics and clothing information acquired using the image from the front-of-cash-line camera 101 at the start of the payment procedure exists in the list of customers, and detects an inconsistency if it does not exist. When a person who has entered the store leaves, the information about that person stored in the list of customers may be deleted or left as is. In addition, the person's attributes and clothing at the time of entry may be acquired using images taken by a camera installed at the entrance of the store to photograph customers, for example, instead of the surveillance camera 102.

[0077] For example, the information stored in the customer list may not be a combination of personal attributes and clothing information, but rather facial information of all people who enter the store, i.e., facial images or facial features. In this example, the information processing device 100 stores the facial information of people who enter the store, acquired using images from the surveillance camera 102, in the customer list. The inconsistency detection unit 320 then determines that there is an inconsistency if the facial information of the person to be recognized, acquired using images from the front-of-cash-line camera 101 at the start of the payment procedure, is not stored in the customer list. In this example as well, when a person who has entered the store leaves, the facial information of that person stored in the customer list may be deleted or left as is. Also, the facial information of people detected at the time of entry may be acquired using images taken by a camera installed at the entrance of the store to photograph customers, for example, instead of the surveillance camera 102.

[0078] The present invention can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. Furthermore, it can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions. The above-described embodiments are merely examples of concrete implementations of the present invention, and the technical scope of the invention should not be limited by them. That is, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0079] Each embodiment of the disclosure includes the following configurations, methods, and programs. (Composition 1) A recognition means that performs recognition processing on the detected person using biometric information obtained from the image of the person detected from the image, An inconsistency detection means for detecting inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, A modification means that changes the security level related to the recognition process in the recognition means in response to the detection of the aforementioned inconsistency, An information processing device characterized by having the following features. (Configuration 2) The second piece of information is information acquired when the recognition process by the recognition means is initiated for the detected person. The information processing device according to configuration 1, characterized in that the first information is information obtained from the image of the person detected before the second information. (Composition 3) The aforementioned inconsistency detection means is As the first piece of information, we obtain the person characteristics acquired from the image of the detected person, The information processing device according to configuration 1 or 2, characterized in that, as the second piece of information, it acquires person characteristics corresponding to person attributes estimated from images of predetermined biological parts of the detected person. (Composition 4) The aforementioned person attributes include one or more of the following: race, gender, age, and physique. The information processing device according to configuration 3, characterized in that the aforementioned person characteristics are one or more of the following: walking speed, physique, posture, gait characteristics, and time spent in different areas of the store. (Composition 5) The information processing device according to configuration 1 or 2, characterized in that the inconsistency detection means detects an inconsistency between the first information and the second information which is provided for a specific person. (Composition 6) The second piece of information provided for the aforementioned specific person is one or more personal characteristics such as walking speed, physique, posture, gait characteristics, time spent in different areas of the store, habits, body sway, and body center of gravity. The information processing device according to configuration 5, characterized in that the inconsistency detection means acquires one or more person characteristics from the detected person's image, such as walking speed, physique, posture, gait characteristics, time spent in different areas of the store, habits, body swaying, and body center of gravity, as the first information, and detects inconsistencies with the person characteristics of the second information. (Composition 7) The information processing apparatus according to any one of configurations 1 to 6, characterized in that the modification means modifies at least one of the recognition method used in the recognition process and the strictness of the recognition process as a change in the security level related to the recognition process. (Composition 8) The aforementioned recognition means is The aforementioned impersonation detection means determines whether the detected person is a living or non-living being, and if it is a non-living being, it detects that impersonation has occurred. A personal identification means that performs personal identification to determine whether the detected person is a registered person or not based on the biometric information obtained from the image of the detected person, It has, The information processing apparatus according to any one of configurations 1 to 6, characterized in that the modification means modifies at least one of the security level related to the recognition process, namely the security level related to the impersonation detection in the impersonation detection means and the security level related to the identity recognition in the identity recognition means. (Composition 9) The information processing device according to configuration 8, characterized in that the identity recognition means performs identity recognition only when impersonation is not detected by the impersonation detection means. (Composition 10) The information processing apparatus according to configuration 8 or 9, characterized in that the modification means changes the security level related to the recognition process using one or more security level modification methods from a plurality of different security level modification methods. (Composition 11) The aforementioned multiple different methods for changing security levels include: A first modification method that changes at least one of the methods for detecting impersonation or the strictness of the impersonation detection, A second method of modification involves changing at least one of the methods of identity recognition or the strictness of identity recognition, A third modification method that, in addition to recognizing a person using biometric information obtained from images of a predetermined biological part of a person, also performs recognition using other biometric information obtained from images of other biological parts other than the predetermined biological part, In addition to identity recognition using biometric information obtained from images of specific biological parts of a person, a fourth modification method is provided to perform identity recognition using non-biometric recognition. A fifth modification method for switching from self-recognition using biometric information obtained from images of predetermined biological parts of a person to self-recognition using other biometric information obtained from images of other biological parts other than the predetermined biological parts, A sixth modification method that disables self-recognition using biometric information obtained from images of a specific biological part of a person and switches to self-recognition using non-biometric recognition, The information processing device according to configuration 10, characterized in that it includes one or more of the above. (Composition 12) The inconsistency detection means also acquires the degree of inconsistency between the first information and the second information, The information processing apparatus according to any one of configurations 1 to 11, characterized in that the modification means changes the security level based on the degree of inconsistency. (Composition 13) The aforementioned inconsistency detection means is As the first piece of information, a combination of person attributes estimated from images of a predetermined biological part of the detected person and information obtained from images of the person other than the predetermined biological part is acquired. As the second piece of information, a combination of person attributes estimated from images of a predetermined biological part of the detected person and information obtained from images of the person other than the predetermined biological part is acquired. The information processing device according to Configuration 1, characterized in that even if the information obtained from images other than the predetermined biological part of the first information and the second information is the same, if the person attributes of the first information and the second information are different, it is detected as an inconsistency. (Composition 14) The information processing device according to configuration 13, characterized in that the inconsistency detection means does not detect an inconsistency if the information obtained from images other than the predetermined biological part of the first information and the second information are different. (Composition 15) The aforementioned inconsistency detection means is For each detected person, the combination of person attributes estimated from images of predetermined biological parts and information obtained from images of the person other than the predetermined biological parts is stored as the first information list. The information processing device according to Configuration 1, characterized in that if the second information, which is a combination of the person attributes estimated from the image of the predetermined biological part of the detected person and the information obtained from the image of the person other than the predetermined biological part, does not exist in the list, it is detected as an inconsistency. (Composition 16) The information processing device according to any one of configurations 13 to 15, characterized in that the information obtained from images other than the predetermined biological parts is information about the clothing worn by the detected person. (Composition 17) The aforementioned inconsistency detection means is The image of a predetermined biological part of the detected person or the characteristic quantity of the predetermined biological part is obtained as a list of the first information. The image of the predetermined biological part of the detected person or the characteristic quantity of the predetermined biological part is acquired as second information. The information processing device according to configuration 1, characterized in that it detects a mismatch if an image of the predetermined biological part of the second information is not present in the list of images of the predetermined biological part, or if a feature quantity of the predetermined biological part of the second information is not present in the list of feature quantities of the predetermined biological part. (Composition 18) The information processing apparatus according to any one of configurations 1 to 17, characterized in that the biological information used in the recognition process is information about at least one of a face, iris, fingerprint, or vein. (Method 1) A recognition step which performs recognition processing on the detected person using biometric information obtained from the image of the person detected from the image, An inconsistency detection step for detecting inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, A modification step that changes the security level related to the recognition process in the recognition step in response to the detection of the aforementioned inconsistency, An information processing method characterized by having the following features. (program) A program that causes a computer to function as an information processing device described in any one of configurations 1 through 18. [Explanation of symbols]

[0080] 100: Information processing device, 300: Image acquisition unit, 301: Person detection unit, 302: Tracking unit, 303: Attribute estimation unit, 304: Characteristic acquisition unit, 305: Impersonation detection unit, 306: Person recognition unit, 307: Level change unit, 310: Biometric recognition unit, 320: Inconsistency detection unit

Claims

1. A recognition means that performs recognition processing on the detected person using biometric information obtained from the image of the person detected from the image, An inconsistency detection means for detecting inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, A modification means that changes the security level related to the recognition process in the recognition means in response to the detection of the aforementioned inconsistency, An information processing device characterized by having the following features.

2. The second piece of information is information acquired when the recognition process by the recognition means is initiated for the detected person. The information processing apparatus according to claim 1, characterized in that the first information is information obtained from the image of the person detected before the second information.

3. The aforementioned inconsistency detection means is As the first piece of information, we obtain the person characteristics acquired from the image of the detected person, The information processing device according to claim 1, characterized in that, as the second piece of information, it acquires person characteristics corresponding to person attributes estimated from images of predetermined biological parts of the detected person.

4. The aforementioned person attributes include one or more of the following: race, gender, age, and physique. The information processing device according to claim 3, characterized in that the aforementioned person characteristics are one or more of the following: walking speed, physique, posture, gait characteristics, and time spent in different areas of the store.

5. The information processing device according to claim 1, characterized in that the inconsistency detection means detects an inconsistency between the first information and the second information which is provided for a specific person.

6. The second piece of information provided for the aforementioned specific person is one or more personal characteristics such as walking speed, physique, posture, gait characteristics, time spent in different areas of the store, habits, body sway, and body center of gravity. The information processing apparatus according to claim 5, characterized in that the inconsistency detection means acquires one or more person characteristics such as walking speed, physique, posture, gait characteristics, time spent in different areas of the store, habits, body swaying, and body center of gravity from the image of the person detected as first information, and detects inconsistencies with the person characteristics of the second information.

7. The information processing apparatus according to claim 1, characterized in that the modification means modifies at least one of the recognition method used in the recognition process and the strictness of the recognition process as a change in the security level related to the recognition process.

8. The aforementioned recognition means is The aforementioned impersonation detection means determines whether the detected person is a living or non-living being, and if it is a non-living being, it detects that impersonation has occurred. A personal identification means that performs personal identification to determine whether the detected person is a registered person or not based on the biometric information obtained from the image of the detected person, It has, The information processing apparatus according to claim 1, characterized in that the modification means modifies at least one of the security level related to the recognition process, namely the security level related to the impersonation detection in the impersonation detection means and the security level related to the identity recognition in the identity recognition means.

9. The information processing device according to claim 8, characterized in that the identity recognition means performs identity recognition only when impersonation is not detected by the impersonation detection means.

10. The information processing apparatus according to claim 8, characterized in that the modification means modifies the security level related to the recognition process using one or more security level modification methods from a plurality of different security level modification methods.

11. The aforementioned multiple different methods for changing security levels include: A first modification method that changes at least one of the methods for detecting impersonation or the strictness of the impersonation detection, A second method of modification involves changing at least one of the methods of identity recognition or the strictness of identity recognition, A third modification method that, in addition to performing identity recognition using biometric information obtained from images of predetermined biological parts of a person, also performs identity recognition using other biometric information obtained from images of other biological parts other than the predetermined biological parts, In addition to identity recognition using biometric information obtained from images of specific biological parts of a person, a fourth modification method is provided to perform identity recognition using non-biometric recognition. A fifth modification method for switching from self-recognition using biometric information obtained from images of predetermined biological parts of a person to self-recognition using other biometric information obtained from images of other biological parts other than the predetermined biological parts, A sixth modification method that disables self-recognition using biometric information obtained from images of a predetermined biological part of a person and switches to self-recognition using non-biometric recognition, The information processing apparatus according to claim 10, characterized in that it includes one or more of the above.

12. The inconsistency detection means also acquires the degree of inconsistency between the first information and the second information, The information processing apparatus according to claim 1, characterized in that the modification means changes the security level based on the degree of inconsistency.

13. The aforementioned inconsistency detection means is As the first piece of information, a combination of person attributes estimated from images of a predetermined biological part of the detected person and information obtained from images of the person other than the predetermined biological part is acquired. As the second piece of information, a combination of person attributes estimated from images of a predetermined biological part of the detected person and information obtained from images of the person other than the predetermined biological part is acquired. The information processing device according to claim 1, characterized in that even if the information obtained from images other than the predetermined biological part of the first information and the second information is the same, if the person attributes of the first information and the second information are different, it is detected as an inconsistency.

14. The information processing apparatus according to claim 13, characterized in that the inconsistency detection means does not detect an inconsistency if the information obtained from the image of a part other than the predetermined biological part of the first information and the second information are different.

15. The aforementioned inconsistency detection means is For each detected person, the combination of person attributes estimated from images of predetermined biological parts and information obtained from images of the person other than the predetermined biological parts is stored as the first information list. The information processing device according to claim 1, characterized in that if the second information, which is a combination of the person attributes estimated from the image of the predetermined biological part of the detected person and the information obtained from the image of a part of the person other than the predetermined biological part, does not exist in the list, it is detected as an inconsistency.

16. The information processing apparatus according to any one of claims 13 to 15, characterized in that the information obtained from images other than the predetermined biological parts is information about the clothing worn by the detected person.

17. The aforementioned inconsistency detection means is The image of a predetermined biological part of the detected person or the characteristic quantities of the predetermined biological part are obtained as a list of the first information. The image of the predetermined biological part of the detected person or the characteristic quantity of the predetermined biological part is acquired as second information. The information processing device according to claim 1, characterized in that if an image of the predetermined biological part of the second information does not exist in the list of images of the predetermined biological part, or if a feature quantity of the predetermined biological part of the second information does not exist in the list of feature quantities of the predetermined biological part, it is detected as an inconsistency.

18. The information processing apparatus according to claim 1, characterized in that the biological information used in the recognition process is information of at least one of a face, iris, fingerprint, or vein.

19. A recognition step which performs recognition processing on the detected person using biometric information obtained from the image of the person detected from the image, An inconsistency detection step for detecting inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, A modification step that changes the security level related to the recognition process in the recognition step in response to the detection of the aforementioned inconsistency, An information processing method characterized by having the following features.

20. Computers, A recognition means that performs recognition processing on the detected person using biometric information obtained from the image of the person detected from the image, An inconsistency detection means for detecting inconsistencies between first information obtained from the image of the detected person and second information obtained as information corresponding to the first information, A modification means that changes the security level related to the recognition process in the recognition means in response to the detection of the aforementioned inconsistency, A program that makes an information processing device function as having a certain feature.

Images