Determination method, determination program, and information processing device

Abstract

In this determination method, a computer performs processing wherein first-type feature information for a first person is extracted using a first image imaged by a first camera, if second-type feature information has been extracted from the first person before or after the imaging of the first image, a person corresponding to the first person is identified using respective second-type feature information for a plurality of persons that is stored in a storage unit, and if first-type feature information for a second person has been extracted using a second image imaged by the first camera or a different, second camera, a person corresponding to the second person is identified using the first-type feature information for the identified first person.　

Description

Determination method, determination program, and information processing device

[0001] The present invention relates to a determination method, a determination program, and an information processing device.

[0002] A technology has been disclosed in which facial features and first whole-body features are extracted from a first image, the facial features are compared with a list, second whole-body features are extracted from a second image, and the first whole-body features are compared with the second whole-body features to identify a tracking target (see, for example, Patent Document 1).

[0003] International Publication No. 2020 / 115910

[0004] However, there are cases where it is not possible to extract features with high accuracy, which reduces the accuracy of person identification.

[0005] In one aspect, the present invention aims to provide a determination method, a determination program, and an information processing device that can improve the accuracy of person determination.

[0006] In one aspect, the determination method includes a computer executing a process in which it extracts first type characteristic information of a first person using a first image captured by a first camera, and if it extracts second type characteristic information from the first person before or after the first image is captured, it identifies a person corresponding to the first person using the second type characteristic information of each of a plurality of people stored in a memory unit, and if it extracts first type characteristic information of a second person using a second image captured by the first camera or another second camera, it identifies a person corresponding to the second person using the first type characteristic information of the identified first person.

[0007] This can improve the accuracy of person determination.

[0008] 1 is a diagram illustrating an overview of a first embodiment; FIG. 1A is a block diagram illustrating an overall configuration of a biometric authentication system 0 according to a first embodiment, and FIG. 1B is a functional block diagram illustrating each function of a server; FIG. 1B is a flowchart illustrating a tracking process; FIG. 1A is a diagram illustrating a table stored in a camera information storage unit, and FIG. 1B is a diagram illustrating a table; FIG. 1B is a diagram illustrating a table stored in an overall information storage unit; FIG. 1C is a flowchart illustrating details of step S5; FIG. 1D is a diagram illustrating table updating in the tracking authentication process; FIG. 1E is a diagram illustrating a temporary storage table stored in a temporary storage unit; FIG. 1F is a flowchart illustrating an authentication process; FIG. 1F is a flowchart illustrating a linking process; FIG. 1G is a diagram illustrating an update of a table stored in an overall information storage unit; FIG. 1G is a diagram illustrating postures when a person approaches an authentication device and performs an authentication operation; FIG. 1F is a flowchart illustrating another example of the tracking authentication process; FIG. 1F is a flowchart illustrating another linking process; and FIG. 1C is a block diagram for explaining the hardware configuration of a server.

[0009] Biometric authentication is a technology that verifies identity using biometric features such as fingerprints, faces, and veins. In biometric authentication, when identity verification is required, biometric feature data for verification acquired by a biometric sensor is compared (matched) with pre-registered biometric feature data, and identity verification is performed by determining whether the similarity is equal to or exceeds an identity verification threshold. Biometric authentication is used in various fields such as bank ATMs and access control, and has recently begun to be used in cashless payments at supermarkets, convenience stores, and other locations.

[0010] These biometric authentication methods are "point" authentication performed at specific authentication spots, such as in front of an authentication machine. However, with "point" authentication, the authentication state is interrupted when the user leaves the authentication spot, and if the user wishes to receive a service again or at a location where authentication is required multiple times, authentication must be performed each time. Therefore, there is a demand for continuous authentication technology that eliminates the need for repeated authentication and allows the user to continue receiving services with a single authentication.

[0011] Here, we will provide an overview of continuous authentication technology. Continuous authentication mainly involves the following three types of authentication:

[0012] The first is authentication at check-in. At the gate, the user undergoes highly accurate authentication using techniques such as palm vein authentication, and when authentication is successful, a camera captures a photo of the user's appearance, which is then linked to the user's ID and the characteristic information acquired by the camera.

[0013] Next, there is linear authentication, which uses feature information at the time of successful authentication and feature information continuously obtained over time using one or more cameras to maintain an authentication state in which the person photographed by the camera is authenticated as a user who has been successfully authenticated.

[0014] Next, re-authentication is performed when the user enters a blind spot of the camera due to, for example, another person or a pillar, and the authentication state is interrupted.

[0015] Continuous authentication can be achieved by performing such authentication. However, when checking in, the user must perform authentication actions for check-in. In this case, the user assumes a posture for performing the authentication action, which differs from normal postures. For example, in palm vein authentication, the user may be in a posture with their hands raised or bent over to hold their hand over the authentication device, which differs from normal postures such as when walking. Even when entering a password, the user assumes a posture to enter the password into an input device, which differs from normal postures. Even in facial authentication, the user's face must be brought close to the camera, so depending on the user's physique, the user may be stretched or bent over, which differs from normal postures. If the user's appearance in a posture that differs from normal postures as described above is stored as feature information, the accuracy of subsequent person identification along the line may be reduced.

[0016] Therefore, in the following embodiments, a determination method, a determination program, and an information processing device that can improve the accuracy of person determination will be described.

[0017] First, the first type of feature information and the second type of feature information will be described. The first type of feature information is feature information used to maintain the authentication state. For example, appearance features such as the color of clothing, physique, facial features, and behavioral features can be used as the first type of feature information. The second type of feature information is a feature for identifying the individual through highly accurate authentication. For example, biometric features such as palm veins, facial features, irises, fingerprints, palm prints, and finger veins can be used as the second type of feature information. Alternatively, attribute information such as an ID card or password can be used as the second type of feature information.

[0018] Next, an overview of the first embodiment will be described. Fig. 1 is a diagram illustrating an overview of the first embodiment. As illustrated in Fig. 1, the linking camera 110a captures an image of the vicinity of the entrance including the authentication device 120. When a user enters the room through the entrance, the user is detected from the image captured by the linking camera 110a, first type characteristic information is extracted, and a temporary ID is issued and linked to the user.

[0019] Next, the user inputs the second characteristic information into the authentication device 120. If authentication by the authentication device 120 is successful, attribute information such as the user's ID is linked to the first type characteristic information acquired by the linking camera 110a.

[0020] Thereafter, as the user moves, the environmental camera 110b capturing the environment acquires first type characteristic information of the user. If the similarity between the first type characteristic information acquired by the linking camera 110a and the first type characteristic information acquired by the environmental camera 110b is equal to or greater than a threshold, the user is identified as the user. Thereafter, as the user moves, authentication is performed using the first type characteristic information using other environmental cameras 110b.

[0021] With this method, the second type of feature information is extracted from the user after the user is photographed by the linking camera 110a, so that when the user is photographed by the linking camera 110a, the user is in a normal posture, such as when moving. This allows for highly accurate authentication using the first type of feature information after the user's ID has been identified. Note that, although the first type of feature information is obtained before the authentication operation is performed by the authentication device in the example of FIG. 1, the first type of feature information may also be obtained after the authentication operation is performed by the authentication device.

[0022] Fig. 2(a) is a block diagram illustrating an example of the overall configuration of a biometric authentication system 200 according to Example 1. As illustrated in Fig. 2(a), the biometric authentication system 200 includes a server 100 that functions as an information processing device, one or more cameras 110, and an authentication device 120. The camera 110 is a device for acquiring first type characteristic information. The authentication device 120 is a device for performing authentication using second type characteristic information.

[0023] Fig. 2(b) is a functional block diagram showing each function of the server 100. As illustrated in Fig. 2(b), the server 100 functions as an acquisition unit 11, a person detection unit 12, an extraction unit 13, a camera information storage unit 14, a first identification unit 15, an overall information storage unit 16, a temporary storage unit 17, an authentication processing unit 18, a registration unit 19, a second identification unit 20, etc.

[0024] 3 is a flowchart showing the tracking process. The tracking process in FIG. 3 is executed at a predetermined cycle. The tracking process in FIG. 3 is executed independently and in parallel for each camera 110.

[0025] As illustrated in FIG. 3, the acquisition unit 11 acquires an image (first image) from the camera 110 (step S1).

[0026] Next, the person detection unit 12 determines whether or not a person has been detected from each image acquired by the acquisition unit 11 (step S2). If the determination in step S2 is "No," the process is executed again from step S1 after a predetermined time.

[0027] If step S2 returns "Yes," the extraction unit 13 extracts the first type characteristic information and location information of the person detected in step S2 from the image acquired in step S1, and generates and associates a temporary ID (step S3). The location information may be location information of a specific part of the person in the image. For example, the position range of the person's two pairs of shoes may be used as the detected location information. Because the installation position and angle of view of the camera 110 are predetermined, the location information can be acquired from the image.

[0028] Next, the camera information storage unit 14 stores the location information of the camera 110 when the person was detected in step S2, the first type of characteristic information extracted in step S3, and the temporary ID generated in step S3 (step S4).

[0029] Thereafter, the first identification unit 15 performs a tracking authentication process (step S5).

[0030] 4A is a diagram illustrating an example of a table stored in the camera information storage unit 14. As illustrated in FIG. 4A, the camera information storage unit 14 stores a temporary ID, location information, and first-type characteristic information associated with each camera. In FIG. 4A, the temporary ID, location information, and first-type characteristic information are stored associated with each other for the cameras with camera IDs 0001 and 0002. For example, for camera ID 0001, three people with temporary IDs aaaa, bbbb, and cccc were detected at a given time.

[0031] Alternatively, the camera information storage unit 14 may store the temporary ID, the location information, and the first type of characteristic information in a single table in association with each other. Fig. 4(b) is a diagram illustrating an example of a table in this case. As illustrated in Fig. 4(b), a camera ID may be added, and the temporary ID, the location information, and the first type of characteristic information may be stored in association with each other.

[0032] 5 is a diagram illustrating a table stored in the overall information storage unit 16. The overall information storage unit 16 stores the temporary IDs being tracked and their first type characteristic information. In the table stored in the overall information storage unit 16, the camera IDs are not linked. As illustrated in FIG. 5, the overall information storage unit 16 stores the temporary IDs linked to the first type characteristic information. For example, at a given time, three people with temporary IDs=aaaa, temporary ID=bbbb, and temporary ID=cccc are being tracked.

[0033] 6 is a flowchart illustrating the details of step S5. The process in FIG. 6 is executed sequentially for each person detected in step S2. First, the first identification unit 15 acquires first characteristic information of the person detected in step S2 from the camera information storage unit 14 (step S11).

[0034] Next, the first identifying unit 15 acquires, from the overall information storage unit 16, first type characteristic information for all temporary IDs stored in the overall information storage unit 16 (step S12).

[0035] Next, the first identification unit 15 calculates the similarity between the first characteristic information acquired in step S11 and the first characteristic information acquired in step S12 (step S13). By executing step S13, it is possible to calculate an index for determining whether the person detected in step S2 is similar to the person with the temporary ID stored in the overall information storage unit 16.

[0036] Next, the first identification unit 15 determines whether each similarity calculated in step S13 exceeds a threshold value (step S14). By executing step S14, it is possible to determine whether the person detected in step S2 is the same person as the person with the temporary ID stored in the overall information storage unit 16.

[0037] If the determination in step S14 is "Yes," the first identification unit 15 updates the stored contents of the camera information storage unit 14 (step S15). Specifically, the temporary ID stored in the camera information storage unit 14 is updated to the temporary ID of the target whose similarity exceeds the threshold in step S14. By executing step S15, the authentication state of the person detected by the camera 110 can be continued.

[0038] If the determination in step S14 is "No," the first identification unit 15 stores information about the temporary IDs whose similarity does not exceed the threshold in the overall information storage unit 16 (step S16). By executing step S16, the newly detected person can be stored in the overall information storage unit 16.

[0039] After step S15 or step S16, the temporary storage unit 17 stores the first characteristic information of the target temporary ID in the temporary storage table (step S17). Then, the process is executed again from step S11. In this case, the next person among the people detected in step S2 is targeted.

[0040] In step S12, the temporary IDs stored in the general information storage unit 16 may be acquired excluding those that have been updated in step S15. In this way, unnecessary authentication processing can be omitted.

[0041] FIG. 7 is a diagram illustrating table updates in the tracking and authentication process described above. The upper left diagram in FIG. 7 is a table for camera ID=0002 stored in the camera information storage unit 14 at a predetermined time. The middle diagram in FIG. 7 is a table stored in the overall information storage unit 16 at that predetermined time. For example, among the temporary IDs stored in the table for camera ID=0002, the similarity between the first type feature information for temporary ID=dddd and each type of first feature information in the table stored in the overall information storage unit 16 is calculated. In this example, the similarity between the first type feature information for temporary ID=dddd and the first type feature information for temporary ID=aaaa is greater than or equal to a threshold. Therefore, as illustrated in the upper right diagram in FIG. 7, the temporary ID=dddd in the table for camera ID=0002 is updated to the temporary ID=aaaa. Furthermore, since there is no first type feature information whose similarity with the first type feature information of temporary ID = eeee is greater than or equal to the threshold value, the information of temporary ID = eeee in the table of camera ID = 0002 is not updated, and the information of temporary ID = eeee is added to the table of the overall information storage unit 16.

[0042] 7, the person with temporary ID=aaaa continues to be authenticated, while the person with temporary ID=eeee is newly added to the tracking targets.

[0043] FIG. 8 is a diagram illustrating a temporary storage table stored in the temporary storage unit 17. The example in FIG. 8 illustrates first-type feature information for temporary ID=aaaa that is temporarily stored. As illustrated in FIG. 8, first-type feature information and location information are stored in chronological order at a predetermined time interval. Note that the first-type feature information at each time point is feature information extracted at each time point, and therefore may change slightly. For example, since the first-type feature information is acquired while the person with temporary ID=aaaa is moving, the first-type feature information may fluctuate depending on the person's posture during movement. Note that the temporary storage table stored in the temporary storage unit 17 is intended to store only temporary data, and therefore past data is deleted after a predetermined time has passed.

[0044] Next, the authentication process and the linking process for identifying the person of each temporary ID using the second type characteristic information will be described. Fig. 9 is a flowchart illustrating the authentication process. As illustrated in Fig. 9, first, the authentication processing unit 18 acquires an authentication request (step S21). For example, when a user performs an action such as holding their hand over the authentication device 120, turning on an authentication request button displayed on the authentication device 120, or entering a password, the authentication request is transmitted from the authentication device 120 to the authentication processing unit 18.

[0045] Next, the authentication processing unit 18 performs authentication processing (step S22). Specifically, the authentication processing unit 18 calculates the similarity between the registration data stored in the registration unit 19 and the matching data acquired by the authentication device 120. Note that in step S22, registration data that has already been successfully authenticated may be excluded from the similarity calculation. In this way, unnecessary authentication processing can be omitted.

[0046] Next, the authentication processing unit 18 determines whether the authentication is successful (step S23). Specifically, it determines whether the similarity calculated in step S22 exceeds a threshold. If the similarity exceeds the threshold, it is determined that the authentication is successful.

[0047] If the determination in step S23 is "No", the process is executed again from step S21. If the determination in step S23 is "Yes", the second identification unit 20 performs the linking process (step S24).

[0048] 10 is a flowchart illustrating the linking process. As illustrated in FIG. 10, the second identification unit 20 acquires the authentication result from the authentication processing unit 18 (step S31).

[0049] Next, the second identification unit 20 acquires the temporary ID of the target camera 110 from the camera information storage unit 14 (step S32).

[0050] Next, the second identification unit 20 obtains from the temporary memory unit 17 the first type of characteristic information extracted from the second image of the target temporary ID taken a predetermined specified time before (a few seconds before) (step S33).

[0051] Next, the second identification unit 20 stores the first characteristic information and the ID acquired in step S33 in a table stored in the overall information storage unit 16 (step S34). After that, the process is executed again from step S31.

[0052] 11 is a diagram illustrating an example of updating the table stored in the overall information storage unit 16. For example, if the temporary ID aaaa is identified as the user name Alice, the temporary ID aaaa is associated with the user name Alice as an ID. Furthermore, the first characteristic information is updated to the first characteristic information acquired in step S33.

[0053] According to this embodiment, first type feature information acquired a predetermined time before the time when authentication using the authentication device 120 was successful is used as the first type feature information of the person who was successfully authenticated using the authentication device 120. As a result, first type feature information in a posture associated with normal movement or movement close to normal movement is used instead of first type feature information in a special posture for the authentication operation of the authentication device 120. As a result, first type feature information can be acquired with high accuracy, and the accuracy of person determination can be improved.

[0054] 12(a) to 12(e) are diagrams illustrating the postures of a person approaching the authentication device 120 to perform authentication. In FIG. 12(a), the person is farther away than the authentication device 120. Therefore, the person has not yet assumed a posture for performing authentication, but is in a normal walking posture. Next, in FIG. 12(b), the person is in a posture for performing authentication because authentication by the authentication device 120 has started. Next, in FIG. 12(c), the person is in a posture for performing authentication because authentication by the authentication device 120 is in progress. Next, in FIG. 12(d), authentication by the authentication device 120 has ended, but the person is still in a posture for performing authentication. Next, in FIG. 12(e), the person is farther away from the authentication device 120, so is in a normal walking posture.

[0055] Considering these movements, for a person with average movement speed, it takes about 4 seconds from the start to the end of authentication. Therefore, it is preferable to use, for example, first type characteristic information acquired 5 to 6 seconds before the time when authentication using the authentication device 120 was successful as the first type characteristic information of the person who was successfully authenticated using the authentication device 120.

[0056] 12(a) to 12(e) have been described for the authentication operation for a person with average movement speed, but movement speed varies depending on the person. For example, some people move quickly to the authentication device 120, while others move slowly. Therefore, the time range of data to be temporarily stored in the temporary storage unit 17 may be determined depending on the distance between the authentication device 120 and the person.

[0057] 13 is a flowchart showing another example of the tracking authentication process. The process in FIG. 13 is executed sequentially for each person detected in step S2. First, the first identification unit 15 acquires first characteristic information of the person detected in step S2 from the camera information storage unit 14 (step S41).

[0058] Next, the first identification unit 15 acquires, from the overall information storage unit 16, first type characteristic information for all temporary IDs stored in the overall information storage unit 16 (step S42).

[0059] Next, the first identification unit 15 calculates the similarity between the first characteristic information acquired in step S41 and the first characteristic information acquired in step S42 (step S43). By executing step S43, it is possible to calculate an index for determining whether the person detected in step S2 is similar to the person with the temporary ID stored in the overall information storage unit 16.

[0060] Next, the first identification unit 15 determines whether each similarity calculated in step S43 exceeds a threshold value (step S44). By executing step S44, it is possible to determine whether the person detected in step S2 is the same person as the person with the temporary ID stored in the overall information storage unit 16.

[0061] If the determination in step S43 is "Yes," the first identification unit 15 updates the stored contents of the camera information storage unit 14 (step S45). Specifically, the first identification unit 15 updates the temporary ID stored in the camera information storage unit 14 to the temporary ID of the target whose similarity exceeds the threshold in step S43. By executing step S45, the authentication state of the person detected by the camera 110 can be continued.

[0062] If the determination in step S44 is "No," the first identification unit 15 stores information about the temporary IDs whose similarity does not exceed the threshold in the overall information storage unit 16 (step S46). By executing step S46, the newly detected person can be stored in the overall information storage unit 16.

[0063] After executing step S45 or step S46, the first identification unit 15 determines whether the detected position information is within a specific range (step S47). By executing step S47, it can be determined whether the person is performing an authentication operation or has not yet performed the authentication operation. If the person has not yet performed the authentication operation, a specific range within which the detected position information falls is determined in advance. If the detected position information falls outside this specific range, it means that the person has approached the authentication device 120, and it can be determined that the person has started the authentication operation.

[0064] If the determination in step S47 is "Yes," it can be determined that the person has not yet undergone authentication. As a result, the temporary storage unit 17 stores the feature amount of the target temporary ID in the temporary storage table (step S47). Then, the process is executed again from step S11. In this case, the next person among the people detected in step S2 is the target.

[0065] If the determination in step S47 is "No", step S47 is not executed and the process is repeated from step S42.

[0066] 12(a) to 12(e) have been described regarding the authentication operation for a person with average movement speed. A person with average movement speed will assume a normal walking posture one to two seconds after the time of successful authentication. Therefore, a case will be described in which first type characteristic information obtained a predetermined time after the time of successful authentication is used.

[0067] 14 is a flowchart illustrating another linking process. As illustrated in FIG. 14, the second identification unit 20 acquires the authentication result by the authentication processing unit 18 (step S51).

[0068] Next, the second identification unit 20 acquires the temporary ID of the target camera 110 from the camera information storage unit 14 (step S52).

[0069] Next, the second identification unit 20 determines whether a predetermined time (several seconds, for example, 1 to 2 seconds) has elapsed since the execution of step S52 (step S53). If the determination in step S53 is "No," step S53 is executed again.

[0070] If the answer in step S53 is "Yes," the first type of feature information extracted from the current image (second image) of the target temporary ID is obtained from the camera information storage unit 14 (step S54).

[0071] Next, the second identification unit 20 stores the first characteristic information and the ID acquired in step S54 in a table stored in the overall information storage unit 16 (step S55). After that, the process is executed again from step S51.

[0072] According to this embodiment, first type feature information acquired a predetermined time after the time when authentication using the authentication device 120 was successful is used as the first type feature information of the person who was successfully authenticated using the authentication device 120. As a result, first type feature information acquired in a posture associated with normal or near-normal movement is used instead of first type feature information acquired in a special posture for the authentication operation of the authentication device 120. As a result, first type feature information can be acquired with high accuracy.

[0073] FIG. 15 is a block diagram illustrating the hardware configuration of the server 100. As illustrated in FIG. 15, the server 100 includes a CPU 101, a RAM 102, a storage device 103, a communication device 104, and the like. These devices are connected via a bus or the like. The CPU (Central Processing Unit) 101 is a central processing unit. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. For example, the storage device 103 may be a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, or a hard disk driven by a hard disk drive. The functions of each part of the server 100 are realized by the CPU 101 executing an authentication program stored in the storage device 103. The functions of each unit of the server 100 may be configured by a dedicated circuit, etc. The communication device 104 is an interface to the electric communication line 300 .

[0074] In each of the above examples, the extraction unit 13 is an example of an extraction unit that extracts first type characteristic information of a first person using a first image captured by a first camera. When the second identification unit 20 extracts second type characteristic information from the first person before or after the first image is captured, this is an example of a second identification unit that identifies a person corresponding to the first person using the second type characteristic information of each of multiple people stored in a storage unit. When the first identification unit 15 extracts first type characteristic information of a second person using a second image captured by the first camera or another second camera, this is an example of a first identification unit that identifies a person corresponding to the second person using the first type characteristic information of the identified first person.

[0075] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention as described in the claims.

[0076] REFERENCE SIGNS LIST 11 Acquisition unit 12 Person detection unit 13 Extraction unit 14 Camera information storage unit 15 First identification unit 16 Overall information storage unit 17 Temporary storage unit 18 Authentication processing unit 19 Registration unit 20 Second identification unit 100 Server 110 Camera 120 Authentication device 200 Biometric authentication system

Claims

1. The computer extracting first characteristic information of a first person using a first image captured by a first camera; When second type characteristic information is extracted from the first person before or after the first image is captured, a person corresponding to the first person is identified using the second type characteristic information of each of a plurality of persons stored in a storage unit; When first type characteristic information of a second person is extracted using a second image captured by the first camera or another second camera, a person corresponding to the second person is identified using the first type characteristic information of the identified first person. A determination method comprising:

2. 2. The method according to claim 1, wherein the time before or after the first image is captured is a time before or a time after a specified time from the time the first image is captured.

3. The determination method according to claim 1, characterized in that before or after the first image is captured means when the first person is detected to be present within a specific position range of the image angle of the first camera.

4. The determination method described in claim 1, characterized in that when identifying a person corresponding to the first person using the second type of characteristic information of each of a plurality of people stored in the memory unit, people from the plurality of people that have already been identified using the second type of characteristic information are excluded.

5. The determination method described in claim 1, characterized in that when identifying a person corresponding to the second person using the first type of characteristic information of the identified first person, determination is made from among the multiple first persons, the other first persons excluding the first person identified as the person corresponding to the second person.

6. The determination method described in claim 1, characterized in that when a person corresponding to the second person is identified, the first type of characteristic information extracted using the second image is stored in the memory unit as identification information of the second person.

7. On the computer, extracting first characteristic information of a first person using a first image captured by a first camera; When second type characteristic information is extracted from the first person before or after the first image is captured, a person corresponding to the first person is identified using the second type characteristic information of each of a plurality of persons stored in a storage unit; When first type characteristic information of a second person is extracted using a second image captured by the first camera or another second camera, a person corresponding to the second person is identified using the first type characteristic information of the identified first person. A determination program that executes a process.

8. an extraction unit that extracts first type characteristic information of a first person using a first image captured by a first camera; a second identification unit that, when second type characteristic information is extracted from the first person before or after the first image is captured, identifies a person corresponding to the first person using the second type characteristic information of each of a plurality of persons stored in a storage unit; an information processing device comprising: a first identification unit that, when first type of characteristic information of a second person is extracted using a second image captured by the first camera or another second camera, identifies a person corresponding to the second person using the first type of characteristic information of the identified first person.