Determination method, determination program, and information processing device.

Abstract

In this assessment method, a computer executes processes including: calculating the trajectory of a person appearing in each of a plurality of first images in a time series captured by a first camera at each first time, and the trajectory of a person appearing in each of a plurality of second images in a time series captured by a second camera at each second time, when the plurality of first images and the plurality of second images are acquired; determining a third time in accordance with the time at which each of the plurality of first images is acquired from the first camera and the time at which each of the plurality of second images is acquired from the second camera; and assessing whether the persons for which the trajectory was calculated from each of the image corresponding to the determined third time among the plurality of first images and the image corresponding to the determined third time among the plurality of second images are the same person.　

Description

Technical Field

[0001] This case relates to a determination method, a determination program, and an information processing apparatus.

Background Art

[0002] Techniques for tracking a person using images acquired by a plurality of cameras and continuing authentication have been disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, when performing person tracking and authentication processing using a plurality of cameras with different shooting intervals, in order to execute the authentication processing using the images taken by each camera at the same timing, a predetermined interval is set, and person tracking and authentication processing are performed at the set interval. However, in this case, images taken at an interval shorter than the set interval are not used for the person tracking process, which deteriorates the accuracy of person tracking and becomes a factor in reducing the determination accuracy of the person.

[0005] In one aspect, an object of this case is to provide a determination method, a determination program, and an information processing apparatus capable of improving the determination accuracy of a person.

Means for Solving the Problems

[0006] In one embodiment, the determination method is characterized in that, when a first camera acquires a plurality of first images in a time series taken every first hour and a second camera acquires a plurality of second images in a time series taken every second hour, the computer calculates the trajectory of a person in each of the plurality of first images and the trajectory of a person in each of the plurality of second images, determines a third time according to the time at which each of the plurality of first images is acquired from the first camera and the time at which each of the plurality of second images is acquired from the second camera, and determines whether the person whose trajectory is calculated from both the image of the plurality of first images corresponding to the determined third time and the image of the plurality of second images corresponding to the determined third time are the same person. [Effects of the Invention]

[0007] This can improve the accuracy of person identification. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram illustrates continuous authentication. [Figure 2] This diagram illustrates the details of Re-ID. [Figure 3] This diagram illustrates the details of Re-ID. [Figure 4] This is a diagram illustrating the outline of Example 1. [Figure 5] This is a diagram illustrating the outline of Example 1. [Figure 6] (a) is a block diagram illustrating the overall configuration of the biometric authentication system according to Example 1, and (b) is a functional block diagram of the information processing device. [Figure 7] This flowchart illustrates the Re-ID process performed by an information processing device. [Figure 8] This diagram illustrates how to determine the execution interval for the second image processing step. [Figure 9] This diagram illustrates images that can be saved to a queue. [Figure 10] This diagram illustrates the process of extracting an image. [Figure 11] This is a block diagram illustrating the hardware configuration of an information processing device. [Modes for carrying out the invention]

[0009] Biometric authentication is a technology that uses biometric characteristics such as fingerprints, faces, and veins to verify identity. In biometric authentication, when identity verification is required, matching biometric characteristic data acquired by a biosensor is compared (matched) with pre-registered biometric characteristic data, and identity verification is performed by determining whether the similarity exceeds a certain threshold. Biometric authentication is used in various fields such as bank ATMs and access control, and in recent years, it has begun to be used in cashless payments in supermarkets and convenience stores.

[0010] These biometric authentication methods are "point-based" authentications performed at specific authentication spots, such as in front of an authentication machine. However, with "point-based" authentication, the authentication state is interrupted when the user leaves the authentication spot, requiring repeated authentication when using the service again or in locations where authentication is required multiple times. Therefore, there is a need for continuous authentication technology that eliminates the need for repeated authentication and allows users to enjoy the service with a single authentication.

[0011] Here, we will explain the overview of continuous authentication technology. Continuous authentication primarily involves the following types of authentication:

[0012] First, there's the authentication process during check-in. At gates and other locations, users undergo highly accurate authentication methods such as palm vein recognition or fingerprint recognition. Upon successful authentication, the user's appearance is captured by a camera, and the user's ID is linked to the characteristic information obtained by the camera and registered.

[0013] Next, it is authentication using lines. By tracking the same person among multiple cameras in the authentication space and performing authentication processing, the authentication state is maintained. As a result, personal services can be provided at any location within the authentication space. Also, since authentication operations are not required each time, services can be provided at the optimal timing.

[0014] For example, as illustrated in FIG. 1, a plurality of cameras 110 are installed at different positions in the authentication space. The cameras 110 acquire images in time series at a predetermined time interval. A person can be detected from the acquired plurality of images. Also, feature information is extracted from the detected person, and the person can be tracked by tracking the person within the same image or by performing identification (Re-ID) between multiple images. Also, by comparing the feature information of the tracked person with the registered feature information, the authentication state of the tracked person can be continued.

[0015] For example, assume that person A moves within the authentication space. In this case, the authentication state of the person being tracked by the first camera 110 can be continued. Next, as person A moves, person A comes into the images acquired by the first camera 110 and the second camera 110. If it is determined that person A being tracked by the first camera 110 and person A being tracked by the second camera 110 with the continued authentication state are the same person, the tracking of person A can be integrated.

[0016] FIG. 2 and FIG. 3 are diagrams illustrating the details of Re-ID. Re-ID will be repeatedly executed at a predetermined time interval. As illustrated in FIG. | 2, a plurality of images are acquired in time series from each camera 110. For example, within a specified time range from the start time of Re-ID, a plurality of images are acquired in time series. For example, images within the time range from the start time of Re-ID to the start time of the next Re-ID are acquired.

[0017] The acquired images are stored in the queue 120. The images stored during the previous Re-ID are discarded.

[0018] Next, as illustrated in FIG. 3, for each of the images stored in the queue 120 that are acquired from each camera 110 at the same timing, image processing is performed. That is, image processing is performed synchronously. Specifically, a person is detected, feature information is extracted from the detected person, and tracking is performed. Tracking is a tracking process for calculating the trajectory of the same person. Known techniques may be used for tracking. If the trajectory can be calculated, even if a single image contains multiple persons, each person can be continuously tracked. For example, it is possible to track which person in the image has been authenticated as which person.

[0019] Next, it is determined which ID of the person each detected person matches. In this case, it is determined that the person has the ID with the highest similarity between the feature information extracted from the image and the registered feature information. In this way, the authentication status of the tracked person can be continued.

[0020] However, the fps (frames per second) of each camera 110 may be different. For example, there may be a mixture of high-fps cameras with a high fps and low-fps cameras with a low fps. In such a case, since the arrival times of the images are shifted, it becomes impossible to determine whether each person photographed by a plurality of cameras and for which a trajectory has been calculated is the same person at the same timing.

[0021] Therefore, for example, by saving images acquired from camera 110 in chronological order and extracting and processing the images to match the lowest fps, it becomes possible to determine whether the person captured by multiple cameras and whose trajectory has been calculated is the same person or not. However, if image processing is performed to match a low fps, the person's trajectory will be calculated at a low fps. In this case, the time interval for calculating the trajectory becomes longer, which reduces the accuracy of tracking the person. If multiple people are captured in a single image, the tracking accuracy will decrease. In particular, if multiple people are captured in a single image and their directions of movement differ, the tracking accuracy will decrease especially. As a result, even if Re-ID is performed, the accuracy of person identification may decrease.

[0022] Therefore, the following embodiments describe a determination method, a determination program, and an information processing device that can improve the accuracy of person identification. [Examples]

[0023] The outline of Example 1 will be described. In this example, we focus on the fact that the tracking process for calculating the trajectory of a person can be performed using images acquired from a single camera. In other words, we focus on the fact that the tracking process can be performed without using images acquired from multiple cameras.

[0024] As an example, let's consider two cameras. The first camera captures multiple first images every first hour (first fps), and the second camera captures multiple second images every second hour (second fps). The shooting ranges of the first camera and the second camera overlap in at least part. This allows both cameras to capture the same person at the same time in areas where their shooting ranges overlap.

[0025] As illustrated in Figure 4, images are acquired from the first and second cameras according to the frame rate (fps) of each camera. Because there is a difference in the fps between the first and second cameras, there is a difference in the number of images acquired from each camera per unit time. Next, a third time is determined based on the time taken to acquire each of the multiple first images from the first camera and the time taken to acquire each of the multiple second images from the second camera. This third time is the execution interval for the second image processing described later.

[0026] Next, as illustrated in Figure 5, the first image processing is performed on each image acquired by each camera. In the first image processing, person detection is performed, feature information is extracted from the detected person, and tracking is performed. This makes it possible to calculate the trajectories of people included in multiple first images and the trajectories of people included in multiple second images. The images after the first image processing are stored in queue 50.

[0027] Next, from queue 50, the image corresponding to the predetermined third time is extracted from among multiple first images. Also, from queue 50, the image corresponding to the predetermined third time is extracted from among multiple second images. Next, second image processing is performed on the images extracted from queue 50. In the second image processing, it is determined which person ID each detected person matches. That is, in the second image processing, it is determined that the detected person is the person with the ID that has the highest similarity between the feature information extracted from that person and the registered feature information. This makes it possible to determine whether the individuals whose trajectories are calculated from multiple first images and multiple second images are the same person or not.

[0028] By doing this, the tracking process can be performed independently of the third time period, thus maintaining a high level of accuracy in tracking individuals. Since Re-ID is performed while maintaining a high level of tracking accuracy, the accuracy of person identification can be improved.

[0029] The details of this embodiment will be described below. Figure 6(a) is a block diagram illustrating the overall configuration of the biometric authentication system 300 according to Embodiment 1. As illustrated in Figure 6(a), the biometric authentication system 300 includes an information processing device 100, multiple cameras 200, and the like. These devices are connected wirelessly or by wire.

[0030] The multiple cameras 200 are cameras installed in the authentication space and are positioned to easily acquire characteristic information about people. For example, they may be installed on the ceiling to facilitate tracking of people. At least one of the multiple cameras 200 may have a different frame rate (fps) than the other cameras.

[0031] Figure 6(b) is a functional block diagram of the information processing device 100. As illustrated in Figure 6(b), the information processing device 100 functions as an image acquisition unit 10, an execution interval determination unit 20, a first image processing unit 30, an image storage unit 40, a queue 50, an image retrieval unit 60, a second image processing unit 70, and so on.

[0032] Next, we will explain each process performed by the information processing device 100.

[0033] Figure 7 is a flowchart illustrating the Re-ID process performed by the information processing device 100. Re-ID will be performed repeatedly at predetermined time intervals. The authentication process will be explained below with reference to the flowchart in Figure 7.

[0034] First, the image acquisition unit 10 acquires multiple images in chronological order from each camera 200 (step S1). For example, the image acquisition unit 10 acquires multiple images in chronological order from each camera 200 within a specified time range from the start time of Re-ID. For example, it acquires images within the time range from the start time of Re-ID to the start time of the next Re-ID. The image acquisition unit 10 may extract images from the video received from each camera 200 according to the fps. Alternatively, each camera 200 may extract images from the video and transmit them as images to the image acquisition unit 10.

[0035] Next, the execution interval determination unit 20 determines the execution interval for the second image processing according to the time interval at which images are acquired from each camera 200 (step S2). For example, the execution interval determination unit 20 determines the minimum value among the fps of each camera 200 as the execution interval. The fps may be set for video, but it may also be calculated from the image generation interval, the image arrival interval, etc. Alternatively, it may be determined from the performance of the cameras 200 and the network in the authentication space. As an example, in the example in Figure 8, if the fps of each camera 200 is 5fps, 15fps, and 30fps, the minimum value of 5fps is determined as the execution interval.

[0036] Next, the first image processing unit 30 performs first image processing on all images acquired in step S1 (step S3). By performing step S3, the trajectory of the same person can be calculated and tracked. If necessary, the processing results of step S3 may be saved in the image storage unit 40.

[0037] Next, queue 50 saves the image after the first image processing in step S3 (step S4). For example, as illustrated in Figure 9, if the next image is acquired from the high fps camera before the second image processing is performed, the image saved during the previous Re-ID may be discarded by overwriting it with the new image, or it may be saved as historical data. For example, for camera A, which is 5 fps, the image saved at 12:00:00.025 on 12 / 19 / 2022 is saved. For camera B, which is 15 fps, the images saved at 12:00:00.021 on 12 / 19 / 2022 and at 12:00:00.028 on 12 / 19 / 2022 are saved. Camera C has saved images taken at 12:00:00.020, 12:00:00.023, 12:00:00.027, and 12:00:00.030 on December 19, 2022.

[0038] Next, the image retrieval unit 60 retrieves an image from the queue 50 at the execution interval of the second image processing determined in step S2 (step S5). For example, it may retrieve the image closest to the time specified by the execution interval of the second image processing, or it may retrieve the most recent image that is after the time specified by the execution interval of the second image processing. For example, images are retrieved as illustrated in Figure 10. Images with a strikethrough indicate that they were not retrieved. In the example in Figure 10, the image taken at 12:00:00.025 on 2022 / 12 / 19 is retrieved from camera A. The image taken at 12:00:00.028 on 2022 / 12 / 19 is retrieved from camera B. The image taken at 12:00:00.030 on 2022 / 12 / 19 is retrieved from camera C. If there are no images to retrieve, it is assumed that there are no images for the target camera. For example, for camera 200 with a frame rate lower than the execution interval, there may be no images to retrieve. Similarly, if delays occur due to network interference, there may be no images to retrieve.

[0039] Next, the second image processing unit 70 performs a second image processing on the image extracted in step S6, as illustrated in Figure 7 (step S6). This makes it possible to determine whether the person captured by multiple cameras 200 and whose trajectory has been calculated is the same person. Alternatively, the second image processing may be performed using the results of the first image processing stored in the image storage unit 40. For example, by using the results of the first image processing stored in the image storage unit 40, the tracking results of the person identified in the previous Re-ID can be utilized.

[0040] In the example above, the first image processing was performed on all images stored in queue 50, but it is not necessary to perform the first image processing on all images. However, it is preferable to make the execution interval for the first image processing shorter than the execution interval for the second image processing. For example, if images are acquired at 30fps, the execution interval for the second image processing may be set to 5fps, and the execution interval for the first image processing may be set to 10fps, which is shorter than the execution interval for the second image processing.

[0041] Furthermore, the execution interval for performing the first image processing may be changed for each camera. For example, in areas with high pedestrian traffic, such as near entrances and exits, it is difficult to maintain high accuracy in tracking individuals. In such cases, it may be possible to set the execution interval for performing the first image processing to a high frame rate, such as 30 fps, for cameras installed in those locations.

[0042] Alternatively, the execution interval for the first image processing may be changed depending on the characteristics of the acquired image. For example, if person detection is performed on the acquired image, the execution interval for the first image processing may be shortened if a large number of people are detected.

[0043] In the example above, the second image processing involved retrieving and processing the most recent image from each camera that was stored. However, it is also possible to select and process images that were acquired at a more recent time. In this case, past images may be retained from the time they are retrieved by the image acquisition unit 10 until they are processed.

[0044] Furthermore, the execution interval determination unit 20 may determine not only the execution interval of the second image processing, but also the execution time when the acquisition timing approaches.

[0045] Figure 11 is a block diagram illustrating the hardware configuration of the information processing device 100. As illustrated in Figure 11, the information processing device 100 includes a CPU 101, RAM 102, and a storage device 103. The CPU (Central Processing Unit) 101 is a central processing unit. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, etc. The storage device 103 is a non-volatile storage device. As the storage device 103, for example, a ROM (Read Only Memory), a solid-state drive (SSD) such as flash memory, or a hard disk driven by a hard disk drive can be used. The functions of each part of the information processing device 100 are realized by the CPU 101 executing the judgment program stored in the storage device 103. Note that the functions of each part of the information processing device 100 may be configured by dedicated circuits, etc.

[0046] In each of the above examples, the first image processing unit 30 is an example of a calculation unit that calculates the trajectory of a person in each of the multiple first images and the trajectory of a person in each of the multiple second images when the first camera acquires a plurality of first images in a time series taken every first hour and a plurality of second images in a time series taken every second hour by the second camera. The execution interval determination unit 20 is an example of a determination unit that determines the third time according to the time at which each of the multiple first images is acquired from the first camera and the time at which each of the multiple second images is acquired from the second camera. The second image processing unit 70 is an example of a determination unit that determines whether the person whose trajectory is calculated from both the image among the plurality of first images corresponding to the determined third time and the image among the plurality of second images corresponding to the determined third time are the same person.

[0047] Although embodiments of the present invention have been described in detail above, the present invention is not limited to these specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0048] 10 Image acquisition unit 20 Execution interval determination unit 30 First Image Processing Unit 40 Image storage section 50 cues 60 Image extraction unit 70. Second Image Processing Unit 100 Information Processing Devices 200 Cameras 300 biometric authentication systems

Claims

1. When the first camera acquires multiple first images in a time series taken every hour, and the second camera acquires multiple second images in a time series taken every two hours, the trajectory of the person in each of the multiple first images and the trajectory of the person in each of the multiple second images are calculated. A third time is determined according to the time taken to acquire each of the plurality of first images from the first camera and the time taken to acquire each of the plurality of second images from the second camera. A determination method characterized in that a computer performs a process to determine whether the person whose trajectory is calculated from both the image of one of the plurality of first images corresponding to the determined third time and the image of one of the plurality of second images corresponding to the determined third time are the same person.

2. The determination method according to claim 1, characterized in that the third time is determined to be the longer of the first time and the second time.

3. The determination method according to claim 1, characterized in that the third time is determined to be the longer of the time intervals for acquiring each of the plurality of first images from the first camera and the time intervals for acquiring each of the plurality of second images from the second camera.

4. The determination method according to any one of claims 1 to 3, characterized in that when calculating the trajectory from the plurality of first images and the plurality of second images, images are extracted from at least one of the plurality of first images and the plurality of second images such that the time interval is shorter than the third time, and the trajectory is calculated.

5. The determination method according to any one of claims 1 to 3, characterized in that when calculating the trajectory from the plurality of first images and the plurality of second images, images are extracted from at least one of the plurality of first images and the plurality of second images such that the time interval becomes shorter as the number of people in the image increases, and the trajectory is calculated.

6. On the computer, When the first camera acquires multiple first images in a time series taken every hour, and the second camera acquires multiple second images in a time series taken every two hours, the trajectory of the person in each of the multiple first images and the trajectory of the person in each of the multiple second images are calculated. A third time is determined according to the time taken to acquire each of the plurality of first images from the first camera and the time taken to acquire each of the plurality of second images from the second camera. A determination program characterized by causing the program to execute a process that determines whether the person whose trajectory is calculated from both the image of one of the plurality of first images corresponding to the determined third time and the image of one of the plurality of second images corresponding to the determined third time are the same person.

7. When a first camera acquires multiple first images in a time series taken every hour, and a second camera acquires multiple second images in a time series taken every two hours, a calculation unit calculates the trajectory of a person in each of the multiple first images and the trajectory of a person in each of the multiple second images. A determination unit that determines a third time according to the time taken to acquire each of the plurality of first images from the first camera and the time taken to acquire each of the plurality of second images from the second camera, An information processing device comprising: a determination unit that determines whether the person whose trajectory is calculated from both the image of one of the plurality of first images corresponding to the determined third time and the image of one of the plurality of second images corresponding to the determined third time are the same person.

Images