Control method, control program, and information processing device

Abstract

This control method performs a process for, when a computer has detected a plurality of persons from an image that includes an authentication target person who has succeeded in personal authentication through the use of the identification information, controlling a target detection region for detecting the target person in the image.　

Description

Control method, control program, and information processing device

[0001] The present invention relates to a control method, a control program, and an information processing device.

[0002] In continuous authentication, highly accurate authentication is performed at the time of check-in to identify the user's ID, and when authentication is successful, information about the user's appearance is acquired and linked to the user's ID as registered information. For example, a method of acquiring information about the user's appearance by photographing the user's appearance with a camera has been disclosed (see, for example, Patent Document 1).

[0003] Special Publication No. 2021-531539

[0004] However, when multiple users check in, there is a risk that people other than the target person will appear in the image captured by the camera. In this case, if the detection accuracy of the target person is low, there is a risk that information about the appearance characteristics of other people will be acquired.

[0005] In one aspect, the present invention aims to provide a control method, a control program, and an information processing device that can improve the accuracy of detecting a target person.

[0006] In one aspect, the control method includes a process in which, when a computer detects multiple people from an image including a subject whose identity has been successfully authenticated using identification information, the computer controls a subject detection area for detecting the subject in the image.

[0007] The accuracy of detecting the target person can be improved.

[0008] 1A and 1B are diagrams for explaining an overview of continuous authentication technology. (a) and (b) are diagrams illustrating details of check-in. (b) is a diagram illustrating a case where multiple people appear in an image acquired by a camera. (a) is a block diagram illustrating an overall configuration of a biometric authentication system according to a first embodiment, and (b) is a functional block diagram illustrating each function of an information processing device. (a) is a diagram illustrating a top view of an entrance area, (b) is a diagram illustrating an example of a registered biometric data table stored in a registered data storage unit, and (c) is a diagram illustrating an example of an image acquired by an acquisition unit from a matching camera. (a) to (c) are diagrams for explaining the direction in which a subject is facing, and (d) is a diagram illustrating an example of the size of a person area in an image. (a) is a diagram illustrating a case where multiple users check in sequentially, (b) is a diagram illustrating a case where multiple person areas are detected, and (c) is a diagram illustrating an example of control of a detection target area. (b) is a diagram illustrating an example of a registered information table stored in a registered information storage unit. (c) is a flowchart illustrating an example of an operation of an information processing device. (a) is a diagram illustrating a case where the angle of view is wide, and (b) is a diagram illustrating a case where the angle of view is narrow. (a) to (c) are diagrams illustrating a case where multiple people are captured in the detection target area. (a) and (b) are diagrams illustrating a case where multiple people overlap in the detection target area. (a) and (b) are diagrams illustrating the angle of the matching camera. (a) and (b) are diagrams illustrating the distance between the biometric sensor and the user. (a) to (c) are diagrams illustrating changes in the person area. A block diagram for explaining the hardware configuration of an information processing device.

[0009] Biometric authentication is a technology that verifies identity using biometric characteristics such as fingerprints, faces, and veins. In biometric authentication, when identity verification is required, biometric data for verification acquired by a biometric sensor is compared (matched) with pre-registered biometric data, and identity verification is performed by determining whether the similarity is equal to or exceeds an identity verification threshold. Biometric authentication is used in various fields such as bank ATMs and access control, and in recent years has also begun to be used for cashless payments at supermarkets, convenience stores, etc.

[0010] These biometric authentication methods are "point" authentication performed at specific authentication spots, such as in front of an authentication machine. However, with "point" authentication, the authentication state is interrupted when the user leaves the authentication spot, and if the user wishes to receive a service again or at a location where authentication is required multiple times, authentication must be performed each time. Therefore, there is a demand for continuous authentication technology that eliminates the need for repeated authentication and allows the user to continue receiving services with a single authentication.

[0011] Here, an overview of continuous authentication technology will be explained. Fig. 1 is a diagram for explaining the overview of continuous authentication technology. Continuous authentication mainly involves the following three types of authentication:

[0012] The first is authentication at check-in. At the gate, the user performs highly accurate authentication using palm vein authentication, facial recognition, etc. to identify the user's ID, and when authentication is successful, a camera captures a photo of the user's appearance to obtain characteristic information about their appearance, which is then linked to the user's ID as registered characteristic information.

[0013] Next, there is line authentication, which uses feature information for matching, which is continuously obtained over time using one or more cameras, and registered feature information to maintain an authentication state in which a person photographed by a camera is authenticated as a user who has been successfully authenticated.

[0014] Next, re-authentication is performed when the user enters a blind spot of the camera due to, for example, another person or a pillar, and the authentication state is interrupted.

[0015] By performing such authentication, continuous authentication can be performed, which allows you to receive services without having to perform authentication multiple times.

[0016] Here, the details of check-in will be described. Figures 2(a) and 2(b) are diagrams illustrating the details of check-in. As illustrated in Figure 2(a), highly accurate biometric data such as a face image acquired by a face camera or a vein image acquired by a vein sensor is acquired from the biometric sensor 201, and authentication is performed using this data. This identifies the ID, name, etc. of the person checking in.

[0017] At this time, an image of the subject is acquired using the camera 202. Next, as illustrated in FIG. 2B, a person area (Bbox: Bounding box) is detected from the image. Next, the person included in the image is analyzed from the person area to extract appearance feature information for identifying the person. In this case, the feature information includes appearance information such as the color of clothing, physique, facial features, and behavioral features. This feature information is linked to the subject's ID as registered feature information. As a result, the registered appearance feature information can be used for subsequent continuous authentication. Note that the behavioral features are, for example, skeletal information including the person's joints.

[0018] However, when a subject is photographed using the camera 202, other people may appear in the image. In this case, it may be difficult to accurately extract the person area corresponding to the subject. For example, check-in is a process of registering that a user has entered a store by performing a predetermined process when the user enters the store. Information about the user's entry into the store may be read using a terminal from an image attached to a predetermined position in the store (e.g., an entrance, etc.). In the store, products for sale are placed for users who have checked in.

[0019] Fig. 3 is a diagram illustrating an example in which multiple people appear in an image captured by the camera 202 in Fig. 2. As illustrated in Fig. 3, three people appear in the image. In this case, three person regions are detected, and there is a risk that the three person regions will be linked to the appearance feature information of people other than the target person.

[0020] Therefore, in the following embodiments, a control method, a control program, and an information processing device that can improve the accuracy of detecting a subject will be described.

[0021] Details will be explained below. Fig. 4(a) is a block diagram illustrating an example of the overall configuration of a biometric authentication system 200 according to the first embodiment. As illustrated in Fig. 4(a), the biometric authentication system 200 includes an information processing device 100, a biometric sensor 110, a linking camera 120, a tracking camera 130, and the like. These devices are connected via electric communication lines.

[0022] The biometric sensor 110 is provided at a gate or the like of the continuous authentication space, and is installed near the linking camera 120. The biometric sensor 110 is not particularly limited as long as it is a sensor that can acquire biometric data of a user checking in with high accuracy. For example, when veins are used as the biometric modality, the biometric sensor 110 is a vein sensor that uses near-infrared rays or the like. When a fingerprint is used as the biometric modality, the biometric sensor 110 is a capacitance fingerprint sensor or the like. When a face is used as the biometric modality, the biometric sensor 110 is a face camera or the like.

[0023] The linking camera 120 is a camera installed at the gate of the continuous authentication space or the like, and is installed in a position where it is easy to obtain information about the appearance characteristics of a person. The tracking camera 130 is a camera for tracking a person in the continuous authentication space, and is installed on the ceiling or the like so that it is easy to track the person. There may be one or more tracking cameras 130.

[0024] 4B is a functional block diagram showing each function of the information processing device 100. As illustrated in FIG. 4B, the information processing device 100 functions as an acquisition unit 11, a person detection unit 12, a number of people determination unit 13, a detection area control unit 14, a direction determination unit 15, a feature extraction unit 16, a registration information storage unit 17, a ReID processing unit 18, an authentication unit 19, a registration data storage unit 20, and the like. The registration data storage unit 20 stores the registered biometric data of each user. This registered biometric data is registered in advance by each user along with their ID, name, and the like.

[0025] FIG. 5( a) is a diagram illustrating an example of a case where one user checks in. FIG. 5( a) is a diagram illustrating an entrance area viewed from above. As illustrated in FIG. 5( a), a biometric sensor 110 and a linking camera 120 are installed at the gate of the entrance area. The user enters the entrance area, approaches the gate, and stands still in a position where the biometric sensor 110 can acquire the user's biometric data. This causes the user to stand still within the shooting range of the linking camera 120.

[0026] First, the biometric sensor 110 acquires the subject's biometric data for matching and sends it to the authentication unit 19. The authentication unit 19 compares the biometric data for matching with each piece of registered biometric data stored in the registered data storage unit 20. FIG. 5B is a diagram illustrating a registered biometric data table stored in the registered data storage unit 20. As illustrated in FIG. 5B, the registered biometric data table stores registered biometric data linked to each user's ID and name. For example, the authentication unit 19 calculates the similarity between the biometric data for matching and each piece of registered biometric data. The authentication unit 19 identifies the subject as a user of registered biometric data whose similarity exceeds a threshold. This makes it possible to identify the subject's ID.

[0027] In parallel, the acquisition unit 11 acquires an image from the matching camera 120. FIG. 5( c) is a diagram illustrating an example of an image acquired by the acquisition unit 11 from the matching camera 120. The person detection unit 12 detects a person area in the image. Methods for detecting a person area include a method of detecting a person area using background difference, and a method of learning person characteristics in advance and detecting person characteristics from an input image. In FIG. 5( c), the area surrounded by a dotted line is the person area. The number of people determination unit 13 determines the number of people appearing in the image. In the example of FIG. 5( c), the number of people determination unit 13 determines the number of people to be one.

[0028] In this case, the detection area control unit 14 does not narrow the detection target area for detecting a human area from the image.

[0029] Next, the direction determination unit 15 determines whether or not the subject is facing a predetermined direction. For example, the direction determination unit 15 determines whether or not the subject is facing the linking camera 120.

[0030] 6(a) to 6(c) are diagrams for explaining the direction in which the subject is facing. For example, as illustrated in FIG. 6(b), if the subject is carrying luggage on his / her back, if the luggage is captured in the image, the subject is facing away from the tying camera 120. Therefore, in this case, it is determined that the subject is not facing in a predetermined direction. Whether luggage is captured in the image can be determined by prior learning of the image, etc.

[0031] For example, as illustrated in Fig. 6(c), the subject is determined not to be facing the predetermined direction even when the subject is facing sideways with respect to the linking camera 120. Whether the subject is facing sideways with respect to the linking camera 120 can be determined by prior learning of the image, etc.

[0032] For example, as illustrated in Fig. 6(a), when the subject faces the direction of the linking camera 120, it is determined that the subject faces a predetermined direction. Whether the subject faces the direction of the linking camera 120 can be determined by prior learning of the image, etc.

[0033] The direction determination unit 15 may determine whether the size of the person region in the image is equal to or larger than a threshold. For example, as illustrated in FIG. 6D , if the size of the person region in the image is equal to or larger than a threshold, the direction determination unit 15 may determine that the size of the person region in the image is equal to or larger than the threshold. If the direction determination unit 15 does not determine that the size of the person region in the image is equal to or larger than the threshold, the direction determination unit 15 may treat the situation in the same way as if it was not determined that the subject is facing a predetermined direction.

[0034] Next, FIG. 7( a) is a diagram illustrating a case where multiple users check in in sequence. FIG. 7( a) is a diagram illustrating an entrance area viewed from above. As illustrated in FIG. 7( a), a user enters the entrance area, approaches the gate, and stops at a position where the biometric sensor 110 can acquire the user's biometric data. This causes the user to stop in the shooting range of the linking camera 120. However, because other users wait their turn behind the user, people other than the target person also appear in the shooting range of the linking camera 120. For example, as illustrated in FIG. 7( b), multiple person areas surrounded by dotted lines may be detected.

[0035] When multiple users check in in sequence, they often appear shifted horizontally in the image. Therefore, the detection area control unit 14 narrows the detection target area for detecting a person area from the image horizontally, as illustrated in FIG. 7( c). In the example of FIG. 7( c), the area between the two hatched areas corresponds to the narrowed detection target area. The vertical height of the detection target area may be the same as that of the original image. For example, the range in the image where the user is facing the linking camera 120 is predetermined. Therefore, the detection target area is narrowed to the range where the user faces the linking camera 120. In the example of FIG. 7( c), the person areas surrounded by dotted lines in the detection target area are merged into one. The direction determination unit 15 then determines whether the person in the person area is facing a predetermined direction.

[0036] Thereafter, when the direction determination unit 15 determines that the subject is facing the linking camera 120, the feature extraction unit 16 extracts appearance feature information from the detected person area as registered feature information. Because the person area is accurately detected, the registered feature information can also be accurately extracted.

[0037] The registration information storage unit 17 stores the registered feature information extracted by the feature extraction unit 16 in association with the ID identified by the authentication unit 19. Fig. 8 is a diagram illustrating an example of a registration information table stored in the registration information storage unit 17. As illustrated in Fig. 8, the registration information table stores the registered feature information in association with the user's ID, name, etc.

[0038] The user then moves through the continuous authentication space. The acquisition unit 11 acquires an image from the tracking camera 130. The person detection unit 12 detects a person region from the image. The feature extraction unit 16 extracts appearance feature information for matching from the detected person region. The ReID processing unit 18 compares the feature information for matching extracted by the feature extraction unit 16 with registered feature information stored in the registered information storage unit 17. For example, the ReID processing unit 18 calculates the similarity between the feature information for matching extracted by the feature extraction unit 16 and each piece of registered feature information. The ReID processing unit 18 identifies a person detected by the tracking camera 130 as a user of registered feature information whose similarity exceeds a threshold. Note that in this case, if the direction determination unit 15 does not determine that a person captured in the image is facing a predetermined direction, the image may not be processed by the ReID processing unit 18. This allows images from other tracking cameras 130 to be prioritized, thereby improving the accuracy of continuous authentication. Furthermore, when the number of people determination unit 13 determines that the number of people appearing in the image is plural, the detection area control unit 14 may narrow the detection target area.

[0039] 9 and 10 are flowcharts showing an example of the operation of the information processing device 100. The flow of operation of the information processing device 100 will be described with reference to FIGS. 9 and 10. The flowcharts of FIGS. 9 and 10 are executed at predetermined intervals. The flowcharts of FIGS. 9 and 10 are also executed independently for each of the linking camera 120 and the tracking camera 130.

[0040] First, the acquisition unit 11 acquires an image from either the linking camera 120 or the tracking camera 130 (step S1).

[0041] Next, the person detection unit 12 detects a person area from the image acquired in step S1 (step S2).

[0042] The person detection unit 12 determines whether or not a person has been detected (step S3). If the determination in step S3 is "No", the process is repeated from step S1.

[0043] If the determination in step S3 is "Yes", the number of people determination unit 13 determines whether the number of people detected is two or more (step S4).

[0044] If the determination in step S4 is "Yes," the detection area control unit 14 narrows the target detection area (step S5), as illustrated in Fig. 7C. If the determination in step S4 is "No," step S5 is not executed. Therefore, the target detection area is not narrowed.

[0045] After step S5 is executed, or if step S4 is determined to be "No," the direction determination unit 15 determines whether the person in the person area included in the target detection area is facing a predetermined direction (step S6). For example, the direction determination unit 15 determines whether the person is facing the camera direction, as described with reference to FIGS. 6( a) to 6(c). In this case, the direction determination unit 15 may also determine whether the size of the person in the target detection area is equal to or larger than a threshold, as described with reference to FIG. 6(d).

[0046] If the determination in step S6 is "No," the process is repeated from step S1. Therefore, unless the person in the target detection area is facing a predetermined direction, feature information is not extracted. If the determination in step S6 is "Yes," the biometric sensor 110 detects biometric data for verification from the user checking in (step S7).

[0047] Next, the authentication unit 19 determines whether the biometric sensor 110 has detected the biometric data for matching (step S8). If the camera from which the image was acquired in step S1 is the tracking camera 130, the determination in step S8 is "No." If the camera from which the image was acquired in step S1 is the linking camera 120, the determination in step S8 is "Yes."

[0048] If the determination in step S8 is "Yes," the authentication unit 19 performs authentication processing (step S9). Specifically, the authentication unit 19 compares the matching biometric data with each registered biometric data stored in the registered data storage unit 20, and identifies the target person as a user of the registered biometric data whose similarity exceeds a threshold. In parallel with step S9, the feature extraction unit 16 extracts appearance feature information from the person area (step S10).

[0049] After steps S9 and S10 are executed, the registration information storage unit 17 stores the characteristic information as registered characteristic information in association with the ID identified in step S9 (step S11). After step S11 is executed, the execution of the flowchart ends.

[0050] If the determination in step S8 is "No", the feature extraction unit 16 extracts and stores appearance feature information for matching from the person area (step S12).

[0051] Next, the ReID processing unit 18 compares the feature information for comparison extracted in step S12 with each piece of registered feature information stored in the registered information storage unit 17, and calculates the degree of similarity (step S13).

[0052] Next, the ReID processing unit 18 identifies the person in the person area as the person of the registered characteristic information with the highest similarity among the similarities calculated in step S13 (step S14).

[0053] Next, the ReID processing unit 18 determines whether or not a person whose ID can be identified has been absent for a period of time equal to or longer than the threshold time (step S15).

[0054] If the determination in step S15 is "No," the process is executed again from step S12. If the determination in step S15 is "Yes," the ReID processing unit 18 erases the ID that had been specified up to that point (step S16).

[0055] According to this embodiment, when multiple people are detected from an image including a person to be authenticated who has been successfully authenticated using identification information such as biometric data, the object detection area for detecting the person in the image is controlled. For example, the object detection area for the image is narrowed. This increases the accuracy of detecting the person area even when multiple people appear in the image. As a result, the accuracy of detecting the person improves. By extracting person feature information from the person and linking it to an identifier such as the ID of the person to be authenticated, highly accurate continuous authentication can be achieved. By extracting person feature information when it is determined that the person to be authenticated is facing a predetermined direction, the accuracy of subsequent continuous authentication can be improved.

[0056] (Modifications) Next, various modifications will be described. For example, the extent to which the detection target area is narrowed may be determined according to the angle of view and focal length of the linking camera 120. For example, when a person with a standard build is standing at a position at the focal length of the linking camera 120, it is preferable to narrow the detection target area to an extent that the entire body of one person is captured.

[0057] FIG. 11( a) is a diagram illustrating a case where the angle of view is wide. In the case of FIG. 11( a), because the angle of view is wide, when a person with a standard build is standing at the focal length, the range in which one person is captured is narrow. Therefore, the detection target area is set to be narrow. FIG. 11( b) is a diagram illustrating a case where the angle of view is narrow. In the case of FIG. 11( b), because the angle of view is narrow, when a person with a standard build is standing at the focal length, the range in which one person is captured is wide. Therefore, the detection target area is set to be wide.

[0058] It is also assumed that check-in users may have a variety of body types. It is also assumed that check-in users may face a variety of directions. When multiple people appear in an image, as illustrated in FIGS. 12( a) to 12(c), multiple people may appear in the detection target area, and the heights and horizontal widths of the person areas may differ. Therefore, when person areas of different heights and widths are detected in the detection target area, the feature extraction unit 16 may extract feature information from the tallest person area.

[0059] Furthermore, in the detection target area, multiple person regions may overlap as illustrated in Figures 13(a) and 13(b). Therefore, when multiple person regions overlap in the detection target area, the feature extraction unit 16 may extract feature information from the person region with the highest height.

[0060] It is preferable that the angle of the linking camera 120 is adjusted so that one user is captured as much as possible. For example, as shown in Fig. 14(b), if the capturing range of the linking camera 120 is a wide space where multiple users can be positioned, there is a high possibility that multiple users will be captured in the image captured by the linking camera 120.

[0061] In contrast, as illustrated in FIG. 14( a), it is preferable that the angle of the linking camera 120 be determined so that it can capture a space that narrows horizontally from a space that widens horizontally. In this case, it is difficult for other users to enter the narrow space while waiting their turn, so the number of people appearing in the image acquired by the linking camera 120 can be reduced. This reduces the number of people appearing in the target detection area, making it possible to accurately extract information about the user's appearance. For example, it is preferable that the narrow space be narrow enough to accommodate only one person.

[0062] In addition, when highly accurate authentication is required at check-in, as illustrated in FIG. 15( a), in the case of face authentication using a face camera, the distance between the user and the biometric sensor 110 used as the face camera is long, which may result in a large variation in the user's standing position. Therefore, there is a risk of variation in the accuracy when feature information is extracted using the linking camera 120. In contrast, as illustrated in FIG. 15( b), in the case of vein authentication or fingerprint authentication, the distance between the user and the biometric sensor 110 is short, which reduces the variation in the user's standing position. Therefore, it is possible to reduce the variation in the accuracy when feature information is extracted using the linking camera 120. Therefore, it is preferable to use vein authentication or fingerprint authentication.

[0063] A person region may be detected using time-series images acquired by the linking camera 120. For example, using time-series images, a person can be tracked by tracking. Detecting a person region for a tracked person allows for accurate detection of the target person's region. For example, in Figures 16(a) to 16(c), the arrows indicate a predetermined time period. As such, people can be tracked using time-series images. For example, using time-series images, the movement direction of each person can be detected. For example, by focusing on a predetermined point, such as the center of each person region, the movement direction of each person can be detected. Since the direction in which the leading person is approaching the linking camera 120 is known in advance, it is possible to determine which person in a person region is the leading person. Alternatively, for example, the leading person stops moving and remains stationary near the biometric sensor 110. Therefore, it is possible to determine whether the person in the person region with the smallest movement per unit time is the leading person. Feature information may be extracted from the person region determined to be the leading person.

[0064] (Application Example) Next, an application example will be described. The biometric authentication system 200 can analyze the behavior of a person who has checked in using an image captured by a camera. The facility may be a railway facility, airport, shop, residence, hotel, castle, amusement park, etc. Furthermore, the gate located at the facility may be located at the entrance to a shop, residence, hotel, castle, amusement park, railway facility, airport boarding gate, etc.

[0065] First, an example will be described in which the check-in target is a railway facility or an airport. In the case of the railway facility or airport, the gate is located at the boarding gate of the railway facility or airport. In this case, if the person's biometric information has been pre-registered as a train or airplane passenger, the information processing device 100 determines that authentication using the person's biometric information has been successful.

[0066] Next, an example will be described in which the check-in target is a store. When the check-in target is a store, the gate is located at the store entrance. In this case, when the biometric information of the person who checked in is registered as a member of the store, the information processing device 100 determines that authentication using the biometric information of the person has been successful.

[0067] Returning to FIG. 1 , an application example will be described assuming that the facility is a store. When checking in, the information processing device 100 acquires biometric information of a person passing through a gate located at a predetermined position within the store. Specifically, the information processing device 100 acquires, from the biometric sensor 110, a vein image acquired by a vein sensor mounted on a gate located at the entrance of the store, and performs authentication. At this time, the information processing device 100 identifies the user's ID, name, etc. from the biometric information.

[0068] The biometric sensor 110 and the linking camera 120 are mounted on a gate located at a predetermined position in the facility and detect biometric information of a person passing through the gate. At this time, the information processing device 100 can also acquire the biometric information of the person using the linking camera 120.

[0069] Next, when authentication based on the person's biometric information is successful, the information processing device 100 generates feature information of the person by analyzing an image including the person passing through the gate. Specifically, the information processing device 100 performs authentication by identifying the user ID and name of the person passing through the gate. Then, when authentication is successful, the information processing device 100 analyzes an image including the person passing through the gate to generate feature information of the person. For example, the feature information of the person is skeletal information including the person's joints. At this time, the information processing device 100 determines whether the person to be authenticated is facing the camera used to capture the image in the direction of travel through the gate passage, and if it is determined that the person to be authenticated is facing the direction of travel through the gate passage, generates feature information from the object detection area. Then, the information processing device 100 associates the ID and name of the user checking in with the generated feature information and stores them in a storage unit.

[0070] The information processing device 100 then uses the feature information stored in the storage unit to track the person moving within the store while identifying the user's ID and name. For example, the information processing device 100 performs gait authentication using skeletal information including the person's joints using an existing skeletal estimation algorithm. The information processing device 100 tracks the person by comparing whether the person who passed through the gate is the same as the person included in the image captured by the tracking camera 130, and identifies the person's trajectory from when they entered the store to when they left. Note that the existing skeletal estimation algorithm is, for example, a skeletal estimation algorithm that uses deep learning, such as HumanPose Estimation, such as DeepPose or OpenPose.

[0071] Furthermore, the information processing device 100 identifies the product acquired by the person from among multiple products placed in the store. Specifically, the information processing device 100 identifies the product acquired by the person from among the multiple products by determining whether or not the person is holding a product placed in the store using skeletal information including the person's joints.

[0072] The information processing device 100 registers in a storage unit the person's identification information and the products acquired by the person in association with each other. The information processing device 100 also generates information that associates the person's trajectory from the time the person enters a store to the time the person leaves the store with the user's ID, name, and the like. For example, the information processing device 100 can identify the items purchased by the person in the store by generating information that associates the ID and name of the user who checks in with the person's trajectory in the store and the products acquired by the person. This makes it possible to analyze the purchasing behavior of the person as they move around the store after they check in.

[0073] FIG. 17 is a block diagram illustrating the hardware configuration of an information processing device 100. As illustrated in FIG. 17, the information processing device 100 includes a CPU 101, a RAM 102, a storage device 103, a communication device 104, and the like. These devices are connected via a bus or the like. The CPU (Central Processing Unit) 101 is a central processing unit. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. For example, the storage device 103 may be a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, or a hard disk driven by a hard disk drive. The CPU 101 executes a control program stored in the storage device 103 to realize the functions of each unit of the information processing device 100. The functions of each unit of the information processing device 100 may be configured by a dedicated circuit etc. The communication device 104 is an interface to an electric communication line.

[0074] In the above example, the user's ID is identified using the user's biometric information acquired from the biometric sensor 110, but this is not limiting. For example, the user's ID may be identified based on whether a password entered by the user using an input device matches a pre-registered password. In this case, too, when the user enters the ID using the input device, visual feature information may be extracted from the image acquired by the linking camera 120.

[0075] In each of the above examples, when the detection area control unit 14 detects multiple people from an image including a person to be authenticated who has been successfully authenticated using identification information, the detection area control unit 14 is an example of a detection area control unit that controls a target detection area for detecting the target person in the image. The feature extraction unit 16 is an example of a feature extraction unit that extracts person feature information from the controlled target detection area. The registration information storage unit 17 is an example of a registration information storage unit that associates the feature information with an identifier of the person to be authenticated as registered feature information and stores it. The direction determination unit 15 is an example of a direction determination unit that determines whether the person to be authenticated is facing a predetermined direction relative to a camera that acquires the image.

[0076] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention as described in the claims.

[0077] REFERENCE SIGNS LIST 11 Acquisition unit 12 Person detection unit 13 Number of people determination unit 14 Detection area control unit 15 Direction determination unit 16 Feature extraction unit 17 Registration information storage unit 18 ReID processing unit 19 Authentication unit 20 Registration data storage unit 100 Information processing device 110 Biometric sensor 120 Linking camera 130 Tracking camera 200 Biometric authentication system

Claims

1. The computer A control method characterized by executing processing to control a target detection area for detecting targets in an image when multiple people are detected from an image containing an authentication target whose identity has been successfully authenticated using identification information.

2. The computer extracting feature information of a person from the controlled object detection area; The control method according to claim 1 , wherein a process is executed to link the characteristic information to an identifier of the person to be authenticated.

3. The control method according to claim 1 , wherein the object detection area is controlled by narrowing the object detection area relative to the range of the image.

4. 2. The control method according to claim 1, wherein the object detection area is narrowed according to the angle of view and focal length of a camera used to acquire the image.

5. 3. The control method according to claim 2, wherein when two or more person regions are detected from the target detection region, the feature information of the person is extracted from the tallest person region.

6. The control method according to claim 1 , wherein a person area is detected from a person being tracked using a time series of images including the person to be authenticated.

7. The control method according to claim 2, further comprising determining whether the person to be authenticated is facing a predetermined direction relative to a camera for acquiring the image, and not extracting feature information from the object detection area if it is not determined that the person to be authenticated is facing the predetermined direction.

8. 2. The control method according to claim 1, wherein biometric information is used as the identification information.

9. The control method according to claim 1 , wherein vein characteristics are used as the identification information.

10. 2. The control method according to claim 1, wherein the camera captures an image of a space that is widening in the horizontal direction and then narrowing in the horizontal direction.

11. A sensor or a camera is mounted on a gate arranged at a predetermined position in a facility, and biometric information of a person passing through the gate is acquired based on the detection result of the sensor or the camera that detects the biometric information of the person; When authentication using the acquired biometric information of the person is successful, an image including the person passing through the gate is analyzed to generate characteristic information of the person; registering in a storage unit, in association with the identification information of the person specified from the biometric information and the generated characteristic information of the person; 2. The control method according to claim 1, further comprising the step of tracking a person moving through the facility using the registered characteristic information.

12. determining whether the person to be authenticated is facing the direction of travel of the gate with respect to a camera for acquiring the image, and generating characteristic information of the person from the target detection area when it is determined that the person to be authenticated is facing the direction of travel of the gate; the person's feature information is skeletal information of the person, 12. The control method according to claim 11, wherein the person is tracked within the facility using skeletal information of the person.

13. the facility is either a railway facility or an airport, the gate is located at the railroad facility or the airport boarding gate; The control method according to claim 11, characterized in that, when the acquired biometric information of the person has been pre-registered as a train or airplane passenger, it is determined that authentication using the biometric information of the person has been successful.

14. the facility is a store, The gate is disposed at an entrance of the store, If the acquired biometric information of the person is registered as a member of the store, it is determined that authentication using the biometric information of the person has been successful; By tracking a person moving through the store, a trajectory of the person from entering the store to leaving the store is identified; The control method according to claim 11, wherein information is generated that associates the trajectory of the identified person with identification information of the person.

15. On the computer, A control program characterized by executing a process to control a target detection area for detecting a target in an image when multiple people are detected from an image containing a target person whose identity has been successfully authenticated using identification information.

16. An information processing device characterized by having a detection area control unit that, when multiple people are detected from an image including an authentication target who has been successfully authenticated using identification information, controls a target detection area for detecting the target in the image.