Information processing device, information processing method, and storage medium

Abstract

The device comprises a memory that stores one or more instructions; and a processor configured to execute the one or more instructions to acquire first images including one or more faces captured at a first time by one or more cameras, each corresponding to a person among one or more persons, detect a first person from the one or more persons by performing a facial recognition operation on the one or more faces in the images acquired from the cameras, detect a body region corresponding to the facial region of the first person, track the body region in a second image captured by the one or more cameras at a second time after the first time, and output information for controlling the barrier to open based on a determination that the body region is approaching a barrier.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a storage medium, and more particularly, to an information processing device, an information processing method, and a storage medium that perform person authentication using an image captured by a camera. [Background technology]

[0002] In recent years, facilities such as airports are provided with gate devices that control passages through which users may pass, such as security inspection areas. In conventional gate devices, face recognition operations may be performed by comparing a face image acquired from a camera with a face image acquired from a registered image such as a passport. However, in conventional gate devices, in order to perform face recognition operations, facility staff must operate a camera to capture face images of passengers or visitors. This method of performing face recognition introduces delays because each passenger or visitor must stop at the camera one by one. Therefore, it is difficult for passengers or visitors to pass through the gate device quickly. Summary of the Invention [Problem to be solved by the invention]

[0003] In view of the above-mentioned problems, one or more aspects of the present disclosure provide a gate device, a control method for the gate device, and a storage medium for enabling a user, such as a passenger or visitor, to pass through the gate device without having to stop in front of a camera, thus reducing user delays. [Means for solving the problem]

[0004] According to one aspect of the present disclosure, an apparatus is provided that includes a memory storing one or more instructions; and a processor configured to execute the one or more instructions to acquire first images including one or more faces captured by one or more cameras at a first time, each corresponding to a person of one or more persons, detect a first person from the one or more persons by performing a facial recognition operation on the one or more faces in the images acquired from the cameras, detect a body region corresponding to a facial region of the first person, track the body region in a second image captured by the one or more cameras at a second time after the first time, and output information for controlling the barrier to open based on a determination that the body region is approaching a barrier.

[0005] According to another aspect of the present disclosure, there is provided an apparatus comprising: a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain a first image including a plurality of faces captured by one or more cameras at a first time, each corresponding to a plurality of persons; detect a first person from the plurality of persons by performing a facial recognition operation on the plurality of faces in the first image; detect a body region corresponding to the facial region of the first person; track the body region in a second image captured by the one or more cameras at a second time after the first time; and output information for controlling the barrier to open based on a determination that the body region is approaching a barrier.

[0006] According to another aspect of the present disclosure, an apparatus is provided comprising: a memory storing one or more instructions; and a processor configured to execute the one or more instructions to acquire an image captured by a camera including a plurality of faces, each corresponding to a plurality of persons approaching a barrier, each including a feature, detect a first person as the person among the plurality of persons closest to the camera based on a size of the feature for the first person, and output information for controlling the barrier based on a result of a comparison between the face information of the first person from the acquired image and a plurality of registered information each corresponding to a person registered prior to acquisition of the image. [Brief description of the drawings]

[0007] [Figure 1A] FIG. 2 illustrates an example of a gate device according to an embodiment. [Figure 1B] FIG. 2 illustrates an example of a gate device according to an embodiment. [Diagram 2] 1 is a functional block diagram of an information processing device 10 according to the present embodiment. [Diagram 3] 4 is a flowchart showing an outline of a process performed by the information processing device according to the present embodiment. [Figure 4A] 4 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 3. [Figure 4B] 4 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 3. [Figure 4C] 4 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 3. [Figure 4D] 4 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 3. [Diagram 5] 11 is an example of a data structure according to another embodiment. [Figure 6] 13 is a flowchart showing a process performed by an information processing device according to yet another embodiment. [Figure 7A]7 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 6. [Figure 7B] 7 is a diagram illustrating an example of a process performed by the information processing device according to the embodiment of FIG. 6. [Figure 8] 13 is a flowchart showing an overview of a process performed by an information processing device according to still another embodiment. [Figure 9] FIG. 1 is a block diagram showing an example of a hardware configuration of a gate control device 1 according to an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] One or more exemplary embodiments of the present disclosure will be described below with reference to the drawings. Throughout the drawings, identical or corresponding components are designated by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0009] 1A and 1B show an example of a gate device 1 according to a non-limiting exemplary embodiment. As shown in FIG. 1A, the gate device 1 may include a gate 2 provided in a passage 3 to control traffic through the passage 3. The gate device 1 may further include a camera 4 and an information processing device 5 for performing face recognition of a user walking through the passage 3. According to an embodiment, the information processing device 5 may acquire a first image of a person or user (U) at a first position P1 while the person is approaching the gate 2. The first image may be captured at a first time (t1) by the camera 4. The first image may include one or more faces of the person near the gate device, each of the one or more faces corresponding to one user among the multiple users.

[0010] According to an embodiment, the information processing device 5 may be configured to detect a first user among a plurality of users by performing a facial recognition operation on a face in a first image. For example, the facial recognition operation may be part of an authentication operation performed in whole or in part by the information processing device 5 configured to authenticate the face of a user, called authenticated user (U), as a person authorized to pass through the gate.

[0011] According to an embodiment, the information processing device 5 may detect an area of ​​the authenticated user (U) as a body area. This body area may be associated with the face area of ​​the authenticated user (U). By associating the body area of ​​the authenticated user (U) with the corresponding face area, the authenticated user (U) may be tracked by tracking the body area. When the user is at a second position P2 of the passage, a second image may be acquired by the camera 4. The second image may be captured at a second time (t2) following the first time (t1) and may be captured by the same camera 4 or may be captured by a different imaging device than the camera 4 that captured the first image. The second image may include the body area but may not include the face area of ​​the authenticated user (U). This may be due to the movement of the authenticated user (U) between time t1 and time t2, or it may be due to the fact that the face of the authenticated user (U) is not sufficiently facing the camera 4 at time t2, or the face is unclear in the second image, or is not present in the second image at time t2. Since the body part region is associated with the authenticated user (U), the authenticated user (U) can be tracked in the second image even if the face region is not present in the second image. The information processing device 5 may output information for controlling the gate 2 to open based on a determination that the tracked body region is approaching the barrier. In this way, the information processing device 5 does not need to repeatedly authenticate the user after the user is authenticated as an authorized person. For example, since the body region of the authenticated user is tracked, the gate 2 may be opened even if the person is not directly facing the camera 4 at the position P2.

[0012] 1B, the information processing device 5 may be configured to prevent the gate 1 from opening if the information processing device 5 determines that the body region tracked in the second image at the position P2 is not associated with a facial region of an authenticated user in the first image. For example, if the information processing device 5 is unable to perform a facial recognition operation on a face in the first image acquired from the camera 4, the information processing device 5 may not track a body region corresponding to the face in the first image. Thus, the information processing device 5 will not open the gate 1 unless an additional facial recognition operation is performed on the user and the user is authenticated.

[0013] FIG. 2 is a functional block diagram of an information processing device 10 according to an exemplary embodiment. The information processing device 10 may be a part of an automatic gate device 1. According to an embodiment, the information processing device may include one or more processors (such as the CPU 102 or other processors in FIG. 9) and a memory (such as the RAM 104 or other memory in FIG. 9). The information processing device 10 may have an image acquisition unit 121, a detection unit 122, an extraction unit 123, an association unit 125, a controller 126, a display image generation unit 127, and a storage unit 129. According to an embodiment, the CPU 102 may execute one or more instructions stored in the memory to implement various units. Although the units and the operations performed by the units are provided for illustration, the present disclosure is not limited to the units or the operations performed by the units. According to other embodiments, the novel features of the present disclosure may be performed by various combinations of units, including the above-mentioned units and other units.

[0014] According to one embodiment, the image acquisition unit 121 may acquire an image from the camera 4, the detection unit 122 may detect a person in the image acquired from the camera 4, the extraction unit 123 may extract features such as facial features and physical features in the image, the association unit 125 may associate the extracted facial features and physical features with each other, the controller 126 may control the gate 2 to open and close, the display image generation unit 127 may generate information to be displayed on the display 110 (shown in FIG. 9 ), and the storage unit 129 may store the information.

[0015] 3 is a flowchart showing an overview of the processing performed by the information processing device 10 according to an exemplary embodiment. The overview of the processing performed by the information processing device 10 will be described along the flowchart of FIG. 3 with reference to FIGS.

[0016] In S310 of FIG. 3, the information processing device 10 may acquire a first image from the camera 4. According to an embodiment, the method of acquiring the first image may include the following operations performed by the image acquisition unit 121. For example, the image acquisition unit 121 acquires a first image from the camera 4. This process may correspond to the diagram of FIG. 4A. FIG. 4A shows a first image captured at a first time and stored in the storage unit 129. According to an embodiment, the acquired first image may include a face of a recognition target. The target may be approaching the gate 1 (shown in FIG. 1A and FIG. 1B). According to another embodiment, the acquired first image may include a plurality of faces among a plurality of recognition targets, each of the plurality of faces corresponding to the target. According to an embodiment, the recognition target may be a person.

[0017] In S320 of Fig. 3, the information processing device 10 detects a person in the first image. This process corresponds to the diagram of Fig. 4B. According to one embodiment, the information processing device 10 may detect a person by performing face recognition on the first image.

[0018] The method for performing face recognition may include the following operations performed by the detection unit 122 and the extraction unit 123. For example, the extraction unit 123 extracts a facial feature from a first image acquired by the image acquisition unit 121, and the detection unit 122 acquires the facial feature extracted by the extraction unit 123, and detects whether the facial feature matches one registered facial feature among a plurality of registered facial features. Each of the plurality of registered facial features may correspond to a face of a person among a plurality of persons previously registered. The registered facial features may be stored in the storage unit 129.

[0019] In S330, the information processing device 10 tracks the body region in the second image captured at the second time. This process corresponds to the diagram of Fig. 4C. According to an embodiment, the information processing device 10 may track the body region in the second image by associating the face region in the first image with the body region in the first image and matching the body region in the second image with the body region in the first image.

[0020] According to one embodiment, the method for performing body tracking may include the following operations performed by the image acquisition unit 121, the detection unit 122, the extraction unit 123 and the association unit 125. For example, the extraction unit 123 may extract a body region of a subject in a first image. The operation of extracting a body region may be performed after the detection unit detects a match between a facial feature in the first image and a registered facial feature. According to another embodiment, the operation of extracting a body region may be performed before the detection unit detects a match between a facial feature in the first image and a registered facial feature, or simultaneously while the detection unit detects a match between a facial feature in the first image and a registered facial feature.

[0021] According to one embodiment, the association unit 125 associates the extracted body region with a face region in the first image. For example, the association unit 125 determines that the body region corresponds to the face region by analyzing one or more characteristics of the face region and the body region. For example, if the association unit 125 determines that one or more characteristics of the face region are similar to one or more characteristics of the body region, the association unit 125 may associate the face region with the body region. According to another embodiment, the association unit 125 may determine that the body region corresponds to the face region based on the proximity of the face region and the body region.

[0022] According to one embodiment, the association unit 125 may store the associated face and body regions in the storage unit 129. For example, as shown in Figure 5, the storage unit 129 may store the face and body regions in association with the identity of the detected person. The associated face and body regions may be temporarily stored in the storage unit 129.

[0023] According to one embodiment, the image acquisition unit 121 may acquire a second image captured by the camera 4 at a second time. The second time is different from the first time. According to one embodiment, the second time is after the first time. According to one embodiment, the second image may be captured by an image capture device, such as a camera, different from the camera capturing the first image.

[0024] According to an embodiment, the extraction unit 123 may extract features of the body region in the second image, and the detection unit 122 may obtain the features of the body region in the second image from the extraction unit 123, and detect whether the features of the body region in the second image match the features of the body region stored in the storage unit 129. Thus, if the body region in the second image and the body region obtained from the storage unit 129 match, the detection unit 122 may obtain the identification of the object previously detected in the first image. Thus, the information processing device 10 may track the object using the extracted body regions in the first image and the second image.

[0025] According to another embodiment, the information processing device 10 may perform a body tracking operation when the facial features in the second image cannot be detected. This process may correspond to the diagram of FIG. 4D. For example, the extraction unit 123 may extract the facial features in the second image, and the detection unit 122 may obtain the facial features in the second image from the extraction unit 123 and determine that the facial features do not have enough information to perform face recognition. For example, if a person approaching the camera does not face the camera directly, the detection unit 122 may determine that the person's face could not extract enough facial features. According to an embodiment, the detection unit 122 may determine that the information is insufficient for face detection if the person's face is deviated from the camera by more than 45 degrees.

[0026] In S340, the information processing device 10 may output information for controlling the gate 2 based on the tracked movement of the body region. This process may correspond to the diagrams of FIG. 1A and FIG. 1B. According to an embodiment, the method for controlling the gate may be performed by the controller 126. For example, the controller 126 may obtain information indicating whether the features of the body region in the second image and the features of the body region obtained from the storage unit 129 match from the detection unit 122. If there is a match, the controller 126 may output a control signal for opening the gate 2 as shown in FIG. 1A. Meanwhile, if there is no match, the controller 126 may keep the gate 2 in a closed state as shown in FIG. 1B.

[0027] 4A to 4D show an example of processing performed by an information processing device according to this embodiment. In Fig. 4A, the information processing device 1 captures an image 401. According to one embodiment, the captured image may be displayed on a display 402.

[0028] In Fig. 4B, the information processing device 10 may detect a person in the first image. According to an embodiment, the information processing device 10 may detect the person by performing face recognition on the first image. For example, the information processing device may extract a facial feature of a face region 403 in the first image and detect whether the facial feature matches one registered facial feature among a plurality of registered facial features.

[0029] 4C, the information processing device 10 may associate the face region 403 with the body region 404 in the first image. According to an embodiment, the information processing device 10 may store the associated face region 403 and body region 404 in the storage unit 129.

[0030] 4D, the information processing device 10 may track the body region 405 in the second image by matching the features of the body region 405 in the second image with the features of the body region in the first image stored in the storage unit 29. In this way, the information processing device 10 may perform a body tracking operation even if the facial features 406 in the second image are not possible.

[0031] 5 shows a data structure that associates facial feature information 501 and physical feature information 502 with each other. For example, the facial feature information 501 and the physical feature information 502 may be associated with each other by identification information 503 and stored in the storage unit 129. The identification information 503 may be used to further identify whether the associated facial feature information 501 and the physical feature information 502 correspond to an authenticated person.

[0032] 6 is a flowchart showing an overview of a process performed by the information processing device 10 according to another exemplary embodiment. The overview of the process performed by the information processing device 10 will be described along the flowchart of FIG. 6 with reference to FIG. 1 and FIG.

[0033] In S610 of FIG. 6, the information processing device 10 may acquire a first image from the camera 4 (as shown in FIG. 1A and FIG. 1B). According to an embodiment, the method for acquiring the first image may include the following operations performed by the image acquisition unit 121. For example, the image acquisition unit 121 may acquire the first image from the camera 4. The acquired first image may include a plurality of faces among a plurality of recognition targets, each of the faces corresponding to a target. According to an embodiment, the recognition target may be a person.

[0034] In S620 of Fig. 6, the information processing device 10 may identify a person in front in the first image, that is, a person who is closest to the imaging device among multiple people. This process may correspond to the diagrams of Fig. 7A and Fig. 7B.

[0035] According to an embodiment, the method for identifying a person in front may include the following operations performed by the detection unit 122 and the extraction unit 123. For example, the extraction unit 123 may extract a plurality of facial features, each of which corresponds to one of a plurality of faces in the first image acquired by the image acquisition unit 121. According to an embodiment, the detection unit 122 may acquire the plurality of facial features extracted by the extraction unit 123, and identify a facial feature corresponding to a person closest to the camera. Among the plurality of people approaching the camera, the person closest to the camera may be identified as a person in front. According to an embodiment, the detection unit 122 may identify a person in front by comparing the sizes corresponding to each of the plurality of facial features. For example, the detection unit 122 may compare the areas occupied by each of the plurality of facial features, and determine the facial feature with the largest area among the plurality of features as the facial feature closest to the camera. According to another embodiment, the detection unit 122 may compare the distance between two feature points in each of the plurality of facial features, and determine the facial feature with the largest distance among the plurality of features as the facial feature closest to the camera. For example, the distance may be the distance between two eyes in a facial feature.

[0036] In S630, the information processing device 10 may perform face recognition on the person in front by comparing the information of the facial features of the person in front with the multiple registered information. According to an embodiment, the method for performing face recognition may include the following operations performed by the detection unit 122. For example, the detection unit 122 may obtain the facial features of the person in front extracted by the extraction unit 123, and detect whether the facial features match one registered facial feature among the multiple registered facial features. Each of the multiple registered facial features may correspond to a face of a person among the multiple previously registered people. The registered facial features may be stored in the storage device 106.

[0037] In S640, the information processing device 10 may output information for controlling the gate 114 based on the result of the face recognition. This process may correspond to the diagrams of FIG. 1A and FIG. 1B. According to an embodiment, the method for controlling the gate may be performed by the controller 126. For example, the controller 126 may obtain information indicating whether the facial features of the person in front match the multiple facial features previously registered from the detection unit 122. If there is a match, the controller 126 may output a control signal for opening the gate 2 as shown in FIG. 1A. On the other hand, if there is no match, the controller 126 may keep the gate 2 in a closed state as shown in FIG. 1B.

[0038] 7A and 7B show an example of a process performed by an information processing device to identify a person in front according to this embodiment.

[0039] In Fig. 7A, the information processing device 10 may acquire an image 701 and extract a plurality of facial features (702a, 702b, and 702c), each of which corresponds to one of a plurality of faces in the image 701. The information processing device 10 may identify a person in front by comparing the size of the feature corresponding to each of the plurality of facial features (702a, 702b, and 702c). For example, as shown in Fig. 7B, the information processing device 10 may compare the area occupied by each of the plurality of facial features and determine the facial feature 702b having the largest area among the plurality of features as the facial feature closest to the camera.

[0040] Fig. 8 is a flowchart showing an outline of processing performed by the information processing device 10 according to another embodiment. The outline of processing performed by the information processing device 10 will be described along the flowchart of Fig. 8 with reference to Figs.

[0041] In S810 of FIG. 8, the information processing device 10 may acquire a first image from the camera 4. According to an embodiment, the image acquisition unit 121 may acquire a first image from the camera 4. The acquired first image may be captured at a first time and stored in the storage unit 129. According to an embodiment, the acquired first image may include a face of a recognition target. The target may be approaching the gate 2. According to another embodiment, the acquired first image may include a plurality of faces of a plurality of recognition targets, each of the faces corresponding to a target. According to an embodiment, the recognition target may be a person.

[0042] In S820 of FIG. 8, the information processing device 10 may identify a person in front in the first image. According to an embodiment, the detection unit 122 may extract a plurality of face regions, each of which corresponds to one of a plurality of faces in the first image acquired by the image acquisition unit 121. According to an embodiment, the detection unit 122 may acquire a plurality of face regions extracted by the extraction unit 123, and identify a face region corresponding to a person closest to the camera. Among a plurality of people approaching the camera, the person closest to the camera may be identified as a person in front. According to an embodiment, the detection unit 122 may identify a person in front by comparing sizes corresponding to each of the plurality of face regions. For example, the detection unit 122 may compare the areas occupied by each of the plurality of face regions, and determine the face region having the largest area among the plurality of features as the face region closest to the camera. According to another embodiment, the detection unit 122 may compare the distance between two feature points in each of the plurality of face regions, and determine the face region having the largest distance among the plurality of regions as the face region closest to the camera. For example, the distance may be the distance between two eyes in a face region.

[0043] 8, the information processing device 10 may extract facial features and physical features of a person in front of the first image. According to another embodiment, the information processing device 10 may extract each of a plurality of facial features and physical features from the first image.

[0044] In S840 of FIG. 8, the information processing device 10 may perform face recognition on the facial features of the forward person. For example, the extraction unit 123 may extract the facial features of the forward person in the acquired first image, and the detection unit 122 may obtain the facial features extracted by the extraction unit 123 and detect whether the facial features match one registered facial feature among the multiple registered facial features. Each of the multiple registered facial features may correspond to a face of a person among the multiple persons previously registered. The registered facial features may be stored in the storage device 106.

[0045] 8, the information processing device 10 may extract physical features of a person in front of the first image and associate the facial features with the physical features. According to one embodiment, the operation of extracting the physical features may be performed after the face recognition operation in S840. According to another embodiment, the operation of extracting the physical features may be performed before the detection unit detects a match between the facial features in the first image and the registered facial features, or simultaneously while the detection unit detects a match between the facial features in the first image and the registered facial features.

[0046] According to one embodiment, the association unit 125 may associate the extracted physical feature with a facial feature in the first image. For example, the association unit 125 determines that the physical feature corresponds to the facial feature by analyzing one or more characteristics of the facial feature and the physical feature. For example, if the association unit 125 determines that one or more characteristics of the facial feature are similar to one or more characteristics of the physical feature, the association unit 125 may associate the facial feature with the physical feature. According to another embodiment, the association unit 125 may determine that the physical feature corresponds to the facial feature based on the proximity of the facial feature and the physical feature.

[0047] According to one embodiment, the association unit may store the associated facial and physical features in the storage unit 129. For example, as shown in Figure 5, the storage unit 129 may store the facial and physical features together with the identity of the detected person. According to one embodiment, the associated facial and physical features may be temporarily stored in the storage unit 129.

[0048] In S860 of FIG. 8, the information processing device 10 may acquire a second image. According to an embodiment, the image acquisition unit 121 may acquire a second image captured by the camera 4 at a second time. The second time is different from the first time. According to an embodiment, the second time is after the first time. According to an embodiment, the second image may be captured by a camera different from the camera capturing the first image. According to another embodiment, the acquired first image may include a plurality of faces among a plurality of recognition targets, each of the faces corresponding to a target. According to an embodiment, the recognition target may be a person.

[0049] 8, the information processing device 10 may extract physical features from the second image. According to an embodiment, the extraction unit 123 may extract the physical features in the second image, and the detection unit 122 may obtain the physical features in the second image from the extraction unit 123.

[0050] 8, the information processing device 10 may compare the physical features extracted from the second image with the physical features stored in the storage unit 129. According to one embodiment, the detection unit 122 may compare the physical features in the second image with the physical features stored in the storage unit 129.

[0051] In S890 of FIG. 8, the information processing device 10 may identify a previously recognized facial feature based on the result of the comparison in S880. According to an embodiment, if the physical feature in the second image matches the physical feature acquired from the storage unit 129, the detection unit 122 may obtain the identification of the target previously recognized in the first image. Thus, the information processing device 10 may identify a previously recognized facial feature. On the other hand, if the physical feature in the second image does not match any of the physical features acquired from the storage unit 129, the detection unit 122 may determine that the physical feature in the second image does not correspond to the facial feature previously recognized in the first image.

[0052] 8, the information processing device 10 may control to open the gate 114 if a facial feature previously recognized in S890 is identified. According to an embodiment, the controller 126 may obtain information from the detection unit 122 indicating whether the physical feature in the second image matches the physical feature obtained from the storage unit 129. If there is a match, the controller 126 may output a control signal to open the gate 2.

[0053] 8, the information processing device 10 may enable closing the gate 2 if the facial feature previously recognized in S890 is not identified. According to one embodiment, the controller 126 may maintain the gate 2 in a closed state if the physical features in the second image do not match any of the physical features obtained from the storage unit 129.

[0054] 9 is a block diagram showing an example of a hardware configuration of the automatic gate device 1. The automatic gate device 1 is a device that can automatically perform face recognition on a person approaching the gate, and can allow a person to pass through without requesting the person to stop for authentication, even if the person does not always face the camera while approaching the gate.

[0055] 9, the automatic gate device 1 includes a CPU 102, a RAM 104, a storage device 106, an input device 108, a display 110, a camera 112, a gate 114, and a communication unit 116. The CPU 102, the RAM 104, the storage device 106, the input device 108, the display 110, the camera 112, the gate 114, and the communication unit 116 are connected to a bus line 118.

[0056] CPU 102 operates by executing a program stored in storage device 106, and may function as a control unit that controls the overall operation of automatic gate device 1. Furthermore, CPU 102 may execute an application program stored in storage device 106 to perform various processes as automatic gate device 1. RAM 104 may provide memory fields necessary for the operation of CPU 102.

[0057] The storage device 106 may be configured with a storage medium such as a non-volatile memory or a hard disk drive, and may function as a storage unit. The storage device 106 may store programs executed by the CPU 102, data referenced by the CPU 102 when the programs are executed, and the like.

[0058] The input device 108 may be, for example, a touch panel built into the display 110. The input device 108 may function as an input unit that accepts input from a user.

[0059] The display 110 may function as a display unit that displays various windows to a user who uses the automatic gate device. For example, the display 110 may display a guide window that shows how to use the automatic gate device 1, a notification window for the user, and the like.

[0060] The camera 112 may capture an image of one person or multiple people. The image may include the face and body regions of one or multiple people. For example, the camera 112 may be a digital camera that captures an image of the area in front of the automatic gate device 1, and captures the face of the user 1 and obtains the facial image when the face of the user standing in front of the automatic gate device 1 is detected in the images captured continuously or periodically.

[0061] According to one embodiment, when personal authentication of a person is successful in the automatic gate device 1, the gate 114 changes from a closed state for standby mode that prevents the person from passing through to an open state that allows the person to pass through. The type of gate 114 is not particularly limited, and may be, for example, a flapper gate in which one or more flappers provided on one or both sides of a passageway open and close, or a turnstile gate in which three bars rotate.

[0062] The communication unit 116 may be connected to a network and may transmit and receive data via the network. The communication unit 116 communicates with a server or the like under the control of the CPU 102.

[0063] The present disclosure is not limited to the above-described exemplary embodiments, and can be modified as appropriate without departing from the spirit of the present disclosure.

[0064] In each of the above-described exemplary embodiments, an information processing device and a system used for face recognition for gate control have been described as examples. However, the present disclosure can be applied to fields such as face recognition and body tracking other than gate control by appropriately modifying the configuration of one or more exemplary embodiments.

[0065] The scope of one or more exemplary embodiments also includes a processing method for storing a program for operating the configuration of the exemplary embodiment to realize the functions of the above-mentioned exemplary embodiment in a storage medium, and reading the program stored in the storage medium as a code and executing it on a computer. That is, a computer-readable storage medium is also included in the scope of each exemplary embodiment. Furthermore, not only the storage medium on which the above-mentioned program is stored, but also the program itself is included in each exemplary embodiment. Furthermore, one or more components included in the above-mentioned exemplary embodiment may be circuits such as application specific integrated circuits (ASICs) and field programmable gate arrays (FPGAs) configured to realize the functions of each component.

[0066] Examples of storage media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, compact disks (CD)-ROMs, magnetic tapes, non-volatile memory cards, ROMs, etc. Furthermore, the scope of each of the exemplary embodiments is not limited to examples in which processing is performed by individual programs stored in storage media, but includes examples in which processing is performed by operating on an operating system (OS) and in cooperation with other software or functions of an add-in board.

[0067] Services implemented by the functionality of one or more of the exemplary embodiments described above may be provided to users in the form of Software as a Service (SaaS).

[0068] It should be noted that the above-described exemplary embodiments are merely illustrative of embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these exemplary embodiments. In other words, the present disclosure can be carried out in various forms without departing from its technical concept or main features.

[0069] Moreover, the following notes may fully or partially describe the above-described exemplary embodiments without being limited thereto.

[0070] (Appendix 1) a memory storing one or more instructions; Executing said one or more instructions obtaining a first image including one or more faces captured by one or more cameras at a first time, each of the first images corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the image captured from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; Based on the determination that the body region is approaching a barrier, output information for controlling the barrier to be opened. A processor configured as An apparatus comprising:

[0071] (Appendix 2) performing the face recognition operation, extracting facial features from the first image corresponding to the one or more faces; Detecting whether the facial feature matches a registered facial feature among a plurality of registered facial features. 2. The apparatus of claim 1, comprising:

[0072] (Appendix 3) Tracking the body region in the second image Associating the face region in the first image with a body region in the first image; matching the body region in the second image with the body region in the first image; 2. The apparatus of claim 1, comprising:

[0073] (Appendix 4) Outputting information for controlling the barrier, 2. The apparatus of claim 1, further comprising: outputting a control signal to open the gate based on a match between features of the body region in the second image and features of one of a plurality of body regions stored in storage.

[0074] (Appendix 5) 5. The apparatus of claim 4, wherein each of the plurality of body regions is pre-associated with a respective facial region from one or more previously captured images.

[0075] (Appendix 6) 2. The apparatus of claim 1, wherein the one or more faces include a plurality of faces captured by the one or more cameras at the first time, the plurality of faces corresponding respectively to a plurality of persons.

[0076] (Appendix 7) 2. The apparatus of claim 1, wherein the body region is detected after the first person is detected by facial recognition.

[0077] (Appendix 8) 2. The apparatus of claim 1, wherein the body region is detected before the first person is detected by facial recognition.

[0078] (Appendix 9) 2. The apparatus of claim 1, wherein the body region is detected at the same time that the first person is detected by facial recognition.

[0079] (Appendix 10) a memory storing one or more instructions; Executing said one or more instructions acquiring an image captured by a camera including a plurality of faces, each of the faces including a feature, corresponding to a plurality of people approaching the barrier; Detecting a first person as a person of the plurality of people closest to the camera based on a size of the feature for the first person; a processor configured to output information for controlling the barrier based on a result of a comparison between the face information of the first person from the captured image and a plurality of registered information each corresponding to a person registered before capturing the image; An apparatus comprising:

[0080] (Appendix 11) 11. The apparatus of claim 10, wherein the size of the feature corresponds to an area of ​​a face region.

[0081] (Appendix 12) 11. The apparatus of claim 10, wherein the size of the features corresponds to eye distance.

[0082] (Appendix 13) obtaining a first image including one or more faces captured by one or more cameras at a first time, each image corresponding to a person of the one or more persons; Detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the image captured from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; A method comprising:

[0083] (Appendix 14) On the computer, obtaining a first image including one or more faces captured by one or more cameras at a first time, each image corresponding to a person of the one or more persons; Detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the image captured from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; A program to execute.

[0084] (Appendix 15) acquiring an image captured by a camera including a plurality of faces, each of the faces including features, corresponding to a plurality of people approaching the barrier; Detecting a first person as a person of the plurality of people closest to the camera based on a size of the feature for the first person; outputting information for controlling the barrier based on a result of comparison information between the face of the first person from the captured image and a plurality of registered information each corresponding to a person registered before capturing the image; A method comprising:

[0085] (Appendix 16) On the computer, acquiring an image captured by a camera including a plurality of faces, each of the faces including features, corresponding to a plurality of people approaching the barrier; Detecting a first person as a person of the plurality of people closest to the camera based on a size of the feature for the first person; outputting information for controlling the barrier based on a result of comparing the face information of the first person from the captured image with a plurality of registered information each corresponding to a person registered before capturing the image; A program to execute.

[0086] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 910,751, filed October 4, 2019, the disclosure of which is incorporated herein in its entirety.

Claims

1. a memory storing one or more instructions; Executing the one or more instructions, acquiring a first image including one or more faces captured by one or more cameras at a first time, each of the first images corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; Based on the determination that the body region is approaching a barrier, output information for controlling the barrier to be opened. A processor configured as Equipped with Tracking the body region in the second image Associating the face region in the first image with a body region in the first image; matching the body region in the second image with the body region in the first image; 13. An apparatus comprising:

2. a memory storing one or more instructions; Executing the one or more instructions, acquiring a first image including one or more faces captured by one or more cameras at a first time, each of the first images corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; Based on the determination that the body region is approaching a barrier, output information for controlling the barrier to be opened. A processor configured as Equipped with Outputting information for controlling the barrier, and outputting a control signal to open the barrier based on a match between a feature of the body region in the second image and a feature of one of a plurality of body regions stored in a storage unit.

3. performing the face recognition operation, extracting facial features from the first image corresponding to the one or more faces; Detecting whether the facial feature matches a registered facial feature among a plurality of registered facial features. The apparatus of claim 2 , comprising:

4. The apparatus of claim 2 , wherein each of the plurality of body regions is pre-associated with a respective face region from one or more previously captured images.

5. The apparatus of claim 1 or 2, wherein the one or more faces include a plurality of faces imaged by the one or more cameras at the first time, the plurality of faces corresponding respectively to a plurality of persons.

6. The apparatus of claim 1 or 2, wherein the body region is detected after the first person is detected by facial recognition.

7. The apparatus of claim 1 or 2, wherein the body region is detected before the first person is detected by facial recognition.

8. The apparatus of claim 1 or 2, wherein the body region is detected at the same time that the first person is detected by facial recognition.

9. The computer obtaining a first image including one or more faces captured at a first time by one or more cameras, each corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; Including, Tracking the body region in the second image Associating the face region in the first image with a body region in the first image; matching the body region in the second image with the body region in the first image; A method comprising:

10. The computer obtaining a first image including one or more faces captured at a first time by one or more cameras, each corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; Including, Outputting information for controlling the barrier, and outputting a control signal to open the barrier based on a match between a feature of the body region in the second image and a feature of one of a plurality of body regions stored in a storage unit.

11. On the computer, obtaining a first image including one or more faces captured at a first time by one or more cameras, each corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; Run the command, Tracking the body region in the second image Associating the face region in the first image with a body region in the first image; matching the body region in the second image with the body region in the first image; Including, the program.

12. On the computer, obtaining a first image including one or more faces captured at a first time by one or more cameras, each corresponding to a person of the one or more persons; detecting a first person from the one or more persons by performing a face recognition operation on the one or more faces in the first image acquired from the camera; Detecting a body region corresponding to a facial region of the first person; tracking the body region in a second image captured by the one or more cameras at a second time after the first time; outputting information for controlling the barrier to open based on a determination that the body region is approaching the barrier; Run the command, Outputting information for controlling the barrier, and outputting a control signal to open the barrier based on a match between a feature of the body region in the second image and a feature of one of a plurality of body regions stored in a storage unit.

13. acquiring a first image and a second image captured at a time later than the time the first image was captured; performing a face recognition operation on a face captured in the first image to detect a first person having the face; Associating a face region of the first person with a body region of the first person in the first image and matching the body region in the first image with a body region of the first person captured in the second image; and outputting information for controlling the barrier to be opened based on a determination that the body region in the second image is approaching a barrier. Processor.

14. acquiring a first image and a second image captured at a time later than the time the first image was captured; performing a face recognition operation on a face captured in the first image to detect a first person having the face; Associating a face region of the first person with a body region of the first person; Matching the body region in the first image with a body region of the first person captured in the second image; outputting a control signal to open the barrier based on a determination that the body region is approaching a barrier and a determination that a characteristic of the body region in the second image matches a characteristic of the body region stored in a storage unit; Processor.

Images