Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises: a plurality of imaging means that each capture an image including an object; an authentication means that authenticates the object using the image; a tracking means that tracks the object using the image; and a control means that, if authentication using the image captured by one imaging means among the plurality of imaging means has failed, controls other imaging means on the basis of tracking information relating to the tracking. According to such an information processing device, for example, even if authentication has failed because the object is partially hidden, an appropriate image can be captured using other imaging means.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include a device that performs authentication processing using a facial image (so-called face authentication). For example, Patent Document 1 discloses a technology that performs face authentication at a gateless entrance of an office or the like.

[0003] JP 2023-136750 A

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises a plurality of imaging means for capturing an image including an object, an authentication means for authenticating the object using the image, a tracking means for tracking the object using the image, and a control means for controlling other imaging means based on tracking information regarding the tracking if the authentication using the image captured by one of the plurality of imaging means fails.

[0006] One aspect of the information processing method disclosed herein involves at least one computer capturing an image including an object using a plurality of imaging means, authenticating the object using the image, and tracking the object using the image, and if the authentication using the image captured by one of the plurality of imaging means fails, controlling the other imaging means based on tracking information regarding the tracking.

[0007] One aspect of the recording medium of this disclosure is a computer program recorded on at least one computer that causes the computer to execute an information processing method, which involves capturing images including an object using multiple imaging means, authenticating the object using the images, tracking the object using the images, and, if the authentication using the image captured by one of the multiple imaging means fails, controlling other imaging means based on tracking information regarding the tracking.

[0008] 1 is a block diagram showing the hardware configuration of a first information processing device. 2 is a block diagram showing the functional configuration of the first information processing device. 3 is a flowchart showing the operation flow of the first information processing device. 4 is a block diagram showing the functional configuration of a second information processing device. 5 is a flowchart showing the operation flow of the second information processing device. 6 is a block diagram showing the functional configuration of a third information processing device. 7 is a block diagram showing the functional configuration of a modified example of the third information processing device. 8 is a flowchart showing the operation flow of the third information processing device. 9 is a flowchart showing the operation flow of a control target determination operation in the third information processing device. 10 is a block diagram showing the functional configuration of a fourth information processing device. 11 is a top view showing an example of gateless authentication to which the fourth information processing device is applied. 12 is a conceptual diagram showing an example of a movement trajectory estimated by the fourth information processing device. 13 is a flowchart showing the operation flow of a priority calculation operation in the fourth information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, the first information processing device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The first information processing device 10 may also include an input device 15 and an output device 16. The first information processing device 10 also includes a camera 18. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, and camera 18 are all connected via a data bus 17. The data bus 17 may be an interface other than a data bus (e.g., a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the first information processing device 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks for executing processing related to authentication of the target are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the first information processing device 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the first information processing device 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the first information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as part of a smartphone, a tablet terminal, an earphone-type terminal, a watch-type terminal, an HMD (Head Mounted Display) terminal, etc. The input device 15 may be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the first information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display or digital signage) that can display information related to the information processing device 10. The output device 16 may also be a speaker or the like that can output information related to the information processing device 10 as audio.

[0020] The camera 18 is configured to be able to capture an image including a target. The target here is a target of the authentication process executed by the first information processing device 10. The target is typically a human, but may also include non-human animals (e.g., dogs, cats, etc.). The camera 18 may be configured to be able to capture a facial image including the face of the target. Alternatively, the camera 18 may be configured to be able to capture an iris image including the iris of the target. The camera 18 is disposed in a position where it can capture an image of an area through which the target passes, for example. The camera 18 may be capable of panning, tilting, and zooming, for example. That is, the camera 18 may be configured to have an adjustable imaging range. The type of the camera 18 is not particularly limited, and may be, for example, a visible light camera or an infrared camera.

[0021] The first information processing device 10 particularly includes a plurality of cameras 18. The number of cameras 18 is not particularly limited. The plurality of cameras 18 may be arranged so that each has a different imaging range. Furthermore, the plurality of cameras 18 may be arranged so that each captures an image of an object from a different angle. The plurality of cameras 18 are configured to be independently controllable. A method for controlling the cameras 18 will be described in detail later.

[0022] 1 may be configured as devices external to the first information processing device 10. For example, the first information processing device 10 may be configured to include a processor 11, a RAM 12, and a ROM 13, and the other devices, such as a storage device 14, an input device 15, an output device 16, and a camera 18, may be configured as external devices. Furthermore, some of the calculation functions of the first information processing device 10 may be realized by an external server, a cloud, or the like.

[0023] (Functional Configuration) Next, the functional configuration of the first information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first information processing device.

[0024] 2, the first information processing device 10 includes a plurality of cameras 18, an authentication unit 110, a tracking unit 120, and a camera control unit 130. Each of the authentication unit 110, the tracking unit 120, and the camera control unit 130 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0025] The authentication unit 110 is configured to be able to authenticate a target using an image captured by the camera 18. The authentication unit 110 may be configured to sequentially or in parallel authenticate multiple images captured by multiple cameras 18. The authentication method performed by the authentication unit 110 is not particularly limited. For example, the authentication unit 110 may extract biometric information of the target contained in the image to perform biometric authentication. More specifically, the authentication unit 110 may perform facial authentication using a facial image of the target. In this case, the authentication unit 110 may authenticate that the target is a registered user by comparing facial features of the target extracted from the image captured by the camera 18 with facial features of a registered user registered in advance. Note that the authentication unit 110 may also be configured to perform biometric authentication other than facial authentication (e.g., iris authentication, fingerprint authentication, etc.). The authentication unit 110 may also be configured to perform biometric authentication using multiple modals (so-called multimodal authentication). The authentication unit 110 may also perform facial authentication of a target tracked by the tracking unit 120. When a single image contains multiple objects, the authentication unit 110 may be configured to authenticate each of the multiple objects. The authentication unit 110 may be configured to output the authentication result. For example, the authentication unit 110 may be configured to display an image or message indicating the authentication result on a display. The authentication result by the authentication unit 110 is also configured to be output to the camera control unit 130.

[0026] The tracking unit 120 is configured to be able to track an object using images captured by the camera 18. The tracking unit 120 may be configured to sequentially or in parallel track an object using multiple images captured by multiple cameras 18. The tracking method performed by the tracking unit 120 is not particularly limited. The tracking unit 120 may be configured to first detect an object from an image and then track the detected object. The tracking unit 120 may store, for example, characteristics related to the appearance of the detected object (e.g., hairstyle or clothing) and track the object based on the stored characteristics. When multiple objects are included in a single image, the tracking unit 120 may track each of the multiple objects. The tracking unit 120 may be configured to output tracking results. For example, the tracking unit 120 may be configured to display an image or message indicating the tracking result on a display. The tracking unit 120 is also configured to output information related to tracking (hereinafter referred to as "tracking information" as appropriate). The tracking information may be, for example, information about the target being tracked, or information about the camera 18 capturing images of the target being tracked. More specifically, the tracking information may be information indicating the past or current position of the target being tracked, or an estimated future position or movement trajectory. The tracking information may also be information about the movement speed of the target being tracked. The tracking information may also be information indicating whether the target is being tracked using images captured by the camera 18. This tracking information is configured to be output to the camera control unit 130.

[0027] The camera control unit 130 is configured to be able to control the operation of the camera 18. The camera control unit 130 may be configured to be able to control the position and angle of the camera 18. For example, the camera control unit 130 may control the position of the camera 18 by moving a drone, a mobile robot, or a camera set on a rail. The camera control unit 130 may also control the angle of the camera 18 to change using a motor or the like. The camera control unit 130 may be configured to be able to control multiple cameras 18 simultaneously in parallel. The camera control unit 130 may also have a function to select the camera 18 to actually control from the multiple controllable cameras 18. The camera control unit 130 controls the camera 18 when authentication by the authentication unit 110 described above fails. Specifically, when authentication using an image captured by one camera 18 fails, the camera control unit 130 controls another camera 18 different from the first camera 18. The camera control unit 130 may control the other camera 18 to capture a new image. The camera control unit 130 particularly controls the other cameras 18 based on the tracking information input from the tracking unit 120 described above. For example, the camera control unit 130 may determine which camera 18 to control from among the multiple cameras 18 based on the tracking information. Specifically, the camera control unit 130 may select a camera 18 capable of capturing an image of a target based on tracking information (e.g., information regarding the target's location) regarding a target whose authentication has failed, and control the selected camera 18. Alternatively, the camera control unit 130 may change the control amount (e.g., the amount of change in position or angle) of the camera 18 based on the tracking information. Specifically, the camera control unit 130 may control the camera 18 to be controlled so as to point in the direction of the target based on tracking information (e.g., information regarding the target's location) regarding a target whose authentication has failed.

[0028] (Flow of Operation) Next, the flow of operation of the first information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the first information processing device.

[0029] 3, when the operation of the first information processing device 10 is started, the camera 18 first captures an image including an object (step S101). The camera 18 may be set to capture an image at all times, or may be set to capture an image when an object present within the capture range is detected.

[0030] Next, the tracking unit 110 detects an object from the image captured by the camera 18 and starts tracking the detected object (step S102). Note that if the detected object is an object that is already being tracked, the tracking unit 110 may continue tracking the object.

[0031] Next, the authentication unit 110 authenticates the target being tracked (step S103). For example, the authentication unit 110 extracts facial features from the face of the target being tracked and compares them with the features of a registered user who has been registered in advance.

[0032] If the authentication by the authentication unit 110 is successful (step S104: YES), the authentication unit 110 outputs and displays the authentication result (step S105). For example, the authentication unit 110 displays information indicating that the target is a registered user. This information may be displayed, for example, superimposed on an image captured by the camera 18.

[0033] On the other hand, if the authentication by the authentication unit 110 fails (step S104: NO), the camera control unit 130 controls another camera 18 (i.e., a camera 18 different from the camera 18 that captured the image used for authentication) (step S106). In this case, the processing from step S101 is repeated by the other camera 18 controlled by the camera control unit 130. That is, a new image may be captured by the other camera 18, and tracking and authentication may be performed using the image.

[0034] If the authentication of the target is not successful even with an image captured by another camera 18, the camera control unit 130 may control yet another camera 18 (i.e., a third camera 18 that has not been controlled so far). Alternatively, if the authentication of the target is not successful even with an image captured by another camera 18, the authentication unit 110 may output information indicating that the authentication of the target has failed.

[0035] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0036] As described with reference to FIGS. 1 to 3 , in the first information processing device 10, if authentication of the target fails, another camera 18 is controlled using tracking information. In this way, even if authentication of the target is not successful, it is possible to continue authentication by controlling another camera 18. Therefore, even if the target cannot be properly captured due to the position or angle of the camera 18 (for example, if the target's face is hidden), it is possible to capture an image of the target using the other camera 18 and authenticate the target. Furthermore, because the first information processing device 10 controls the other camera 18 based on the tracking information of the target, it is possible to control the camera 18 more appropriately than when not using tracking information.

[0037] Second Embodiment A second information processing device 10 will be described with reference to Figures 4 and 5. The second information processing device 10 differs in some configurations and operations from the first information processing device 10 described above, but other parts may be similar to the first information processing device 10. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0038] (Functional Configuration) First, the functional configuration of the second information processing device 10 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 4, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0039] 4, the second information processing device 10 includes a plurality of cameras 18, an authentication unit 110, a tracking unit 120, a camera control unit 130, and a face direction determination unit 140. That is, the second information processing device 10 includes the configuration of the first information processing device (see FIG. 2) and further includes a face direction determination unit 140. Note that the face direction determination unit 140 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0040] The face orientation determination unit 140 is configured to receive the authentication result from the authentication unit 110. The face orientation determination unit 140 is configured to determine whether or not the cause of a failure in authentication of a target by the authentication unit 110 is the target's face orientation. For example, when the authentication unit 110 performs face authentication or iris authentication, if the target's face cannot be captured by the camera 18 due to the target's face orientation (i.e., if the target's face is hidden from the camera 18), proper authentication cannot be performed and authentication fails. The face orientation determination unit 140 is configured to be able to determine whether or not such a situation has occurred. The face orientation determination unit 140 may determine the cause of authentication failure based on, for example, the target's facial features or image quality. Note that causes of authentication failure other than face orientation include, for example, the target's hairstyle or accessories (e.g., sunglasses, a mask, a hat, etc.), blurred images, or unregistered face information. The face orientation determination unit 140 may determine whether or not the cause of authentication failure is the target's face orientation by determining whether or not such causes other than face orientation exist.

[0041] Furthermore, when it is determined that the cause of authentication failure is the facial direction of the target, the facial direction determination unit 140 estimates the facial direction of the target. The facial direction determination unit 140 may estimate the facial direction of the target based on the unobstructed portion of the target in the image, the target's movement, etc. Information regarding the facial direction estimated by the facial direction determination unit 140 (hereinafter referred to as "facial direction information" as appropriate) is configured to be output to the camera control unit 130. The camera control unit 130 in the second information processing device 10 controls the camera 18 based on the facial direction information input from the facial direction determination unit 140 in addition to the tracking information input from the tracking unit 120.

[0042] Note that the face direction determination unit 140 may be configured to estimate the face direction of the target without relying on the authentication result of the authentication unit 110. For example, the face direction determination unit 140 may acquire an image of the target captured by the camera 18, estimate the face direction, and output face direction information before the authentication unit 110 performs authentication. In this case, the camera control unit 130 may control the camera 18 based on the face direction information without relying on the authentication result of the authentication unit 110. That is, the camera control unit 130 may acquire the face direction information from the face direction determination unit 140 before the authentication unit 110 performs authentication, and control the camera 18 based on the face direction information before the authentication unit 110 performs authentication.

[0043] (Operation Flow) Next, the operation flow of the second information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation flow of the second information processing device. Note that in Fig. 5, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0044] 5, when the operation of the second information processing device 10 is started, the camera 18 first captures an image including a target (step S101). Then, the tracking unit 110 detects the target from the image captured by the camera 18 and starts tracking the detected target (step S102).

[0045] Next, the authentication unit 110 authenticates the target being tracked (step S103). If the authentication by the authentication unit 110 is successful (step S104: YES), the authentication unit 110 outputs and displays the authentication result (step S105).

[0046] On the other hand, if authentication by the authentication unit 110 fails (step S104: NO), the face orientation determination unit 140 determines the cause of the authentication failure (step S201). If the cause of the authentication failure is the facial orientation of the subject (step S202: YES), the camera control unit 130 controls another camera 18 based on the tracking information and facial orientation information (step S203). In this case, the processing from step S101 may be repeated by the other camera 18 controlled by the camera control unit 130.

[0047] If the authentication failure is not due to the facial orientation of the subject (step S202: NO), the authentication unit 110 outputs and displays the authentication result (step S105). In this case, information indicating that the authentication failed is displayed. The authentication unit 110 may output and display information indicating the cause of the authentication failure determined by the facial orientation determination unit 140 (i.e., a cause other than facial orientation). Furthermore, if the cause of the authentication failure is a cause other than facial orientation, the subject may be prompted to take action to resolve the cause. For example, if the cause of the authentication failure is an attachment of the subject, a message may be output to the subject to remove the attachment.

[0048] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0049] As described with reference to Figures 4 and 5, the second information processing device 10 determines whether the cause of authentication failure is facial orientation, and if the cause is facial orientation, controls another camera 18. In this way, the other camera 18 can be controlled to capture an image of the target from a different angle, making it possible to properly perform facial authentication of the target. Furthermore, the second information processing device 10 controls the other camera 18 using facial orientation information in addition to tracking information, making it possible to more properly capture an image of the target's face. For example, by selecting and controlling the camera 18 according to the target's facial orientation, it becomes possible to easily capture an image of a hidden target's face.

[0050] <Third Embodiment> A third information processing device 10 will be described with reference to Figures 6 to 9. The third information processing device 10 differs in some configurations and operations from the first and second information processing devices 10 described above, but other parts may be similar to the first and second information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0051] (Functional Configuration) First, the functional configuration of the third information processing device 10 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 6, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0052] 6, the third information processing device 10 includes a plurality of cameras 18, an authentication unit 110, a tracking unit 120, and a camera control unit 130. In particular, the camera control unit 130 in the third information processing device 10 includes a control object determination unit 135. Note that the third information processing device 10 may be configured to include a face direction determination unit 140, similar to the second information processing device (see FIG. 4). In the case where the face direction determination unit 140 is included, an operation by the control object determination unit 135 may be executed when the face direction determination unit 140 determines that the cause of authentication failure is face direction.

[0053] The control target determination unit 135 is configured to be able to determine, from among the multiple cameras 18, the camera 18 to be controlled by the camera control unit 130. Specifically, the control target determination unit 135 determines whether or not the camera 18 is in a state in which it may be controlled (hereinafter referred to as a "controllable state" as appropriate) based on tracking information about the other camera 18 to be controlled. For example, the control target determination unit 135 may determine whether or not the other camera 18 to be controlled is in a controllable state depending on whether or not the other camera 18 is tracking an object. A more specific method for determining whether or not the other camera 18 is in a controllable state will be described in detail later.

[0054] (Variation of Functional Configuration) Next, a functional configuration according to a variation of the third information processing device 10 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration according to a variation of the third information processing device. Note that in Fig. 7, elements similar to those described in Fig. 6 are assigned the same reference numerals.

[0055] 7, the modified example of the third information processing device 10 includes a plurality of cameras 18, an authentication unit 110, a tracking unit 120, a camera control unit 130, and a control determination unit 135. In other words, the control target determination unit 135 according to the modified example is provided independently of the camera control unit 130.

[0056] The control object determination unit 135 according to the modified example is configured to acquire an authentication result from the authentication unit 110 and acquire tracking information from the tracking unit 120. The control object determination unit 135 according to the modified example may be configured to determine the camera 18 to be controlled by the camera control unit 130 based on the acquired authentication result and tracking information. The control object determination unit 135 according to the modified example may be configured to output control instruction information (i.e., information indicating how to control the camera 18) to the camera control unit 130. In this case, the camera 130 may control itself based on the control instruction information acquired from the control object determination unit 135.

[0057] Note that, in the embodiment described later, it is possible to employ a configuration similar to the configuration described in Fig. 7. That is, in the embodiment described later, the control object determination unit 135 may be provided independently of the camera control unit 130.

[0058] (Operation Flow) Next, the operation flow of the third information processing device 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 8, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0059] 8, when the operation of the third information processing device 10 is started, the camera 18 first captures an image including a target (step S101). Then, the tracking unit 110 detects the target from the image captured by the camera 18 and starts tracking the detected target (step S102).

[0060] Next, the authentication unit 110 authenticates the target being tracked (step S103). If the authentication by the authentication unit 110 is successful (step S104: YES), the authentication unit 110 outputs and displays the authentication result (step S105).

[0061] On the other hand, if authentication by the authentication unit 110 fails (step S104: NO), the control target determination unit 135 determines the state of the camera 18 to be controlled (step S301). Then, if the camera 18 to be controlled is in a controllable state, the controllability determination unit 135 determines that camera 18 as the control target (step S302). Thereafter, the camera control unit 130 controls another camera 18 determined as the control target based on the tracking information (step S303). In this case, the processing from step S101 may be repeated by the other camera 18 controlled by the camera control unit 130.

[0062] If the camera 18 to be controlled is not in a controllable state, the control target determination unit 135 may determine whether another camera 18 is in a controllable state. That is, the camera 18 to be controlled may be changed, and the processing from step S301 may be repeated. Alternatively, the control target determination unit 135 may cancel the determination of the control target if the camera 18 to be controlled is not in a controllable state. In this case, the series of operations may end without the camera control unit 130 controlling the other camera 18.

[0063] (Control Target Determining Operation) Next, the flow of the control target determining operation (i.e., the operation of the control target determining unit 135) in the third information processing device 10 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the control target determining operation in the third information processing device.

[0064] 9 , when the control target determination operation is started in the third information processing device 10, the control target determination unit 135 first determines a control target candidate (i.e., the camera 18 to be controlled). The control target candidate may be determined based on the tracking information output by the tracking unit 120 and the face direction information output by the face direction determination unit 140.

[0065] Next, the control target determination unit 135 determines whether the camera 18, which is a control target candidate, is not tracking an object (step S352). If the camera 18, which is a control target candidate, is not tracking an object (step S352: YES), the control target determination unit 135 determines that the camera 18, which is a control target candidate, is in a controllable state and determines the camera 18 as a control target (step S353). If there is no tracking object, even if the camera 18 is controlled, there is little chance of a problem occurring. For example, even if the position or angle of the camera 18 is changed, tracking will not be interrupted. Therefore, by determining that the camera 18 is in a controllable state when there is no tracking object, it is possible to appropriately control the camera 18 while suppressing the occurrence of inconveniences.

[0066] On the other hand, if the camera 18 that is a candidate for control is not tracking any targets (step S352: YES), the control target determination unit 135 determines whether authentication of all targets tracked by the camera 18 has been successful (step S354). If authentication of all targets tracked by the camera 18 has been successful (step S354: YES), the control target determination unit 135 determines that the camera 18 that is a candidate for control is in a controllable state and determines the camera 18 as a control target (step S353). If authentication of all targets tracked is successful, even if the camera 18 is controlled, there is little likelihood of a problem occurring. For example, since new authentication is not required, there is no delay in authentication even if the position or angle of the camera 18 is changed. Therefore, if the camera 18 is determined to be in a controllable state when authentication of all targets tracked is successful, it is possible to appropriately control the camera 18 while suppressing the occurrence of inconveniences.

[0067] On the other hand, if authentication of all tracked targets has not been successful (step S354: YES), the control target determination unit 135 determines whether the unauthenticated first target captured by the camera 18 that captured the image first (i.e., the camera 18 that captured the image in step S101 of FIG. 8 ) should be authenticated in preference to the unauthenticated second target captured by the camera 18 that is a candidate for control (step S355). If it is determined that the first target should be authenticated in preference to the second target (step S355: YES), the control target determination unit 135 determines that the camera 18 that is a candidate for control is in a controllable state and determines it as a control target (step S353). In a situation where the first target should be authenticated in preference to the second target, successful authentication of the first target takes priority, even if controlling the camera 18 that is a candidate for control may cause inconvenience. Therefore, if the camera 18 that is a candidate for control is determined to be in a controllable state when the first target should be authenticated in preference to the second target, authentication can be performed appropriately according to the priority order. Which target should be given priority in authentication can be determined by calculating a priority, for example. The method of calculating the priority will be described in detail in another embodiment below.

[0068] On the other hand, if it is determined that the first target should not be authenticated with priority over the second target (step S355: NO), the control target determination unit 135 determines that the camera 18, which is a control target candidate, is not controllable and removes it from the control targets (step S356). In this case, the control target candidate camera 18 may be changed, and the process from step S351 may be repeated. Alternatively, the control target determination unit 135 may cancel the determination of the control target.

[0069] (Technical Effects) Next, technical effects obtained by the third information processing apparatus 10 will be described.

[0070] 6 to 9, the third information processing device 10 determines whether the camera 18 that is a candidate for control is in a controllable state, and then determines the camera 18 as a candidate for control. In this way, it is possible to prevent unintended inconveniences from occurring due to the control of another camera 18.

[0071] <Fourth embodiment> A fourth information processing device 10 will be described with reference to Figures 10 to 13. The fourth information processing device 10 differs in some configurations and operations from the first to third information processing devices 10 described above, but other parts may be similar to the first to third information processing devices 10. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0072] (Functional Configuration) First, the functional configuration of the fourth information processing device 10 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the fourth information processing device. Note that in Fig. 10, the same elements as those described in Fig. 6 are denoted by the same reference numerals.

[0073] 9, the fourth information processing device 10 includes a plurality of cameras 18, an authentication unit 110, a tracking unit 120, and a camera control unit 130. In particular, the camera control unit 130 in the fourth information processing device 10 includes a priority calculation unit 136 in addition to a control target determination unit 135. Note that the fourth information processing device 10 may be configured to include a face direction determination unit 140, similar to the second information processing device (see FIG. 4).

[0074] The priority calculation unit 136 is configured to calculate priorities used by the control object determination unit 135 when determining the priority order of the cameras 18. Specifically, the priority calculation unit 136 is configured to calculate the priority of the first object and the priority of the second object in step S355 of FIG. 9 . The priority calculation unit 136 estimates the movement trajectory and movement speed of the objects to calculate the priority. Specifically, the priority calculation unit 136 estimates the movement trajectory and movement speed of the first object to calculate the priority of the first object. Furthermore, the priority calculation unit 136 estimates the movement trajectory and movement speed of the second object to calculate the priority of the second object. Note that the movement trajectory and movement speed of the objects may be estimated using, for example, multiple frames of images captured in time series. The movement trajectory may be, for example, up to several seconds in the future. A more specific method for calculating the priority will be described in detail in the operation description below.

[0075] (Gateless authentication) Next, gateless authentication to which the fourth information processing apparatus 10 is applied will be described with reference to Fig. 11. Fig. 11 is a top view showing an example of gateless authentication to which the fourth information processing apparatus is applied.

[0076] 11, the fourth information processing device 10 is configured to perform gateless authentication to authenticate a plurality of targets 50 passing through a preset authentication area. The authentication area is set, for example, at the entrance of various facilities, and targets 50 whose authentication is successful are permitted to enter the facility, while targets 50 whose authentication is unsuccessful are prohibited from entering the facility.

[0077] A plurality of cameras 18 are installed around the authentication area. The cameras 18 are installed so that the authentication area is at least partially included in their imaging range. Preferably, the cameras 18 are installed so that they each image the authentication area from different positions and at different angles. For example, cameras 18A, 18B, and 18C in Fig. 11 are installed so that they each image the authentication area from different positions and at different angles.

[0078] (Priority) Next, the priority of authentication in gateless authentication will be described with reference to Fig. 12. Fig. 12 is a conceptual diagram showing an example of a movement trajectory estimated by the fourth information processing apparatus.

[0079] As shown in FIG. 12 , the fourth information processing device 10 divides objects passing through the authentication area into three patterns and calculates priorities. Note that a predetermined line is set in the authentication area to classify the objects into each pattern. The predetermined line is a line set to determine whether or not the object wants to be authenticated. For example, an object that wants to be authenticated is likely to go some distance deep into the authentication area. On the other hand, an object that does not want to be authenticated (e.g., an object that is simply walking near the authentication area) is likely to exit the authentication area without going deep into it, even if it temporarily enters the authentication area. Therefore, if the predetermined line is set at an appropriate position in advance, it can be appropriately determined whether or not the object wants to be authenticated. Note that the predetermined line may be set, for example, about halfway through the authentication area.

[0080] The first pattern is an object whose movement trajectory falls within the imaging range, such as object 50A. For the first pattern, the movement speed of object 50 is used to calculate the remaining time until the object leaves the imaging range. Then, a priority is determined according to the calculated remaining time. Specifically, the shorter the calculated remaining time, the higher the priority value is determined. For example, for object 50 with a relatively short remaining time, if authentication is not completed quickly, it will not be possible to capture an image. Therefore, the priority related to authentication is set relatively high. On the other hand, for object 50 with a relatively long remaining time, it is possible to continue capturing images for a while. Therefore, the priority related to authentication is set relatively low. In this way, by using the remaining time until the object leaves the imaging range, it is possible to appropriately determine the priority.

[0081] The second pattern is an object, such as object 50B, whose movement trajectory crosses a predetermined line before it leaves the imaging range. For the second pattern, as with the first pattern, the remaining time until object 50 leaves the imaging range is calculated, and the shorter the calculated remaining time, the higher the priority value is determined. In the second pattern, although object 50 is expected to leave the imaging range relatively early, since its movement trajectory crosses the predetermined line, it can be determined that the object desires authentication. Therefore, by utilizing the remaining time until it leaves the imaging range, the priority can be appropriately determined. Note that the first and second patterns use the same priority calculation method, but since the movement trajectory of the second pattern leaves the imaging range, the remaining time is likely to be calculated as shorter than that of the first pattern. Therefore, the second pattern tends to have a higher priority than the first pattern.

[0082] The third pattern is an object, such as object 50C, whose movement trajectory leaves the imaging range without crossing a predetermined line. The priority of the third pattern is determined to be the lowest value. Here, the "lowest value" refers to a value lower than the priorities calculated for the first pattern and the second pattern. For example, if the priorities of the first and second patterns are set between 1 and 10, the priority of the third pattern may be 0. Note that the third pattern may be treated as essentially the lowest value by not setting a priority. In the third pattern, since the object 50 is expected to leave the imaging range without crossing a predetermined line, it can be determined that the object does not wish to be authenticated. Therefore, by setting the priority to the lowest value, it is possible to prevent interference with the authentication of other objects 50. Objects 50 classified into the third pattern may be removed from the list of objects to be authenticated upon classification.

[0083] (Priority Calculation Operation) Next, the flow of the priority calculation operation (i.e., the operation of the priority calculation unit 136) in the fourth information processing device 10 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of the priority calculation operation in the fourth information processing device.

[0084] As shown in FIG. 13, when the priority determination operation in the fourth information processing apparatus 10 is started, first, the priority calculation unit 136 estimates the movement trajectory and movement speed of the target (step S451).

[0085] Next, the priority calculation unit 136 determines whether the movement trajectory of the target falls within the image capture range (step S452). If the movement trajectory of the target falls within the image capture range (step S452: YES), the priority calculation unit 136 calculates the remaining time until the target leaves the image capture range using the movement speed of the target (step S453). Then, the priority calculation unit 136 sets a priority according to the calculated remaining time (step S454).

[0086] On the other hand, if the movement trajectory of the object does not fall within the image capture range (step S452: NO), the priority calculation unit 136 determines whether the object will leave the image capture range after crossing a predetermined line (step S455). If the object will leave the image capture range after crossing a predetermined line (step S455: YES), the priority calculation unit 136 calculates the remaining time until the object leaves the image capture range using the movement speed of the object (step S453). Then, the priority calculation unit 136 sets a priority according to the calculated remaining time (step S454).

[0087] On the other hand, if the target goes out of the imaging range without crossing the predetermined line (step S455: NO), the priority calculation unit 136 sets the priority to the lowest value (step S456). Alternatively, the priority calculation unit 136 may not set a priority.

[0088] (Technical Effects) Next, technical effects obtained by the fourth information processing apparatus 10 will be described.

[0089] 10 to 13, in the fourth information processing device 10, priorities are set based on the movement trajectory and movement speed of the target, which makes it possible to appropriately determine which target should be given priority in authentication.

[0090] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0091] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0092] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0093] (Supplementary Note 1) The information processing device described in Supplementary Note 1 is an information processing device that includes a plurality of imaging means that capture images including an object, an authentication means that authenticates the object using the images, a tracking means that tracks the object using the images, and a control means that, if the authentication using the image captured by one of the plurality of imaging means fails, controls the other imaging means based on tracking information related to the tracking.

[0094] (Supplementary Note 2) The information processing device described in Supplementary Note 2 is the information processing device described in Supplementary Note 1, further comprising a determination means for determining whether the cause of the authentication failure is the facial orientation of the subject when the authentication using the image captured by the one imaging means fails, and the control means controls the other imaging means based on the tracking information and facial orientation information regarding the facial orientation of the subject when it is determined that the cause of the authentication failure is the facial orientation of the subject.

[0095] (Supplementary Note 3) The information processing device described in Supplementary Note 3 is the information processing device described in Supplementary Note 1 or 2, further comprising a control determination means for determining whether the other imaging means is in a controllable state based on the tracking information regarding the other imaging means, and the control means controls the other imaging means when it is determined that the other imaging means is in the controllable state.

[0096] (Supplementary Note 4) The information processing device described in Supplementary Note 4 is the information processing device described in Supplementary Note 3, in which the control determination means determines that the other imaging means is in the controllable state when there is no target being tracked using an image captured by the other imaging means.

[0097] (Appendix 5) The information processing device described in Appendix 5 is the information processing device described in Appendix 4, in which the control determination means determines that the other imaging means is in a controllable state if the authentication is successful for all of the targets being tracked, even if there are targets being tracked using images captured by the other imaging means.

[0098] (Supplementary Note 6) The information processing device described in Supplementary Note 6 is the information processing device described in Supplementary Note 4 or 5, wherein the control determination means determines which of a first target included in the image captured by the one imaging means or a second target included in the image captured by the other imaging means should be given priority in the authentication when the targets being tracked using the image captured by the other imaging means include an target for which the authentication has not been successful, and the control means controls the other imaging means when it is determined that the authentication should be given priority to the first target over the second target.

[0099] (Appendix 7) The information processing device described in Appendix 7 is the information processing device described in Appendix 6, wherein the authentication means performs the authentication on an object present in a predetermined authentication area included in the imaging range of the first imaging means and the second imaging means, and the control determination means determines whether to give priority to the authentication of the first object or the second object based on a priority determined from the movement trajectory and movement speed of the object in the imaging range.

[0100] (Appendix 8) The information processing device described in Appendix 8 is the information processing device described in Appendix 7, in which the control determination means calculates the remaining time until an object whose movement trajectory falls within the imaging range leaves the authentication area using the movement trajectory and movement speed, and determines the priority as a value that becomes higher the shorter the remaining time.

[0101] (Appendix 9) The information processing device described in Appendix 9 is the information processing device described in Appendix 7 or 8, in which the control determination means calculates the remaining time until a target moves out of the imaging range after its movement trajectory has crossed a predetermined line set within the authentication area, using the movement trajectory and movement speed, and determines the priority as a value that becomes higher the shorter the remaining time.

[0102] (Appendix 10) The information processing device described in Appendix 10 is the information processing device described in any one of Appendices 7 to 9, wherein the control determination means determines the priority to be the lowest value for an object whose movement trajectory goes outside the imaging range without crossing a predetermined line set within the authentication area.

[0103] (Appendix 11) The information processing method described in Appendix 11 is an information processing method in which, by at least one computer, images including an object are captured by a plurality of imaging means, the images are used to authenticate the object, the images are used to track the object, and if the authentication using the image captured by one of the plurality of imaging means fails, the other imaging means are controlled based on tracking information related to the tracking.

[0104] (Appendix 12) The recording medium described in Appendix 12 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which involves capturing images including an object using a plurality of imaging means, authenticating the object using the images, tracking the object using the images, and, if the authentication using the image captured by one of the plurality of imaging means fails, controlling the other imaging means based on tracking information related to the tracking.

[0105] (Appendix 13) The computer program described in Appendix 13 is a computer program that causes at least one computer to execute an information processing method that captures images including an object with a plurality of imaging means, authenticates the object using the images, tracks the object using the images, and, if the authentication using the image captured by one of the plurality of imaging means fails, controls other imaging means based on tracking information related to the tracking.

[0106] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0107] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 18 Camera 50 Object 110 Authentication unit 120 Tracking unit 130 Camera control unit 135 Control object determination unit 136 Priority calculation unit 140 Face direction determination unit

Claims

1. An information processing device comprising: a plurality of imaging means for capturing images including an object; authentication means for authenticating the object using the images; tracking means for tracking the object using the images; and control means for controlling other imaging means based on tracking information regarding the tracking when the authentication using the image captured by one of the plurality of imaging means fails.

2. The information processing device of claim 1, further comprising a determination means for determining whether the cause of the authentication failure using the image captured by the one imaging means is the facial direction of the subject when the authentication fails, and wherein the control means controls the other imaging means based on the tracking information and facial direction information regarding the facial direction of the subject when it is determined that the cause of the authentication failure is the facial direction of the subject.

3. An information processing device as described in claim 1 or 2, further comprising a control determination means for determining whether the other imaging means is in a controllable state based on the tracking information regarding the other imaging means, and the control means controls the other imaging means when it is determined that the other imaging means is in the controllable state.

4. The information processing device according to claim 3, wherein the control determination means determines that the other imaging means is in the controllable state when there is no target being tracked using an image captured by the other imaging means.

5. The information processing device of claim 4, wherein the control determination means determines that the other imaging means is in the controllable state if the authentication is successful for all of the tracking targets, even if there are targets being tracked using images captured by the other imaging means.

6. The information processing device described in claim 4, wherein the control determination means determines which of a first object included in the image captured by the one imaging means or a second object included in the image captured by the other imaging means should be given priority in the authentication when the objects being tracked using the image captured by the other imaging means include an object for which the authentication has not been successful, and the control means controls the other imaging means when it determines that the authentication should be given priority to the first object over the second object.

7. The information processing device according to claim 6, wherein the authentication means performs the authentication on an object present in a predetermined authentication area included in the imaging range of the first imaging means and the second imaging means, and the control determination means determines whether to give priority to the authentication of the first object or the second object based on a priority determined from the movement trajectory and movement speed of the object within the imaging range.

8. The information processing device according to claim 7, wherein the control determination means calculates the remaining time until an object whose movement trajectory falls within the imaging range leaves the authentication area using the movement trajectory and movement speed, and determines the priority as a value that increases as the remaining time decreases.

9. The information processing device of claim 7, wherein the control determination means calculates the remaining time until an object exits the authentication area using the movement trajectory and movement speed when the object exits the imaging range after the movement trajectory crosses a predetermined line set within the authentication area, and determines the priority as a value that increases as the remaining time decreases.

10. The information processing device according to claim 7, wherein the control determination means determines the priority to be the lowest value for an object whose movement trajectory leaves the imaging range without crossing a predetermined line set within the authentication area.

11. An information processing method comprising: using at least one computer to capture images including an object using multiple imaging means; authenticating the object using the images; tracking the object using the images; and, if the authentication using the image captured by one of the multiple imaging means fails, controlling the other imaging means based on tracking information regarding the tracking.

12. A recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which comprises capturing images including an object using multiple imaging means, authenticating the object using the images, tracking the object using the images, and, if the authentication using the image captured by one of the multiple imaging means fails, controlling other imaging means based on tracking information regarding the tracking.

Images