Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present invention comprises: an acquisition means for acquiring position information of a target and terminal information related to a terminal carried by an inspector; a determination means for determining, on the basis of the position information of the target and the terminal information, the visibility of the target from the inspector; and a generation means for generating, on the basis of the visibility determination result, a display image of the terminal related to authentication processing using biometric information of the target. By using this information processing device, it is possible to display an appropriate image on the terminal held by the inspector.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] One such device is known that performs access control without setting up a gate by performing an authentication process on an object passing through a predetermined area. For example, Patent Document 1 discloses a device that acquires an image including a predetermined area through which a person to be authenticated passes, tracks the person, and performs facial authentication if the image captured during tracking includes the person's face.

[0003] WO 2023 / 152970

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises an acquisition means for acquiring location information of a target and terminal information relating to a terminal held by a monitor, a determination means for determining the visibility of the target from the monitor based on the location information of the target and the terminal information, and a generation means for generating a display image of the terminal relating to authentication processing using biometric information of the target based on the visibility determination result.

[0006] One aspect of the information processing method disclosed herein involves using at least one computer to acquire location information of a target and terminal information regarding a terminal held by a monitor, determining the visibility of the target from the monitor based on the location information of the target and the terminal information, and generating a display image of the terminal related to authentication processing using the biometric information of the target based on the visibility determination result.

[0007] One aspect of the recording medium of this disclosure is a computer program recorded on at least one computer that causes the computer to execute an information processing method, which acquires location information of a target and terminal information regarding a terminal held by an observer, determines the visibility of the target from the observer based on the location information of the target and the terminal information, and generates a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result.

[0008] 1 is a block diagram showing the hardware configuration of a gateless authentication system. FIG. 2 is a top view showing an example of the operation of the gateless authentication system. FIG. 3 is a flowchart showing the flow of authentication operation in the gateless authentication system. FIG. 4 is a block diagram showing the functional configuration of a first information processing device. FIG. 5 is a flowchart showing the flow of display image generation processing by the first information processing device. FIG. 6 is a flowchart showing the flow of display image generation processing by the second information processing device. FIG. 7 is a top view showing an example of a case where an unauthenticated person is present outside the field of view of an observer in the gateless authentication system. FIG. 8 is a flowchart showing the flow of display image generation processing by the second information processing device. FIG. 9 is a plan view showing an example of transition of a display image generated by a third information processing device. FIG. 10 is a plan view (part 1) showing an example of a display image generated by the third information processing device. FIG. 11 is a plan view (part 2) showing an example of a display image generated by the third information processing device. FIG. 12 is a plan view (part 3) showing an example of a display image generated by the third information processing device. FIG. 13 is a plan view (part 4) showing an example of a display image generated by the third information processing device. FIG. 14 is a plan view (part 5) showing an example of a display image generated by the third information processing device. FIG. 15 is a plan view (part 6) showing an example of a display image generated by the third information processing device. FIG. 16 is a plan view (part 7) showing an example of a display image generated by the third information processing device. FIG. 8 is a plan view (part 8) showing an example of a display image generated by the third information processing apparatus.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Gateless Authentication Apparatus) First, the hardware configuration of a gateless authentication apparatus to which the first information processing apparatus is applied will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of a gateless authentication system.

[0012] 1, the gateless authentication system 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, and an output device 16. The gateless authentication system 1 also includes a user terminal 20 and an authentication device 30. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, user terminal 20, and authentication device 30 are all connected via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN or a USB).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may obtain (i.e., load) the computer program from a device (not shown) located outside the gateless authentication system 1 via a network interface. The processor 11 executes the loaded computer program to perform various processes. In particular, in this embodiment, when the processor 11 executes the loaded computer program, functional blocks related to the authentication process performed by the gateless authentication system 1 are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the gateless authentication system 1.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the gateless authentication system 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the gateless authentication system 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be, for example, a device that allows voice input, including a microphone.

[0019] The output device 16 is a device that outputs information related to the gateless authentication system 1 to the outside. For example, the output device 16 may be a display device (e.g., a display or digital signage) that can display information related to the gateless authentication system 1. The output device 16 may also be a speaker or the like that can output information related to the gateless authentication system 1 as audio.

[0020] The user terminal 20 is a terminal used by a user (specifically, a monitor) of the gateless authentication system 1. The user terminal 20 includes a display unit 201 and a terminal camera 202. The display unit 201 is a display mounted on the user terminal 20. The display unit 201 according to this embodiment is particularly configured to be capable of displaying images (described in detail below) generated by a first information processing device. The terminal camera 202 is a camera mounted on the user terminal 20. The terminal camera 202 is configured to be capable of capturing an image in the direction in which the user terminal 20 is pointed (for example, an image corresponding to the field of view of the monitor). The user terminal 20 may be configured as, for example, an HMD (Head Mounted Display). The HMD may use AR (Augmented Reality), VR (Virtual Reality), or MR (Mixed Reality). By using the HMD, it is possible to properly capture an image corresponding to the field of view of the observer and properly present the image generated by the first information processing device to the user. Note that the user terminal 20 may be a terminal other than an HMD, and may be configured as, for example, a smartphone, a tablet, a laptop computer, etc.

[0021] The authentication device 30 is a device that executes authentication processing in the gateless authentication system 1. The authentication device 30 includes a gateless camera 301. The gateless camera 301 is positioned so as to capture an image of the authentication area through which the target passes. The authentication device 30 executes authentication processing using the image captured by the gateless camera 301. For example, the authentication device 30 detects a target included in the image and acquires biometric information from the target. The authentication device 30 then executes authentication processing by comparing the biometric information of the target with pre-registered registration information (i.e., biometric information of registered users). The authentication device 30 may have a function for storing registration information related to multiple registered users. Alternatively, the registration information may be stored in an external storage device or the like, and the authentication device 30 may read it to execute authentication processing. The authentication device 30 may execute authentication processing by calculating a score according to the degree of match between the biometric information and the registration information. In this case, the authentication device 30 may determine that the authentication process is successful (i.e., the target and the registered user match) if the score is higher than a predetermined threshold, and may determine that the authentication process is unsuccessful (i.e., the target and the registered user do not match) if the score is lower than the predetermined threshold. The type of biometric information used by the authentication device 30 is not particularly limited, and the authentication process may be performed using biometric information related to the face or iris, for example.

[0022] The first information processing device may be configured to include some of the components described in FIG. 1 . For example, the first information processing device may be configured to include a processor 11, a RAM 12, and a ROM 13. The first information processing device may also be configured to have some of the functions of the authentication device 30 (e.g., the function of executing authentication processing). Some of the calculation functions of the first information processing device 10 may be realized by an external server, a cloud, or the like. The specific configuration of the first information processing device will be described in detail later.

[0023] (Gateless Authentication) Next, gateless authentication, which is an authentication process executed by the gateless authentication system 1, will be specifically described with reference to Fig. 2. Fig. 2 is a top view showing an example of the operation of the gateless authentication system.

[0024] As shown in Fig. 2, the gateless authentication system 1 is configured to perform authentication processing (hereinafter referred to as "gateless authentication") on multiple targets 50 passing through a preset authentication area. In the example shown in the figure, multiple targets 50 are walking from the left side to the right side of the figure. The authentication area is set, for example, at the entrance of various facilities, and targets 50 who are successfully authenticated are permitted to enter the facility, while targets 50 who are unsuccessful in authentication are prohibited from entering the facility.

[0025] An authentication device 30 equipped with a gateless camera 301 is installed around the authentication area. The gateless camera 301 is installed so that it at least partially includes the authentication area within its imaging range. Note that although an example in which one gateless camera 301 is installed is given here, multiple gateless cameras 301 may be installed. In this case, it is preferable that the multiple gateless cameras 301 are installed so that they each capture images of the authentication area from different positions and at different angles.

[0026] At least one monitor 60 is assigned to the authentication device 30. The monitor 60 uses the user terminal 20 to monitor the target 50 undergoing gateless authentication. For example, if there is a target 50 for which the authentication process has not been successful (hereinafter referred to as an "unauthenticated individual"), the monitor 60 calls out to the unauthenticated individual and prohibits them from passing through. The area indicated by the dashed line in the figure is the field of view of the monitor 60. When multiple targets pass through the authentication area at the same time, it is preferable for the monitor 60 to be positioned so that multiple targets are within his or her field of view. However, when multiple targets 50 are present, the multiple targets may overlap from the monitor's perspective. For example, in the example shown in the figure, targets 50A and 50B appear to overlap from the monitor's perspective. In this case, the monitor 60 can see target 50A, which is located in the background, but has difficulty seeing target 50B. The information processing device according to this embodiment displays an image relating to the authentication process on the display unit 201 of the user terminal 20 so that the monitor 60 can appropriately monitor the target 50 even in the above-mentioned situation.

[0027] (Authentication Operation) Next, the flow of authentication operation by the gateless authentication system 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of authentication operation in the gateless authentication system.

[0028] As shown in FIG. 3 , when the authentication operation by the gateless authentication system 1 is started, the authentication device 30 first captures an image of the target 50 using the gateless camera 301, and acquires an image including the target 50 (step S101). Then, the authentication device 30 detects the target 50 from the image captured by the gateless camera 301 and starts tracking it (step S102). The tracking here is a process of tracking the same target 50 included in multiple frames of images captured in time series, and an individual tracking ID is assigned to each target 50 being tracked. Note that the authentication device 30 may detect multiple targets 50 from a single image. In this case, the authentication device 30 may track multiple targets 50 simultaneously in parallel. The subsequent processes from step S103 onwards are executed for each target.

[0029] The authentication device 30 determines whether a new tracking ID has been assigned (step S202). That is, the authentication device 30 determines whether tracking of a new target 50 has begun. If it is determined that a new tracking ID has not been assigned (step S202: NO), the authentication device 30 determines whether the target 50 being tracked has been authenticated (step S203). That is, the authentication device 30 determines whether an authentication process has been executed on the target 50 being tracked and whether the authentication process has been successful. If the target 50 being tracked has been authenticated (step S203: YES), the authentication device 30 determines that the target 50 is allowed to pass (step S204).

[0030] On the other hand, if it is determined that a new tracking ID has been assigned (step S202: YES), or if the target 50 being tracked has not yet been authenticated (step S203: NO), the authentication device 30 executes authentication processing for the target 50 (step S205). If the authentication processing is successful (step S205: YES), the authentication device 30 associates the tracking ID assigned to the target 50 with the target ID identified by the authentication (i.e., the ID set for each registered user) and stores them (step S207). Then, for the target 50 that has been successfully authenticated, it is determined that the target 50 is allowed to pass (step S204).

[0031] On the other hand, if the authentication process fails (step S205: NO), the authentication device 30 determines whether an alert has already been issued for the target 50 (step S208). An alert is issued, for example, when the number of authentication failures exceeds a predetermined threshold N. Examples of the number of authentication failures include the number of consecutive frames of authentication failure and the duration of consecutive authentication failures. For a target 50 for which an alert has already been issued (step S208: YES), the authentication device 30 determines that the target 50 is not permitted to pass (step S209). If an alert has not been issued (step S208: NO), the authentication device 30 determines whether the number of consecutive frames of authentication failure exceeds a predetermined threshold N (step S210). If the number of consecutive frames of authentication failure exceeds the predetermined threshold N (step S210: YES), the authentication device 30 issues an alert for the target 50 and determines that the target 50 is not permitted to pass (step S209). On the other hand, if the number of consecutive frames of authentication failure does not exceed the predetermined threshold N (step S210: NO), the authentication device 30 continues tracking the target 50 as being under evaluation (step S211).

[0032] When the processing for each control up to step S104 is completed, the first information processing device 10 executes a display image generation process (step S105). That is, the first information processing device 10 generates a display image to be displayed on the user terminal 20 held by the monitor 60 according to the authentication result of the target 50. Thereafter, the authentication device 30 determines whether or not the input of the image captured by the gateless camera 301 is completed (step S106). If the input of the image is completed (step S106: YES), the series of operations is completed. On the other hand, if the input of the image is not completed (step S106: NO), the processing is repeated from step S101.

[0033] (Configuration of Information Processing Apparatus) Next, the configuration of the first information processing apparatus applied to the gateless authentication system 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the first information processing apparatus.

[0034] 4, the first information processing device 10 includes an information acquisition unit 110, a visibility determination unit 120, and a display image generation unit 130. Each of the information acquisition unit 110, the visibility determination unit 120, and the display image generation unit 130 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0035] The information acquisition unit 110 is configured to acquire object position information indicating the position of the object 50. The information acquisition unit 110 is also configured to acquire terminal information regarding the user terminal 20 held by the monitor 60. The terminal information is information for determining the visibility of the object 50 from the monitor 60. The terminal information may be, for example, information indicating the position and orientation of the user terminal 20. Alternatively, the terminal information may be information regarding an image captured by the terminal camera 202 of the user terminal 20. By using the above-described terminal information, it is possible to appropriately estimate the field of view of the monitor 20 holding the user terminal 20. For example, by using information regarding the position and orientation of the user terminal 20, it is possible to estimate from which position and in which direction the monitor 20 is looking. Furthermore, the image captured by the terminal camera 202 can be used as a field of view corresponding to the field of view of the monitor 60. The information acquisition unit 110 may acquire the terminal information from, for example, an image captured by the gateless camera 301. For example, the information acquisition unit 110 may detect the monitor 60 and the user terminal 20 included in the image captured by the gateless camera 301 and acquire the terminal information. By using the image of the gateless camera 301 in this way, it is possible to appropriately acquire the terminal information. The terminal information may be acquired from the user terminal 20 itself. For example, the terminal information may be acquired using a GPS function, a Wi-Fi positioning function, a Bluetooth function, or the like installed in the user terminal 20. The terminal information may also be acquired from the image captured by the terminal camera 202.

[0036] The visibility determination unit 120 is configured to determine the visibility of the target 50 from the observer 60 based on the target position information and terminal information acquired by the information acquisition unit 110. The visibility determination unit 120 may, for example, determine whether the target is present in the field of view 50 of the observer 60 estimated from the terminal information. Furthermore, when the target 50 is present in the field of view of the observer 60, the visibility determination unit 120 may determine whether the target 50 is overlapped with an obstructing object. In this case, the obstructing object is typically another target 50, but may also be, for example, an obstacle (e.g., a pillar, ornament, etc.) present between the observer 60 and the target 50. The visibility determination unit 120 may determine the visibility of the target 50 in two stages: "good" or "poor." Alternatively, the visibility determination unit 120 may determine the visibility by calculating a score (e.g., a score out of 100) that increases as the visibility increases. A more specific method for determining visibility will be described in another embodiment below.

[0037] The display image generation unit 130 is configured to generate a display image to be displayed on the user terminal 20 based on the determination result by the visibility determination unit 120. The display image is an image related to the authentication process of the target 50. For example, the display image may be an image indicating the location of an unauthenticated person. The display image generation unit 130 may generate the display image by superimposing information corresponding to the authentication result on an image captured by the terminal camera 202. For example, the display image generation unit 130 may generate an image that highlights only the unauthenticated target 50 among multiple targets 50 included in the image captured by the terminal camera 202. In this case, the display image generation unit 130 reflects the visibility determination result by the visibility determination unit 120. For example, the display image generation unit 130 may generate the display image by selecting a display mode corresponding to the current visibility from multiple display modes prepared in advance. Specific examples of the display image will be described in detail in other embodiments described below.

[0038] (Display Image Generation Processing) Next, the display image generation processing executed by the first information processing device 10 (i.e., the processing of step S105 in FIG. 2) will be specifically described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the display image generation processing by the first information processing device.

[0039] 5 , when the display image generation process by the first information processing device 10 starts, the information acquisition unit 110 first determines whether there are one or more targets 50 (i.e., unauthenticated individuals) whose authentication process results indicate that they are not permitted to pass or are under evaluation (step S301). If it is determined that there are one or more unauthenticated individuals (step S301: YES), the information acquisition unit 110 then determines whether there are two or more targets 50 (step S302). If there are no unauthenticated individuals (step S301: NO) or if the number of targets 50 is one or less (step S302: NO), it can be determined that there are no unauthenticated individuals who will be hidden from the monitor 60. Therefore, the display image generation unit 130 selects default display content (step S307) and generates a display image (step S308).

[0040] On the other hand, if the number of targets 50 is two or less (step S302: YES), there is a possibility that unauthenticated individuals may become invisible to the observer 60 due to overlapping of the targets 50. Therefore, the information acquisition unit 110 acquires target position information (step S303). The information acquisition unit 110 also acquires terminal information (S304). Then, the visibility determination unit 120 determines the visibility of the targets 50 from the observer 60 based on the target position information and terminal information acquired by the information acquisition unit 110 (step S305).

[0041] For example, the visibility determination unit 120 may use the target position information of the unauthenticated person and other target position information to calculate the distance between the unauthenticated person and other people or obstacles, and determine that visibility is poor if the calculated distance is within a predetermined distance. Alternatively, the visibility determination unit 120 may detect people or obstacles from an image captured by the terminal camera 202, and determine that visibility is poor if the position of the unauthenticated person in the image and the positions of the other people or obstacles are within a predetermined distance. Alternatively, the visibility determination unit 120 may use the target position information and the terminal position information to calculate a straight line connecting the target position of the unauthenticated person and the terminal position, and determine that visibility is poor if other people or obstacles are present within a predetermined distance from the calculated straight line. The terminal information may be acquired from the user terminal 20, or may be acquired from an image captured by the terminal camera 202 or the gateless camera 301.

[0042] Next, the display image generation unit 130 determines the display content based on the visibility determination result by the visibility determination unit 120 (step S306). Then, the display image generation unit 130 generates a display image according to the determined display content (step S307). Thereafter, the display image generation unit 130 outputs the generated display image to the user terminal 20 (step S308). As a result, the display image generated by the display image generation unit 130 is displayed on the display unit 201 of the user terminal 20.

[0043] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0044] 1 to 5 , in the first information processing device 10, a display image of the user terminal 20 held by the monitor 60 is generated based on the visibility of the target 50 for which gateless authentication is to be performed. In this way, the monitor 60 can appropriately monitor the target by checking the display image displayed on the user terminal 20. For example, even if an unauthenticated person is difficult for the monitor 60 to see because the unauthenticated person overlaps with other targets 50, the monitor 60 can appropriately grasp information about the unauthenticated person.

[0045] Second Embodiment A second information processing device 10 will be described with reference to Fig. 6. The second information processing device 10 differs in some of its operations from the first information processing device 10 described above, but other parts may be the same as the first information processing device 10. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0046] (Display Image Generation Process) First, the flow of the display image generation process executed by the second information processing device 10 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the display image generation process by the second information processing device.

[0047] 6 , when the display image generation process by the second information processing device 10 starts, the information acquisition unit 110 first determines whether there are one or more objects 50 (i.e., unauthenticated individuals) whose authentication process results indicate that they are not permitted to pass or are under evaluation (step S301). If it is determined that there are one or more unauthenticated individuals (step S301: YES), the information acquisition unit 110 then determines whether there are two or more objects 50 (step S302). If there are no unauthenticated individuals (step S301: NO) or if the number of objects 50 is one or less (step S302: NO), it can be determined that there are no unauthenticated individuals who will be hidden from the monitor 60. Therefore, the display image generation unit 130 selects default display content (step S307) and generates a display image (step S308).

[0048] On the other hand, if the number of targets 50 is two or less (step S302: YES), there is a possibility that unauthenticated individuals may be hidden from the monitor 60 due to overlapping of targets 50. For this reason, the information acquisition unit 110 acquires target position information (step S303). The information acquisition unit 110 also acquires terminal information (S304). Note that in the second information processing device 10, the information acquisition unit 110 acquires both an image captured by the gateless camera 301 and an image captured by the terminal camera 202.

[0049] Next, the visibility determination unit 120 performs matching of the target 50 (step S401). Specifically, the visibility determination unit 120 compares the target 50 in the image captured by the gateless camera 301 with the target 50 in the image captured by the terminal camera 202 to confirm that they are the same person. The visibility determination unit 120 may perform matching by comparing the position of the target 50 in the image captured by the gateless camera 301 with the position of the target 50 in the image captured by the terminal camera 202. Alternatively, the visibility determination unit 120 may perform matching by comparing the biometric information of the target 50 in the image captured by the gateless camera 301 with the biometric information of the target 50 in the image captured by the terminal camera 202.

[0050] Next, the visibility determination unit 120 calculates the percentage of overlap between the unauthenticated person matched in step S401 and other targets (hereinafter referred to as the "overlap percentage") (step S402). The visibility determination unit 120 may calculate the overlap percentage using target position information and terminal information. For example, the visibility determination unit 120 may calculate the overlap percentage from an image captured by the gateless camera 301 or an image captured by the terminal camera 202.

[0051] Next, the visibility determination unit 120 determines the visibility of the target 50 based on the calculated overlap ratio (step S403). For example, if the overlap ratio of the target 50 exceeds 60% (i.e., if the hidden and invisible part exceeds 60% of the whole), the visibility determination unit 120 may determine that the visibility of the target 50 is poor.

[0052] Next, the display image generation unit 130 determines the display content based on the visibility determination result by the visibility determination unit 120 (step S306). Then, the display image generation unit 130 generates a display image according to the determined display content (step S307). Thereafter, the display image generation unit 130 outputs the generated display image to the user terminal 20 (step S308). As a result, the display image generated by the display image generation unit 130 is displayed on the display unit 201 of the user terminal 20.

[0053] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0054] As described in FIG. 6 , in the second information processing device 10, the target 50 whose visibility is to be determined is compared using an image. This prevents the same target 50 from being treated as a different person, or prevents k different targets 50 from being treated as the same person, which could result in an inappropriate determination of visibility. Furthermore, in the second information processing device 10, visibility is determined based on the overlap ratio of the target 50. This makes it possible to easily and accurately determine whether the visibility of the target 50 is good or bad.

[0055] <Third Embodiment> A third information processing device 10 will be described with reference to Figures 7 to 17. Note that the third information processing device 10 differs in some of its operations from the first and second information processing devices 10 described above, but other parts may be similar to the first and second information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0056] (Unauthenticated Person Outside the Field of View) First, a specific situation assumed by the third information processing device 10 (specifically, a situation in which an unauthenticated person is outside the field of view of an observer) will be described with reference to Fig. 7. Fig. 7 is a top view showing an example of a case in which an unauthenticated person is outside the field of view of an observer in a gateless authentication system.

[0057] As shown in Fig. 7, when there are multiple targets 50, there may be cases where an unauthenticated person is outside the field of view of the monitor 60. For example, in the example shown in the figure, the monitor is facing the direction of target 50C. Therefore, unauthenticated target 50A is outside the field of view of the monitor 60. Therefore, if this continues, it is difficult for the monitor 60 to recognize the presence of the unauthenticated person. In such a situation, the third information processing device 10 generates a display image to guide the gaze of the monitor 60 toward the unauthenticated person.

[0058] (Display Image Generation Process) Next, the flow of the display image generation process executed by the third information processing apparatus 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of the display image generation process by the third information processing apparatus.

[0059] 8 , when the display image generation process by the third information processing device 10 starts, the information acquisition unit 110 first determines whether there are one or more targets 50 (i.e., unauthenticated individuals) whose authentication process results indicate that they are not permitted to pass or are under evaluation (step S301). If it is determined that there are one or more unauthenticated individuals (step S301: YES), the information acquisition unit 110 then determines whether there are two or more targets 50 (step S302). If there are no unauthenticated individuals (step S301: NO) or if the number of targets 50 is one or less (step S302: NO), it can be determined that there are no unauthenticated individuals who will be hidden from the monitor 60. Therefore, the display image generation unit 130 selects default display content (step S307) and generates a display image (step S308).

[0060] On the other hand, if the number of targets 50 is two or less (step S302: YES), there is a possibility that unauthenticated individuals may become invisible to the observer 60 due to overlapping of the targets 50. Therefore, the information acquisition unit 110 acquires target position information (step S303). The information acquisition unit 110 also acquires terminal information (S304). Then, the visibility determination unit 120 determines the visibility of the targets 50 from the observer 60 based on the target position information and terminal information acquired by the information acquisition unit 110 (step S305).

[0061] Here, the third information processing device 10 particularly determines whether or not an unauthenticated person is present within the field of view of the monitor 60 (step S501). Whether or not an unauthenticated person is present within the field of view can be determined based on the position information of the target 50 and the field of view estimated from the terminal information. The field of view of the monitor may be set based on, for example, the imaging range of the terminal camera 202 or the terminal information (the direction in which the user terminal 20 is facing). Specifically, the viewing angle may be set to ±100 degrees based on the imaging range of the terminal camera 202 or the orientation of the user terminal 20. The process of estimating the field of view may be a process of identifying an expected field of view based on the terminal information (i.e., the position and direction of the user terminal 20).

[0062] If it is determined that the unauthenticated person is not within the field of view (in other words, is outside the field of view) (step S501: NO), the display image generation unit 130 generates a display image for guiding the gaze of the observer 60 toward the unauthenticated person (step S502). For example, the display image generation unit 130 generates a display image including an arrow indicating the direction in which the unauthenticated person is present. The display image generation unit 130 then outputs the generated display image (step S503). As a result, a display image for guiding the gaze of the observer 60 is displayed on the display unit 201 of the user terminal 20. After the gaze of the observer 60 has been guided, processing resumes from step S304.

[0063] On the other hand, if it is determined that an unauthenticated person is present within the field of view (step S501: YES), the display image generation unit 130 determines the display content based on the visibility determination result by the visibility determination unit 120 (step S306). Then, the display image generation unit 130 generates a display image according to the determined display content (step S307). Thereafter, the display image generation unit 130 outputs the generated display image to the user terminal 20 (step S308). As a result, the display image generated by the display image generation unit 130 is displayed on the display unit 201 of the user terminal 20.

[0064] (Example of Display Image) Next, a specific example of a display image (particularly a display image for guiding the line of sight of the monitor 60) generated by the third information processing device 10 will be described with reference to Fig. 9. Fig. 9 is a plan view showing an example of transition of a display image generated by the third information processing device.

[0065] The image in Fig. 9(a) corresponds to the field of view of the observer 60 shown in Fig. 7 (in other words, the field of view of the user terminal 20). In this case, the field of view of the observer 60 includes the authenticated target 50C, but does not include the unauthenticated target 50A. Therefore, the display image generation unit 130 generates a display image including an arrow indicating the direction in which the target 50A, which is outside the field of view, is located. This makes it possible to guide the gaze of the observer 60 to the left (i.e., in the direction of the arrow).

[0066] 9B, as the observer 60's gaze moves to the left, the target 50A gradually becomes visible. In this case, the display image generating unit 130 may reduce the size of the arrow. That is, the size of the arrow may be reduced in stages as the observer 60's gaze approaches the unauthenticated individual. Furthermore, the size of the arrow may be changed taking into account the movement status of the unauthenticated individual (e.g., real-time position, speed, movement direction, etc.).

[0067] 9(c), when the observer 60 moves his / her gaze further to the left, the unauthenticated target 50A comes completely within the field of view of the observer 60. In this case, the display image generating unit 130 erases the arrow that had been displayed up until then from the display image. The display image generating unit 130 may also highlight the unauthenticated target 50A by surrounding it with a frame, making it easier for the observer 60 to recognize that the unauthenticated target 50A is unauthenticated.

[0068] (Other Display Modes) Next, other examples of display images generated by the third information processing device 10 will be described with reference to Fig. 10 to Fig. 17. Fig. 10 to Fig. 17 are plan views showing examples of display images generated by the third information processing device. Note that the display examples shown in Fig. 10 to Fig. 17 are displayed when an unauthenticated person is present within the field of view of the monitor 60.

[0069] In the example shown in FIG. 10, unauthenticated individuals are displayed relatively brightly, and other authenticated individuals (hereinafter referred to as "authenticated individuals") are displayed relatively darkly.

[0070] In the example shown in FIG. 11, an authenticator who is completely hidden by multiple authenticators is displayed as a silhouette.

[0071] In the example shown in FIG. 12, in addition to the display mode described with reference to FIG. 10, a bird's-eye view is displayed at the bottom right of the screen.

[0072] In the example shown in FIG. 13, the authenticated person is deleted and only the unauthenticated person is displayed (in reality, there are multiple authenticated people around the unauthenticated person).

[0073] In the example shown in FIG. 14, the route taken by the monitor 60 to reach the unauthenticated person is displayed.

[0074] 15, the direction in which the observer 60 should move (the place to move to) to eliminate the overlap of unauthenticated persons is displayed. The direction and place in which the observer should move can be determined from the degree of overlap of unauthenticated persons and the position of authenticated persons.

[0075] In the example shown in Figure 16, there are two unauthenticated persons, and the unauthenticated person with a large degree of overlap is surrounded by a thick, dark frame. On the other hand, the unauthenticated person with a small degree of overlap is surrounded by a thin, light frame. The frame of the authenticated person with a large degree of overlap may be made to blink.

[0076] In the example shown in FIG. 17, two unauthenticated persons are included, and both unauthenticated persons are enclosed in the same frame, but an icon is displayed above the head of the unauthenticated person who overlaps to a large extent.

[0077] The above-described display modes are merely examples, and the display image generating unit 130 may generate a display image in a display mode other than those mentioned here.

[0078] (Technical Effects) Next, technical effects obtained by the third information processing apparatus 10 will be described.

[0079] As described with reference to Figures 7 to 17, the third information processing device 10 generates a display image that guides the gaze of the monitor 60 when an unauthenticated person is outside the field of view of the monitor 60. In this way, the gaze of the monitor 60 can be shifted to reliably bring the unauthenticated person into the field of view of the monitor 60. Furthermore, the third information processing device 10 highlights the unauthenticated person in the display image. In this way, the monitor 60 can easily recognize the presence of the unauthenticated person.

[0080] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0081] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0082] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0083] (Appendix 1) The information processing device described in Appendix 1 is an information processing device that includes an acquisition means for acquiring location information of a target and terminal information related to a terminal held by an observer, a determination means for determining the visibility of the target from the observer based on the location information of the target and the terminal information, and a generation means for generating a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result.

[0084] (Appendix 2) The information processing device described in Appendix 2 is the information processing device described in Appendix 1, in which the target information includes at least one of location information of the terminal, information regarding the orientation of the terminal, and information regarding an image captured by a first camera mounted on the terminal.

[0085] (Appendix 3) The information processing device described in Appendix 3 is the information processing device described in Appendix 1 or 2, wherein the acquisition means acquires at least one of the target's location information and the terminal information from an image captured by a second camera installed so as to be able to capture an authentication area through which the target passes when executing the authentication process.

[0086] (Supplementary Note 4) The information processing device described in Supplementary Note 4 is the information processing device described in any one of Supplementary Note 3, wherein the determination means compares the target using an image captured by the first camera and an image captured by the second camera, and determines the visibility based on the comparison result of the target.

[0087] (Supplementary Note 5) The information processing device described in Supplementary Note 5 is the information processing device described in any one of Supplementary Notes 1 to 4, wherein the determination means calculates the percentage of overlap between the target and an obstruction, and determines the visibility based on the percentage.

[0088] (Appendix 6) The information processing device described in Appendix 6 is the information processing device described in any one of Appendices 1 to 5, wherein the generation means generates the display image to include information indicating the direction in which the target is located when an unauthenticated person who has not succeeded in the authentication process is not within the field of view of the monitor.

[0089] (Supplementary Note 7) The information processing device described in Supplementary Note 7 is the information processing device described in any one of Supplementary Notes 1 to 6, wherein the generation means generates the display image so as to highlight unauthenticated individuals whose authentication process has not been successful.

[0090] (Supplementary Note 8) The information processing device according to Supplementary Note 8 is the information processing device according to any one of Supplementary Notes 1 to 7, wherein the terminal is a head-mounted display worn by the monitor.

[0091] (Appendix 9) The information processing method described in Appendix 9 is an information processing method that, using at least one computer, acquires location information of a target and terminal information related to a terminal held by an observer, determines the visibility of the target from the observer based on the location information of the target and the terminal information, and generates a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result.

[0092] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which includes acquiring location information of a target and terminal information related to a terminal held by an observer, determining the visibility of the target from the observer based on the location information of the target and the terminal information, and generating a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result.

[0093] (Supplementary Note 11) The computer program described in Supplementary Note 11 is a computer program that causes at least one computer to execute an information processing method of acquiring location information of a target and terminal information related to a terminal held by an observer, determining the visibility of the target to the observer based on the location information of the target and the terminal information, and generating a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result. (Supplementary Note 12) The gateless authentication system described in Supplementary Note 12 is a gateless authentication system that includes: an imaging means that captures an image of the target by capturing an image of the target; an authentication means that performs authentication processing using biometric information of the target acquired from the image of the target; a terminal held by an observer that monitors the authentication processing; an acquisition means that acquires location information of the target and terminal information related to the terminal held by the observer; a determination means that determines the visibility of the target to the observer based on the location information of the target and the terminal information; and a generation means that generates a display image of the terminal related to authentication processing using biometric information of the target based on the visibility determination result.

[0094] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0095] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 18 Camera 20 User terminal 201 Display unit 202 Terminal camera 30 Authentication device 30 301 Gateless camera 50 Target 60 Surveillance officer 110 Information acquisition unit 120 Visibility determination unit 130 Display image generation unit

Claims

1. An information processing device comprising: an acquisition means for acquiring location information of a target and terminal information relating to a terminal held by an observer; a determination means for determining the visibility of the target from the observer based on the location information of the target and the terminal information; and a generation means for generating a display image of the terminal relating to authentication processing using biometric information of the target based on the visibility determination result.

2. The information processing device according to claim 1, wherein the target information includes at least one of location information of the terminal, information regarding the orientation of the terminal, and information regarding an image captured by a first camera mounted on the terminal.

3. The information processing device according to claim 2, wherein the acquisition means acquires at least one of the target's location information and the terminal information from an image captured by a second camera installed so as to be able to capture an image of the authentication area through which the target passes when the authentication process is performed.

4. An information processing device according to any one of claims 3, wherein the determination means compares the object using an image captured by the first camera and an image captured by the second camera, and determines the visibility based on the result of the comparison of the object.

5. The information processing device according to any one of claims 1 to 4, wherein the determining means calculates the proportion of overlap between the target and an obstructing object, and determines the visibility based on the proportion.

6. An information processing device according to any one of claims 1 to 4, wherein the generation means generates the display image to include information indicating the direction in which the target is located when an unauthenticated person who has not been successful in the authentication process is not within the field of view of the monitor.

7. The information processing device according to any one of claims 1 to 4, wherein the generation means generates the display image so as to highlight unauthenticated individuals who have not been successfully authenticated.

8. The information processing device according to any one of claims 1 to 4, wherein the terminal is a head-mounted display worn by the monitor.

9. An information processing method comprising: acquiring, by at least one computer, location information of an object and terminal information relating to a terminal held by an observer; determining the visibility of the object from the observer based on the location information of the object and the terminal information; and generating a display image of the terminal relating to authentication processing using the biometric information of the object based on the visibility determination result.

10. A recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, which comprises acquiring location information of a target and terminal information relating to a terminal held by an observer, determining the visibility of the target from the observer based on the location information of the target and the terminal information, and generating a display image of the terminal relating to authentication processing using the biometric information of the target based on the visibility determination result.

Images