Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present invention comprises: an acquisition means for acquiring biometric information of a user; an authentication means for using the biometric information to execute an authentication process for the user; a storage means for storing appearance information indicating an appearance feature of the user subjected to the authentication process; a tracking means for using the appearance information to track the user; and an output means for outputting guidance information tailored to the user on the basis of the result of the tracking. According to such an information processing device, appropriate guidance information can be presented for each user.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known devices of this type output guidance information to users. For example, Patent Document 1 discloses a technology that identifies visitors to a facility using facial images and then displays guidance information based on the visitor's location, reservation details, etc., by floor projection.

[0003] Japanese Patent Application Laid-Open No. 2020-181525

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises an acquisition means for acquiring biometric information of a user, an authentication means for performing an authentication process for the user using the biometric information, a storage means for storing appearance information indicating the appearance characteristics of the user who performed the authentication process, a tracking means for tracking the user using the appearance information, and an output means for outputting guidance information tailored to the user based on the results of the tracking.

[0006] One aspect of the information processing method disclosed herein involves using at least one computer to acquire biometric information of a user, perform an authentication process for the user using the biometric information, store appearance information indicating the external characteristics of the user who performed the authentication process, track the user using the appearance information, and output guidance information tailored to the user based on the results of the tracking.

[0007] One aspect of the recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which includes acquiring biometric information of a user, performing an authentication process for the user using the biometric information, storing appearance information indicating the external characteristics of the user who performed the authentication process, tracking the user using the appearance information, and outputting guidance information tailored to the user based on the results of the tracking.

[0008] 1 is a block diagram showing the hardware configuration of a first information processing device. FIG. 2 is a block diagram showing the functional configuration of the first information processing device. FIG. 3 is a flowchart showing the operation flow of the first information processing device. FIG. 4 is a block diagram showing the functional configuration of a second information processing device. FIG. 5 is a flowchart showing the operation flow of the second information processing device. FIG. 6 is a plan view showing an example of a display in the second information processing device. FIG. 7 is a block diagram showing the functional configuration of a third information processing device. FIG. 8 is a flowchart showing the operation flow of the third information processing device. FIG. 9 is a plan view showing an example of a warning in a restricted area by the third information processing device. FIG. 10 is a plan view showing an example of a warning in a belonging restriction area by the third information processing device. FIG. 11 is a plan view showing an example of a warning in an accompanied person required area by the third information processing device. FIG. 12 is a plan view showing an example of a display in an accompanied person required area by the third information processing device. FIG. 13 is a conceptual diagram showing an example of operation in a facility to which a fourth information processing device is applied.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, a first information processing device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing device 10 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing device 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, a functional block for executing a process of outputting guidance information to the user is realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing device 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that the information processing device 10 stores for a long period of time. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as part of a smartphone, a tablet terminal, an earphone-type terminal, a watch-type terminal, an HMD (Head Mounted Display) terminal, etc. The input device 15 may be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display or digital signage) that can display information related to the information processing device 10. The output device 16 may also be a speaker or the like that can output information related to the information processing device 10 as audio.

[0020] 1 may be configured to be included in a device external to the first information processing device 10. For example, the first information processing device 10 may be configured to include a processor 11, a RAM 12, and a ROM 13, and the other devices, such as a storage device 14, an input device 15, and an output device 16, may be configured as external devices. That is, the first information processing device 10 may be configured as an information processing system including a plurality of different devices. Furthermore, some of the calculation functions of the first information processing device 10 may be realized by an external server, a cloud, or the like.

[0021] (Functional Configuration) Next, the functional configuration of the first information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first information processing device.

[0022] 2, the first information processing device 10 is configured to include, as processing blocks for realizing its functions, a biometric information acquisition unit 110, an authentication unit 120, an appearance information storage unit 130, a tracking unit 140, and a guidance information output unit 150. Note that each of the biometric information acquisition unit 110, the authentication unit 120, the appearance information storage unit 130, the tracking unit 140, and the guidance information output unit 150 may be realized by, for example, the above-mentioned processor 11 or storage device 14 (see FIG. 1).

[0023] The biometric information acquisition unit 110 is configured to acquire biometric information of a user. The biometric information is information that can be used for user authentication processing. For example, the biometric information may include the user's face information, iris information, fingerprint information, palmprint information, and voiceprint information. The biometric information may include multiple types of information. The biometric information acquisition unit 110 may acquire the biometric information from an image of the user captured by a camera, for example. In this case, the biometric information acquisition unit 110 may acquire biometric features extracted from the image of the user as the user's biometric information. The biometric information may be configured to be acquired when the user reaches a predetermined location. For example, the biometric information acquisition unit 110 may be configured to acquire the biometric information when the user passes through a predetermined gate, entrance, or the like. The user's biometric information acquired by the biometric information acquisition unit 110 is configured to be output to the authentication unit 120.

[0024] The authentication unit 120 is configured to be able to perform user authentication processing (i.e., biometric authentication) using the user's biometric information acquired by the biometric information acquisition unit 110. For example, the authentication unit 120 may be configured to be able to perform face authentication using the user's face information. Alternatively, the authentication unit 120 may be configured to be able to perform iris authentication using the user's iris information. The authentication processing may be processing to determine whether the user whose biometric information has been acquired is a pre-registered user. In this case, the authentication unit 120 may compare the biometric information acquired from the user with the biometric information of the pre-registered registered user to determine whether the user is a pre-registered user. For example, the authentication unit 120 may determine that the user is a pre-registered user if the degree of match between the biometric information acquired from the user and the biometric information of the pre-registered registered user exceeds a predetermined value.

[0025] The appearance information storage unit 130 is configured to store appearance information indicating the external characteristics of a user who has executed authentication processing in the authentication unit 120. The appearance information is information for identifying a user from their appearance. The appearance information may be, for example, information about the user's face, hairstyle, and body shape. Alternatively, the appearance information may be information about the user's clothing, accessories, and belongings. The appearance information storage unit 130 may acquire the appearance information from, for example, biometric information acquired by the biometric information acquisition unit 110. For example, the appearance information storage unit 130 may acquire appearance information about the user's face (e.g., appearance information about the entire face, appearance information about each part of the face, such as the eyes, nose, ears, and mouth) from the user's facial information acquired by the biometric information acquisition unit 110. This eliminates the need to separately acquire the user's appearance information. However, the appearance information storage unit 130 may acquire the appearance information via a route separate from the biometric information. For example, the appearance information storage unit 130 may acquire an image of the user separately and acquire the user's appearance information from the image. The appearance information storage unit 130 may store the appearance information of a user whose authentication process has been successful. The appearance information storage unit 130 may also store the appearance information of a user whose authentication process has been unsuccessful.

[0026] The tracking unit 140 tracks the user using the user's appearance information stored in the appearance information storage unit 130. Specifically, the tracking unit 140 identifies the user using the stored appearance information and tracks the location of the identified user. The tracking unit 140 may also track the user while storing the route taken by the user. The tracking unit 140 may track the user using, for example, multiple cameras. Note that the specific tracking method is not particularly limited, and the user may be tracked using existing technology as appropriate. The tracking unit 140 may be configured to track the user within a predetermined area (e.g., within a predetermined facility), for example. In this case, the tracking unit 140 may be configured to automatically end tracking of the user when the user leaves the predetermined area.

[0027] The guidance information output unit 150 outputs guidance information tailored to the user based on the tracking results by the tracking unit 140. The guidance information output unit 150 may output guidance information based on the current location of the tracked user. For example, the guidance information output unit 150 may output guidance information when the user reaches a predetermined location. The guidance information is information that is individually changed or adjusted for each user. Therefore, if there are multiple users, the guidance information output unit 150 outputs different guidance information for each user. The guidance information may include various information to be notified to each user. For example, the guidance information may include information indicating the user's destination. Alternatively, the guidance information may be information indicating the user authentication result by the authentication unit 120. The guidance information may be information that displays various messages, images, etc. on the display. In this case, the guidance information output unit 150 may output guidance information when the user approaches the display. The display of guidance information will be described in detail in another embodiment below.

[0028] (Flow of Operation) Next, the flow of operation in the first information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the first information processing device.

[0029] 3, when the operation of the first information processing device 10 is started, the biometric information acquisition unit 110 first acquires biometric information of the user (step S101). Then, the authentication unit 120 executes authentication processing of the user using the biometric information acquired by the biometric information acquisition unit 110 (step S102).

[0030] Next, the appearance information storage unit 130 stores the appearance information of the user who has executed the authentication process (step S103).Then, the tracking unit 140 tracks the user using the appearance information stored in the appearance information storage unit 130 (step S104).

[0031] Next, the guidance information output unit 150 outputs guidance information based on the tracking result by the tracking unit 140 (step S105). Specifically, guidance information tailored to the user being tracked is output.

[0032] The guidance information may be output only once at a predetermined timing, or may be output multiple times at different timings. When the guidance information is output multiple times, the content of the guidance information may change each time. For example, the content of the guidance information may be changed depending on the tracking result (e.g., the user's position when the guidance information is output).

[0033] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0034] As described with reference to Figures 1 to 3, the first information processing device 10 tracks the user who has performed the authentication process and outputs guidance information tailored to the user. In this way, it is possible to present appropriate guidance information for each user. Therefore, more appropriate guidance can be provided than, for example, when the same guidance information is output to multiple users. Furthermore, because the guidance information is output based on the tracking results, it is not necessary to perform the authentication process again when outputting the guidance information.

[0035] Second Embodiment A second information processing device 10 will be described with reference to Figures 4 to 6. The second information processing device 10 differs in some configurations and operations from the first information processing device 10 described above, but other parts may be similar to the first information processing device 10. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0036] (Functional Configuration) First, the functional configuration of the second information processing device 10 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 4, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0037] 4, the second information processing device 10 is configured to include, as processing blocks for realizing its functions, a biometric information acquisition unit 110, an authentication unit 120, an appearance information storage unit 130, a tracking unit 140, a guidance information output unit 150, and a display unit 200. That is, the second information processing device 10 is configured to further include the display unit 200 in addition to the configuration of the first information processing device (see FIG. 2). The display unit 200 may be realized by, for example, the above-mentioned output device 16 (see FIG. 1). However, the display unit 200 may also be a device provided outside the information processing device 10 (for example, a display device installed in a location separate from the information processing device 10).

[0038] The display unit 200 is configured to be able to display guidance information output from the guidance information. The display unit 200 may be provided in the information processing device 10 itself. Alternatively, the display unit 200 may be a display device configured as a device different from the information processing device 10. For example, the display unit 200 may be a large, stationary display or digital signage. Alternatively, the display unit 200 may be a display of a terminal owned by the user. A plurality of display units 200 may be provided. For example, the display units 200 may be a plurality of display devices installed in different locations.

[0039] The display unit 200 may display guidance information using a different display for each user. For example, guidance information for user A may be displayed on a first display, and guidance information for user B may be displayed on a second display installed separately from the first display.

[0040] The display position of the guidance information may be determined depending on its content or type. In this case, the display position of the guidance information may be set as an absolute position on the display, or may be set as a position relative to the person being displayed. For example, information about the destination may always be displayed in the upper right corner of the display. Furthermore, information about the time may be displayed in the upper left corner of the user. In this way, the visibility of the guidance information can be improved. More specifically, the user can predict the position where the guidance information for the user will be displayed by referring to the guidance information displayed for other users in front of the user.

[0041] The display unit 200 may also use different displays for different types of guidance information. For example, destination-related information may be displayed on the third display, and time-related information may be displayed on the fourth display. In this case, the type of information displayed by each display is predetermined, making it easier for the user to understand the displayed content.

[0042] The display unit 200 may change the display position of the guidance information depending on the importance of the guidance information. For example, the display unit 200 may display guidance information with a relatively high level of importance in a prominent position, while displaying guidance information with a relatively low level of importance in an inconspicuous position. Alternatively, the display unit 200 may change the size of the guidance information depending on the importance of the guidance information. For example, the display unit 200 may display guidance information with a relatively high level of importance in a large size, while displaying guidance information with a relatively low level of importance in a small size.

[0043] The display unit 200 may change the display mode of the guidance information depending on the user's position and movement. For example, the display unit 200 may estimate the user's line of sight and enlarge the guidance information that the user is actually looking at. Alternatively, the display unit 200 may cause the guidance information to move in a direction encouraging the user to take action (for example, guidance information encouraging the user to move to the right moves in an animated manner to the right) when the user approaches within a predetermined distance from the display.

[0044] The display unit 200 may display preliminary guidance information before displaying guidance information for the user. The preliminary guidance information is displayed based on the components of the guidance information. The components of the guidance information may be a display position, type, content, etc., or a combination thereof. For example, the preliminary guidance information may be a symbol related to the guidance information. The symbol may be a pictogram corresponding to the type or content of the guidance information. Specifically, when information related to time is displayed as the guidance information, a pictogram showing a clock may be displayed as preliminary guidance information at the display position of the guidance information, and then the pictogram may change to the guidance information.

[0045] The preliminary guidance information may be information indicating the relationship between the first guidance information for user A and the second guidance information for user B. Specifically, when the second guidance information for user B is displayed based on the first guidance information for user A, the preliminary guidance information is displayed before the second guidance information. The preliminary guidance information is displayed based on the components of the first guidance information and the second guidance information. For example, when the display positions of the first guidance information and the second guidance information change, an arrow may be displayed from the display position of the first guidance information to the display position of the second guidance information. Furthermore, the preliminary guidance information may display a frame or symbol indicating that the first guidance information was displayed, and then display a frame or symbol at the display position of the second guidance information, or the frame or symbol may move from the display position of the first guidance information to the display position of the second guidance information. Alternatively, when the types of the first guidance information and the second guidance information do not change but the content changes, the color or shape of the pictogram displayed as the preliminary guidance information may change to make it clear that the content has changed. By displaying the preliminary guidance information, the user can predict what kind of guidance information will be displayed by referring to the preliminary guidance information.

[0046] The display unit 200 may be configured to include a camera. In this case, the display unit 200 may be configured to capture an image of an approaching user with the camera and display the image. The display unit 200 may then be configured to display guidance information superimposed on the captured image. For example, the display unit 200 may be configured to display a message based on the guidance information around the user in the captured image. Note that if there are multiple users in the image, the display unit 200 may simultaneously display different guidance information for each of the multiple users. In this way, guidance information can be presented efficiently to multiple users. Note that specific display examples will be described in detail later.

[0047] (Operation Flow) Next, the operation flow in the second information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation flow of the second information processing device. Note that in Fig. 5, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0048] 5, when the operation of the second information processing device 10 is started, the biometric information acquisition unit 110 first acquires biometric information of the user (step S101). Then, the authentication unit 120 executes authentication processing of the user using the biometric information acquired by the biometric information acquisition unit 110 (step S102).

[0049] Next, the appearance information storage unit 130 stores the appearance information of the user who has executed the authentication process (step S103).Then, the tracking unit 140 tracks the user using the appearance information stored in the appearance information storage unit 130 (step S104).

[0050] Next, the guidance information output unit 150 determines whether the user being tracked is approaching the display unit 200 (step S201). In addition to the tracking result, the guidance information output unit 150 may detect the approach of the user from, for example, an image captured by a camera included in the display unit 200. Note that if it is determined that the user is not approaching the display unit 200 (step S201: NO), tracking by the tracking unit 140 continues.

[0051] On the other hand, if it is determined that the user is approaching the display unit 200 (step S201: YES), the guidance information output unit 150 outputs guidance information (step S105) based on the tracking result by the tracking unit 140. Specifically, guidance information tailored to the user approaching the display unit 200 is output.

[0052] Next, the display unit 200 displays the guidance information output by the guidance information output unit 150 to the user (step S202). The display unit 200 may be configured to display the guidance information when the user can view the display unit 200. Specifically, the display unit 200 may be configured to display the guidance information when the user reaches a visible position where the display can be viewed. The visible position may be set in advance based on the distance from the display, etc. For example, the display unit 200 may display the guidance information when the user approaches within 3 meters of the display. The user's approach may be detected by various sensors. Alternatively, an image of the user may be captured and the user's approach may be detected when the size of the user's face or the distance between the user's eyes exceeds a predetermined value. Furthermore, the display unit 200 may estimate the user's line of sight and display the guidance information when it is determined that the user's line of sight is directed toward the display.

[0053] The display unit 200 may display guidance information based on conditions other than the approach of the user. For example, the display unit 200 may display guidance information when the user makes a specific gesture (e.g., tilting their head, crossing their arms, remaining in place, etc.). Alternatively, the display unit 200 may display guidance information when a specific emotion (e.g., anxiety, confusion, etc.) is estimated from the user's facial expression.

[0054] The display unit 200 may display guidance information by using both information about the user's approach and the above-described information about the user's gestures and emotions. For example, the display unit 200 may first display simple guidance information when detecting the user's approach, and then display more detailed guidance information when detecting the user's gestures or specific emotions.

[0055] (Display Example) Next, a display example of the second information processing device 10 will be specifically described with reference to Fig. 6. Fig. 6 is a plan view showing a display example of the second information processing device.

[0056] 6 , the second information processing device 10 displays guidance information on a display 210. The display 210 is a specific example of the display unit 200. The display 210 includes a camera 310 capable of capturing an image of the area around the display 210. The display 210 displays the guidance information by superimposing it on the image captured by the camera 310. The display 210 may be configured to display the image captured by the camera 310 in a horizontally inverted manner so that the user can easily recognize themselves.

[0057] 6 shows an example of guidance in a movie theater. Users 51 and 61 are present in front of the display 210. In this case, the display 210 displays a user image 61 corresponding to the user 51 and a user image 62 corresponding to the user 61. A guidance display corresponding to the user 51 is superimposed and displayed around the user image 61.

[0058] Specifically, a left-pointing arrow is displayed above the user image 61 along with the message "Screen 1 Seat A24." This allows the user 51 to know the screen and seat number to which he or she should go, and the direction in which he or she should proceed. Similarly, a right-pointing arrow is displayed above the user image 62 along with the message "Screen 3 Seat B06." This allows the user 52 to know the screen and seat number to which he or she should go, and the direction in which he or she should proceed.

[0059] The above-described guidance information is an example. The second information processing device 10 may be configured to be able to appropriately display guidance information tailored to the user in various venues, facilities, and the like.

[0060] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0061] 4 to 6, in the second information processing device 10, guidance information is displayed superimposed on the image of the user. In this way, the user can easily understand that the displayed guidance information is being displayed for that user. This technical effect is particularly pronounced when there are multiple users in the image.

[0062] <Third Embodiment> A third information processing device 10 will be described with reference to Figures 7 to 12. The third information processing device 10 differs in some configurations and operations from the first and second information processing devices 10 described above, but other parts may be similar to the first and second information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0063] (Functional Configuration) First, the functional configuration of the third information processing device 10 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 7, the same elements as those described in Fig. 4 are denoted by the same reference numerals.

[0064] 7, the third information processing device 10 is configured to include, as processing blocks for realizing its functions, a biometric information acquisition unit 110, an authentication unit 120, an appearance information storage unit 130, a tracking unit 140, a guidance information output unit 150, a determination unit 160, a warning unit 170, and a display unit 200. That is, the third information processing device 10 is configured to further include the determination unit 160 and the warning unit 170 in addition to the configuration of the second information processing device (see FIG. 4). Each of the determination unit 160 and the warning unit 170 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0065] The determination unit 160 is configured to determine whether a user's behavior is permitted. The behavior permitted for a user is set in advance depending on the location and situation. For example, a specific facility may have designated areas off-limits to users (specifically, areas that only facility staff can enter). In such a facility, users are permitted to enter areas other than the off-limits areas, but are not permitted to enter the off-limits areas. In this case, the determination unit 160 may determine whether the user's location is in a off-limits area based on the tracking result, and then determine whether the user's behavior is permitted. However, the determination unit 160 may also determine whether the user's behavior is permitted using information other than the tracking result. For example, the determination unit 160 may determine whether the user's behavior is permitted based on an image captured of the user.

[0066] The warning unit 170 is configured to be able to output warning information when the determination unit 160 determines that the user's behavior is not permitted. The warning unit 170 may be configured to display the warning information on the display unit 200, for example. When the warning information is output, the display unit 200 may display the warning information instead of the guidance information. Alternatively, the display unit 200 may display both the guidance information and the warning information. The warning information may also be output as a sound such as an alarm. The warning unit 170 may output the warning information based on the result of tracking by the tracking unit 140. For example, the warning information may be output when the user reaches a position where the warning information can be seen.

[0067] The warning information may be output to the user, or may be output to a party other than the user. For example, the warning unit 170 may be configured to output the warning information to facility staff or the like. In this case, the warning unit 170 may output the warning information only to facility staff, or may output the warning information to both facility staff and the user. Note that, when the user's behavior is permitted, the warning unit 170 may output information indicating that the user's behavior is permitted. For example, the warning unit 170 may output a message or the like to a user who is about to enter an area that is not a restricted area, informing the user that they may enter.

[0068] (Operation Flow) Next, the operation flow in the third information processing device 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 8, the same processes as those described in Fig. 5 are denoted by the same reference numerals.

[0069] 8, when the operation of the third information processing device 10 is started, the biometric information acquisition unit 110 first acquires biometric information of the user (step S101). Then, the authentication unit 120 executes a user authentication process using the biometric information acquired by the biometric information acquisition unit 110 (step S102).

[0070] Next, the appearance information storage unit 130 stores the appearance information of the user who has executed the authentication process (step S103).Then, the tracking unit 140 tracks the user using the appearance information stored in the appearance information storage unit 130 (step S104).

[0071] Next, the guidance information output unit 150 determines whether the user being tracked is approaching the display unit 200 (step S201). If it is determined that the user is not approaching the display unit 200 (step S201: NO), the tracking unit 140 continues tracking.

[0072] On the other hand, if it is determined that the user is approaching the display unit 200 (step S201: YES), the determination unit 160 determines whether the user's action is permitted (step S301). If it is determined that the user's action is permitted (step S301: YES), the guidance information output unit 150 outputs guidance information based on the tracking result by the tracking unit 140 (step S105). On the other hand, if it is determined that the user's action is not permitted (step S301: NO), the warning unit 170 outputs warning information (step S302).

[0073] Next, the display unit 200 displays at least one of the guidance information output by the guidance information output unit 150 and the warning information output by the warning unit 170 to the user, etc. (step S202). Note that if the user stops the unauthorized behavior, the display unit 170 may stop displaying the warning information. Also, if the user does not stop the unauthorized behavior, the display unit 170 may change the warning information to a stronger one. Alternatively, the display unit 170 may increase the number of destinations to which the warning information is output. For example, the warning information that was initially output only to the user may also be output to facility staff, etc.

[0074] (Display Examples) Next, display examples by the third information processing device 10 will be specifically described with reference to Figs. 9 to 12. Fig. 9 is a plan view showing an example of a warning in a no-entry area by the third information processing device. Fig. 10 is a plan view showing an example of a warning in a belonging restriction area by the third information processing device. Fig. 11 is a plan view showing an example of a warning in a companion-required area by the third information processing device. Fig. 12 is a plan view showing an example of a display in a companion-required area by the third information processing device.

[0075] The example shown in FIG. 9 displays warning information on a display 220, which is a specific example of the display unit 200. The display 220 is provided with a camera 320. The display 220 is configured to display warning information superimposed on an image captured by the camera 320. The display 220 displays warning information informing users that the area ahead is a restricted area. Specifically, a message reading "Staff only allowed" is displayed above the user image 63 corresponding to the user 53. In addition, a cross mark indicating that entry is prohibited is displayed over the user image 63. This prevents users from accidentally entering the restricted area.

[0076] In the example shown in FIG. 10 , whether a user's behavior is permitted is determined based on information about the user's belongings. In this example, the size of the user's luggage is estimated from an image captured by camera 320, and warning information is output if the user is carrying luggage that is prohibited. User 54 is carrying a large suitcase. Therefore, a message stating "Please leave large luggage in a locker" is displayed above user image 64 corresponding to user 54. In addition, a cross mark indicating that the suitcase is prohibited is displayed over the large suitcase in user image 63. This prevents the user from accidentally bringing in large luggage. Note that the small bag in user image 63 does not display a cross mark indicating that the bag is prohibited. This notifies the user that the bag is permitted. By using information about the user's belongings, warning information can be output appropriately in places where restrictions on belongings are set.

[0077] The user's belongings may be determined using factors other than size (i.e., whether the user's actions are permitted). For example, information about the luggage may be read from an IC tag attached to the luggage, and whether the luggage is permitted to be brought in may be determined based on the read information. Alternatively, an object detection process may be performed on the captured image, and warning information may be output if a specific object that is prohibited from being brought in (e.g., food, drink, pets, etc.) is detected. For the object detection process, an object detection model configured, for example, by a neural network may be used.

[0078] In the example shown in FIG. 11 , whether the user's behavior is permitted is determined based on information about other users accompanying the user. In this example, whether the user is accompanied by an attendant is determined from an image captured by camera 320, and warning information is output if the user is not accompanied by an attendant. User 55 is not accompanied by an attendant. Therefore, a message saying "Please wait a moment until the attendant arrives" is displayed above user image 65 corresponding to user 55. In addition, a STAY mark indicating that the user will wait there is displayed so as to overlap user image 63. This prevents the user from proceeding without an attendant. By using information about other users accompanying the user, warning information can be appropriately output in places where restrictions on other users are set.

[0079] 12 is accompanied by an attendant 57. In this case, the warning information is not displayed. Instead, a message saying "Welcome Mr. / Ms. XX" is displayed above a user image 66 corresponding to the user 56. This message is the guidance information output by the guidance information output unit 150.

[0080] (Technical Effects) Next, technical effects obtained by the third information processing apparatus 10 will be described.

[0081] As described with reference to FIGS. 7 to 12 , the third information processing device 10 outputs warning information when a user's behavior is not permitted. This makes it possible to prevent the user from engaging in unauthorized behavior. Note that the warning information does not directly restrict the user's behavior, as in the case of a physical gate, for example, and therefore can encourage the user to voluntarily change their behavior. Therefore, compared to when direct behavioral restrictions are imposed, hospitality toward the user can be improved. Furthermore, since there is no need to deploy security guards or the like on-site, costs such as labor costs can be reduced.

[0082] <Fourth embodiment> A fourth information processing device 10 will be described with reference to Fig. 13. Note that the fourth information processing device 10 represents a specific application example of the first to third information processing devices 10 described above, and its configuration and operation may be similar to those of the first to third information processing devices 10. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0083] (Application Example to Facilities) First, the configuration and operation of the fourth information processing device 10 will be described with reference to Fig. 13. Fig. 13 is a conceptual diagram showing an operation example in a facility to which the fourth information processing device is applied.

[0084] 13 , in the fourth information processing device 10, the biometric information acquisition unit 110 first acquires biometric information of a user attempting to enter a facility, and the authentication unit 120 executes authentication processing using the user's biometric information. These processes may be performed, for example, at an entrance gate equipped with a camera installed at the entrance of the facility.

[0085] Next, the tracking unit 140 tracks the user moving within the facility. Then, when the user approaches a display unit 200 installed within the facility, the guidance information output unit 150 outputs guidance information. This guidance information is changed or adjusted individually to suit the user. A plurality of display units 200 may be installed within the facility. In this case, the guidance information may be displayed on each of the plurality of display units 200.

[0086] The content of the guidance information may change depending on the results of tracking by the tracking unit 140. For example, when the authentication unit 120 executes the authentication process, multiple pieces of guidance information corresponding to the authenticated user may be stored, and guidance information to be actually displayed may be selected from the multiple pieces of guidance information depending on the tracking results of the authenticated user (e.g., the position of the display the user approached). Note that if the user performs an action in accordance with the guidance information after the guidance information is displayed, the multiple pieces of guidance information stored and the user information corresponding to the guidance information may be deleted.

[0087] The above-mentioned facility may be, for example, a movie theater. In a movie theater, as described in the second embodiment ( FIG. 6 ), the screen and seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the above-mentioned facility may be a stadium where sports such as baseball or soccer are played, a live concert venue, or an event venue. In these facilities, the seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the above-mentioned facility may be a transportation hub such as an airport. In an airport, the boarding gate and seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the above-mentioned facility may be an accommodation facility such as a hotel. In a hotel, the user's room number and the direction to proceed may be output as guidance information.

[0088] Facilities to which the fourth information processing device 10 can be applied are not limited to the above-mentioned examples. The fourth information processing device 10 can be applied to any facility that is required to present some kind of guidance information to users who enter the facility.

[0089] (Technical Effects) Next, technical effects obtained by the fourth information processing apparatus 10 will be described.

[0090] As described in FIG. 13 , in the fourth information processing device 10, after the authentication process is performed on a user entering a facility, guidance information is output based on the tracking results within the facility. In this manner, it is possible to provide appropriate guidance to each user entering the facility. Note that, although the above-described embodiment illustrates an example in which the authentication process is performed at the entrance to the facility, the authentication process may also be performed at a location other than the entrance. For example, the authentication process may be performed in a predetermined area within the facility (e.g., a hotel check-in counter, an airport boarding gate, etc.).

[0091] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0092] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0093] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0094] (Supplementary Note 1) The information processing device described in Supplementary Note 1 is an information processing device that includes an acquisition means for acquiring biometric information of a user, an authentication means for performing authentication processing of the user using the biometric information, a storage means for storing appearance information indicating the appearance characteristics of the user who performed the authentication processing, a tracking means for tracking the user using the appearance information, and an output means for outputting guidance information tailored to the user based on the results of the tracking.

[0095] (Supplementary Note 2) The information processing device according to Supplementary Note 2 is the information processing device according to Supplementary Note 1, wherein the output means causes a display unit to display the guidance information together with an image representing the user.

[0096] (Supplementary Note 3) The information processing device according to Supplementary Note 3 is the information processing device according to Supplementary Note 1 or 2, wherein the output means simultaneously displays guidance information corresponding to a plurality of the users on one display unit.

[0097] (Appendix 4) The information processing device described in Appendix 4 is the information processing device described in any one of Appendices 1 to 3, further comprising a determination means for determining whether the user's action is permitted or not, and a warning means for outputting warning information when the user's action is not permitted.

[0098] (Supplementary Note 5) The information processing device described in Supplementary Note 5 is the information processing device described in Supplementary Note 4, in which the determination means determines whether the user's behavior is permitted based on information about the user's belongings.

[0099] (Supplementary Note 6) The information processing device described in Supplementary Note 6 is the information processing device described in Supplementary Note 4 or 5, wherein the determination means determines whether the user's behavior is permitted based on information about other users accompanying the user.

[0100] (Supplementary Note 7) The information processing device according to Supplementary Note 7 is the information processing device according to any one of Supplementary Notes 1 to 6, wherein the storage unit acquires appearance information of the user from the biometric information of the user.

[0101] (Appendix 8) The information processing device described in Appendix 8 is the information processing device described in any one of Appendices 1 to 7, wherein the authentication means executes the authentication process for the user entering a facility, the tracking means tracks the user in the facility, and the output means displays guidance information tailored to the user on a display unit installed in the facility.

[0102] (Appendix 9) The information processing method described in Appendix 9 is an information processing method that, using at least one computer, acquires biometric information of a user, performs authentication processing of the user using the biometric information, stores appearance information indicating external characteristics of the user who performed the authentication processing, tracks the user using the appearance information, and outputs guidance information tailored to the user based on the results of the tracking.

[0103] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which includes acquiring biometric information of a user, performing an authentication process for the user using the biometric information, storing appearance information indicating external characteristics of the user who performed the authentication process, tracking the user using the appearance information, and outputting guidance information tailored to the user based on the results of the tracking.

[0104] (Appendix 11) The computer program described in Appendix 11 is a computer program that causes at least one computer to execute an information processing method that acquires biometric information of a user, performs authentication processing of the user using the biometric information, stores appearance information that indicates external characteristics of the user that performed the authentication processing, tracks the user using the appearance information, and outputs guidance information tailored to the user based on the results of the tracking.

[0105] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0106] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 50 User 60 User image 110 Biometric information acquisition unit 120 Authentication unit 130 Appearance information storage unit 140 Tracking unit 150 Guidance information output unit 160 Determination unit 170 Warning unit 200 Display unit 210, 220 Display 310, 320 Camera

Claims

1. An information processing apparatus comprising: an acquisition means for acquiring biometric information of a user; an authentication means for performing an authentication process of the user using the biometric information; a storage means for storing appearance information indicating characteristics of the appearance of the user who has performed the authentication process; a tracking means for tracking the user using the appearance information; and an output means for outputting guidance information adapted to the user based on the result of the tracking.

2. The information processing apparatus according to claim 1, wherein the output means causes the guidance information to be displayed on a display unit together with an image indicating the user.

3. The information processing apparatus according to claim 1 or 2, wherein the output means causes guidance information corresponding to a plurality of the users to be simultaneously displayed on one display unit.

4. The information processing apparatus according to claim 1 or 2, further comprising: a determination means for determining whether or not the action of the user is permitted; and a warning means for outputting warning information when the action of the user is not permitted.

5. The information processing apparatus according to claim 4, wherein the determination means determines whether or not the action of the user is permitted based on information regarding the belongings of the user.

6. The information processing apparatus according to claim 4, wherein the determination means determines whether or not the action of the user is permitted based on information regarding another user accompanying the user.

7. The information processing apparatus according to claim 1 or 2, wherein the storage means acquires the appearance information of the user from the biometric information of the user.

8. The information processing apparatus according to claim 1 or 2, wherein the authentication means performs the authentication process on the user entering the facility, the tracking means tracks the user in the facility, and the output means causes the guidance information adapted to the user to be displayed on a display unit installed in the facility.

9. An information processing method, wherein at least one computer acquires biometric information of a user, executes authentication processing of the user using the biometric information, stores appearance information indicating characteristics of the appearance of the user who has executed the authentication processing, performs tracking of the user using the appearance information, and outputs guidance information tailored to the user based on the result of the tracking.

10. A recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, the method including acquiring biometric information of a user, executing authentication processing of the user using the biometric information, storing appearance information indicating characteristics of the appearance of the user who has executed the authentication processing, performing tracking of the user using the appearance information, and outputting guidance information tailored to the user based on the result of the tracking.

Images