Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises: an acquisition means for acquiring an image of a user; a conversion means for generating a converted image by replacing at least a part of the user included in the image of the user with an avatar image corresponding to the user; and an output means for outputting a guide image in which guidance information for the user is superimposed on the converted image. Such an information processing device makes it possible to appropriately present guidance information to a user while respecting the user's privacy.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include one that outputs information related to authentication processing to a user who is the target of authentication. For example, Patent Document 1 discloses a technology that, when a user's approach is detected, displays an initial screen indicating that face authentication is in a standby state, and, when the user's face is detected from a live image, displays a notification screen indicating the progress of face authentication.

[0003] JP 2023-101822 A

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein includes an acquisition means for acquiring an image of a user, a conversion means for generating a converted image by replacing at least a portion of the user included in the image of the user with an avatar image corresponding to the user, and an output means for outputting a guidance image in which guidance information for the user is superimposed on the converted image.

[0006] One aspect of the information processing method disclosed herein involves using at least one computer to acquire an image of a user, generate a converted image by replacing the facial portion of the user contained in the image of the user with an avatar image corresponding to the user, and output a guidance image in which guidance information for the user is superimposed on the anonymized image.

[0007] One aspect of the recording medium of this disclosure is a computer program recorded on at least one computer that causes the computer to execute an information processing method, which includes acquiring an image of a user, generating a converted image by replacing the user's facial portion included in the image of the user with an avatar image corresponding to the user, and outputting a guidance image in which guidance information for the user is superimposed on the anonymized image.

[0008] 1 is a block diagram showing a hardware configuration of a first information processing device. 2 is a block diagram showing a functional configuration of the first information processing device. 3 is a flowchart showing the flow of operation of the first information processing device. 4 is a plan view showing an example of each image processed in the first information processing device. 5 is a block diagram showing a functional configuration of a second information processing device. 6 is a flowchart showing the flow of avatar generation operation in the second information processing device. 7 is a diagram showing example user information in the second information processing device. 8 is a plan view showing an example of coloring of avatar images in the second information processing device. 9 is a block diagram showing a functional configuration of a third information processing device. 10 is a flowchart showing the flow of operation of the third information processing device. 11 is a schematic diagram showing an example of operation of the third information processing device. 12 is a plan view showing an example of display of a guide image in the third information processing device. 13 is a block diagram showing a functional configuration of a fourth information processing device. 14 is a flowchart showing the flow of operation of the fourth information processing device. 15 is a conceptual diagram showing an example of operation in a facility to which the fourth information processing device is applied. 16 is a plan view showing an example of display of an avatar image in a fifth information processing device. 17 is a plan view showing an example of display of a guide image in the fifth information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 4. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, a first information processing device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing device 10 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing device 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, a functional block for executing a process of outputting a guidance image to the user is realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing device 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that the information processing device 10 stores for a long period of time. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as part of a smartphone, a tablet terminal, an earphone-type terminal, a watch-type terminal, an HMD (Head Mounted Display) terminal, etc. The input device 15 may be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display or digital signage) that can display information related to the information processing device 10. The output device 16 may also be a speaker or the like that can output information related to the information processing device 10 as audio.

[0020] 1 may be configured to be included in a device external to the first information processing device 10. For example, the first information processing device 10 may be configured to include a processor 11, a RAM 12, and a ROM 13, and the other devices, such as a storage device 14, an input device 15, and an output device 16, may be configured as external devices. That is, the first information processing device 10 may be configured as an information processing system including a plurality of different devices. Furthermore, some of the calculation functions of the first information processing device 10 may be realized by an external server, a cloud, or the like.

[0021] (Functional Configuration) Next, the functional configuration of the first information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first information processing device.

[0022] 2, the first information processing device 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, an image conversion unit 120, and a guide image output unit 130. Note that each of the image acquisition unit 110, the image conversion unit 120, and the guide image output unit 130 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0023] The image acquisition unit 110 is configured to be able to acquire an image including a user. The image acquisition unit 110 acquires an image of a user, for example, by taking a picture of the user attempting to enter a predetermined facility or a predetermined area with a camera. Hereinafter, the image acquired by the image acquisition unit 110 will be referred to as a "captured image" as appropriate. The captured image may include the entire body of the user, or may include only a part of the user (for example, the face). Furthermore, the captured image may include one user, or may include multiple users. The captured image acquired by the image acquisition unit 110 is configured to be output to the image conversion unit.

[0024] The image conversion unit 120 is configured to generate a converted image by replacing at least a portion of a user included in a captured image with an avatar image corresponding to the user. Here, "at least a portion" refers to a portion of the user's body part included in the captured image that requires consideration of the user's privacy. Hereinafter, this portion will be referred to as a "converted portion" as appropriate. The converted portion may be, for example, the user's face. However, the converted portion may also be the user's entire body. The image conversion unit 120 replaces the converted portion of the user in the captured image with an avatar image. The avatar image is an image that can protect the privacy of the converted portion. An avatar image is prepared for each user, and different avatar images are used for different users. For example, if the user image includes multiple users, the image conversion unit 120 replaces the facial portions of each of the multiple users with avatar images with different faces. The avatar images may be pre-registered images. Alternatively, the avatar images may be generated by the first information processing device 10. The configuration for generating avatar images will be described in detail in another embodiment described below.

[0025] The guidance image output unit 130 is configured to generate a guidance image by superimposing guidance information on the converted image generated by the image conversion unit 120. The guidance information may be, for example, guidance information for the user (e.g., information indicating a destination or a direction to go, etc.). The guidance information may also be information indicating the user's situation or state. The guidance information is information individually changed or adjusted for each user. Therefore, if there are multiple users, the guidance information may be different information corresponding to each of the multiple users. The guidance image output unit 130 may superimpose the guidance image around the user on the converted image. That is, the guidance image output unit 130 may superimpose the guidance information at a position where it is easy to understand that the guidance information is intended for that user. If the converted image includes multiple users, the guidance image output unit 130 may superimpose different guidance information around each of the multiple users. The guidance information may include, for example, a message or an image. The guidance information may also be included in the avatar image itself. That is, the avatar image itself may be an image indicating guidance information for the user. In this case, the image conversion unit 120 also functions as the guide image output unit 130 .

[0026] The guide image output unit 130 is configured to be able to output the generated guide image. The guide image may be displayed, for example, by a display device (e.g., the output device 16 shown in FIG. 1 ) included in the first information processing device 10. Alternatively, the guide image may be displayed by a display device provided separately from the first information processing device 10. Note that multiple display devices may be provided to display the guide image. The guide image may be displayed, for example, when the user can see it. Specifically, the guide image may be displayed when the user approaches the display device. In this case, the image acquisition unit 110 may acquire an image of the user by photographing the user with, for example, a camera provided in the display device. The display device may have a function to display the guide image in a horizontally inverted manner.

[0027] (Flow of Operation) Next, the flow of operation in the first information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the first information processing device.

[0028] 3, when the operation of the first information-processing device 10 is started, the image acquisition unit 110 first acquires a captured image including a user (step S101). Then, the image conversion unit 120 replaces a converted portion of the user included in the captured image with an avatar image to generate a converted image (step S102).

[0029] Next, the guide image output unit 130 generates a guide image by superimposing the guide information on the converted image (step S103). Then, the guide image output unit 130 outputs the generated guide image (step S104). The guide image output by the guide image output unit 130 may be displayed on a display device, as already described.

[0030] (Image Example) Next, each image processed by the first information processing device 10 will be specifically described with reference to Fig. 4. Fig. 4 is a plan view showing an example of each image processed by the first information processing device.

[0031] 4, the first information processing device 10 first acquires a captured image by the image acquisition unit 110. The captured image includes a part of the user whose privacy should be considered. For example, the captured image includes the user's face.

[0032] The first information processing device 10 then replaces the user's face in the captured image with an avatar image to generate a converted image. In the converted image, the user's face is hidden by the avatar image, as shown in Fig. 4. Therefore, even if another user views the converted image, the user's face cannot be recognized.

[0033] Thereafter, the first information processing device 10 generates a guide image by superimposing guide information on the captured image. In the example shown in Fig. 4, a message saying "Your seat is on the right" is displayed above the user. Also, an arrow for guiding the user to the right is displayed. Both the message and the arrow are guide information. In this way, the guide image may be generated to include multiple pieces of guide information.

[0034] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0035] As described with reference to FIGS. 1 to 4 , the first information-processing device 10 outputs a guidance image in which at least a portion of the user is replaced with an avatar image and guidance information is superimposed on the avatar image. This makes it possible to appropriately present guidance information to the user while respecting the user's privacy. Specifically, by replacing at least a portion of the user with an avatar image, the user's privacy is protected. Furthermore, by superimposing guidance information, it becomes possible to provide appropriate guidance to each user.

[0036] Second Embodiment A second information processing device 10 will be described with reference to Figures 5 to 8. The second information processing device 10 differs in some configurations and operations from the first information processing device 10 described above, but other parts may be similar to the first information processing device 10. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0037] (Functional Configuration) First, the functional configuration of the second information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 5, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0038] 5, the second information processing device 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, an image conversion unit 120, a guide image output unit 130, a user information acquisition unit 140, and an avatar generation unit 150. That is, the second information processing device 10 is configured to further include, in addition to the configuration of the first information processing device (see FIG. 2), a user information acquisition unit 140 and an avatar generation unit 150. Each of the user information acquisition unit 140 and the avatar generation unit 150 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0039] The user information acquisition unit 140 is configured to be able to acquire user information, which is information related to the user. The user information acquisition unit 140 may acquire the user information from, for example, a captured image acquired by the image acquisition unit 110. Alternatively, the user information acquisition unit 140 may acquire the user information from personal information input in advance by a user or the like. The user information acquisition unit 140 may acquire multiple types of user information. Specific examples of user information will be described in detail later.

[0040] The avatar generation unit 150 is configured to generate an avatar image based on the user information acquired by the user information acquisition unit 140. The avatar generation unit 150 may generate an avatar image that reflects the user's characteristics based on the user information. For example, the avatar generation unit 150 may generate an avatar image that resembles the user. In this way, the user can easily identify their own appearance, which is replaced by the avatar image in the guidance image. However, the avatar generation unit 150 may also intentionally generate an avatar image that does not resemble the user. In this case, the anonymity of the user in the guidance image can be improved. Specific examples of avatar images generated by the avatar generation unit 150 will be described in detail later.

[0041] (Avatar Generation Operation) Next, the flow of the avatar generation operation in the second information processing device 10 (i.e., the operation when the avatar generation unit 150 generates an avatar) will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the avatar generation operation in the second information processing device.

[0042] 6, when the avatar generation operation in the second information-processing device 10 is started, the user information acquisition unit 140 first acquires user information (step S201). When acquiring user information from a captured image, the user information acquisition unit 140 may acquire the user information after the image acquisition unit 110 acquires the captured image (i.e., after step S101 in FIG. 3). On the other hand, when acquiring user information from a source other than the captured image, the user information acquisition unit 140 may acquire the user information before the image acquisition unit 110 acquires the captured image (i.e., before step S101 in FIG. 3).

[0043] Next, avatar generation unit 150 generates an avatar image based on the user information (step S202). Then, avatar generation unit 150 outputs the generated avatar image to image conversion unit 120. Image conversion unit 120 in second information-processing device 10 generates a converted image using the avatar image generated by avatar generation unit 150.

[0044] (User Information) Next, the user information acquired by the user information acquisition unit 140 will be specifically described with reference to Fig. 7. Fig. 7 is a diagram showing an example of user information in the second information processing apparatus.

[0045] As shown in FIG. 7 , the user information acquisition unit 140 acquires at least one of attribute information and appearance information of the user as user information. The attribute information is, for example, information about the user's age, gender, etc. The attribute information may be acquired, for example, from information input by the user, or may be estimated by analyzing a captured image that includes the user. The appearance information is, for example, information about the user's hairstyle and headwear. Examples of headwear include glasses, earrings, piercings, hair clips, hats, masks, etc.

[0046] By using attribute information and appearance information as user information, it is possible to generate an avatar image that resembles the user. This allows the user to easily recognize the avatar image. In particular, by acquiring appearance information from a captured image, it is possible to reflect the user's appearance on that day, thereby generating an avatar image that is more easily recognized by the user.

[0047] (Example of Avatar Generation) Next, a method for generating an avatar by avatar generation unit 150 will be described in detail with reference to Fig. 8. Fig. 8 is a plan view showing an example of coloring of avatar images in the second information processing device.

[0048] 8, the second information-processing device 10 generates an avatar image by combining several pre-prepared features (hereinafter referred to as "common features"), for example, common features are prepared for each part of the face (hairstyle, face shape, eyes, nose, mouth, etc.). The avatar generation unit 150 combines the common features for each part based on the user information to generate the base of the avatar image.

[0049] The avatar generation unit 150 then generates an avatar image corresponding to each user by applying different colors to the common features of each body part. In this way, even if the number of combinations of common features is limited (for example, if there are only a few types of common features), a large number of avatar images can be generated by varying the colors of each body part. Therefore, even if there are multiple users, different avatar images can be generated for each user.

[0050] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0051] 5 to 8, second information-processing device 10 generates an avatar based on user information, so that even if no avatar is prepared in advance, an avatar corresponding to the user can be appropriately generated.

[0052] <Third Embodiment> A third information processing device 10 will be described with reference to Figures 9 to 12. The third information processing device 10 differs in some configurations and operations from the first and second information processing devices 10 described above, but other parts may be similar to the first and second information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0053] (Functional Configuration) First, the functional configuration of the third information processing device 10 will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 9, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0054] 9, the third information processing device 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, an image conversion unit 120, a guide image output unit 130, and a first authentication unit 160. That is, the third information processing device 10 is configured to further include the first authentication unit 160 in addition to the configuration of the first information processing device (see FIG. 2). The first authentication unit 160 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0055] The first authentication unit 160 is configured to be able to perform user authentication processing. Specifically, the first authentication unit 160 is configured to be able to perform biometric authentication using the user's biometric information. For example, the first authentication unit 160 is configured to acquire biometric information from a captured image acquired by the image acquisition unit 110 and perform user authentication processing. More specifically, the first authentication unit 160 may be configured to be able to perform face authentication using the user's facial information. Alternatively, the authentication unit 160 may be configured to be able to perform iris authentication using the user's iris information. However, the authentication processing performed by the first authentication unit 160 is not limited to the above-described example, and the authentication processing may be performed using various types of biometric information such as a fingerprint or palm print.

[0056] The authentication process performed by the first authentication unit 160 may be a process for determining whether the user whose biometric information has been acquired is a pre-registered user. In other words, the authentication process may be a process for identifying the user. In this case, the authentication unit 120 may compare the biometric information acquired from the user with the biometric information of the pre-registered registered user to determine whether the user is a pre-registered user. For example, the authentication unit 120 may determine that the user is a pre-registered user if the degree of match between the biometric information acquired from the user and the biometric information of the pre-registered registered user exceeds a predetermined value.

[0057] The image conversion unit 120 in the third information processing device 10 is configured to generate a converted image based on the result of the authentication process performed by the first authentication unit 160. Specifically, the image conversion unit 120 generates the converted image by replacing at least a portion of a user who has successfully been authenticated with an avatar image. On the other hand, the image conversion unit 120 generates the converted image by replacing at least a portion of a user who has not successfully been authenticated with a silhouette image. The silhouette image may be, for example, an image in which the user's face is filled in with a single color. Alternatively, the silhouette image may be an image in which the user's face is pixelated. Similar to an avatar image, the silhouette image makes the user's transformed parts indistinguishable from other users. However, the silhouette image does not have to be an image prepared separately for each user, as with an avatar image. In other words, the silhouette image may be an image that does not have any distinctiveness.

[0058] The image conversion unit 120 may replace a portion of the user with a silhouette image before the authentication process is performed by the first authentication unit 160 (for example, when the user is detected or when the user enters the image capture range). In this case, the silhouette image before the authentication process and the silhouette image after the authentication process may be different. For example, the image conversion unit 120 may first replace a portion of a user for whom the authentication process has not been performed with a first silhouette image, and then, for users for whom the authentication process is not successful, replace the first silhouette image with a second silhouette image. For users for whom the authentication process is successful, the image conversion unit 120 may replace the first silhouette image with an avatar image.

[0059] (Operation Flow) Next, the operation flow in the third information processing device 10 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 10, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0060] 10 , when the operation of the third information processing device 10 starts, the image acquisition unit 110 first acquires a captured image including the user (step S101). The image acquisition unit 110 may acquire the captured image when the user approaches a display device that displays a guidance image. When the captured image is acquired, the first authentication unit 160 acquires biometric information of the user from the captured image and starts authentication processing (step S301).

[0061] Next, the image conversion unit 120 generates a converted image by replacing the converted body parts of the user included in the captured image with avatar images (step S302). At this time, the image conversion unit 120 replaces the converted body parts of users who have been successfully authenticated by the first authentication unit 160 with avatar images. On the other hand, the image conversion unit 120 replaces the converted body parts of users who have not been successfully authenticated by the first authentication unit 160 with silhouette images.

[0062] Next, the guide image output unit 130 generates a guide image by superimposing the guide information on the converted image (step S103).The guide image output unit 130 then outputs the generated guide image (step S104).The guide image output by the guide image output unit 130 is displayed on a display device.

[0063] As described above, in the third information processing device 10, a converted image is generated in real time after a photographed image is acquired, and a guide image is displayed on the display device in real time. In this way, it is possible to execute authentication processing for a user and immediately notify the user of the result. This technical effect does not require complete real-time processing, and can be reasonably achieved even when there is some delay due to various processes, for example.

[0064] The authentication process by the first authentication unit 160 does not necessarily have to be performed in real time. For example, the first authentication unit 160 may acquire biometric information of the user from a source other than the captured image before acquiring the captured image and perform the authentication process. In this case, the result of the authentication process performed in advance is notified to the user by a guide image.

[0065] (Display Example) Next, a display example by the third information processing device 10 will be specifically described with reference to Fig. 11 and Fig. 12. Fig. 11 is a schematic diagram showing an operation example of the third information processing device. Fig. 12 is a plan view showing a display example of a guide image in the third information processing device.

[0066] 11 , in the third information processing device, an image of a user approaching the display device 200 is acquired by the image acquisition unit 110. Specifically, the image acquisition unit 110 acquires a captured image including the user by capturing an image of the user using the camera 300 provided in the display device 200. In the example of FIG. 11 , a captured image including users 51, 52, and 53 is acquired.

[0067] It should be noted that users 51, 52, and 53 are proceeding toward the display device 200. In this manner, the first authentication unit 160 may be configured to appropriately authenticate users approaching the display device 200. Here, it is assumed that the authentication process for users 51 and 52 has been successful, but the authentication process for user 53 has not been successful.

[0068] 12 , the face of a user image 61 corresponding to a user 51 who has been successfully authenticated has been replaced with an avatar image. Also, a message saying "Your seat is on the right" and an arrow encouraging the user to move to the right are displayed around the user image 61 as guidance information. Similarly, the face of a user image 62 corresponding to a user 52 who has been successfully authenticated has been replaced with an avatar image. Also, a message saying "Your seat is on the left" and an arrow encouraging the user to move to the left are displayed around the user image 62 as guidance information.

[0069] On the other hand, the face portion of the user image 63 corresponding to the user 53 who has not been successfully authenticated has been replaced with a silhouette image. Also, a message stating "Cannot be used because authentication has not been successful" is displayed around the user image 63 as guidance information. Note that even for the user 53 who has been replaced with a silhouette image, if the subsequent authentication process is successful, the silhouette image may be replaced with an avatar image. In this way, it is possible to inform the user that the authentication process, which initially failed, has succeeded in the subsequent process.

[0070] (Technical Effects) Next, technical effects obtained by the third information processing apparatus 10 will be described.

[0071] As described with reference to FIGS. 9 to 12 , in the third information processing device 10, the converted body parts of users who have been successfully authenticated are replaced with avatar images, and the converted body parts of users who have not been successfully authenticated are replaced with silhouette images. In this way, it is possible to confirm whether the authentication process has been successful or not based on whether the converted body parts have been replaced with avatar images or silhouette images. That is, a user whose part of himself or herself has been replaced with an avatar image can recognize that his or her authentication process has been successful. On the other hand, a user whose part of himself or herself has been replaced with a silhouette image can recognize that his or her authentication process has not been successful. Therefore, for example, a user whose authentication process has not been successful can be prompted to take an action that will result in the authentication process being successful (for example, turning his or her face toward the camera or removing an accessory from his or her head).

[0072] <Fourth embodiment> A fourth information processing device 10 will be described with reference to Figures 13 to 15. The fourth information processing device 10 differs in some configurations and operations from the first to third information processing devices 10 described above, but other parts may be similar to the first to third information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0073] (Functional Configuration) First, the functional configuration of the fourth information processing device 10 will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the functional configuration of the fourth information processing device. Note that in Fig. 13, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0074] 13, the fourth information processing device 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, an image conversion unit 120, a guide image output unit 130, a second authentication unit 170, and a tracking unit 180. That is, the fourth information processing device 10 is configured to further include, in addition to the configuration of the first information processing device (see FIG. 2), the second authentication unit 170 and the tracking unit 180. Each of the second authentication unit 170 and the tracking unit 180 may be realized, for example, by the above-mentioned processor 11 (see FIG. 1).

[0075] The second authentication unit 170 is configured to be able to perform user authentication processing. Specifically, the second authentication unit 170 is configured to be able to perform biometric authentication using the user's biometric information. That is, the second authentication unit 170 has the same functions as the first authentication unit 160 (see FIG. 9 ) in the third information processing device 10. However, the second authentication unit 170 is configured to be able to acquire the user's biometric information from a source other than the captured image acquired by the image acquisition unit 110. For example, the second authentication unit 170 may acquire the user's biometric information from an image captured separately before the captured image was captured (for example, an image captured at a location different from the captured image).

[0076] The tracking unit 180 is configured to be able to track a user who has executed the authentication process using the second authentication unit 170. The tracking unit 180 may track the user, for example, using appearance information of the user who executed the authentication process. Specifically, the tracking unit 180 stores appearance information of the user who executed the authentication process and tracks the location of the user who matches the stored appearance information. The tracking unit 180 may also track the user while storing the route taken by the user. The tracking unit 180 may track the user, for example, using multiple cameras. Note that the specific tracking method is not particularly limited, and the user may be tracked using existing technology as appropriate. The tracking unit 180 may be configured to track the user, for example, within a predetermined area (e.g., within a predetermined facility). In this case, the tracking unit 180 may be configured to automatically end tracking of the user who leaves the predetermined area.

[0077] The results of tracking by the tracking unit 180 are used by both the image conversion unit 120 and the guide image output unit 130. A specific method for using the results of tracking will be described in detail in the following operation description.

[0078] (Operation Flow) Next, the operation flow in the third information processing device 10 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 10, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0079] 10 , when the operation of the third information processing device 10 is started, the second authentication unit 170 first acquires biometric information of the user (step S401). Then, the second authentication unit 170 executes authentication processing of the user using the acquired biometric information (step S402).

[0080] Next, the tracking unit 180 starts tracking the user for whom the authentication process has been executed (step S403). The tracking unit determines whether the user has approached the display device 200 while tracking the user (step S404). Whether the user has approached the display device 200 may be determined using the tracking result (e.g., the user's current location). Note that if the user has not approached the display device 200 (step S404: NO), the tracking unit 180 continues tracking the user.

[0081] On the other hand, if the user is approaching the display device 200 (step S404: YES), the image acquisition unit 110 acquires a captured image including the user (step S101). Then, the image conversion unit 120 generates a converted image by replacing the converted body part of the user included in the captured image with an avatar image (step S405). At this time, the image conversion unit 120 generates the converted image based on the tracking result of the tracking unit 180. Specifically, the image conversion unit 120 identifies the user included in the captured image based on the tracking result, and replaces the converted body part of the user included in the captured image with an avatar image corresponding to the identified user.

[0082] The image conversion unit 120 may convert the converted body parts of the user into an avatar image and a silhouette image, as described in the third information processing device (see FIG. 12 ). That is, the converted image may be generated by replacing the facial portion of a user who has been successfully authenticated by the second authentication unit 170 with an avatar image, and by replacing the facial portion of a user who has not been successfully authenticated by the second authentication unit 170 with a silhouette image.

[0083] Next, the guide image output unit 130 generates a guide image by superimposing guide information on the converted image (step S406). At this time, the guide image output unit 130 generates the guide image based on the tracking result of the tracking unit 180. Specifically, the guide image output unit 130 identifies a user included in the converted image based on the tracking result, and superimposes guide information corresponding to the identified user.

[0084] Then, the guide image output unit 130 outputs the generated guide image (step S104). As a result, the guide image output by the output unit 130 is displayed to the user identified based on the tracking result.

[0085] (Application Example to Facilities) Next, a specific application example of the fourth information processing device 10 will be described with reference to Fig. 15. Fig. 15 is a conceptual diagram showing an operation example in a facility to which the fourth information processing device is applied.

[0086] 15 , in the fourth information processing device 10, the second authentication unit 170 first acquires biometric information of a user attempting to enter a facility and performs authentication processing using the biometric information. This processing may be performed, for example, at an entrance gate equipped with a camera installed at the entrance of the facility.

[0087] Next, the tracking unit 180 tracks the user moving within the facility. Then, when the user approaches a display device 200 installed within the facility, the image acquisition unit 110 acquires a captured image. Then, the image conversion unit 120 generates a converted image from the captured image. After that, the guide image output unit 130 outputs a guide image in which the guide image is superimposed on the converted image. This guide image includes guide information that has been individually changed or adjusted to suit the user. A plurality of display devices 200 may be installed within the facility. In this case, a guide image may be displayed on each of the plurality of display devices 200.

[0088] The facility described above may be, for example, a movie theater. In a movie theater, for example, the screen and seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the facility described above may be a stadium where sports such as baseball or soccer are played, a live music venue, or an event venue. In these facilities, for example, the seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the facility described above may be a transportation hub such as an airport. In an airport, for example, the boarding gate and seat number corresponding to the user's ticket and the direction to proceed may be output as guidance information. Alternatively, the facility described above may be an accommodation facility such as a hotel. In a hotel, for example, the user's room number and the direction to proceed may be output as guidance information.

[0089] The image conversion unit 120 may change the avatar image depending on the type of facility. For example, the image conversion unit 120 may add items to the avatar image depending on the facility. The items may include, for example, earrings, sunglasses, hats, clothes, and other accessories, as well as bags, various goods, and other belongings. More specifically, earrings with soccer balls may be added to the avatar image of a user visiting a soccer stadium. A T-shirt with a logo related to the concert may be added to the avatar image of a user visiting a live music venue. A merchandise corresponding to the ticket (for example, a mask of a character appearing in the movie) may be added to the avatar image of a user visiting a movie theater.

[0090] Furthermore, even within the same facility, the avatar image may be different for each display device. For example, image conversion unit 120 may add a red hat to the avatar image when displayed on a first display device installed near the entrance gate, and add a blue bag to the avatar image when displayed on a second display device installed in an area after passing through the entrance gate.

[0091] The above-described items may be provided according to the result of the authentication process by the first authentication unit 160 or the second authentication unit 170. For example, an individual may be identified by first performing facial authentication on the user, and then information linked to the identified individual (e.g., ticket information) or an item corresponding to the location information of the identified individual may be added to the avatar image.

[0092] Furthermore, items may be changed depending on the number of visits to the facility. For example, the number of visits may be counted and linked to personal information, and the more visits a user makes, the more luxurious the items may appear or the more items may be added. For example, sunglasses may be added to the avatar image of a user who has visited the facility one to three times, a bag may be added to the avatar image of a user who has visited the facility four to nine times, and a fur coat may be added to the avatar image of a user who has visited the facility ten or more times.

[0093] The item added to the avatar described above may include guidance information. That is, the item may be guidance information superimposed on the avatar image by the guidance image output unit 130. Specifically, the shape of the item may indicate a destination. In this case, for example, the item may have a shape including an arrow indicating the direction of the destination. Alternatively, the color of the item may correspond to the destination. In this case, for example, a color may be assigned to each area within the relocation (e.g., red zone, blue zone, yellow zone, etc.), and the color of the item may be the color of the destination area.

[0094] Facilities to which the fourth information processing device 10 can be applied are not limited to the above-mentioned examples. The fourth information processing device 10 can be applied to any facility that is required to present some kind of guidance information to users who enter the facility. By applying the fourth information processing device, it is possible to provide appropriate guidance to multiple users entering the facility while taking privacy into consideration.

[0095] (Technical Effects) Next, technical effects obtained by the fourth information processing apparatus 10 will be described.

[0096] As described in FIG. 13 , in the fourth information processing device 10, tracking of the user for whom authentication processing has been performed is performed. Then, a converted image is generated based on the tracking results. Furthermore, when generating a converted image and a guide image in which a guide image is generated based on the tracking results, it is required to identify the user in the image. For this reason, for example, in the third information processing device 10 already described, the first authentication unit 160 performs user authentication processing after acquiring a captured image (see, for example, FIG. 10 ). On the other hand, in the fourth information processing device, it is possible to identify the user in the image based on the tracking results by the tracking unit 180. Therefore, it is not necessary to perform authentication processing again after acquiring a captured image. Therefore, by eliminating the need to perform authentication processing each time, it is possible to efficiently output a guide image.

[0097] Fifth Embodiment A fifth information processing device 10 will be described with reference to Figures 16 and 17. Note that the fifth information processing device 10 differs in some of its operations from the first to fourth information processing devices 10 described above, but other parts may be similar to the first to fourth information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0098] <Avatar Image Output> As shown in FIG. 16 , guide image output unit 130 outputs an avatar image to user terminal 500 (e.g., a smartphone) owned by the user. Therefore, the avatar image is displayed on the display of user terminal 500. The avatar image displayed in this manner may be used by the user to confirm the avatar image corresponding to the user. If an avatar image has been registered in advance, guide image output unit 130 may output the avatar image to user terminal 500 before an image of the user is captured (e.g., before the user visits the facility). Furthermore, when generating an avatar image, guide image output unit 130 may output the avatar image to user terminal 500, for example, at the timing of generating an avatar, creating a converted image, or outputting a guide image.

[0099] <Guide Image Output> As shown in FIG. 17 , the guide image output unit 130 outputs a guide image to the user terminal 500. Therefore, the guide image is displayed on the display of the user terminal 500. In this case, the user can check the guide image on the user terminal 500 that the user owns. Note that the image output to the user terminal may be different from the guide image output to the fixed display device 200. For example, the display of the user terminal 500 is likely to be smaller in size than the display of the display device 200. Therefore, the guide image output unit 130 may generate a guide image suitable for a small display. For example, the guide image output unit 130 may trim a portion of the guide image or change the size of the characters.

[0100] (Technical Effects) Next, technical effects obtained by the fifth information processing apparatus 10 will be described.

[0101] As described with reference to FIGS. 16 and 7 , in the fifth information processing device 10, avatar images and guide images are output to the user terminal 500. By outputting avatar images to the user terminal 500, the user can easily recognize the avatar corresponding to the user. This technical effect is particularly pronounced when an avatar image that does not resemble the user is generated. Furthermore, by outputting a guide image to the user terminal 500, the user can check the guide image on the user terminal 500. Therefore, for example, the guide image can be checked without looking at the display device 200. This eliminates the need for a fixed display device 200, thereby reducing costs.

[0102] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0103] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0104] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0105] (Supplementary Note 1) The information processing device described in Supplementary Note 1 is an information processing device that includes an acquisition means for acquiring an image of a user, a conversion means for generating a converted image by replacing at least a part of the user included in the image of the user with an avatar image corresponding to the user, and an output means for outputting a guidance image in which guidance information for the user is superimposed on the converted image.

[0106] (Appendix 2) The information processing device described in Appendix 2 is the information processing device described in Appendix 1, further comprising an information acquisition means for acquiring user information about the user, and an avatar generation means for generating the avatar image based on the user information.

[0107] (Supplementary Note 3) The information processing device according to Supplementary Note 3 is the information processing device according to Supplementary Note 2, in which the user information includes at least one of information on attributes of the user and information on appearance of the user.

[0108] (Supplementary Note 4) The information processing device described in Supplementary Note 4 is the information processing device described in Supplementary Note 2 or 3, wherein the avatar generation means generates the avatar image by painting at least one of the parts constituting a pre-prepared face in a color different from that of other users.

[0109] (Appendix 5) The information processing device described in Appendix 5 is the information processing device described in any one of Appendices 1 to 4, further comprising a first authentication means that executes an authentication process for the user, and the conversion means replaces at least a portion of the user whose authentication process has been successful with the avatar image, and replaces at least a portion of the user whose authentication process has not been successful with a silhouette image.

[0110] (Appendix 6) The information processing device described in Appendix 6 is the information processing device described in any one of Appendices 1 to 5, wherein the acquisition means acquires an image of a user approaching a display device by taking a photograph with a camera, the conversion means generates the converted image in real time, and the output means displays the guidance image on the display device in real time.

[0111] (Appendix 7) The information processing device described in Appendix 7 is the information processing device described in Appendix 6, further comprising a second authentication means that performs authentication processing of the user outside the shooting range of the camera, and a tracking means that tracks the user who has performed the authentication processing, wherein the conversion means generates the converted image based on the results of the tracking, and the output means generates the guidance image based on the results of the tracking.

[0112] (Appendix 8) The information processing device described in Appendix 8 is the information processing device described in any one of Appendices 1 to 7, wherein the acquisition means acquires an image of the user entering a facility, and the output means displays the guidance image including guidance information within the facility on a display device installed within the facility.

[0113] (Supplementary Note 9) The information processing device according to Supplementary Note 9 is the information processing device according to any one of Supplementary Notes 1 to 8, wherein the output means outputs the avatar image to a user terminal held by the user.

[0114] (Supplementary Note 10) The information processing device according to Supplementary Note 10 is the information processing device according to any one of Supplementary Notes 1 to 9, wherein the output means outputs the guidance image to a user terminal held by the user.

[0115] (Appendix 11) The information processing method described in Appendix 11 is an information processing method that, by using at least one computer, acquires an image of a user, generates a converted image by replacing a facial portion of the user included in the image of the user with an avatar image corresponding to the user, and outputs a guidance image in which guidance information for the user is superimposed on the anonymized image.

[0116] (Appendix 12) The recording medium described in Appendix 12 is a recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, which includes acquiring an image of a user, generating a converted image by replacing a facial portion of the user included in the image of the user with an avatar image corresponding to the user, and outputting a guidance image in which guidance information for the user is superimposed on the anonymized image.

[0117] (Appendix 13) The computer program described in Appendix 13 is a computer program that causes at least one computer to execute an information processing method, which acquires an image of a user, generates a converted image by replacing a facial portion of the user included in the image of the user with an avatar image corresponding to the user, and outputs a guidance image in which guidance information for the user is superimposed on the anonymized image.

[0118] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0119] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 50 User 60 User image 110 Image acquisition unit 120 Image conversion unit 130 Guidance image output unit 140 User information acquisition unit 150 Avatar generation unit 160 First authentication unit 170 Second authentication unit 180 Tracking unit 200 Display device 500 User terminal

Claims

1. An information processing apparatus comprising: an acquisition unit that acquires an image of a user; a conversion unit that generates a converted image by replacing at least a part of the user included in the image of the user with an avatar image corresponding to the user; and an output unit that outputs a guidance image in which guidance information for the user is superimposed on the converted image.

2. The information processing apparatus according to claim 1, further comprising: an information acquisition unit that acquires user information about the user; and an avatar generation unit that generates the avatar image based on the user information.

3. The information processing apparatus according to claim 2, wherein the user information includes at least one of information regarding an attribute of the user and information regarding an appearance of the user.

4. The information processing apparatus according to claim 2 or 3, wherein the avatar generation unit generates the avatar image by painting at least one of parts constituting a face, which are prepared in advance, in a color different from that of other users.

5. The information processing apparatus according to claim 1 or 2, further comprising a first authentication unit that executes an authentication process for the user, wherein the conversion unit replaces at least a part of the user for whom the authentication process has succeeded with the avatar image, and replaces at least a part of the user for whom the authentication process has not succeeded with a silhouette image.

6. The information processing apparatus according to claim 1 or 2, wherein the acquisition unit acquires an image of the user approaching a display device by photographing with a camera, the conversion unit generates the converted image in real time, and the output unit causes the display device to display the guidance image in real time.

7. The information processing apparatus according to claim 6, further comprising: a second authentication unit that executes an authentication process for the user outside a shooting range of the camera; and a tracking unit that tracks the user for whom the authentication process has been executed, wherein the conversion unit generates the converted image based on a result of the tracking, and the output unit generates the guidance image based on the result of the tracking.

8. The information processing apparatus according to claim 1 or 2, wherein the acquisition unit acquires an image of the user entering a facility, and the output unit causes a display device installed in the facility to display the guidance image including guidance information in the facility.

9. The information processing apparatus according to claim 1 or 2, wherein the output means outputs the avatar image to a user terminal owned by the user.

10. The information processing apparatus according to claim 1 or 2, wherein the output means outputs the guidance image to a user terminal owned by the user.

11. An information processing method, comprising: obtaining an image of a user by at least one computer; generating a converted image by replacing a face portion of the user included in the image of the user with an avatar image corresponding to the user; and outputting a guidance image in which guidance information for the user is superimposed on the anonymized image.

12. A recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, the information processing method comprising: obtaining an image of a user; generating a converted image by replacing a face portion of the user included in the image of the user with an avatar image corresponding to the user; and outputting a guidance image in which guidance information for the user is superimposed on the anonymized image.

Images