Imaging device, information processing method, and program

Abstract

To provide an image capturing device, an information processing device, an information processing method, and a program, which can be applied to photo creation game machines configured to provide a series of games for printing captured images on sticker paper sheets and allow for appropriately modifying content of a game if a user using the game is a repeat user.SOLUTION: In an information processing system, an image capturing unit of a photo sticker creation device 1 is configured to capture an image of multiple users U1, U2 using a game together, obtain a result of determination as to whether or not the users are repeat users who have used the game before based on the image acquired by photographing the multiple users, and modify content of the game according to the result of determination as to whether the users are repeat users or not.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This technology relates to a photographing device 、 an information processing method, and a program, and particularly to a photographing device that can appropriately change the content of a game when the user using the game is a repeat user 、 an information processing method, and a program.

Background Art

[0002] Conventionally, a photo sticker creating device is known. The photo sticker creating device photographs a user, allows the user to perform editing on the photographed image, and prints and provides the edited image on sticker paper. The photo sticker creating device is installed in a play facility or the like.

[0003] The flow of one game provided by the photo sticker creating device usually involves photographing a user in the photographing booth as a subject, moving the user to the editing booth, editing the image according to the operations performed in the editing booth, and printing the edited image on sticker paper.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The game of the photo sticker creating device is usually used by a plurality of users. When all of the users are repeat users, it may be preferable to change the content of the game and proceed.

[0006] This technology has been made in view of such a situation, and enables appropriate change of the content of a game when the user using the game is a repeat user. [Means for solving the problem]

[0007] One aspect of this technology is a shooting device which includes a shooting unit that photographs multiple users who are playing the game together as subjects, an acquisition unit that obtains a determination result of whether or not a user is a repeat user who has played the game before, based on images obtained by photographing the multiple users, and a control unit that changes the content of the game based on the determination result of whether or not each of the multiple users is a repeat user.

[0008] The information processing device for another aspect of this technology includes an authentication unit that authenticates a person in an image transmitted from a camera that photographs multiple users playing the game together; a determination unit that determines, based on the person authentication result, whether or not the multiple users are repeat users who have played the game before; and a communication unit that transmits information regarding the determination result of whether or not each of the multiple users is a repeat user to the camera.

[0009] In one aspect of this technology, multiple users playing the game together are photographed as subjects, and a determination is made based on the images obtained from photographing these multiple users to determine whether or not they are repeat users who have played the game before. Furthermore, the content of the game is modified based on the determination result for each of the multiple users as to whether or not they are repeat users.

[0010] In another aspect of this technology, the person in the image transmitted from the camera, which captures multiple users playing the game together, is authenticated, and based on the person authentication result, it is determined whether or not the multiple users are repeat users who have played the game before. Information regarding the determination result of whether or not each of the multiple users is a repeat user is also transmitted to the camera. [Effects of the Invention]

[0011] According to the present technology, when a user who plays a game is a repeat user, the content of the game can be appropriately changed.

Brief Description of Drawings

[0012] [Figure 1] It is a diagram showing a configuration example of an information processing system according to an embodiment of the present technology. [Figure 2] It is a diagram showing an example of input / output of a face authentication engine. [Figure 3] It is a diagram showing the flow of processing during a game. [Figure 4] It is a diagram showing an example of game management information. [Figure 5] It is a diagram showing an example of a display screen. [Figure 6] It is a diagram showing an example of store installation information. [Figure 7] It is a perspective view showing the appearance of a photo sticker creation device. [Figure 8] It is a block diagram showing a configuration example of a photo sticker creation device. [Figure 9] It is a diagram showing an example of generation of a face image. [Figure 10] It is a diagram explaining the standing position of a user. [Figure 11] It is a block diagram showing a hardware configuration example of an information processing server. [Figure 12] It is a block diagram showing a functional configuration example of an information processing server. [Figure 13] It is a flowchart explaining the photo sticker creation game providing process executed by a photo sticker creation device. [Figure 14] It is a flowchart explaining the shooting process in step S2 of FIG. 13. [Figure 15] It is a diagram showing an example of display of an analysis result screen. [Figure 16] It is a diagram showing an example of display at the start of face diagnosis. [Figure 17] It is a diagram showing an example of display of a face diagnosis screen. [Figure 18] It is a diagram showing another example of display of a face diagnosis screen. [Figure 19] It is a diagram showing an example of the display of an editing screen. [Figure 20] It is a diagram showing the state in an editing booth. [Figure 21] It is a diagram showing an example of the display of an editing palette. [Figure 22] It is a flowchart for explaining the operation of an information processing server. [Figure 23] It is a diagram showing an example of the timing for performing face authentication. [Figure 24] It is a diagram showing another example of the timing for performing face authentication. [Figure 25] It is a diagram showing an example of a face authentication engine. [Figure 26] It is a diagram showing another configuration example of a photo sticker creation device. [Figure 27] It is a diagram showing an example of the presentation of information regarding the standing position.

Embodiments for Carrying Out the Invention

[0013] <Provision of a Game Using Face Authentication> ·System Configuration FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment of the present technology.

[0014] As shown in FIG. 1, an information processing system according to an embodiment of the present technology is configured by connecting a photo sticker creation device 1 and an information processing server 2 via a network 3 such as the Internet.

[0015] The photo sticker creation device 1 is a game machine that provides a photo creation game. The photo sticker creation device 1 is installed in a store such as a game center. Although one photo sticker creation device 1 is shown in FIG. 1, a plurality of photo sticker creation devices 1 installed in a plurality of stores are connected to the network 3. There may be a plurality of photo sticker creation devices 1 installed in one store.

[0016] The users of the photo sticker creation device 1 are primarily high school girls and young women. The photo sticker creation device 1 allows multiple users, such as two or three people per group, to enjoy the game. Of course, a single user can also enjoy the game using the photo sticker creation device 1.

[0017] In the photo creation game using the photo sticker creation device 1, the user takes a photograph of themselves. The user then edits the resulting photograph by adding composite images such as handwritten text and stamp images, thereby editing the photograph into a colorful image. The user receives a sticker sheet printed with the edited photograph, and the game concludes.

[0018] Here, the photo sticker creation device 1 is, for example, a device compatible with 5G communication (fifth-generation mobile communication system). Since the photo sticker creation device 1 is connected to the network 3 via 5G communication, communication between the photo sticker creation device 1 and the information processing server 2 is high-speed and low-latency. During gameplay on the photo sticker creation device 1, various types of information, described later, are sent and received between the photo sticker creation device 1 and the information processing server 2.

[0019] The information processing server 2 is an internet server managed by the manufacturer of the photo sticker creation device 1, etc. The information processing server 2 has a face recognition engine 11. The face recognition engine 11 is an image processing engine that assigns a face ID, which is a unique ID, to the face of a person in an image. The face recognition engine 11 is composed of, for example, an inference model generated by machine learning.

[0020] Figure 2 shows an example of the input and output of the facial recognition engine 11.

[0021] As shown in Figure 2, the facial recognition engine 11 is an inference model that takes a facial image as input and outputs a facial ID. In the example in Figure 2, when the facial image of user U1 is input, the ID "000001" is output as the facial ID. If the same person's facial image is input, the facial recognition engine 11 will output the same facial ID. In training the facial recognition engine 11, facial images of various people are used as training data.

[0022] Methods other than those using inference models may be employed, as long as they can assign a unique ID to each face. For example, an image processing engine that assigns a face ID to a face in an authentication face image by matching it with pre-registered face images may be used as the face recognition engine 11. Face image matching is performed, for example, by calculating the degree of agreement between features extracted from pre-registered face images and features extracted from the authentication face image.

[0023] In the facial recognition engine 11, facial ID detection (assignment of facial ID) is performed on the facial images transmitted from the photo sticker creation device 1. The photo sticker creation device 1 transmits the facial images of each user who uses the game on the photo sticker creation device 1.

[0024] • Flow of the photo creation game Figure 3 is a diagram showing the processing flow during gameplay.

[0025] Here, we will describe a case where two users, User U1 and User U2, use the photo sticker creation device 1 together, as shown on the left side of Figure 3. For example, User U1 and User U2 are friends.

[0026] When users U1 and U2 pay a predetermined amount of money and enter the shooting booth according to the instructions, a photograph is taken of them. A camera is installed in the shooting booth. The game of the photo sticker creation device 1 mainly proceeds by having the users perform a series of tasks: pre-selection before shooting, entering the shooting booth and taking the photograph, and moving from the shooting booth to the editing booth to edit the captured image.

[0027] The photo sticker creation device 1 generates separate facial images of user U1 and user U2 based on an image in which user U1 and user U2 are pictured together, and transmits them to the information processing server 2 as shown by arrow #1.

[0028] The information processing server 2 inputs the facial images transmitted from the photo sticker creation device 1 into the facial recognition engine 11 and detects the facial IDs of user U1 and user U2 as described above.

[0029] Furthermore, the information processing server 2 determines, based on the detected face ID, whether user U1 and user U2, who are playing the game using the photo sticker creation device 1, are repeat users who have used the photo sticker creation device 1 in the past.

[0030] The information processing server 2 manages game management information, including the face IDs of users who have used the photo sticker creation device 1. Repeat user determination, which determines whether a user playing a game using the photo sticker creation device 1 is a repeat user, is performed based on whether the detected face ID is already registered in the game management information.

[0031] Figure 4 shows an example of game management information.

[0032] As shown in Figure 4, a game management information DB 12 is constructed on the information processing server 2. For example, each time a game is played in the photo sticker creation device 1, game management information is generated and stored in the game management information DB 12.

[0033] As shown in the speech bubble, game management information is a collection of linked pieces of information, such as the date and time of game use, arcade machine ID, face ID, game information, and attribute information.

[0034] The game usage date and time refers to the date and time the game was used (the date and time the game was provided).

[0035] The enclosure ID is the ID of the enclosure of the photo sticker creation device 1. Each photo sticker creation device 1 installed in each store has a unique enclosure ID assigned to it.

[0036] The face ID is the face ID of the user who played the game. If user U1 and user U2 played the game together, the face IDs of user U1 and user U2 will be registered. Since the face IDs of all users who played the game in each session are registered, it is possible to identify who played the game together based on the game management information.

[0037] Game information represents the user's choices in past games. Various choices are made by the user during pre-shoot selection, shooting in the shooting booth, and editing in the editing booth. For example, during editing in the editing booth, the user selects the tools to use for editing the captured images, and selects content.

[0038] Various pieces of information representing the user's situation during past gameplay, such as their position in the photo booth (whether they were standing on the right or left), their position in the editing booth, and the number of users who played the game together, may be managed as game information.

[0039] Attribute information refers to user attribute information such as gender and age. The face recognition engine 11 may be equipped with functions to estimate the gender of a person in a face image and to estimate their age. In this case, the face recognition engine 11 is provided with an inference model that takes a face image as input and outputs gender information, and an inference model that takes a face image as input and outputs age information.

[0040] If the same face ID detected based on the face image transmitted from the photo sticker creation device 1 is registered in the game management information which includes such information, the user playing the game using the photo sticker creation device 1 is determined to be a repeat user. On the other hand, if the same face ID detected based on the face image is not registered in the game management information, the user playing the game using the photo sticker creation device 1 is determined not to be a repeat user. The determination of whether or not a user is a repeat user is performed for each individual user.

[0041] The results of this repeat user detection are transmitted from the information processing server 2 to the photo sticker creation device 1 along with the face ID, as shown by arrow #2 in Figure 3. Based on the information transmitted from the information processing server 2, the photo sticker creation device 1 can determine whether the user currently using the game on the photo sticker creation device 1 is a user who has used it in the past.

[0042] In the photo sticker creation device 1, the content of the game provided to users U1 and U2 is changed based on the result of the repeat judgment.

[0043] Figure 5 shows an example of a display screen.

[0044] As will be detailed later, in the game of the photo sticker creation device 1, the user's face is analyzed when the photo is taken in the shooting booth. Face analysis is the process of identifying which of several pre-defined face categories the user's face belongs to. The screen shown in Figure 5 is the analysis results screen used to display the results of the facial feature analysis. The shooting booth is equipped with a shooting monitor used to display various screens, such as the live view screen during shooting and the analysis results screen.

[0045] If user U1 and user U2 are playing the game together, the analysis results screen will display the analysis results for user U1's facial features and user U2's facial features. The left side of the analysis results screen displays the analysis results for user U1, who is standing to the left of the camera (the monitor used for recording), and the right side displays the analysis results for user U2, who is standing to the right of the camera. In this example, user U1 is standing to the left of the camera, and user U2 is standing to the right of the camera.

[0046] For example, if user U1 is a repeat user and user U2 is not, as shown in Figure 5A, a message for repeat users, "Thank you for taking another photo," will be displayed to user U1 in the left-hand area of ​​the analysis results screen along with the analysis results. In this case, the message for repeat users will not be displayed in the right-hand area of ​​the analysis results screen.

[0047] User U1 can feel a sense of specialness when they realize that the photo sticker creation device 1 recognizes them. Information for repeat users, such as messages, may also be presented using voice.

[0048] On the other hand, if both user U1 and user U2 are repeat users, as shown in Figure 5B, a message for repeat users will be displayed to both user U1 and user U2 in the left and right areas of the analysis results screen, along with the analysis results.

[0049] Users U1 and U2 each realize that the photo sticker creation device 1 recognizes them, and can feel a sense of specialness.

[0050] Thus, the display of the analysis results screen changes based on the repeat user determination result, depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users. The display position and number of messages for repeat users change depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users. At least one of the display position or the number of messages for repeat users may also be changed.

[0051] Not only can the display position and number of messages be changed, but the content of the messages may also be modified. For example, if all users playing the game together are repeat users, a message such as "Thank you both for taking pictures again" could be displayed.

[0052] In addition to displaying the analysis results screen, it is also possible to change the content of the photo sticker creation device 1 at each stage of the game depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users.

[0053] After reviewing the analysis results, users U1 and U2 proceed with the game according to the instructions given by the photo sticker creation device 1.

[0054] In the above, a user who has used the game on Photo Sticker Creation Device 1, which is a photo sticker creation device with a specific cabinet ID set, is determined to be a repeat user. However, it is also possible to determine a repeat user as a user who has used the game on a photo sticker creation device installed in the same store where Photo Sticker Creation Device 1 is installed.

[0055] Whether or not a user has previously used a photo sticker creation device installed in the same store where the photo sticker creation device 1 is installed is determined using store installation information, as shown in Figure 6, along with game management information.

[0056] Store installation information is information that links the ID of the store where the photo sticker creation device is installed, the model ID of the photo sticker creation device, etc., to the enclosure ID. Information processing server 2 is equipped with store installation information DB13, which is a database of store installation information. By referring to the store installation information, information processing server 2 can identify the store where the photo sticker creation device is installed based on the enclosure ID of the photo sticker creation device that sent the facial image.

[0057] Additionally, users who have previously used a photo sticker creation device of the same model as Photo Sticker Creation Device 1 may be identified as repeat users. The determination of whether or not a user has previously used a photo sticker creation device of the same model as Photo Sticker Creation Device 1 is made using the model ID included in the store installation information.

[0058] Thus, repeat users may be users who have used the same type of photo sticker printing machine before, or users who have used the same type of photo sticker printing machine at the same store, or users who have used the same model of photo sticker printing machine before.

[0059] The store installation information DB13 may be built on a server different from the information processing server 2.

[0060] <Configuration of each device> • Configuration of the photo sticker making device 1 Figure 7 is a perspective view showing the external appearance of the photo sticker making device 1.

[0061] As shown in Figure 7, a pre-operation unit 21 is provided on the side of the photo sticker creation device 1, which has a box-shaped casing. The pre-operation unit 21 is a device used for pre-selection operations such as selecting the number of users to play the game together and selecting the shooting course. A touch panel monitor 21A is provided above the pre-operation unit 21. A coin slot 21B for paying the play fee is provided below the touch panel monitor 21A.

[0062] The screen used for various selections is displayed on the touch panel monitor 21A. After the user makes various selections according to the display on the touch panel monitor 21A, they enter the shooting booth through the entrance / exit G and take a photograph. A box-shaped shooting unit 31 containing a camera 31A is provided at the front of the shooting booth. The shooting unit 31 is equipped with a shooting monitor 31B.

[0063] An editing unit 41 is provided to the left of the pre-operation unit 21. The editing unit 41 is a device used for editing the captured images obtained by shooting with the camera 31A. Users who have taken pictures in the shooting booth will move to the editing booth in front of the editing unit 41 to edit the captured images. An editing monitor 41A is provided on a slope formed above the editing unit 41. The screen used for editing the captured images is displayed on the editing monitor 41A.

[0064] Figure 8 is a block diagram showing an example of the configuration of the photo sticker making device 1. Components in Figure 8 that are the same as those described above are denoted by the same reference numerals. Repetitive explanations are omitted as appropriate.

[0065] As shown in Figure 8, the photo sticker creation device 1 is configured by connecting a pre-operation unit 21, a shooting unit 31, an editing unit 41, a communication unit 52, and a printing unit 53 to a control unit 51 which is configured as a PC.

[0066] The control unit 51 executes a program using the CPU and controls the overall operation of the photo sticker creation device 1 according to the progress of the game. In the control unit 51, the execution of a predetermined program realizes the game control unit 61 and the authentication result acquisition unit 62.

[0067] The game control unit 61 controls a series of game processes, including pre-selection, shooting, editing, and printing.

[0068] Pre-selection processing is a process that allows the user to perform pre-selection tasks, such as selecting a shooting course.

[0069] The shooting process involves having the user take photographs of themselves. In addition to the regular images provided to the user, the shooting process also includes the capture of images used for facial recognition. During facial recognition, the game control unit 61 controls the camera 31A to photograph the user playing the game and acquire images for facial recognition. The images for facial recognition are supplied to the authentication result acquisition unit 62.

[0070] The editing process is a process in which the user performs editing tasks on the captured images obtained through the shooting process.

[0071] The printing process involves arranging the edited images obtained through the editing process on sticker paper in a predetermined layout. The sticker paper with the edited images printed on it is provided to the user through an opening located at a predetermined position in the housing of the editing unit 41.

[0072] The game control unit 61 modifies the content of each process based on the results of the repeat user determination obtained by the authentication result acquisition unit 62. The game control unit 61 functions as a control unit that modifies the game content based on the determination results of whether each of the multiple users is a repeat user or not. Furthermore, the game control unit 61 functions as a control unit that modifies the game content depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users.

[0073] Furthermore, the game control unit 61 controls the communication unit 52 and transmits game information, such as the user's selections in the game, to the information processing server 2. The game information, along with information such as the date and time the game was played and the user's face ID, is transmitted to the information processing server 2 when the game ends.

[0074] The authentication result acquisition unit 62 generates a face image showing the face of each user based on the face recognition image supplied from the game control unit 61.

[0075] Figure 9 shows an example of face image generation.

[0076] As shown in the upper part of Figure 9, when an image P containing users U1 and U2 is taken as an image for facial recognition, the facial images of user U1 and user U2 are extracted from image P, as indicated by the tip of arrow #11.

[0077] In the example shown in Figure 9, the person standing to the left of camera 31A (the person shown on the right) is user U1, and the person standing to the right of camera 31A (the person shown on the left) is user U2. Images of the rectangular regions containing the faces of each user, user U1 and user U2, are extracted as face images.

[0078] As shown in Figure 10, if user U1 is standing to the left of camera 31A and user U2 is standing to the right of camera 31A, it is possible to determine the position of each user based on their position in the image.

[0079] As shown in image P in Figure 9, the original image before any processing is applied is used for facial recognition. While typical captured images provided to users after editing or other processing are processed images, the facial image extracted from image P, which has not undergone such processing, is sent to the information processing server 2 and used for facial recognition.

[0080] The processing includes at least one of the following: processing to enlarge the eyes, processing to reduce the size of the face, processing to brighten the skin tone of the face, and processing to adjust the hair color.

[0081] For example, if facial recognition is performed based on an image that has been processed to enlarge the eyes, false positives may occur, such as different facial IDs being detected even when using images of the same user. By performing facial recognition based on the original image, it is possible to prevent such false positives.

[0082] An image P showing users U1 and U2 may be sent to the information processing server 2, and face image generation may be performed at the information processing server 2. The face recognition engine 11 provided on the information processing server 2 may be configured as an image processing engine that takes image P as input and outputs the face ID of user U1 and the face ID of user U2.

[0083] The authentication result acquisition unit 62 in Figure 8 controls the communication unit 52 and transmits the generated face image to the information processing server 2. The authentication result acquisition unit 62 acquires the authentication result transmitted from the information processing server 2. The authentication result acquired by the authentication result acquisition unit 62 includes the face ID and the result of the repeat judgment. Other information managed in conjunction with the face ID, such as game information and attribute information, is also provided to the photo sticker creation device 1 as appropriate.

[0084] The pre-operation unit 21 displays a predetermined screen on the touch panel monitor 21A, allowing the user to make various selections before starting shooting. The pre-operation unit 21 receives the user's selections and outputs information representing the selections to the control unit 51.

[0085] The shooting unit 31 drives the camera 31A to take pictures of the user in the shooting booth. For example, multiple shots are taken to acquire multiple images. The images acquired by the shooting unit 31 are supplied to the control unit 51.

[0086] The editorial unit 41 performs editing for users in the editing booth. The editing process includes image processing to edit the captured images. Editing of the captured images is performed using the editing screen displayed on the editing monitor 41A. After finishing editing the captured images, the editorial unit 41 generates print data by arranging the edited images in a predetermined layout and outputs it to the control unit 51.

[0087] The communication unit 52 communicates with external devices such as the information processing server 2. The communication unit 52 transmits the facial image to the information processing server 2. Along with the facial image, information such as the date and time of game use and the arcade cabinet ID is also transmitted as appropriate.

[0088] The printing unit 53 has a printer. The printing unit 53 drives the printer and prints the edited photographic image onto sticker paper based on the print data.

[0089] • Configuration of Information Processing Server 2 Figure 11 is a block diagram showing an example of the hardware configuration of the information processing server 2.

[0090] As shown in Figure 11, the information processing server 2 is composed of computers. The information processing server 2 may also be composed of multiple computers. When the information processing server 2 is composed of multiple computers, the various processes of the information processing server 2 are realized through the cooperation of each computer.

[0091] The CPU (Central Processing Unit) 101, ROM (Read Only Memory) 102, and RAM (Random Access Memory) 103 are interconnected by a bus 104.

[0092] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0093] The input unit 106 consists of a keyboard, mouse, etc. The output unit 107 consists of a display, etc.

[0094] The storage unit 108 consists of a hard disk, non-volatile memory, and the like. The storage unit 108 stores programs executed by the CPU 101 and various types of information used to provide facial recognition services.

[0095] Game management information DB12 and store installation information DB13 are built in, for example, the storage unit 108. The storage unit 108 functions as a storage unit that stores game management information, which links various pieces of information such as game usage date and time, cabinet ID, face ID, game information, and attribute information, and store installation information, which links cabinet ID, store ID, and machine ID. The combination of information stored in the storage unit 108 can be changed arbitrarily. If the game information includes information representing the user's selections in the game and information on the user's standing position at the time of each photo, the storage unit 108 stores at least one of the following: the face ID assigned to each user as a result of face recognition, the information representing the user's selections in the game, and the information on the user's standing position at the time of each photo.

[0096] The communication unit 109 is an interface to the network 3. For example, the communication unit 109 communicates with the photo sticker creation device 1 and receives facial images transmitted from the photo sticker creation device 1. The communication unit 109 also transmits information such as authentication results to the photo sticker creation device 1.

[0097] Drive 110 controls the writing of data to and reading of data from the removable media 111.

[0098] Figure 12 is a block diagram showing an example of the functional configuration of the information processing server 2.

[0099] As shown in Figure 12, an information processing unit 121 is implemented in the information processing server 2. The information processing unit 121 consists of an image acquisition unit 131, a face recognition unit 132, a determination unit 133, and a game information acquisition unit 134. At least some of the functional units shown in Figure 12 are implemented by the execution of a predetermined program by the CPU 101. The information processing unit 121 is also provided with the aforementioned game management information DB 12.

[0100] The image acquisition unit 131 acquires the face image transmitted from the photo sticker creation device 1 and received by the communication unit 109. The face image acquired by the image acquisition unit 131 is supplied to the face recognition unit 132.

[0101] The facial recognition unit 132 is implemented by the facial recognition engine 11. The facial recognition unit 132 inputs the facial image supplied from the image acquisition unit 131 to the facial recognition engine 11 and detects the facial ID. The facial recognition unit 132 outputs the facial ID detected using the facial recognition engine 11 to the determination unit 133.

[0102] Furthermore, the facial recognition unit 132 outputs the facial ID detected using the facial recognition engine 11 to the game management information DB 12 and registers it in association with other information in the game management information related to the current game.

[0103] The determination unit 133 performs a repeat user determination by referring to the game management information stored in the game management information DB 12 and determining whether the same face ID as the face ID supplied by the face recognition unit 132 is already registered. If the same face ID as the face ID supplied by the face recognition unit 132 is already registered, the user to whom that face ID is assigned is determined to be a repeat user. On the other hand, if the same face ID as the face ID supplied by the face recognition unit 132 is not registered, the user to whom that face ID is assigned is determined to be a non-repeat user.

[0104] The determination unit 133 transmits information including the face ID supplied from the face recognition unit 132 and the result of the repeat judgment as an authentication result to the photo sticker creation device 1.

[0105] The game information acquisition unit 134 receives game information transmitted from the photo sticker creation device 1 at predetermined times, such as when the game ends, and registers it in the game management information.

[0106] <Operation of each device> • Operation of the photo sticker creation device 1 Referring to the flowchart in Figure 13, the process of providing a photo sticker creation game performed by the photo sticker creation device 1 will be explained. The process in Figure 13 starts, for example, when a predetermined amount of money is paid.

[0107] In step S1, the game control unit 61 performs a pre-selection process for users who are outside the shooting booth. This pre-selection process involves selecting the number of users and the shooting course. All selections are made using the selection screen displayed on the shooting monitor 31B of the pre-operation unit 21. When the pre-selection process is complete, users are instructed to move to the shooting booth.

[0108] In step S2, the game control unit 61 performs a shooting process targeting the user in the shooting booth. The shooting process includes taking photos of the user, facial analysis, and facial diagnosis. Details of the shooting process will be described later with reference to the flowchart in Figure 14.

[0109] In step S3, the game control unit 61 performs editing processing for the user in the editing booth. Specifically, the game control unit 61 displays the editing screen on the editing monitor 41A and allows the user to perform editing work on the captured image obtained through the shooting process. Editing work is performed using various tools such as the pen tool and stamp tool. The game control unit 61 performs image processing in response to the user's operations and generates the edited image.

[0110] In step S4, the game control unit 61 outputs print data to the printing unit 53 and prints the edited image onto sticker paper. When printing is complete, the sticker paper with the edited image printed on it is ejected from the dispenser of the casing and provided to the user.

[0111] In step S5, the game control unit 61 controls the communication unit 52 to upload the captured images and edited images to an image management server (not shown). The image management server is an internet server managed by the manufacturer of the photo sticker creation device 1, etc. Users can access the image management server by operating a mobile device such as a smartphone to view and download images created in the game.

[0112] In step S6, the game control unit 61 controls the communication unit 52 and transmits game information, including information representing the user's selections in the game, to the information processing server 2. The transmitted information is then linked to a facial ID and registered in the game management information.

[0113] Next, referring to the flowchart in Figure 14, we will explain the shooting process that takes place in step S2 of Figure 13. Here, we assume that users U1 and U2 are playing the game together.

[0114] In step S21, the game control unit 61 controls the camera 31A to perform face analysis photography. Face analysis photography is performed to capture a face analysis image, which is an image used to analyze facial features. For example, as explained with reference to Figure 9, a single image showing both user U1 and user U2 together is captured as the face analysis image. The face image generated based on the face analysis image is also used for face recognition. The face analysis image is also the image used for face recognition.

[0115] In step S22, the authentication result acquisition unit 62 generates face images showing the faces of user U1 and user U2 based on the face analysis image.

[0116] In step S23, the game control unit 61 analyzes the facial images and analyzes the facial features of user U1 and user U2.

[0117] For example, the game control unit 61 analyzes each face image to detect feature points of specific facial features such as contours, eyes, nose, and mouth. The game control unit 61 also quantifies the facial features of the users in the face images and maps them onto the feature space to identify the facial lineage of user U1 and user U2. The faces of user U1 and user U2 are identified as belonging to one of several lineages, such as cute, baby-like, fashionable, or mature.

[0118] In step S24, the authentication result acquisition unit 62 controls the communication unit 52 to send the face image to the information processing server 2. In response to the transmission of the face image, the information processing server 2 sends the authentication result. By using 5G communication or the like, low-latency communication becomes possible.

[0119] In step S25, the authentication result acquisition unit 62 receives the authentication results and obtains the respective face IDs of user U1 and user U2, as well as the result of the repeat user determination. If at least one of user U1 or user U2 is determined to be a repeat user, the repeat user's game information is included in the authentication result. Based on the game information obtained here, for example, the various selections made during the previous game are identified.

[0120] In step S26, the game control unit 61 displays an analysis results screen on the capture monitor 31B that corresponds to the result of the repeat judgment. As described above, the analysis results screen is used to present the results of the analysis of the user's facial features.

[0121] Figure 15 shows an example of the analysis results screen.

[0122] The analysis results screen displayed on the shooting monitor 31B has two display areas 201L and 201R arranged side by side. Display area 201L is the display area for the analysis results of the facial features of user U1, who is standing to the left of camera 31A (shooting monitor 31B). Display area 201L is the area assigned to user U1, who is standing to the left of camera 31A.

[0123] Display area 201R is the display area for the analysis results of the facial features of user U2, who is standing to the right of camera 31A. Display area 201R is the area assigned to user U2, who is standing to the right of camera 31A.

[0124] In the example in Figure 15, the analysis result of user U1's facial features is displayed in display area 201L as "cute." Above the facial feature analysis result, a message for repeat users, "Thank you for taking another photo," is displayed along with an illustration of a girl.

[0125] On the other hand, in the example in Figure 15, the analysis result of user U2's facial features is displayed in display area 201R as "baby-like." No message for repeat users is displayed in display area 201R. In other words, the analysis results screen shown in Figure 15 is the analysis results screen that is displayed when user U1 is a repeat user and user U2 is not a repeat user, as explained with reference to Figure 5A.

[0126] After this analysis results screen is displayed, in step S27 of Figure 14, the game control unit 61 displays the face diagnosis screen on the capture monitor 31B. Face diagnosis proceeds as the user answers questions about their preferences for how their face looks. Multiple options are presented to the user along with the questions. Depending on the results of the face diagnosis, the image processing, such as processing applied to the face, is changed.

[0127] Figure 16 shows an example of the display at the start of a facial diagnosis.

[0128] At the start of the facial analysis, a message is displayed instructing the user to begin the analysis with the person standing on the left, as shown in Figure 16. The facial analysis is performed on one person at a time. After a predetermined amount of time has elapsed, the facial analysis screen will switch to the display shown in Figure 17.

[0129] In this example, user U1, standing on the left, is a repeat user. When a face diagnosis is initiated for a repeat user, the face diagnosis screen displays a message asking whether to apply the same selections made during the previous game to the current game, as shown in Figure 17. Below the message, there are two buttons: a YES button 211 to press if the selections are to be applied, and a NO button 212 to press if the selections are not to be applied.

[0130] In other words, the game information provided by the information processing server 2 includes information about what user U1 selected during the face diagnosis in the previous game. By pressing the YES button 211, user U1 can use the same selections from the previous game as the face diagnosis result. User U1 can also skip making any selections related to face diagnosis.

[0131] Furthermore, by pressing the NO button 212, user U1 can make selections regarding facial appearance in the same way as a non-repeat user, without using the same selections as in the previous game.

[0132] In the lower right corner of the face recognition screen, a false authentication button 213 is displayed, containing the text "I've never taken a picture before." The false authentication button 213 is pressed when a screen like the one shown in Figure 17 is displayed to a non-repeat user. When the false authentication button 213 is pressed, the same face recognition process as for non-repeat users begins.

[0133] If a false positive occurs during facial ID detection using the facial recognition engine 11, a screen for repeat users, as shown in Figure 17, may be displayed to a non-repeat user. Non-repeat users can select their preferred facial image by pressing the false positive button 213.

[0134] For example, if user U1 presses the YES button 211 and chooses to use the same selections as in the previous game as the facial diagnosis result, the facial diagnosis for user U1 will end. Following the facial diagnosis for user U1, the facial diagnosis for user U2 will begin. When the facial diagnosis for user U2 begins, a facial diagnosis screen will be displayed that includes a message instructing the system to diagnose user U2, who is standing on the right. After a predetermined amount of time has elapsed, the facial diagnosis screen will switch to the display shown in Figure 18.

[0135] Here, user U2, standing on the right, is not a repeat user. Facial recognition for non-repeat users is performed by user U2 answering questions as shown in Figure 18.

[0136] At the top of the face diagnosis screen in Figure 18, a message is displayed asking what kind of face the user wants to have. Below the message, buttons 221 and 222 are displayed as options. User U2 answers the question by pressing either button 221 or button 222. These questions are repeated multiple times with variations in content.

[0137] Repeat user U1 can also perform this facial analysis again, separate from the facial analysis performed during the previous game, by pressing the NO button 212 in Figure 17.

[0138] If both User U1 and User U2 are repeat users, a screen will appear during the facial recognition process for User U2 allowing them to choose to apply the same selections made during the previous game. The number of times this screen is displayed will differ depending on whether all users playing the game together are repeat users or if there are non-repeat users among them.

[0139] Returning to the explanation of Figure 14, in step S28, the game control unit 61 accepts the user's selection for the face diagnosis screen. The processing in steps S27 and S28 is performed for each user.

[0140] Once all users' facial recognition is complete, in step S29, the game control unit 61 controls the camera 31A to perform normal shooting. Normal shooting is the process of generating images to be provided to the users. Normal shooting is performed multiple times, for example, five times.

[0141] In step S30, the game control unit 61 processes the face portion of each user in the captured image based on face parameters corresponding to the combination of the analysis results of each user's facial features and the face diagnosis results. Face parameters corresponding to the combination of the analysis results of facial features and the face diagnosis results are set in advance.

[0142] The face portion of user U1 in the captured image obtained through normal shooting is processed using face parameters corresponding to the combination of the analysis results of user U1's facial features and the face diagnosis results of user U1. Similarly, the face portion of user U2 in the captured image is processed using face parameters corresponding to the combination of the analysis results of user U2's facial features and the face diagnosis results of user U2.

[0143] As a result, the appearance of user U1's face in the captured image will be appropriate according to the analysis results of user U1's facial features and the facial diagnosis results for user U1. Similarly, the appearance of user U2's face in the captured image will be appropriate according to the analysis results of user U2's facial features and the facial diagnosis results for user U2.

[0144] In step S31, the game control unit 61 determines whether or not the shooting is finished. If it is determined that the shooting is not finished, the process returns to step S29 and normal shooting is repeated.

[0145] For example, if it is determined in step S31 that the shooting is complete because five images have been taken, the process returns to step S2 in Figure 13, and the shooting process ends. After the shooting process, the editing process is performed on the user who has moved to the editing booth.

[0146] • Display of editing screen Figure 19 shows an example of the editing screen.

[0147] During the editing of captured images (step S3 in Figure 13), the editing monitor 41A in the editing booth displays an editing screen with editing areas 301L and 301R arranged side by side. Between the editing areas 301L and 301R is a thumbnail display area 302, which is a display area for thumbnail images used to select the captured image to be edited.

[0148] Editing area 301L is the area used by user U1, who is standing to the left of the editing monitor 41A. Editing area 301R is the area used by user U2, who is standing to the right of the editing monitor 41A.

[0149] Editing area 301L is assigned to user U1, who is standing to the left of the editing monitor 41A, and editing area 301R is assigned to user U2, who is standing to the right of the editing monitor 41A. As shown in Figure 20, the editing screen is displayed assuming that users U1 and U2 are editing from the same positions as when they were shooting. If users U1 and U2 are standing in opposite positions to those shown in Figure 20, it is possible to change the screen display so that user U2 uses editing area 301L and user U1 uses editing area 301R.

[0150] In the editing area 301L used by user U1, a repeat user, a message for repeat users, "Thank you for taking another photo," is displayed overlaid on the photograph being edited. This also allows user U1 to realize that the photo sticker creation device 1 recognizes them, giving them a sense of being special.

[0151] Although not shown in the example in Figure 19, if user U2 is a repeat user, a message for repeat users will also be displayed in the editing area 301R used by user U2.

[0152] The message for repeat users disappears, for example, after a predetermined amount of time has elapsed since the editing screen was first displayed.

[0153] As described above, the display of the editing screen also changes depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users. The display position and number of messages for repeat users also change depending on whether all users playing the game together are repeat users or whether there are users who are not repeat users.

[0154] Below the editing areas 301L and 301R, the editing palettes 303L and 303R are displayed, respectively. The editing palettes 303L and 303R display buttons used to select various editing tools, such as the pen tool and stamp tool. Users U1 and U2 operate the buttons displayed on the editing palettes 303L and 303R using the stylus pen 41B (Figure 20), respectively, to edit the captured image to be edited. Near the editing monitor 41A, a stylus pen 41B is provided for user U1 and a stylus pen 41B for user U2.

[0155] To the left of the editing area 301L, the personal chart 304L is displayed, which is used to select recommended makeup treatments set for user U1. The personal chart 304L displays buttons and other elements used to select makeup treatments based on the combination of the analysis results of user U1's facial features and the results of user U1's facial diagnosis. Makeup treatments include processes such as adding highlights and eye color through image processing.

[0156] Similarly, on the right side of the editing area 301R, the personal chart 304R is displayed, which is used to select recommended makeup treatments set for user U2. The personal chart 304L displays buttons and other elements used to select makeup treatments based on the combination of the analysis results of user U2's facial features and user U2's facial diagnosis results.

[0157] Editing of the captured images is performed by two different users using this editing screen.

[0158] Figure 21 shows an example of the display of the editing palette 303L.

[0159] Information for repeat users may be displayed overlaid on the editing palette 303L used by user U1, who is a repeat user. In the example in Figure 21, a message is displayed asking whether to apply the same makeup processing settings used in the previous game to the current game.

[0160] Below the message, there is a YES button 321 to press to apply the changes and a NO button 322 to press to reject them. To the right of the NO button 322, there is a false authentication button 323 containing the text "I've never taken a picture before."

[0161] In other words, in this example, user U1, a repeat user, can choose to reflect the content of the make-up process. The game information sent from the information processing server 2 as an authentication result includes information about the make-up process selected by user U1 during the previous game.

[0162] In this way, information representing various choices made in past games can be linked to facial ID and managed for use in the current game.

[0163] • Operation of Information Processing Server 2 The operation of the information processing server 2 will be explained with reference to the flowchart in Figure 22.

[0164] In step S51, the image acquisition unit 131 (Figure 12) acquires the face image transmitted from the photo sticker creation device 1.

[0165] In step S52, the facial recognition unit 132 inputs the facial image acquired by the image acquisition unit 131 to the facial recognition engine 11 and detects the facial ID.

[0166] In step S53, the determination unit 133 refers to the game management information stored in the game management information DB 12 and performs a repeat determination by determining whether the same face ID as the face ID detected by the face recognition unit 132 is already registered.

[0167] In step S54, the determination unit 133 transmits information including the face ID detected by the face recognition unit 132 and the result of the repeat determination as an authentication result to the photo sticker creation device 1.

[0168] In step S55, the game information acquisition unit 134 determines whether or not game information has been transmitted from the photo sticker creation device 1, and waits until game information is transmitted.

[0169] If step S55 determines that game information has been transmitted from the photo sticker creation device 1, in step S56 the game information acquisition unit 134 includes the game information transmitted from the photo sticker creation device 1 in the game management information and registers it in the game management information DB 12.

[0170] In this way, facial ID can be included in game management information and used for repeat user detection.

[0171] If facial recognition using images obtained through facial analysis photography fails (i.e., facial ID cannot be detected), the game on the photo sticker creation device 1 can proceed in the same way as if it were for a non-repeat user, even if the user is actually a repeat user.

[0172] Possible reasons for facial recognition failure include, for example, poor image quality preventing face recognition from the image, or an error in communication between the photo sticker creation device 1 and the information processing server 2. In either case, the game on the photo sticker creation device 1 proceeds in the same manner as when targeting a non-repeat user.

[0173] <Variation> • Examples of images used for facial recognition The image for facial recognition (image for facial analysis) is captured at the start of the shooting process and facial recognition is performed using an image that is not provided to the user. However, facial recognition may also be performed based on the image obtained through normal shooting. The image obtained through normal shooting is the image that is provided to the user either by printing it on sticker paper or as data via an image management server.

[0174] Figure 23 shows an example of the timing of facial recognition.

[0175] After users U1 and U2 enter the shooting booth, a live view image is displayed on the shooting monitor 31B located in the shooting booth. After a certain period of time has elapsed since the display of the live view image, the first shooting is performed, and facial recognition is performed based on the captured image P1, which is a still image obtained from the first shooting. The facial image extracted from captured image P1 is transmitted to the information processing server 2.

[0176] Then, a second photo is taken, and facial recognition is performed based on the image P2 obtained from the second photo. In this way, facial recognition is repeated each time a photo is taken. When facial recognition is performed based on the image P5 obtained from the final photo, for example, the fifth photo, both the photo and the facial recognition process end.

[0177] Depending on the orientation of the face, it may not be possible to detect the facial ID with a single facial recognition attempt. Also, there may be cases where only user U2, who is using the system with user U1, is visible in the image. Repeated facial recognition attempts can improve the accuracy of repeat user detection.

[0178] Figure 24 shows another example of when facial recognition is performed.

[0179] As shown in Figure 24, facial recognition may be performed based on the five images obtained from the five captures after all captures have been completed. When the fifth capture is completed, the facial images extracted from each of the captured images P1 to P5 are sent to the information processing server 2, and facial recognition is performed based on each facial image.

[0180] For example, if, in five facial recognition attempts, a predetermined facial ID is detected three times as the user's facial ID, and a different facial ID is detected twice, the former facial ID, which was detected three times, will be assigned as the user's facial ID. By using the facial ID that is detected most frequently, in a kind of majority vote manner, it becomes possible to improve the accuracy of repeat detection.

[0181] The confidence level of the face ID may be calculated based on the state of the face in the face image used as input to the face recognition engine 11. The confidence level of the face ID is calculated based on parameters that represent the state of the face, such as face size, face orientation, and facial expression. For example, a face ID detected using a face image in which a face facing forward is large will have a higher confidence level than a face ID detected using a face image in which a face is turned to the side, or a face ID detected using a face image in which a small face is visible.

[0182] In this case, for example, repeat user detection is performed using a facial ID with a calculated confidence level above a certain threshold.

[0183] The reliability information of the facial ID may be included in the authentication result and provided from the information processing server 2 to the photo sticker creation device 1, and used to control the game content. Information for repeat users, such as changes in message content or message size, presented to users determined to be repeat users is modified based on the reliability of the facial ID used for repeat user determination.

[0184] For example, users identified as repeat users based on a facial ID with a confidence level higher than the threshold will be shown the message "Thank you for taking my picture again," as described above. On the other hand, users identified as repeat users based on a facial ID with a confidence level lower than the threshold will be shown a different message, such as "Did you take my picture last time too?"

[0185] This makes it possible to display messages based on the reliability of facial recognition, that is, the reliability of repeat customer detection.

[0186] • Examples of facial recognition engines As described above, the facial recognition engine 11 is an image processing engine that takes a facial image as input and outputs a unique facial ID. There is a limit to the number of facial IDs (number of faces that can be classified) that the facial recognition engine 11 can output. As shown in Figure 25, the information processing server 2 may be provided with facial recognition engines for each attribute of the photo sticker creation device, such as a facial recognition engine for each housing, a facial recognition engine for each model, a facial recognition engine for each store, and a facial recognition engine for the region where the store is located.

[0187] The facial recognition engines 11-1A to 11-1C shown in Figure 25 are facial recognition engines for different housings. Which facial recognition engine to use for facial recognition is determined, for example, based on the housing ID of the photo sticker creation device 1 that transmitted the facial image.

[0188] The facial recognition engines 11-2A to 11-2C are facial recognition engines for different models. Which facial recognition engine to use for facial recognition is determined, for example, based on the model ID corresponding to the model of the photo sticker creation device 1 that transmitted the facial image.

[0189] The facial recognition engines 11-3A to 11-3C are each different facial recognition engines for different stores. Which facial recognition engine to use for facial recognition is determined, for example, based on the store ID of the store where the photo sticker creation device 1 that sent the facial image is installed.

[0190] The facial recognition engines 11-4A to 11-4C are each facial recognition engines for different regions. Which facial recognition engine to use for facial recognition is determined, for example, based on a regional ID corresponding to the region where the store where the photo sticker creation device 1 that sent the facial image is installed is located.

[0191] By providing a separate facial recognition engine for each attribute of the photo sticker creation device, the number of faces that can be classified can be increased, thereby improving the accuracy of repeat detection.

[0192] The facial ID detected by facial recognition may be managed in conjunction with identification information such as the Bluetooth® ID of the smartphone carried by the user. The smartphone's identification information may be managed, for example, by being included in game management information.

[0193] The photo sticker creation device 1 acquires the Bluetooth® ID of the user's smartphone at the start of the game and sends it to the information processing server 2 along with the facial image. The information processing server 2 determines that the user is a repeat user if the facial ID detected by inputting the facial image into the facial recognition engine 11 is the same as the facial ID associated with the Bluetooth ID sent from the photo sticker creation device 1 in the game management information.

[0194] By using smartphone identification information, it becomes possible to improve the accuracy of repeat customer detection.

[0195] As shown in Figure 26, the facial recognition engine 11 and the game management information DB 12 may be provided in the photo sticker creation device 1. This allows the photo sticker creation device 1 to independently determine whether a user is a repeat customer of its own device.

[0196] The facial recognition engine 11 may be provided in the photo sticker creation device 1, and the game management information DB 12 may be provided in the information processing server 2. Conversely, the facial recognition engine 11 may be provided in the information processing server 2, and the game management information DB 12 may be provided in the photo sticker creation device 1. In this way, it is possible to provide at least one of the functional units of the information processing server 2 shown in Figure 12 on the photo sticker creation device 1 side.

[0197] • Example of repeat detection A period for determining whether a user is a repeat user may be set. This period can be set to a predetermined duration, such as one day, one week, one month, or one year. Users who use the game again before the end of the determination period since their last use will be identified as repeat users.

[0198] This prevents the system from mistakenly classifying users who haven't played the game for a long time as repeat users.

[0199] The number of repeat visits may be determined based on game management information. For example, if three game management entries containing the same face ID are registered, the user with that face ID will be determined to be a repeat user for the third time.

[0200] By combining the number of repeat plays with the date and time of game usage included in the game management information, it becomes possible to display messages tailored to the number of repeat plays, such as "This is your third time today," as information for repeat users.

[0201] By using the game usage date and time, it becomes possible to display more detailed messages tailored to usage, such as "Thank you for taking pictures again today," to users who also played the game the previous day, as information for repeat users.

[0202] In this way, it is possible to change the content of the game based on various pieces of information included in the game management information.

[0203] • Example of game changes 1 The game's content is designed to change depending on whether all users playing together are repeat users or if there are non-repeat users among them. However, the game's content may be further modified based on attribute information. For example, the types of makeup available may change depending on the user's age, or parameters used in various image processing operations such as editing may be changed.

[0204] Information other than gender and age, such as hair color, whether or not colored contact lenses are worn, and whether or not makeup is worn, may be estimated based on the facial image and managed as attribute information.

[0205] • Example 2 of game changes If all users playing the game together are determined to be repeat users, the system may further determine whether or not those users have played together in the past.

[0206] For example, if users U1 and U2 have played the game together, the game management information DB12 on the information processing server 2 will register game management information including the combination of user U1's face ID and user U2's face ID. Based on the face ID combination, it will also be possible to determine whether or not they are repeat users who have played together in the past. Since the same friends often play together, multiple game management entries containing the same face ID combination will be registered.

[0207] If a user playing the game with another user is determined to be a repeat user who has played the game with the other user in the past, the game will be different from when all users playing together are repeat users, or when some users playing together are repeat users. For example, a message such as "We've played together before" will be displayed to all of the friends playing the game together.

[0208] Repeat users who use the game together will notice that the photo sticker creation device 1 recognizes them, giving them a sense of specialness.

[0209] • Example 3 of game changes If the game management information includes information about the player's position at the time of shooting, repeat users who are playing the game together may be shown information about their position during their previous game session.

[0210] Figure 27 shows an example of how information regarding standing positions is presented.

[0211] As shown on the left side of Figure 27, if, during the previous game, user U1 stood to the left of camera 31A and user U2 stood to the right of camera 31A while taking pictures, then information indicating this will be registered in the game information of the game management information. The game management information also includes the combination of user U1's face ID and user U2's face ID.

[0212] In this state, if user U1 and user U2 use the photo sticker creation device 1 game again, in the repeat game determination for this game, user U1 and user U2 will be determined to be repeat users who used the game together.

[0213] As shown on the right side of Figure 27, in this game, if user U1 is standing to the right of camera 31A and user U2 is standing to the left of camera 31A and taking a picture, the game control unit 61 of the photo sticker creation device 1 determines that the standing positions of user U1 and user U2 are opposite to their previous positions based on the standing position information contained in the game information transmitted from the information processing server 2. When the game control unit 61 determines that the standing positions of user U1 and user U2 are opposite to their previous positions, it presents a message to user U1 and user U2 indicating this.

[0214] This allows users U1 and U2 to realize that their positions are opposite to those they had when taking photos in the previous game, and they can switch positions. When taking photos with the same friends, their positions are often predetermined. By providing information about their positions, they can take photos with the same composition as the photos taken in the previous game.

[0215] Instead of only displaying information about the user's position if it's the opposite of their previous position, the information about their position may always be displayed when user U1 and user U2 are using the system together. In this case, a message such as "You were standing on the left last time" would be displayed to user U1, and a message such as "You were standing on the right last time" would be displayed to user U2.

[0216] Information regarding a user's position is provided, for example, when user U1 and user U2 are using the service together. If user U1 is using the service with a different friend than user U2, or if user U2 is using the service with a different friend than user U1, information regarding their positions will not be provided. However, it may be possible to provide information regarding their positions even when user U1 is using the service with a different friend than user U2, or when user U2 is using the service with a different friend than user U1.

[0217] ·others If all users playing the game together are repeat users, the guidance on how to proceed with the game may be omitted or simplified. Normally, guidance is provided at various points during the game, explaining what to do at that time.

[0218] The guidance displayed will change depending on whether all users playing the game together are repeat users or if there are some non-repeat users among them. This will make it possible to shorten game time.

[0219] While user authentication is currently based on facial recognition, it may also be based on other biometric information such as iris scans or fingerprints. Alternatively, user authentication may be based on information such as an ID entered by the user.

[0220] • About the program The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed from a program storage medium onto a computer that is built into dedicated hardware, or a general-purpose personal computer.

[0221] The program to be installed is provided by recording it on removable media 111, as shown in Figure 11, which consists of optical discs (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or semiconductor memory. Alternatively, it may be provided via wired or wireless transmission media such as a local area network, the internet, or digital broadcasting. The program can be pre-installed in ROM 102 or storage unit 108.

[0222] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when they are called.

[0223] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device containing multiple modules within a single enclosure, are both considered systems.

[0224] The effects described herein are illustrative and not limited to those described herein, and other effects may also occur.

[0225] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0226] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0227] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0228] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices. [Explanation of Symbols]

[0229] 1 Photo sticker creation device, 2 Information processing server, 11 Face recognition engine, 12 Game management information DB, 13 Store installation information DB, 51 Control unit, 61 Game control unit, 62 Authentication result acquisition unit, 121 Information processing unit, 131 Image acquisition unit, 132 Face recognition unit, 133 Judgment unit, 134 Game information acquisition unit

Claims

1. The photography team photographs multiple users playing the game together, A display unit that displays the screen of the aforementioned game, An acquisition unit that determines whether or not a user is a repeat user who has used the aforementioned game, based on images obtained by photographing multiple such users, Based on the determination result for each of the multiple users whether or not they are repeat users, the control unit displays information for repeat users at a position on the screen corresponding to the standing position of the user determined to be a repeat user. A photographic device equipped with the following features.

2. The control unit changes the screen display depending on whether all of the users are repeat users or whether any of the users are not repeat users. The imaging apparatus according to claim 1.

3. The control unit displays information for repeat users on the left side of the screen if the user standing on the left side is a repeat user, and displays information for repeat users on the right side of the screen if the user standing on the right side is a repeat user. The imaging apparatus according to claim 1.

4. The acquisition unit acquires information regarding the selections made by the repeat user in the past games. The control unit displays information used to determine whether or not to reflect the same selections made in previous games in the current game, as information for repeat users. The imaging apparatus according to any one of claims 1 to 3.

5. The control unit changes the screen display depending on whether the multiple users are repeat users who have been playing the game together. The imaging apparatus according to any one of claims 1 to 4.

6. The acquisition unit acquires information regarding the standing position of the repeat user during past gameplay. The control unit presents information regarding the user's position if multiple users are repeat users who have been playing the game together. The imaging apparatus according to claim 5.

7. The control unit presents information regarding the standing positions if the standing positions of multiple users during filming in the current game differ from their standing positions during filming in past games. The imaging apparatus according to claim 6.

8. The acquisition unit authenticates a person based on a facial image and transmits each of the multiple facial images of the user generated based on the image to an information processing device that determines whether the authenticated person is a repeat user, and acquires information from the information processing device regarding the determination result of whether or not the person is a repeat user. The imaging apparatus according to any one of claims 1 to 7.

9. The imaging device, The photographs were taken of multiple users playing the game together, The screen of the aforementioned game is displayed, The determination of whether or not a user is a repeat user who has used the aforementioned game is obtained based on images taken of multiple such users. Based on the determination result for each of the multiple users whether or not they are repeat users, information for repeat users is displayed on the screen at a position corresponding to the location of the user determined to be a repeat user. Information processing methods.

10. On the computer, The photographs were taken of multiple users playing the game together, The screen of the aforementioned game is displayed, The determination of whether or not a user is a repeat user who has used the aforementioned game is obtained based on images taken of multiple such users. Based on the determination result for each of the multiple users whether or not they are repeat users, information for repeat users is displayed on the screen at a position corresponding to the location of the user determined to be a repeat user. A program to execute a process.

Images