Information processing device, information processing method, program, and photographing device

Abstract

To provide an information processing apparatus which authenticates a user of an imaging apparatus on the basis of a face image, an information processing method, a program, and an imaging apparatus.SOLUTION: An information processing apparatus stores a face image registered by a user, and performs user authentication to determine whether the user is included in a captured image which is captured in an imaging apparatus that provides a game including imaging processing to image the user as a subject, on the basis of the captured image and the face image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present technology relates to an information processing device, an information processing method, a program, and an imaging device, and particularly to an information processing device, an information processing method, a program, and an imaging device that enable authentication of a user of the imaging device based on a facial image. [Background technology]

[0002] Photo sticker creation devices are known in the art. These devices photograph a user, allow the user to edit the photographed image, and print the edited image on sticker paper. These photo sticker creation devices are installed in amusement facilities and the like.

[0003] The flow of one game provided by the photo sticker creation device usually involves taking a photo of the user in the photo booth as the subject, then moving the user to the editing booth, editing the image according to the operations performed in the editing booth, and printing the edited image on sticker paper.

[0004] Some stores that have installed these photo sticker creation devices offer a subscription service that allows users to use the photo sticker creation device's games as many times as they like after paying a certain monthly fee. Users who pay the monthly fee and register as members are issued a paper membership card.

[0005] The membership card contains information such as the membership number, the expiration date of the subscription service, the stores where the service can be used, etc. A user who uses the photo sticker creation device presents the membership card to the store staff and has them perform an operation on the device to start the game before they can begin using the game.

[0006] Patent Document 1 discloses a technique for realizing various services, such as providing free play tickets for games on a photo sticker creation device, by using a membership card application. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-142773 Summary of the Invention [Problem to be solved by the invention]

[0008] There is a possibility of fraudulent activity, such as a user who has registered as a member lending their membership card to a friend, who then uses the membership card to use the photo sticker creation device. Typically, the terms of use for subscription services such as those described above include a condition that the photo sticker creation device can only be used by the person who registered as a member or when used together with a friend.

[0009] The present technology has been made in view of such circumstances, and makes it possible to authenticate a user of an image capturing device based on a facial image. [Means for solving the problem]

[0010] An information processing device according to one aspect of the present technology includes a memory unit that stores a facial image registered by a user, and an authentication unit that performs user authentication based on the captured image and the facial image, to determine whether the user appears in an image captured by a photography device that provides a game including a photography process that photographs the user as a subject.

[0011] Another aspect of the present technology is an imaging device that includes an imaging unit that captures an image of a user as a subject, an authentication unit that performs user authentication to determine whether the user appears in the captured image based on the captured image captured by the imaging unit and a facial image provided from an information processing device that stores facial images registered by the user, and a control unit that controls the progress of a game that includes capturing an image of the user as a subject, depending on the result of the user authentication.

[0012] In one aspect of the present technology, a facial image registered by a user is stored, and user authentication is performed based on the captured image and the facial image to determine whether the user appears in an image captured by a photography device that provides a game including a photography process that photographs the user as a subject.

[0013] In another aspect of the present technology, a photograph is taken of a user as a subject, and user authentication is performed to determine whether the user appears in the photographed image based on the photographed image and a facial image provided from an information processing device that stores facial images registered by the user, and the progress of a game including the photographing of the user as a subject is controlled according to the result of the user authentication. [Effects of the Invention]

[0014] According to the present technology, it is possible to authenticate a user of an image capturing device based on a facial image. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram showing a flow when using a photo sticker creation device according to an embodiment of the present technology; [Figure 2] FIG. 10 is a diagram showing a selfie scene. [Figure 3] FIG. 10 is a diagram illustrating an example of registration of a face image. [Figure 4] FIG. 10 is a diagram illustrating an example of member information stored in a member information DB. [Figure 5] FIG. 10 is a diagram illustrating an example of a flow of user authentication. [Figure 6] FIG. 10 illustrates an example of user authentication. [Figure 7] FIG. 10 is a diagram illustrating another example of user authentication. [Figure 8] FIG. 10 is a diagram illustrating an example of timing for performing user authentication. [Figure 9] FIG. 10 is a diagram illustrating another example of the timing for performing user authentication. [Figure 10] 1 is a perspective view showing the appearance of a photo sticker creating device; [Figure 11] 1 is a block diagram showing an example of the configuration of a photo sticker creation device; [Figure 12] FIG. 1 is a block diagram illustrating an example of the configuration of a smartphone. [Figure 13] FIG. 2 is a block diagram illustrating an example of the hardware configuration of an image management server. [Figure 14] FIG. 2 is a block diagram illustrating an example of a functional configuration of an image management server. [Figure 15] 10 is a flowchart illustrating a series of processes including user authentication. [Figure 16] FIG. 10 is a diagram showing an example of the flow of user authentication when user authentication is performed by the photo sticker creating device. [Figure 17] FIG. 10 is a diagram illustrating an example of image management in an image management service. [Figure 18] FIG. 10 is a diagram showing a flow of user authentication when an image is acquired. [Figure 19] FIG. 10 is a diagram showing an example of an image provided when user authentication is successful. [Figure 20] FIG. 10 is a diagram showing another example of an image provided when user authentication is successful. DETAILED DESCRIPTION OF THE INVENTION

[0016] <Subscription service> Flow at the start of the game FIG. 1 is a diagram showing a flow of use of a photo sticker creation device according to an embodiment of the present technology.

[0017] The photo sticker creating device 1 shown in Fig. 1 is a game machine that provides a photo creating game. The photo sticker creating device 1 is installed in a store such as a game center.

[0018] Users of the photo sticker creation device 1 are mainly high school girls and young women. Multiple users, such as two or three people per group, can enjoy the game with the photo sticker creation device 1. Of course, a single user can also enjoy the game with the photo sticker creation device 1.

[0019] In the photo creation game of the photo sticker creation device 1, the user takes a photo of themselves as the subject. A camera is provided in the housing of the photo sticker creation device 1. The user edits the captured image obtained by shooting to combine it with composite images such as handwritten characters and stamp images, and edits the captured image into a colorful image. The user receives a sticker paper on which the edited captured image is printed, completing the series of games.

[0020] In the photo sticker creation device 1, a game can be started by inserting a predetermined amount of money, such as 400 yen, into a slot provided in the housing, or by presenting a membership card, as shown in the speech bubble in Figure 1. Users can use the membership card functions by installing an application provided by the manufacturer of the photo sticker creation device 1 on their smartphone and registering as a member.

[0021] By paying a predetermined fee such as 1,000 yen / month, a user can play the game on the photo sticker creation device 1 any number of times during the month for which the fee is paid. In other words, the photo sticker creation device 1 in Fig. 1 is a device that can provide games as a so-called subscription service.

[0022] When starting a game using the membership card function, the user operates the smartphone to display code information such as a two-dimensional code on the application screen. The code information includes a user ID, which is the user's identification information. The user performs user authentication by having the photo sticker creation device 1 read the code information displayed on the screen. For example, the housing of the photo sticker creation device 1 is provided with a code reader that reads the code information displayed on the application screen.

[0023] User authentication may be performed by having the photo sticker creation device 1 read the user ID stored in a contactless IC chip installed in the smartphone, or by having the smartphone read code information such as a two-dimensional code displayed on the photo sticker creation device 1, as described below.

[0024] User membership information is managed on a server on the Internet. The photo sticker creation device 1 reads the code information presented by the user and transmits information such as the user ID to the server, requesting user authentication. If the user authentication on the server is successful, information indicating that the user authentication was successful is transmitted from the server to the photo sticker creation device 1, allowing the user to use a regular game as a subscription service with the same content as when the user starts by inserting a predetermined amount of money, such as 400 yen.

[0025] In this way, by registering as a regular user who can use the subscription service, users do not have to pay money each time they start playing a game. Furthermore, users can play games as many times as they like within a specified period, such as one month.

[0026] Instead of a limited-time subscription service with a set usage period of a predetermined period such as one month, a membership card may be used to provide services with various usage conditions, such as a subscription service limited to a specific model or a subscription service limited to a specific store.

[0027] A model-specific subscription service is a service that allows you to play games on a specific model of photo sticker creation device as many times as you like from among the multiple models installed.A store-specific subscription service is a service that allows you to play games on multiple photo sticker creation devices installed in a specific store as many times as you like, regardless of the model.

[0028] The conditions for use permitted to authorized users may be set by combining various conditions such as period, model, store, etc. Conditions such as the number of times the device can be used may also be set.

[0029] ·Register a face image There is a possibility that another user who borrows a smartphone from a legitimate user may use the legitimate user's membership card to play games, which could lead to fraud. Subscription services are effective when a legitimate user plays a game by themselves or when a legitimate user plays a game together with a friend.

[0030] To prevent such fraudulent acts, the above-mentioned user authentication is performed based on a face image registered by the user.

[0031] For example, when registering as a member of a subscription service, the user takes a picture of their face using the camera on their smartphone 2, as shown in Figure 2. The application for using the membership card function is equipped with a function that sends the selfie image of their face to a server and registers them as a member.

[0032] FIG. 3 is a diagram showing an example of registration of a face image.

[0033] As shown in Fig. 3, an image management application 2A is installed on the smartphone 2 used by the user. The image management application 2A is an application provided by the manufacturer of the photo sticker creation device 1 or the like. The image management application 2A realizes functions such as membership registration and membership card functions. As will be described later, the image management application 2A also realizes functions such as viewing images uploaded from the photo sticker creation device 1 to the image management server 3 and downloading images.

[0034] The facial image taken by taking a selfie when registering as a member is sent to the image management server 3 as indicated by arrow A1. The image management server 3 is an internet server managed by the manufacturer of the photo sticker creation device 1, etc. The image management server 3 is provided with a member information DB11 and an image DB12. The facial image sent to the image management server 3 is managed in the member information DB11 as member information. The uploading of images stored in the image DB12 will be described later.

[0035] FIG. 4 is a diagram showing an example of member information stored in the member information DB 11. As shown in FIG.

[0036] As shown in Figure 4, the user ID, face image, subscription expiration date, email address, and attribute information are linked and stored as member information. The attribute information includes information such as gender and age.

[0037] User authentication at the start of a game using the membership card function is performed, for example, based on an image of the user taken during the game and a facial image that has been registered as membership information.

[0038] If the user who starts the game using the membership card function has the same face image as the registered user, the user who starts the game is determined to be an authorized user. If the user who starts the game is determined to be an authorized user, user authentication is successful, and the user who starts the game can continue playing the game as usual.

[0039] On the other hand, if the user who starts the game using the membership card function does not have the same facial image as the registered user, the user who starts the game is determined to be an unauthorized user. If the user who starts the game is determined to be an unauthorized user, user authentication will fail and the game will be played with content different from the normal game.

[0040] In this way, in the photo sticker creating device 1, the progress of the game is controlled according to the result of user authentication.

[0041] The game provided by the photo sticker creation device 1 is a game that includes a photography process in which a user is photographed as a subject. By using the image obtained by the photography process during the game, the image management server 3 can perform user authentication without the user being aware of it.

[0042] <User authentication (example of user authentication performed on the server side)> FIG. 5 is a diagram illustrating an example of the flow of user authentication.

[0043] Here, we will explain the case where two users, user U1 and user U2, use the game of the photo sticker creation device 1 together. User U1 is a user who has subscribed to the subscription service and registered a face image when registering as a member. User U2 is a friend of user U1. User U2 is not subscribed to the subscription service.

[0044] As shown on the left side of Figure 5, when user U1 operates image management application 2A and presents his / her membership card, the game begins. The game of photo sticker creation device 1 mainly involves the user performing a series of tasks: pre-selection before taking a photo, entering a photo booth to take a photo, and moving from the photo booth to an editing booth to edit the captured image.

[0045] After presenting their membership cards and having the code information read into the photo sticker creation device 1, users U1 and U2 select the number of people, the photography course, and other pre-selection tasks. Outside the photography booth, a monitor or the like is provided to display a screen used for the pre-selection tasks. A code reader or the like used to read the code information is also provided outside the photography booth.

[0046] After completing the pre-selection work, users U1 and U2 follow the instructions to enter the photo booth and take their photos. In the photo booth, multiple still images, such as five, are taken at predetermined time intervals. The photo booth is equipped with a photo monitor that displays a live view image, which is a moving image being captured by the camera. Users U1 and U2 pose for the camera while looking at the display on the photo monitor provided in the photo booth, and then take their photos.

[0047] For example, when the first photograph is taken, as shown by arrow A11, the photo sticker creating device 1 transmits the first photographed image P1 obtained by the first photograph to the image management server 3. In the photographed image P1, user U1 appears on the right and user U2 appears on the left. The user ID obtained by reading the code information is also transmitted to the image management server 3.

[0048] The image management server 3 receives the photographed image P1 sent from the photo sticker creating device 1 and reads out the member information of the user U1 from the member information DB 11 based on the user ID. The image management server 3 also performs user authentication based on the photographed image P1 and the face image of the user U1 included in the member information. That is, user authentication is performed based on the photographed image sent from the photo sticker creating device 1 during the game.

[0049] FIG. 6 is a diagram illustrating an example of user authentication.

[0050] As indicated by the tip of arrow A21, a face image of user U1 and a face image of user U2 are extracted from captured image P1. In the example of Fig. 6, an image of a rectangular area including the face of each user is extracted as the face image.

[0051] Furthermore, as indicated by arrow A22, it is determined whether the person in the captured image P1 is the same as user U1, who is registered as an authorized user, based on the facial images of users U1 and U2 and the facial images stored in the image DB 12 as member information. For example, feature points are extracted from each facial image, and it is determined whether they are the same person based on the positional relationship of the feature points. The determination of whether they are the same person may be performed using an inference model generated by machine learning.

[0052] It should be noted that the images provided to the user as the outcome of the game of the photo sticker creation device 1 are images that have undergone various processing processes. The processed images, or images that have been further edited from processed images, are printed on sticker paper or provided to the user via a server.

[0053] It is possible to use the original image before any processing is performed for user authentication, as shown in photographed image P1 in Fig. 6. In this case, the first photographed image provided to the user and the image to be edited is an image that has been processed, while the image that has not been processed is sent to image management server 3 and used for user authentication.

[0054] The processing includes at least one of the following: enlarging the eyes, reducing the size of the face, adjusting the skin color to be lighter, and adjusting the hair color.Facial images registered as member information are not subjected to such processing.

[0055] For example, if user authentication is performed based on a processed captured image that enlarges the eyes, even if the captured image shows a legitimate user, there is a possibility that the captured image may be mistakenly determined to show a person different from the person in the facial image in the member information. By performing user authentication based on the original captured image, it is possible to prevent such mistaken determination from occurring.

[0056] When a determination is made based on the captured image P1 and a registered face image, as shown in Fig. 7, user U1 in the captured image P1 is determined to be a legitimate user, and user U2 is determined to be an unauthorized user. Because user U1, who is a legitimate user, is in the image, user authentication is successful. In other words, as shown in the lower part of Fig. 7, when multiple people appear in the captured image, user authentication is successful as long as at least one legitimate user is in the image.

[0057] The result of the user authentication is sent from the image management server 3 to the photo sticker creation device 1, as shown by arrow A12 in FIG. 5. The photo sticker creation device 1 controls the progress of the game according to the result of the user authentication. User U1, who is an authorized user, can use the game of the photo sticker creation device 1 as a subscription service together with user U2. If user authentication using the first captured image is successful, processing from the second captured image onwards is carried out.

[0058] The above-described user authentication is performed, for example, each time a photograph is taken, based on the photographed image obtained by the photograph.

[0059] FIG. 8 is a diagram showing an example of timing for performing user authentication.

[0060] After users U1 and U2 enter the photo booth and photoshooting begins, a live view image is displayed on the photoshoot monitor installed in the photoshoot booth. After a certain time has passed since the live view image was displayed, the first photoshoot is performed, and user authentication is performed based on the photoshoot image P1 obtained by the first photoshoot.

[0061] Then, a second photograph is taken, and user authentication is performed based on the photographed image P2 obtained by the second photograph. In this manner, user authentication is repeated each time a photograph is taken. When user authentication is performed based on the photographed image P5 obtained by the final photograph, for example, the fifth photograph, user authentication ends together with the photographing.

[0062] Depending on the direction of the face, etc., user authentication may not be successful in one attempt even if the authorized user is taking the photo. Also, there may be cases where the photo is taken in a state where only authorized user U1, user U2, who is playing together with the authorized user, is in the photo.

[0063] In this way, by repeatedly performing user authentication, it becomes possible to more reliably determine whether the legitimate user is in the photograph. A flag indicating whether the legitimate user is in the photograph may be set for each photographed image.

[0064] Once user authentication is successful, subsequent user authentication may not be performed. If user authentication is performed for each photograph, user authentication for the entire game may be determined to be successful if at least one photographed image shows a legitimate user.

[0065] In addition, the user may be guided to take a photo in a state where the authorized user is in the photo. Such guidance may be presented, for example, by a sound from a speaker provided in the photo booth of the photo sticker creating device 1 or by a screen display on the monitor for photography.

[0066] FIG. 9 is a diagram showing another example of the timing at which user authentication is performed.

[0067] 9, user authentication may be performed based on the five captured images obtained by five captures after all captures have been completed. When the fifth capture is completed, the captured images P1 to P5 are sent from the photo sticker creating device 1 to the image management server 3 along with the user ID, and user authentication is performed based on the facial images extracted from each captured image and the facial image of user U1 that has been registered as member information.

[0068] For example, if user U1, who is an authorized user, appears in at least one of the five captured images, user authentication is successful. The criteria for successful user authentication can be set arbitrarily, such as if the authorized user appears in two or more captured images, or if the authorized user appears in all captured images.

[0069] In this way, user authentication can be performed only once using multiple captured images after the last capture.

[0070] The details of the processing of the photograph sticker creating device 1 that performs user authentication as described above and controls the progress of the game based on the result of user authentication will be described later.

[0071] <Configuration of each device> Photo sticker creation device 1 configuration FIG. 10 is a perspective view showing the appearance of the photograph sticker creating device 1. As shown in FIG.

[0072] As shown in FIG. 10 , a pre-operation unit 21 is provided on the side of the photo sticker creation device 1 having a box-shaped housing. The pre-operation unit 21 is a device used for pre-selection operations such as selecting the number of users to play with and selecting a photography course. A touch panel monitor 21A is provided above the pre-operation unit 21. A coin slot 21B for paying a play fee is provided below the touch panel monitor 21A. A code reader 21C for reading code information displayed on the smartphone 2 is provided to the left of the coin slot 21B.

[0073] A screen used for various selections is displayed on touch panel monitor 21A. After making various selections according to the display on touch panel monitor 21A, the user enters the photo booth through entrance G and begins taking photos. A box-shaped photo unit 31 with a built-in camera 31A is provided at the front of the photo booth. Photo unit 31 is provided with a photo monitor 31B.

[0074] An editing unit 41 is provided to the left of the pre-operation unit 21. The editing unit 41 is a device used to edit the captured images obtained by taking pictures with the camera 31A. A user who has taken pictures in the photo booth moves to the editing booth in front of the editing unit 41 and edits the captured images. An editing monitor 41A is provided on a slope formed above the editing unit 41. A screen used to edit the captured images is displayed on the editing monitor 41A.

[0075] Fig. 11 is a block diagram showing an example of the configuration of the photo sticker creating device 1. Of the configuration shown in Fig. 11, the same components as those described above are given the same reference numerals, and duplicated explanations will be omitted where appropriate.

[0076] As shown in FIG. 11, the photo sticker creating device 1 is configured by connecting a pre-operation unit 21, a photographing unit 31, an editing unit 41, a communication unit 102, and a printing unit 103 to a control unit 101 configured by a PC.

[0077] The control unit 101 executes a predetermined program using the CPU, and in accordance with the progress of the game, controls the overall operation of the photo sticker creating device 1. In the control unit 101, a game control unit 111 and an authentication result acquisition unit 112 are realized by executing the predetermined program.

[0078] The game control unit 111 controls a series of game processes such as pre-selection processing, photography processing, editing processing, and printing processing.

[0079] The pre-selection process is a process that allows the user to perform pre-selection tasks such as selecting a photography course.

[0080] The photographing process is a process in which the user performs the task of photographing the user as a subject.

[0081] The editing process is a process in which the user edits the captured image obtained by the photographing process.

[0082] The printing process is a process in which the edited images obtained by the editing process are arranged in a predetermined layout and printed on sticker paper. The sticker paper on which the edited images are printed is provided to the user through an outlet provided at a predetermined position on the housing of the editing unit 41.

[0083] The authentication result acquisition unit 112 controls the communication unit 102 to request user authentication by transmitting the captured image supplied from the imaging unit 31 to the image management server 3. The authentication result acquisition unit 112 acquires the result of user authentication transmitted from the image management server 3.

[0084] The pre-operation unit 21 displays a predetermined screen on the touch panel monitor 21A and allows the user to make various selections before starting shooting. The pre-operation unit 21 accepts the user's selections and outputs information indicating the selections to the control unit 101.

[0085] The photographing unit 31 drives the camera 31A to photograph the user in the photographing booth as a subject. For example, by taking photographs multiple times, multiple photographed images are acquired. The photographed images acquired by the photographing unit 31 are supplied to the control unit 101.

[0086] The editing unit 41 performs editing processing for a user in the editing booth. The editing processing includes image processing for editing the captured image. The captured image is edited using an editing screen displayed on the editing monitor 41A. After editing the captured image, the editing unit 41 generates print data by arranging the edited image in a predetermined layout and outputs the print data to the control unit 101. The above-mentioned processing may be selected when editing the captured image.

[0087] The communication unit 102 communicates with external devices such as the image management server 3. The communication unit 102 transmits captured images used for user authentication to the image management server 3. The communication unit 102 also transmits images to be provided to the user via the server to the image management server 3.

[0088] The printing unit 103 has a printer. The printing unit 103 drives the printer and prints the edited photographed image on sticker paper based on the print data.

[0089] Smartphone 2 configuration FIG. 12 is a block diagram showing an example of the configuration of the smartphone 2.

[0090] As shown in FIG. 12, the smartphone 2 includes a control unit 201, a communication unit 202, a memory 203, a display 204, an operation unit 205, and a camera 206.

[0091] The control unit 201 is configured with a CPU, a ROM, a RAM, etc. The control unit 201 controls the overall operation of the smartphone 2 by executing a predetermined program.

[0092] An application execution unit 201A is implemented in the control unit 201. By executing the image management application 2A by the application execution unit 201A, processes such as member registration and display of a membership card screen are implemented. The image management application 2A manages information such as the user ID for using the services provided by the image management server 3. When a membership card is presented, the image management application 2A causes the display 204 to display a membership card screen including code information generated based on the user ID.

[0093] The communication unit 202 communicates with the image management server 3 via the Internet. The communication unit 202 receives information transmitted from the image management server 3 and outputs it to the control unit 201.

[0094] The memory 203 is configured by a flash memory, etc. The memory 203 stores various information such as images downloaded from the image management server 3 and applications executed by the control unit 201.

[0095] The display 204 is configured by an organic EL display, an LCD, etc. The display 204 displays various screens such as the screen of the image management application 2A.

[0096] The operation unit 205 is configured with various buttons and a touch panel provided over the display 204. The operation unit 205 outputs to the control unit 201 a signal indicating the content of a user's operation.

[0097] The camera 206 takes pictures of various subjects, including the user when registering as a member. The camera 206 may be used to read code information displayed on the photo sticker creating device 1.

[0098] Image management server 3 configuration FIG. 13 is a block diagram showing an example of the hardware configuration of the image management server 3. As shown in FIG.

[0099] 13, the image management server 3 is configured by a computer. The image management server 3 may be configured by multiple computers. When the image management server 3 is configured by multiple computers, the various processes of the image management server 3 are realized by the respective computers working together.

[0100] A CPU (Central Processing Unit) 301 , a ROM (Read Only Memory) 302 , and a RAM (Random Access Memory) 303 are interconnected by a bus 304 .

[0101] An input / output interface 305 is further connected to the bus 304. An input unit 306, an output unit 307, a storage unit 308, a communication unit 309, and a drive 310 are connected to the input / output interface 305.

[0102] The input unit 306 is configured with a keyboard, a mouse, etc. The output unit 307 is configured with a display, etc.

[0103] The storage unit 308 is configured with a hard disk, non-volatile memory, etc. The storage unit 308 stores various information used to provide the image providing service, such as programs executed by the CPU 301 and images uploaded from the photo sticker creating device 1. A member information DB 11 and an image DB 12 are constructed in the storage unit 308, for example.

[0104] The communication unit 309 is an interface to the Internet. For example, the communication unit 309 communicates with the photo sticker creating device 1 and receives images uploaded from the photo sticker creating device 1. The communication unit 309 also communicates with the smartphone 2 and transmits various information to the smartphone 2, such as information used for screen display and images selected to be downloaded.

[0105] The drive 310 controls writing of data to and reading of data from the removable medium 311 .

[0106] FIG. 14 is a block diagram showing an example of the functional configuration of the image management server 3. As shown in FIG.

[0107] 14, an information processing unit 321 is implemented in the image management server 3. The image management server 3 is an information processing device having the information processing unit 321.

[0108] The information processing unit 321 is made up of an authentication unit 331 and an image management unit 332. At least some of the functional units shown in Fig. 14 are realized by the CPU 101 executing a predetermined program.

[0109] The authentication unit 331 acquires the captured image sent from the photo sticker creating device 1 and received by the communication unit 309. The authentication unit 331 performs user authentication based on the captured image sent from the photo sticker creating device 1 and the face image managed in the member information DB 11, and sends the result of the user authentication to the photo sticker creating device 1. The result of the user authentication by the authentication unit 331 is supplied to the image management unit 332 as appropriate.

[0110] The image management unit 332 associates images uploaded from the photo sticker creation device 1 and received by the communication unit 109 with management information and stores them in the image DB 12. The image management unit 332 controls the communication unit 109 to send images of users who use the image providing service to the smartphone 2. The image providing service is a service that provides images created by users during games to users who use the photo sticker creation device 1. As will be described later, the provision of images in the image providing service is controlled according to the result of user authentication.

[0111] <Operation of each device> 15, a series of processes including user authentication performed between the photo sticker creation device 1 and the image management server 3 will be described. The process in FIG. 15 starts, for example, when a user operates the smartphone 2 to display the membership card screen on the display 204 and points it at the code reader 21C.

[0112] In step S1, the code reader 21C of the photo sticker creating device 1 reads the code information displayed on the membership card screen.

[0113] In step S2, the game control unit 111 of the control unit 101 controls the pre-operation unit 21 to perform a pre-selection process.

[0114] In step S3, the game control unit 111 controls the photographing unit 31 to perform photographing processing.

[0115] In step S4, the authentication result acquisition unit 112 transmits the captured image to the image management server 3. Information such as the user ID is also transmitted to the image management server 3 along with the captured image as appropriate. If user authentication is performed for each image capture, the authentication result acquisition unit 112 transmits the captured image each time an image is captured. If user authentication is performed after image capture is completed, the authentication result acquisition unit 112 transmits the captured image when image capture is completed.

[0116] In step S11, the authentication unit 331 of the image management server 3 receives the photographed image sent from the photo sticker creating device 1.

[0117] In step S12, the authentication unit 331 performs user authentication based on the received photographed image and the face image managed in the member information DB11.

[0118] In step S13, the authentication unit 331 transmits the result of the user authentication to the photo sticker creating device 1.

[0119] In step S5, the authentication result acquisition unit 112 of the photo sticker creating device 1 acquires the result of user authentication sent from the image management server 3.

[0120] In step S6, the game control unit 111 controls the progress of the game in accordance with the result of the user authentication.

[0121] If the user authentication is successful, the game control unit 111 proceeds with the game from that point on. If the user authentication is successful, the game is played with the same content as when the game is started by inserting money.

[0122] For example, if user authentication is performed for each photo taken and user authentication using the first photo taken is successful, the second and subsequent photos can be taken. Also, if user authentication is performed after photo taking is completed and user authentication using the photo taken is successful, the user is guided to move to the editing booth, and editing and subsequent steps can be continued.

[0123] The user in the photo booth may be notified that the user authentication has been successful.

[0124] On the other hand, if the user authentication fails, the game control unit 111 performs a process different from that when the user authentication is successful, i.e., a process different from that when money is inserted and the game is started.

[0125] For example, if user authentication is performed for each photograph and user authentication using the first photographed image fails, the game control unit 111 stops (interrupts) the progress of the game before the second photograph is taken. Similarly, if user authentication is performed after photographing is completed and user authentication using the photographed image fails, the progress of the game is stopped.

[0126] If user authentication fails, the user may be prompted to continue playing the game by paying money. If the user follows the prompt and pays money, the game resumes. In this case, a money slot is provided inside the photo booth. Payment may be made by cashless payment by operating the smartphone 2.

[0127] When the user authentication is successful, or when the user authentication is unsuccessful and the payment is made, in step S7, the game control unit 111 controls the editing unit 41 to perform editing processing.

[0128] In step S8, the game control unit 111 controls the communication unit 102 to upload the photographed image to the image management server 3. For example, the photographed image before editing and the photographed image after editing are transmitted to the image management server 3.

[0129] In step S14, the image management section 332 of the image management server 3 receives the image uploaded from the photo sticker creating device 1.

[0130] In step S15, the image management unit 332 stores the received image in the image DB 12.

[0131] Meanwhile, in step S9, the game control unit 111 controls the printing unit 103 to perform printing processing. The series of processes ends when the user receives the sticker paper on which the edited photographed image or the like is printed.

[0132] In this way, the user can play the game of the photo sticker creating device 1 provided as a subscription service without being aware that face authentication is being performed.

[0133] The subscription service provider can also prevent the game on the photo sticker creation device 1 from being used illegally as a subscription service by users other than authorized users. That is, even if a user borrows a smartphone from an authorized user and starts a game by presenting the authorized user's membership card, the user who started the game will be determined to be an unauthorized user through user authentication using a facial image, and will not be able to continue playing the game as usual. Preventing unauthorized use makes it possible to increase the number of subscribers to the subscription service.

[0134] <Other examples of game progression> If user authentication fails when taking a photo, the game may continue as is and uploading to the image management server 3 may not be performed. Alternatively, printing may not be performed and sticker paper with the captured image printed on it may not be provided. In this case, a user who fails user authentication will not be able to receive the image created during the game as data via the server or printed on sticker paper.

[0135] Instead of restricting both uploading to the image management server 3 and printing, only one of uploading to the image management server 3 and printing may be restricted.

[0136] Instead of restricting the uploading of all captured images, it is also possible to restrict only images for which user authentication has failed from being uploaded, and upload images for which user authentication has succeeded.

[0137] Also, instead of restricting printing of all captured images, only images that fail user authentication may not be printed, and images that succeed in user authentication may be printed on sticker paper and provided to the user.

[0138] <User authentication (an example where user authentication is performed on the photo sticker creation device)> User authentication may be performed by the photo sticker creating device 1 instead of by the image management server 3.

[0139] 16 is a diagram showing an example of the flow of user authentication when the user authentication is performed by the photo sticker creating device 1. Description of the same processes as those described with reference to FIG. 5 will be omitted as appropriate.

[0140] As shown on the left side of Fig. 16, the game begins when user U1 operates image management application 2A and presents their membership card. After presenting their membership card and having the code information read into photo sticker creation device 1, users U1 and U2 select the number of people and the photography course as pre-selection tasks.

[0141] After completing the pre-selection work, users U1 and U2 follow the instructions to enter the photo booth and take their photos.

[0142] After the code information is read, the photo sticker creating device 1 transmits the user ID included in the code information to the image management server 3, as shown by an arrow A31.

[0143] The image management server 3 receives the user ID sent from the photo sticker creating device 1 and, based on the user ID, reads out the member information of user U1 from the member information DB 11. The image management server 3 sends the face image of user U1 included in the member information to the photo sticker creating device 1, as shown by arrow A32.

[0144] The photo sticker creating device 1 receives the face image sent from the image management server 3 and performs user authentication based on the received face image and the photographed image obtained by the photographing process. User authentication by the photo sticker creating device 1 is also performed for each face appearing in the photographed image, as explained with reference to Fig. 6. User authentication by the photo sticker creating device 1 may also be performed each time a photograph is taken, or after the photographing is completed.

[0145] After the user authentication, the photo sticker creating device 1 controls the progress of the game according to the result of the user authentication.

[0146] In this way, it is also possible for the registered face image to be provided from the image management server 3 to the photograph sticker creating device 1, and for the photograph sticker creating device 1 to perform user authentication.

[0147] <Other processes based on the result of user authentication> User authentication may be performed not during the game, but when an image is obtained (image acquisition) after the game ends.

[0148] Image Get is a process of downloading images uploaded from the photo sticker creation device 1 to the image management server 3 to the user's smartphone 2 or saving them on the My Page screen. The My Page screen is a user-dedicated screen (area) assigned to each user who has registered as a member of the image management service provided by the image management server 3. Images saved on the My Page screen can be viewed by using the image management application 2A. Providing images through Image Get is providing images via the server described above.

[0149] Image management services Fig. 17 is a diagram showing an example of image management in an image management service. In the example of Fig. 17, only one photo sticker creation device 1 is shown, but in reality, multiple photo sticker creation devices 1 sold by the same manufacturer are installed.

[0150] The photo sticker creation device 1 not only prints the images created in the game onto sticker paper and provides them to the user, but also makes the images viewable on the smartphone 2, thereby providing the user with the images. As shown in Fig. 17, the images created in the game are transmitted (uploaded) from the photo sticker creation device 1 to the image management server 3. For example, five images created in the game are uploaded.

[0151] Management information is added to images uploaded from the photo sticker creating device 1. The management information is information that is generated in the photo sticker creating device 1 to manage the uploaded images and is transmitted to the image management server 3. In the image management server 3, the uploaded images are stored in the image DB 12 in association with each piece of information included in the management information.

[0152] The management information includes the sticker ID, email address, etc.

[0153] The sticker ID is an ID that is issued for each game according to a predetermined rule by the photo sticker creating device 1. Because it is an ID that is issued each time a game is played, it can also be said to be an ID for identifying the game.

[0154] For example, the sticker ID is printed on the edge of the sticker paper on which the edited image is printed and is notified to the user. When a user of smartphone 2 who has performed a login operation to the image management service on image management app 2A looks at the edge of the sticker paper and enters the sticker ID, the image created by the user of smartphone 2 during the game is identified in image management server 3. In this way, the sticker ID is used to identify the image created by each user in each game.

[0155] The email address is the email address of the user entered during the game.

[0156] The user's email address is used by the image management server 3, for example, to notify the user of the URL assigned as the image storage destination by email. If there are two users, the management information includes the email addresses of each user. For example, the email addresses are input at a predetermined time, such as when editing of a captured image is completed.

[0157] After such uploading has been performed, the user can access the image management server 3 by operating the image management application 2A, for example, and obtain the images created in the game. The screen of the image management application 2A displays thumbnails of images uploaded from the photo sticker creation device 1, images that can be downloaded, and images that can be saved on the My Page screen.

[0158] FIG. 18 is a diagram showing the flow of user authentication when getting an image.

[0159] As shown on the left side of Figure 18, when user U1 is displaying uploaded images and instructs to get a specific image, image management application 2A requests image management server 3 to get the image selected by user U1, as shown by arrow A41.

[0160] Based on the user ID of user U1, the image management server 3 reads the member information of user U1 from the member information DB 11. When the image management application 2A, which manages the user ID of user U1, accesses the image management server 3, the user ID of user U1 has been identified by the image management server 3.

[0161] The image management server 3 also performs user authentication based on the photographed image for which an image get is requested and the facial image of user U1 included in the membership information. If user U1, who is an authorized user, appears in the photographed image for which an image get is requested, user authentication is successful. If user U1, who is an authorized user, does not appear in the photographed image for which an image get is requested, user authentication is unsuccessful.

[0162] If the user authentication is successful, the image management server 3 permits the user to get the requested captured image and provides the captured image to the user. In this case, the captured image is downloaded to the smartphone 2, saved on the My Page screen, and so on.

[0163] On the other hand, if the user authentication fails, the image management server 3 restricts the image get of the requested photographed image and does not provide the photographed image to the user. The image is not downloaded to the smartphone 2, nor is it saved on the My Page screen.

[0164] Fig. 19 is a diagram showing an example of images provided when user authentication is successful. It is assumed that five photographed images P1 to P5 shown on the left side of Fig. 19 have been uploaded from the photo sticker creating device 1 as images taken in one game.

[0165] If the image requested for image get contains a picture of the authorized user U1, user authentication is successful. As shown on the right side of Figure 19, the pictures P1 to P3, for example, in which the user U1 requesting image get is included, are available images.

[0166] On the other hand, if the user U1, who is the authorized user, is not included in the captured image for which the image get is requested, the user authentication fails. As shown on the right side of Figure 19, captured images P4 and P5, for example, that do not include the user U1 requesting the image get, are not available.

[0167] In this example, even if an image was taken during the same game, if the image does not include user U1, the image will be treated as an image that cannot be provided.

[0168] As shown in Fig. 20, even if a captured image does not include user U1, the image may be treated as an available image if it was captured in the same game as an image that includes user U1, who is an authorized user. This allows user U1, who is an authorized user, to obtain an image that includes only user U2, who is a friend who played a game with him.

[0169] <Modification> Although the membership card for using the subscription service is provided as a function of the image management application 2A, it may also be provided physically as a card made of paper, resin, or other material. Code information including a user ID is printed on the front of the membership card. When using a game through the subscription service, the user reads the code information printed on the membership card into the photo sticker creation device 1 and starts the game.

[0170] Although it is possible to start the game on the photo sticker creation device 1 by reading the code information displayed on the screen of the smartphone 2 into the code reader 21C of the photo sticker creation device 1, it is also possible to start the game on the photo sticker creation device 1 by reading the code information displayed on the monitor of the photo sticker creation device 1 on the smartphone 2 side.

[0171] In this case, for example, code information such as a two-dimensional code including a housing ID assigned to the housing of the photo sticker creating device 1 is displayed on the touch panel monitor 21A provided in the advance operation unit 21 of the photo sticker creating device 1. The user photographs the code information with the camera 206 of the smartphone 2, and causes the image management application 2A to read the housing ID of the photo sticker creating device 1.

[0172] The housing ID of the photo sticker creation device 1 read by the image management application 2A is sent to the image management server 3 together with the user ID managed by the image management application 2A, and the user ID of the user using the photo sticker creation device 1 is thereby identified by the image management server 3. In the image management server 3, the user's facial image is identified based on the user ID, and user authentication and the like are performed.

[0173] Although the image used for user authentication is taken during the photography process using camera 31A installed in the photography booth, it may also be taken using a camera installed outside the photography booth to photograph the user before entering the photography booth. For example, it is possible to use a camera installed in pre-operation unit 21 to photograph the user before entering the photography booth.

[0174] About the program The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0175] The program to be installed is provided by being recorded on removable media 311 shown in FIG. 13, which may be an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. Alternatively, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be installed in advance in ROM 302 or storage unit 308.

[0176] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0177] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device with multiple modules housed in a single housing, are both systems.

[0178] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0179] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0180] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0181] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0182] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. [Explanation of symbols]

[0183] 1 Photo sticker creation device, 2 Smartphone, 3 Image management server, 11 Member information DB, 12 Image DB, 21 Pre-operation unit, 21C Code reader, 31 Photography unit, 41 Editing unit, 101 Control unit, 111 Game control unit, 112 Authentication result acquisition unit, 321 Information processing unit, 331 Authentication unit, 332 Image management unit

Claims

1. A storage unit that stores a facial image of a user who uses a game subscription service that includes photography of the user as a subject, the facial image of the user being photographed using a mobile device before using a photography device, and the facial image transmitted from the mobile device, in association with a user ID; an authentication unit that performs user authentication to determine whether a user appearing in an image taken by the photographing device and transmitted from the photographing device after the start of use of the photographing device is a legitimate user of the subscription service, based on a face image acquired by the photographing device when the start of use of the photographing device is started and linked to a user ID transmitted from the photographing device; Equipped with When the authentication unit determines that the user appearing in the photographed image transmitted from the photographing device is the authorized user, the authentication unit allows the photographing device to continue providing the game. Information processing device.

2. The captured image used for the user authentication is an image that has undergone predetermined image processing and is provided to the user as an outcome of the game. The information processing device according to claim 1 .

3. If the result of the user authentication indicates that the user is not the authorized user, the progress of the game is stopped in the photographing device.

3. The information processing device according to claim 1.

4. The game is resumed upon payment of the fee. The information processing device according to claim 3 .

5. The authentication unit performs the user authentication based on the captured image transmitted from the image capturing device each time a photograph is taken.

5. The information processing device according to claim 1.

6. The authentication unit performs the user authentication based on each of the plurality of captured images transmitted from the photographing device after the completion of a plurality of photographing operations.

5. The information processing device according to claim 1.

7. The authentication unit determines that the user is the authorized user if there is at least one image of the user who is the authorized user among a plurality of images taken during one game.

7. The information processing device according to claim 1.

8. The authentication unit determines that the user is the authorized user when at least one of the users whose face images have been registered is included among a plurality of people appearing in the photographed image.

7. The information processing device according to claim 1.

9. The authentication unit performs the user authentication based on an original image of the captured image that has not been subjected to the predetermined image processing. The information processing device according to claim 2 .

10. The information processing device a facial image of a user who uses a game subscription service that includes photography of the user as a subject, the facial image of the user being photographed using a mobile terminal before using a photography device, the facial image of the user being photographed using the mobile terminal, and the facial image of the user being transmitted from the mobile terminal being associated with a user ID and stored; user authentication is performed to determine whether a user appearing in an image taken by the photographing device and transmitted from the photographing device after the start of use of the photographing device is an authorized user of the subscription service, based on a face image acquired by the photographing device when the use of the photographing device is started and linked to a user ID transmitted from the photographing device; If it is determined that the user appearing in the photographed image transmitted from the photographing device is the authorized user, provision of the game by the photographing device continues. Information processing methods.

11. On the computer, a facial image of a user who uses a game subscription service that includes photography of the user as a subject, the facial image of the user being photographed using a mobile terminal before using a photography device, the facial image of the user being photographed using the mobile terminal, and the facial image of the user being transmitted from the mobile terminal being associated with a user ID and stored; user authentication is performed to determine whether a user appearing in an image taken by the photographing device and transmitted from the photographing device after the start of use of the photographing device is an authorized user of the subscription service, based on a face image acquired by the photographing device when the use of the photographing device is started and linked to a user ID transmitted from the photographing device; If it is determined that the user appearing in the photographed image transmitted from the photographing device is the authorized user, provision of the game by the photographing device continues. A program for executing a process.

12. an imaging unit that captures an image of a user as a subject; an authentication unit that performs user authentication to determine whether a user appearing in an image taken by the photographing unit is a legitimate user of the subscription service, based on a facial image that is taken by a user using a mobile terminal before using the photographing device and that is provided by an information processing device that stores a facial image of the user transmitted from the mobile terminal and that is associated with a user ID acquired when the user starts using the photographing device; and a control unit that continues providing the game when it is determined that the user appearing in the photographed image is the authorized user; An imaging device comprising:

Images