Image processing device, image processing method, and program

Abstract

To enable a user to select his or her favorite face while checking his or her face and other faces.SOLUTION: An image processing device as one aspect of this technique analyzes the features of a user's face based on an image obtained by photographing the user as a subject, displays a result of the analysis of the features of the user's face, and makes the user select a favorite face out of the face with the displayed features and the other faces. The technique can be applied to a photograph creation game machine which provides a series of games to print photographed images on a seal paper.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] This technology is particularly useful for users to: It is possible to perform image processing suited to facial features. Image processing device and image processing method ,program Regarding. [Background technology]

[0002] Conventionally, photo sticker creation devices are known. The photo sticker creation devices take a picture of a user, allow the user to edit the photographed image, and print the edited image on sticker paper for provision. The photo sticker creation devices are installed in amusement facilities and the like.

[0003] The flow of a game provided by the photo sticker creation device typically involves taking a photo of the user in the shooting space as the subject, then moving the user to the editing space, editing the image according to the operations performed in the editing space, and printing the edited image on sticker paper.

[0004] Some of these photo sticker creation devices have a function to adjust the appearance of the user in the image by modifying parts of the face or body. For example, the user can slim down the shape of his or her face or legs while looking at the image of himself or herself.

[0005] Patent Document 1 discloses a technique for calculating a facial tendency by referring to a reference face shape generated by statistically processing the face shapes of a plurality of people. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2017 / 103985 Summary of the Invention [Problem to be solved by the invention]

[0007] In order to adjust the appearance appropriately, it is desirable for the user to know the characteristics of their face and body. For example, if the contours of the face are objectively thin enough but the user is not aware of this, the appearance will become unnatural if the adjustment is made to make them even thinner.

[0008] In addition, in order to adjust the appearance appropriately, it is desirable for the user to know the objective characteristics of the face they want to have. Even if they imagine the face they want and adjust it by intuition, the face may not turn out as they wish.

[0009] This technology was developed in light of these circumstances, and allows users to: It is possible to perform image processing suited to facial features. It is intended to make it possible. [Means for solving the problem]

[0010] According to one aspect of the present technology, there is provided an image processing device, comprising: an analysis unit that acquires a first captured image in which a plurality of persons are captured as subjects, and analyzes facial features of each of the persons captured in the first captured image; and the results of an analysis of the facial features of each of said persons. and an image processing unit that performs image processing on the first captured image, including deformation of specific parts of the face of each of the persons captured in the first captured image, based on the analysis results of the facial features of each of the persons and a user's selection operation regarding the appearance of each of the persons' faces, and performs image processing with the same content as the image processing on the first captured image on each of a plurality of second captured images. The display control unit displays the captured image after image processing each time image processing including deformation of a specific part of the person's face is performed on each captured image including the first captured image and the multiple second captured images.

[0011] In one aspect of the present technology, a first captured image in which a plurality of people are captured as subjects is acquired, and facial features of each of the people captured in the first captured image are analyzed. and the results of an analysis of the facial features of each of said persons.is displayed, and image processing including deformation of specific parts of the face of each of the persons appearing in the first photographed image is performed on the first photographed image based on the analysis result of the facial features of each of the persons and the user's selection operation for the appearance of the face of each of the persons. Also, image processing with the same content as the image processing on the first photographed image is performed on each of the plurality of second photographed images. Each time image processing including deformation of a specific part of the face of the person is performed on each of the captured images including the first captured image and the plurality of second captured images, the captured image after image processing is displayed. Effect of the Invention

[0012] According to this technology, the user can: Perform image processing appropriate to facial features can be done. [Brief description of the drawings]

[0013] [Figure 1] 1 is a perspective view showing an example of the external configuration of a photo sticker creation device according to the present technology; [Diagram 2] FIG. 2 is a perspective view of the photo sticker creating device as seen from another angle. [Diagram 3] FIG. 1 is a diagram illustrating the movement of a user. [Figure 4] FIG. 13 illustrates an example of the configuration of a pre-selection unit; [Diagram 5] FIG. 2 is a diagram illustrating an example of the configuration of an imaging unit. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of the front side of the editing unit. [Figure 7] 1 is a block diagram showing an example of the configuration of a photo sticker creating device; [Figure 8] 2 is a block diagram showing an example of a functional configuration of the photo sticker creating device; [Figure 9] 9 is a block diagram showing an example of the configuration of an image capture processing unit in FIG. 8. [Figure 10] 9 is a block diagram showing an example of the configuration of an edit processing unit in FIG. 8. [Figure 11] 13 is a flowchart illustrating a photo sticker creation game process. [Figure 12] 12 is a flowchart illustrating the photographing process performed in step S2 of FIG. 11. [Figure 13]FIG. 13 is a diagram showing a display example of a live view screen. [Figure 14] FIG. 13 is a diagram showing an example of a display screen for analysis results. [Figure 15] FIG. 2 is an enlarged view of a facial feature map. [Figure 16] FIG. 13 is a diagram showing another example of a display screen of the analysis result. [Figure 17] FIG. 13 is a diagram showing an example of a guide screen. [Figure 18] FIG. 13 is a diagram showing a display example of a selection screen for a desired face. [Figure 19] FIG. 13 is a diagram showing a display example of a desired face selection result. [Figure 20] FIG. 13 is a diagram showing an example of mapping according to eye spacing. [Figure 21] FIG. 13 is a diagram showing an example of mapping according to the shape of the face contour. [Figure 22] FIG. 13 is a diagram illustrating an example of how to calculate an evaluation value. [Figure 23] FIG. 13 is a diagram showing a display example of a live view screen. [Figure 24] FIG. 13 is a diagram showing a display example of a photographing result. [Diagram 25] FIG. 13 is a diagram showing an example of a live view screen when a course for three people is selected. [Figure 26] FIG. 13 is a diagram showing an example of a display of analysis results when a course for three people is selected. [Figure 27] FIG. 13 is a diagram showing a display example of a selection result. [Figure 28] FIG. 13 is a diagram showing a display example of the live view screen when a course for four or more people is selected. [Figure 29] FIG. 13 is a diagram showing a display example of a selection screen for selecting a desired face when a course for four or more people is selected. [Diagram 30] FIG. 13 is a diagram showing a display example of an edit screen. [Diagram 31] FIG. 13 is a diagram showing another display example of the editing screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <<1. About the photo sticker creation game>> <Exterior of the photo sticker creation device> 1 and 2 are perspective views showing an example of the external configuration of a photo sticker creating device 1. FIG.

[0015] The photo sticker creation device 1 is a photo creation game machine that provides captured images and edited images. The photo sticker creation device 1 provides images to users by printing images on sticker paper and making the images viewable on the user's mobile terminal. The photo sticker creation device 1 is installed in amusement facilities such as game arcades and in stores.

[0016] Users of the photo sticker creating device 1 are mainly high school girls and young women. In the photo sticker creating device 1, multiple users, mainly two or three people per group, can enjoy the game. Of course, in the photo sticker creating device 1, a single user can also enjoy the game.

[0017] In the photo sticker creation device 1, the user takes a photo of himself / herself as the subject. Through editing, the user selects an image from the photographed images and combines it with a composite image such as handwritten characters or a stamp image. This edits the photographed image into a colorful image. The user receives a sticker paper on which the edited image, which is the edited image, is printed, and the series of games is completed.

[0018] The photo sticker creating device 1 can be said to be a photo creating game machine that provides a photo creating game.

[0019] 1, the photo sticker creating device 1 is basically configured by installing a photographing unit 11 and an editing unit 12 in contact with each other. The photographing unit 11 and the editing unit 12 are electrically connected to each other.

[0020] The photographing unit 11 has a box-shaped housing large enough for a person to enter, and is composed of a pre-selection section 20, a photographing section 21, and a background section 22. The pre-selection section 20 is installed on the side of the photographing section 21. The space in front of the pre-selection section 20 becomes a pre-selection space where pre-selection processing is performed. In addition, the photographing section 21 and the background section 22 are installed a predetermined distance apart. The space formed between the photographing section 21 and the background section 22 becomes a photographing space where photographing processing is performed.

[0021] As a pre-selection process, the pre-selection unit 20 performs guidance to introduce the game provided by the photo sticker creating device 1 and performs various settings for the photographing process performed in the photographing space. The pre-selection unit 20 is provided with a coin insertion / return slot where users insert coins, a touch panel monitor used for various operations, and the like. The pre-selection unit 20 appropriately guides users in the pre-selected space to the photographing space according to the availability of the photographing space.

[0022] The photographing unit 21 is a device for photographing a user as a subject. The photographing unit 21 is located in front of the user who has entered the photographing space. A camera, a touch panel monitor used for various operations, and the like are provided in front of the photographing unit 21 facing the photographing space.

[0023] If the left side as viewed from a user facing forward in the photographing space is the left side and the right side is the right side, then the left side of the photographing unit 21 is formed by side panel 41A, and the right side is formed by side panel 41B. Furthermore, the front of the photographing unit 21 is formed by front panel 42. The above-mentioned pre-selection unit 20 is installed on side panel 41A. Note that the pre-selection unit 20 may also be installed on side panel 41B.

[0024] Background section 22 is composed of rear panel 51, side panel 52A, and side panel 52B. Rear panel 51 is a plate-like member located behind a user facing forward. Side panel 52A is attached to the left end of rear panel 51 and is a plate-like member narrower in width than side panel 41A. Side panel 52B is attached to the right end of rear panel 51 and is a plate-like member narrower in width than side panel 41B.

[0025] The side panels 41A and 52A are provided on approximately the same plane. The upper parts of the side panels 41A and 52A are connected by a connecting portion 23A, which is a plate-shaped member. The lower parts of the side panels 41A and 52A are connected by a connecting portion 23A', which is, for example, a metal member provided on the floor surface. The side panels 41B and 52B are similarly provided on approximately the same plane. The upper parts of the side panels 41B and 52B are connected by a connecting portion 23B. The lower parts of the side panels 41B and 52B are connected by a connecting portion 23B'.

[0026] An opening formed by being surrounded by side panel 41A, connecting portion 23A, and side panel 52A serves as an entrance to the photography space. In addition, an opening formed by being surrounded by side panel 41B, connecting portion 23B, and side panel 52B also serves as an entrance to the photography space.

[0027] Above the shooting space, a ceiling is formed, surrounded by the front of shooting unit 21, connecting part 23A, and connecting part 23B. A ceiling strobe unit 24 is provided in a part of the ceiling. One end of ceiling strobe unit 24 is fixed to connecting part 23A, and the other end is fixed to connecting part 23B.

[0028] The ceiling strobe unit 24 has a built-in strobe that irradiates light into the shooting space in accordance with shooting. In addition to the strobe, a fluorescent lamp is provided inside the ceiling strobe unit 24. This allows the ceiling strobe unit 24 to function as an illumination for the shooting space.

[0029] The editing unit 12 is a device for editing the captured image. The editing unit 12 is connected to the photographing unit 11 with one side surface in contact with the front panel 42 of the photographing section 21.

[0030] 1 is the configuration on the front side of the editing unit 12, components used for editing work are provided on both the front side and the rear side of the editing unit 12. With this configuration, two groups of users can perform editing work simultaneously.

[0031] The front side of the editing unit 12 is composed of a surface 61 and an inclined surface 62 formed above the surface 61. The surface 61 is perpendicular to the floor surface and is substantially parallel to the side panel 41A of the imaging section 21. A tablet built-in monitor and a touch pen are provided on the inclined surface 62 as components used for editing work. A columnar support 63A that supports one end of the lighting device 64 is provided on the left side of the inclined surface 62. A columnar support 63B that supports the other end of the lighting device 64 is provided on the right side of the inclined surface 62. A support 65 that supports the curtain rail 26 is provided on the upper surface of the support 63A.

[0032] A curtain rail 26 is attached above the editing unit 12. The curtain rail 26 is configured by combining three rails 26A to 26C. The three rails 26A to 26C are combined so that they form a U-shape when viewed from above. One ends of rails 26A and 26B, which are provided in parallel, are fixed to connecting parts 23A and 23B, respectively, and the other ends of rails 26A and 26B are joined to both ends of rail 26C, respectively.

[0033] A curtain is attached to the curtain rail 26 so that the space in front of the front and the space in front of the rear of the editing unit 12 cannot be seen from the outside. The space in front of the front and the space behind the rear of the editing unit 12 surrounded by the curtain become the editing space where the user performs editing work.

[0034] An outlet through which the printed sticker paper is discharged is provided on the left side of the editing unit 12. The space in front of the left side of the editing unit 12 is a print waiting space where the user waits for the printed sticker paper to be discharged.

[0035] <About user movement> Here, the flow of the photo sticker creation game and the accompanying movement of the user will be described.

[0036] FIG. 3 is a plan view of the photo sticker creating device 1 seen from above.

[0037] The user inserts coins into a coin slot in the pre-selection space A0, which is the space in front of the pre-selection unit 20. Next, the user performs various settings according to the screen displayed on the touch panel monitor. For example, the user performs a pre-selection task such as selecting a course for photography to be performed in the photography space.

[0038] After completing the pre-selection task, the user enters the photographing space A1 formed between the photographing unit 21 and the background unit 22 from the doorway G1 between the side panel 41A and the side panel 52A, as indicated by the outlined arrow #1. The user performs photographing work using the camera and touch panel monitor provided in the photographing unit 21.

[0039] After completing the photo shoot, the user exits the photo shoot space A1 through the entrance G1 as shown by the white arrow #2 and moves to the editing space A2-1, or exits the photo shoot space A1 through the entrance G2 as shown by the white arrow #3 and moves to the editing space A2-2.

[0040] Editing space A2-1 is an editing space on the front side of the editing unit 12. On the other hand, editing space A2-2 is an editing space on the rear side of the editing unit 12. The user is guided as to which of editing space A2-1 and editing space A2-2 to move to by, for example, a screen display on the touch panel monitor of the imaging unit 21. For example, the user is guided to the one of the two editing spaces that is vacant as the destination. The user who moves to editing space A2-1 or editing space A2-2 starts editing work. The user of editing space A2-1 and the user of editing space A2-2 can perform editing work at the same time.

[0041] After the editing work is completed, printing of the edited image begins. When printing begins, a user who has finished editing work in the editing space A2-1 moves from the editing space A2-1 to the print waiting space A3 as shown by the outlined arrow #4. Also, a user who has finished editing work in the editing space A2-2 moves from the editing space A2-2 to the print waiting space A3 as shown by the outlined arrow #5.

[0042] The user moves to the print waiting area A3 and waits for the printing to finish. When the printing is finished, the user receives the sticker paper that was discharged from the discharge port provided on the right side of the editing unit 12, and the series of photo sticker creation games is completed.

[0043] Next, the configuration of each device will be described.

[0044] <Configuration of pre-selection part> FIG. 4 is a diagram showing an example of the configuration of the pre-selection unit 20. As shown in FIG.

[0045] A touch panel monitor 71 is provided above the pre-selection unit 20. The touch panel monitor 71 is composed of a monitor such as an LCD (Liquid Crystal Display) and a touch panel laminated thereon. The touch panel monitor 71 has a function of displaying various GUIs (Graphical User Interfaces) and accepting selection operations by the user. A screen used for selecting the number of users, etc. is displayed on the touch panel monitor 71.

[0046] A speaker 72 is provided below the touch panel monitor 71. The speaker 72 outputs voice guidance for the pre-selection process, background music, sound effects, etc. Adjacent to the speaker 72, a coin insertion / return slot 73 into which a user inserts coins is provided.

[0047] <Composition of the shooting team> 5 is a diagram showing an example of the configuration of the front side of the imaging unit 21. The imaging unit 21 is configured so as to be surrounded by a side panel 41A, a side panel 41B, and a front panel .

[0048] A camera unit 81 is provided slightly above the center of the front panel 42. In front of the camera unit 81, a camera 91 and a touch panel monitor 92 are provided.

[0049] The camera 91 is, for example, a single-lens reflex camera, and is attached inside the camera unit 81 so that the lens is exposed. The camera 91 has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and captures an image of a user in the imaging space A1. The camera 91 functions as an imaging unit that captures an image of the user as a subject.

[0050] A moving image (hereinafter also referred to as a live view image) captured by the camera 91 is displayed in real time on the touch panel monitor 92. A still image captured by the camera 91 at a predetermined timing, such as when an instruction to capture is given, is saved as a captured image.

[0051] The touch panel monitor 92 is provided below the camera 91. The touch panel monitor 92 is composed of a monitor such as an LCD and a touch panel laminated on it. The touch panel monitor 92 has a function as a live view monitor that displays moving images captured by the camera 91, and a function of displaying various GUIs and accepting selection operations by the user.

[0052] An upper strobe unit 82 with a curved light-emitting surface facing the user is installed above the camera unit 81. The upper strobe unit 82 irradiates light onto the face and upper body of the user from above and in front of the user.

[0053] Below the camera unit 81, a foot strobe unit 83 is provided which irradiates light onto the lower body and feet of the user.

[0054] Fluorescent lamps and light-emitting tubes capable of strobe emission are provided inside upper strobe unit 82 and foot strobe unit 83. Upper strobe unit 82 and foot strobe unit 83 constantly illuminate the shooting space with fluorescent lamps, and emit light from their light-emitting tubes at the timing of shooting to irradiate the user who is the subject with strobe light.

[0055] A box-shaped portion 84 is provided between the foot strobe unit 83 and the side panel 41A. Also, a box-shaped portion 85 is provided between the foot strobe unit 83 and the side panel 41B. The top surfaces of the box-shaped portions 84 and 85 are substantially horizontal with the installation surface of the photo sticker creating device 1, and are used as a luggage storage area for a user performing photography work to place baggage and the like.

[0056] Although not shown, a speaker is provided, for example, near the ceiling of the front panel 42. The speaker outputs voice guidance for the shooting process, background music, sound effects, and the like.

[0057] <Editing unit composition> FIG. 6 is a diagram showing an example of the configuration of the front side (editing space A2-1 side) of the editing unit 12. As shown in FIG.

[0058] A tablet built-in monitor 131 is provided approximately in the center of the inclined surface 62. A touch pen 132A is provided on the left side of the tablet built-in monitor 131. A touch pen 132B is provided on the right side of the tablet built-in monitor 131.

[0059] The tablet built-in monitor 131 is configured by stacking a tablet and a display. The tablet allows operation input using touch pens 132A and 132B. For example, an editing screen used for editing work is displayed on the tablet built-in monitor 131. When two users perform editing work at the same time, the touch pen 132A is used by the user on the left side as viewed from the tablet built-in monitor 131, and the touch pen 132B is used by the user on the right side as viewed from the tablet built-in monitor 131.

[0060] Note that operations using touch pen 132A and operations using touch pen 132B are distinguished. As appropriate, only operations using touch pen 132A are possible for the display on the left side of tablet built-in monitor 131, and only operations using touch pen 132B are possible for the display on the right side. Hereinafter, when there is no need to distinguish between touch pen 132A and touch pen 132B, they will be collectively referred to as touch pen 132.

[0061] A sticker paper outlet is provided on the left side of the editing unit 12. A printer is provided inside the editing unit 12. The printer prints an image of a user of the editing space A2-1 or an image of a user of the editing space A2-2 on sticker paper in a predetermined layout, and the sticker paper is discharged from the sticker paper outlet.

[0062] <Internal configuration of the photo sticker creation device> Fig. 7 is a block diagram showing an example of the configuration of the photo sticker creating device 1. In Fig. 7, the same components as those described above are given the same reference numerals, and duplicated explanations will be omitted as appropriate.

[0063] 7, the photo sticker creating device 1 is configured by connecting a pre-selection unit 302, a photographing unit 304, editing units 305A and 305B, and a printing unit 306 to a PC (Personal Computer) unit 301 via an external input / output interface 303. The PC unit 301, which is an information processing unit, is housed inside the housing of the editing unit 12, for example.

[0064] The PC section 301 includes a central processing unit (CPU) 311, a read only memory (ROM) 312, and a random access memory (RAM) 313, which are connected to each other via a bus 314.

[0065] The CPU 311 executes a predetermined program and controls the overall operation of the photo sticker creating apparatus 1. The ROM 312 stores the programs and data executed by the CPU 311. The RAM 312 temporarily stores the data and programs processed by the CPU 311.

[0066] An input / output interface 315 is further connected to the bus 314. A storage unit 316, a communication unit 317, and a drive 318 are connected to the input / output interface 315.

[0067] The storage unit 316 is a non-volatile storage medium such as a hard disk, a flash memory, etc. The storage unit 316 stores various setting information and the like supplied from the CPU 311. The information stored in the storage unit 316 is read out by the CPU 311 as appropriate.

[0068] The communication unit 317 is an interface for a network such as the Internet. The communication unit 317 communicates with an external device under the control of the CPU 311. The communication unit 317 transmits an image created by a user during a game to, for example, an image management server managed by the manufacturer of the photo sticker creating device 1. The image transmitted from the communication unit 317 is assigned a predetermined memory area and stored, and is displayed on a mobile terminal that has accessed the image management server or downloaded to the mobile terminal.

[0069] Removable media 319, such as an optical disk or semiconductor memory, is appropriately loaded into the drive 318. Programs and data read from the removable media 319 by the drive 318 are supplied to the CPU 311, and are stored in the storage unit 316 or installed.

[0070] An external input / output interface 303 is connected to the input / output interface 315. The PC unit 301 controls each unit via the external input / output interface 303.

[0071] The pre-selection unit 302 realizes a pre-selection process for users in the pre-selection space A0. The pre-selection unit 302 is composed of a touch panel monitor 71, a speaker 72, and a coin processing unit 321.

[0072] The touch panel monitor 71 displays various selection screens under the control of the PC unit 301, and accepts operations by the user on the selection screens. An input signal representing the content of the user's operation is supplied to the PC unit 301, and various settings are made.

[0073] The coin processing unit 321 detects the insertion of coins into the coin insertion / return slot 73. When the coin processing unit 321 detects that a predetermined amount of coins has been inserted, it outputs a start signal to the PC unit 301 to instruct the start of a game.

[0074] The image capturing unit 304 realizes image capturing processing for capturing an image of a user in the image capturing space A1. The image capturing unit 304 is composed of a lighting device 331, a camera 91, a touch panel monitor 92, and a speaker 332.

[0075] The lighting devices 331 are strobe units in the photographing space A1, and emit light in accordance with a lighting control signal supplied from the PC section 301.

[0076] The camera 91 captures images according to shutter control by the PC section 301, and outputs the captured images (image data) obtained by the capture to the PC section 301.

[0077] Editing unit 305A realizes editing processing for a user in editing space A2-1. Editing unit 305A is composed of tablet built-in monitor 131, touch pens 132A and 132B, and speaker 341. Editing unit 305B realizes editing processing for a user in editing space A2-2, and has the same configuration as editing unit 305A. In the following, when there is no need to distinguish between editing units 305A and 305B, they will simply be referred to as editing unit 305.

[0078] The tablet built-in monitor 131 displays an editing screen and accepts operations on the editing screen by the user under the control of the PC section 301. An input signal representing the content of the user's operation is supplied to the PC section 301, and the captured image to be edited is edited.

[0079] The printing section 306 is configured to include a printer 351. A sticker paper unit 352 is attached to the printer 351.

[0080] Printer 351 drives built-in head 361 and prints an image on rolled sticker paper 363 stored in sticker paper unit 352 based on print data supplied from PC section 301. Printer 351 also cuts sticker paper 363 with the image printed thereon at a predetermined cutting position with cutter 362 and discharges it to a sticker paper outlet. As the print medium, various media other than sticker paper that can be printed with photos can be used.

[0081] <Example of functional configuration of photo sticker creation device> Fig. 8 is a block diagram showing an example of a functional configuration of the photo sticker creating device 1. At least a part of the functional units shown in Fig. 8 is realized by the CPU 311 executing a predetermined program.

[0082] In the photo sticker creating device 1, a pre-selection processing unit 401, a photography processing unit 402, an edit processing unit 403, a print processing unit 404, and a game progress control unit 405 are realized.

[0083] The pre-selection processing unit 401 executes pre-selection processing by controlling each unit of the pre-selection unit 302. The pre-selection processing selects the number of users, etc. Information indicating the contents of the selection by the users is supplied to the shooting processing unit 402.

[0084] The shooting processing unit 402 performs shooting processing by controlling each unit of the shooting unit 304. The shooting processing unit 402 outputs data of shot images obtained by performing the shooting processing to the editing processing unit 403. Data of multiple shot images obtained by performing shooting multiple times is supplied from the shooting processing unit 402 to the editing processing unit 403.

[0085] The photographing processing unit 402 also analyzes the facial features of the user based on the images obtained by photographing. As will be described in detail later, in the photographing process, before photographing a plurality of photographed images, photographing is performed for analyzing the facial features. Information representing the facial features of the user analyzed by the photographing processing unit 402 is supplied to the editing processing unit 403 and the game progress control unit 405.

[0086] The edit processing unit 403 performs edit processing by controlling each unit of the edit unit 305. In the edit processing, the analysis result of the facial features of the user by the photograph processing unit 402 is used as appropriate. After editing the photographed image, the edit processing unit 403 generates print data by arranging the edited image in a predetermined layout, and outputs the print data to the print processing unit 404.

[0087] A print processing unit 404 outputs print data to the printer 351, and prints the edited image.

[0088] The game progress control unit 405 controls the progress of the game based on the analysis result of the facial features of the user by the shooting processing unit 402. For example, the game progress control unit 405 controls the progress of the game by changing the contents of the shooting process performed by the shooting processing unit 402 and the editing process performed by the editing processing unit 403 in accordance with the facial features of the user.

[0089] In this way, the photo sticker creating device 1 not only functions as an image processing device that processes photographed images, but also functions as a control device that controls the progress of the game based on the analysis results of the user's facial features.

[0090] <Detailed configuration of the imaging processing section> FIG. 9 is a block diagram showing an example of the configuration of the image capture processing unit 402.

[0091] The photographing processing unit 402 is made up of a photographing control unit 411 , a face analysis unit 412 , an image processing unit 413 , a display control unit 414 , and an image storage unit 415 .

[0092] The shooting control unit 411 captures moving images by controlling the camera 91, and outputs the images obtained by capturing images to the image processing unit 413. The moving images output from the shooting control unit 411 are used to generate live view images.

[0093] Furthermore, the shooting control unit 411 shoots still images in sync with the countdown, and outputs the still images obtained by shooting to the image processing unit 413. The still images output from the shooting control unit 411 are used to generate a shot image.

[0094] The photographing control unit 411 photographs a face analysis image, which is a still image used to analyze the facial features of the user, before photographing a photographed image, which is a still image to be edited. In this way, in the photo sticker creating device 1, after the user moves to the photographing space A1, photographing of a face analysis image is performed before photographing a normal photographed image to be used as an editing target.

[0095] As described later, image processing according to the selection of the user who confirmed the facial features analyzed using the face analysis image is performed on each captured image. The shooting control unit 411 outputs the face analysis image captured before shooting the normal captured image to the face analysis unit 412.

[0096] The face analysis unit 412 analyzes the facial features of the user based on the face analysis image supplied from the photography control unit 411. The analysis of the facial features is performed by detecting feature points and mapping the user's face onto a preset feature space. A face type is set for each area of ​​the feature space. By mapping the user's face onto the feature space, the user's face type is identified. The face analysis unit 412 outputs the analysis result to the display control unit 414, which displays it to the user.

[0097] The user can view the presented analysis results to see their facial features along with those of other face types.

[0098] Moreover, the user can view the analysis results and select a desired face type. The desired face here refers to a face that the user wishes to appear in the photographed image. The desired face is selected by selecting a face type set in each area of ​​the feature space. According to the desired face selected by the user, parameters used for image processing to be applied to the facial portion of the user appearing in the photographed image are determined in the image processing unit 413.

[0099] The user's operation of selecting a desired face is an operation of selecting parameters for image processing to be applied to the facial portion shown in the photographed image. Hereinafter, the parameters determined in response to the selection of a desired face will be referred to as "face parameters" as appropriate.

[0100] The face analysis unit 412 outputs the face analysis results and information on the desired face selected by the user to the image processing unit 413. The face analysis unit 412 also outputs the face analysis results to the editing processing unit 403 and the game progress control unit 405. An image for face analysis is also output to the editing processing unit 403.

[0101] The image processing unit 413 cuts out a rectangular area by trimming from each frame of the moving image supplied from the shooting control unit 411, and generates a moving image made up of the cut-out images. The moving image generated by the image processing unit 413 is used as a live view image to be displayed on a live view screen.

[0102] Furthermore, the image processing unit 413 cuts out a rectangular area by trimming from the still image supplied from the shooting control unit 411, thereby generating a shot image with a predetermined angle of view.

[0103] The image processing unit 413 determines face parameters according to the desired face selected by the user based on the information supplied from the face analysis unit 412. The image processing unit 413 uses the face parameters to perform image processing on the face parts appearing in each captured image, and adjusts the appearance of the face so as to approach the desired face selected by the user.

[0104] The image processing unit 413 outputs the photographed image generated by adjusting the appearance of the face to the image storage unit 415, and stores it therein. In this manner, in the photo sticker creation device 1, the photographed image is generated in which the appearance has been adjusted so as to approximate the desired face selected by the user, and is used for editing, etc.

[0105] The display control unit 414 controls the display of various screens on the touch panel monitor 92, such as a live view screen.

[0106] The image storage unit 415 stores the captured image generated by the image processing unit 413. When the capturing process is completed, the captured image stored in the image storage unit 415 is supplied to the edit processing unit 403.

[0107] <Detailed configuration of the editing processing section> FIG. 10 is a block diagram showing an example of the configuration of the edit processing unit 403 in FIG.

[0108] The editing processing unit 403 is composed of an editing control unit 431, a display control unit 432, an analysis result image generating unit 433, a print data generating unit 434, and a communication control unit 435. The captured image supplied from the shooting processing unit 402 is input to the editing control unit 431 and the display control unit 432. In addition, the facial feature analysis results and the face analysis image supplied from the shooting processing unit 402 are input to the display control unit 432 and the analysis result image generating unit 433.

[0109] The editing control unit 431 edits the captured image in response to a user's operation on the editing screen. The editing performed by the editing control unit 431 is image processing for compositing a compositing image, such as handwritten characters or a stamp image, with the captured image to be edited.

[0110] The above-mentioned image processing using face parameters is a process of applying image processing according to the selection made by the user to all captured images, whereas editing is a process of applying image processing according to the selection made by the user to one captured image that is currently being edited. Of the editing performed using various editing tools described below, image processing according to an operation using a makeup tool may be applied to all captured images. The user may be able to select by operating a selection button whether image processing according to an operation using a makeup tool is applied to all captured images or to one captured image that is currently being edited.

[0111] The editing control unit 431 outputs information indicating the editing result to the display control unit 432, and causes the information to be reflected in the display of the editing screen. When the editing is completed, the editing control unit 431 outputs an edited image, which is the photographed image obtained by performing editing, to the print data generation unit 434 and the communication control unit 435.

[0112] The display control unit 432 displays an operation screen used for operation by the user on the tablet built-in monitor 131, and controls the display of the operation screen in response to the user's operation. For example, the display control unit 432 displays an editing screen, which is a screen used for editing, on the tablet built-in monitor 131. The captured image output from the shooting processing unit 402, the edited image output from the editing control unit 431, etc. are used to display the editing screen.

[0113] As described below, the content of the editing screen changes according to the analysis result of the user's facial features. The editing screen provides synthesis images that can be used to edit the captured image. For example, the type of synthesis image that can be used to edit the captured image changes according to the analysis result of the facial features.

[0114] The analysis result image generating unit 433 generates an analysis result image based on the face analysis result and the face analysis image supplied from the photograph processing unit 402. The analysis result image is an image that represents the analysis result of the facial features. For example, the analysis result image generating unit 433 generates the analysis result image by combining characters expressing the face analysis result in words with an illustrated image of the face analysis image.

[0115] The analysis result image generating unit 433 generates an analysis result image for each user and outputs it to the communication control unit 435. That is, in this example, the analysis result image is not printed on the sticker paper 363, but is sent to the image management server and provided to the user via the image management server.

[0116] The print data generation unit 434 generates print data by arranging, in a predetermined layout, the edited image supplied from the edit control unit 431. The print data generation unit 434 outputs the print data to the print processing unit 404, which prints the print data on the sticker paper 363.

[0117] The communication control unit 435 controls the communication unit 317 to transmit to the image management server the edited image supplied from the editing control unit 431 and the analysis result image supplied from the analysis result image generation unit 433. The images transmitted by the communication control unit 435 are linked to the user's account and managed in the image management server.

[0118] <How the photo sticker creation device works> Here, the operation of the photo sticker creating device 1 that provides the photo sticker creating game will be described with reference to the flowchart of Fig. 11. The process of Fig. 11 is started, for example, when a predetermined amount of coins is inserted.

[0119] In step S1, the pre-selection processing unit 401 performs a pre-selection process by controlling the pre-selection unit 302. Through the pre-selection process, for example, a course is selected according to the number of users.

[0120] The photo sticker creating device 1 has three types of courses prepared: a two-player course, which is selected when two people play the game, a three-player course, which is selected when three people play the game, and a four or more players course, which is selected when four or more people play the game. The course selection according to the number of players is performed using a selection screen displayed on the touch panel monitor 71.

[0121] In step S2, the image capturing processing unit 402 controls the image capturing unit 304 to perform image capturing processing.

[0122] For example, after guiding the user to a proper standing position, the photography processing unit 402 causes the touch panel monitor 92 to display a live view screen based on the moving image captured by the camera 91. After displaying the live view screen, the photography processing unit 402 captures still images of the user in the photography space A1 as a subject. Details of the photography processing will be described later with reference to the flowchart in FIG.

[0123] In step S3, the editing processing unit 403 performs editing processing by controlling the editing unit 305 corresponding to either the editing space A2-1 or the editing space A2-2, whichever is the destination of the user who has completed the shooting processing.

[0124] For example, the display control unit 432 of the editing processing unit 403 displays an editing screen on the tablet built-in monitor 131, and allows the user to edit the captured image. On the editing screen, the captured image to be edited and buttons to be operated when selecting a compositing image to be composited with the captured image are displayed. The editing screen will be described in detail later.

[0125] The edit control unit 431 edits each captured image in response to an operation by a user to generate an edited image.

[0126] When editing of the captured image is completed, the print data generation unit 434 generates print data based on the edited image generated by the edit control unit 431. Furthermore, the communication control unit 435 links the edited image generated by the edit control unit 431 and the analysis result image generated by the analysis result image generation unit 433 with the user's account information and transmits them to the image management server.

[0127] In step S4, the print processing unit 404 performs print processing by controlling the printer 351. The print processing unit 404 outputs the print data generated by the edit processing unit 403 to the printer 351, and causes it to print on the sticker paper 363.

[0128] When printing is completed, in step S5, the printer 351 discharges the sticker paper 363 cut off by the cutter 362 to the sticker paper discharge port.

[0129] A photo creation game consisting of the above-mentioned series of processes is provided to each group of users of the photo sticker creating device 1. When a user of one group starts using the photo sticker creating device 1 followed by a user of another group, the photo sticker creating game of the user who started using the photo sticker creating device 1 first and the photo sticker creating game of the user who started using the device 1 later are played in parallel as appropriate. The photo sticker creating game is also referred to simply as a game as appropriate.

[0130] <Shooting process> Example of a course for two Next, the photographing process performed in step S2 in FIG. 11 will be described with reference to the flowchart in FIG.

[0131] In step S11, the display control unit 414 displays a live view screen for face analysis photography on the touch panel monitor 92 based on the moving image captured by the camera 91. As described above, before shooting a photographed image, face analysis photography, which is shooting an image for face analysis, is performed.

[0132] FIG. 13 is a diagram showing a display example of a live view screen for face analysis photography.

[0133] A live view image is displayed in a horizontally long rectangular display area 501 formed in the upper center of the live view screen. In the example of Fig. 13, a live view image showing two users, user H1 and user H2, is displayed.

[0134] The live view screen shown in Fig. 13 is a screen that is displayed when a two-player course is selected. Below, we will mainly explain the processing when two users, user H1 and user H2, are playing the game.

[0135] In the live view image, roughly square frames F1 and F2 are displayed side by side. While viewing such a live view image, users H1 and H2 adjust their standing positions so that their respective faces fit within frames F1 and F2.

[0136] Face recognition may be performed based on a live view image, and the frames F1 and F2 may be displayed so as to follow the recognized face.

[0137] In step S12, the photography control unit 411 performs face analysis photography, and outputs a still image obtained by the face analysis photography to the face analysis unit 412 as an image for face analysis.

[0138] In step S13, the face analysis unit 412 analyzes the facial features of each user based on the face analysis image captured by the photography control unit 411.

[0139] For example, the face analysis unit 412 generates face images of the users H1 and H2 by cropping out the areas within the frames F1 and F2 of the face analysis image captured by the capture control unit 411. The face analysis unit 412 detects facial feature points by analyzing the face images of the users. For example, feature points of specific parts of the face, such as the outline, eyes, nose, and mouth, are detected. The analysis for detecting facial feature points is performed on the face image that has been subjected to image processing.

[0140] In addition, the face analysis unit 412 quantifies the facial features of the user shown in the face image. For example, the shape of the facial outline and the distance between the eyes (left eye and right eye) are quantified. The face analysis unit 412 maps the user's face on a feature space based on the numerical values ​​representing the facial features. Information representing the analysis results by the face analysis unit 412 is supplied to the display control unit 414.

[0141] In step S14, the display control unit 414 causes the touch panel monitor 92 to display the analysis results of the user's facial features.

[0142] FIG. 14 is a diagram showing an example of a display screen for the analysis results.

[0143] An area 511L is formed in the left half of the analysis result display screen, and an area 511R is formed in the right half. Area 511L is an area for the analysis results of user H1 shown on the left side. Area 511R is an area for the analysis results of user H2 shown on the right side. Areas 511L and 511R have the same configuration. Duplicate explanations will be omitted as appropriate.

[0144] A facial image of user H1 is displayed in a facial image area 521L formed above the area 511L. As shown in Fig. 14, small dots representing the facial feature points of user H1 and straight lines connecting each dot are superimposed on the facial image so that the feature points can be visually recognized. The dots representing the feature points and the straight lines connecting each dot are displayed based on the detection result of the feature points by the face analysis unit 412.

[0145] In the example of Figure 14, dots representing the characteristic points on the facial contour and the characteristic points on the edges of the left and right eyes, and the straight lines connecting them are displayed. For example, each dot is displayed one by one in order, and an effect is performed to present the analysis results of the characteristics.

[0146] Below the face image area 521L, a face feature map 522L is displayed, which represents the mapping result of the face of the user H1 in the feature space.

[0147] FIG. 15 is an enlarged view of facial feature map 522L.

[0148] The facial feature map 522L is configured as a map of a two-dimensional space in which the vertical and horizontal axes intersect at the origin.

[0149] The vertical axis represents the shape of the facial contour. Faces with higher values ​​are expressed as "baby faces," and faces with lower values ​​are expressed as "adult faces."

[0150] The horizontal axis represents the distance between the eyes. A face with a value further to the right is expressed as having "close-set eyes," and a face with a value further to the left is expressed as having "wide-set eyes."

[0151] The position of user H1's face on the vertical axis is determined based on a numerical value representing the shape of the facial contour. For example, the numerical value representing the shape of the facial contour analyzed by face analysis unit 412 includes a value representing the length of the face and a value representing thinness. For example, the smaller the value representing the length of the face (shorter face) and the smaller the value representing thinness (rounder face), the closer the position on the vertical axis to a "baby face." Conversely, the larger the value representing the length of the face (longer face) and the larger the value representing thinness (thinner face), the closer the position on the vertical axis to a "adult face."

[0152] The position of the face of user H1 on the horizontal axis is determined based on the value representing the distance between the eyes. For example, the smaller the value representing the distance between the eyes analyzed by the face analysis unit 412, the closer the position on the horizontal axis to the "closer eyes". Conversely, the larger the value representing the distance between the eyes, the closer the position on the horizontal axis to the "wider eyes".

[0153] The heart-shaped image P represents the mapping result of the face of the user H1. In the example of Fig. 15, the face of the user H1 is mapped to a position closer to the "baby face" than the center and closer to the "wide-set eyes".

[0154] In this way, two types of features, the shape of the facial outline and the distance between the eyes, are analyzed, and the user's face is mapped onto the facial feature map 522L, which is a two-dimensional space based on the axes of the two types of features. Details of the mapping will be described later.

[0155] By looking at the facial feature map 522L, the user H1 can confirm what features his or her face has. On the facial image in the facial image area 521L in Fig. 14, characters expressing such facial features are also displayed together with information on the feature points.

[0156] Similarly, user H2 can check what features his or her face has from the display of face image area 521R and face feature map 522R in area 511R.

[0157] After the screen of Fig. 14 is displayed for a predetermined time, a facial feature map 522 as shown in Fig. 16 is enlarged and displayed on the touch panel monitor 92 in place of the screen of Fig. 14. The facial feature map 522 is also a map based on two types of features, the shape of the facial outline and the distance between the eyes. The facial feature map 522 simultaneously displays the analysis results of the facial features of user H1 and the facial feature analysis results of user H2.

[0158] The face image of user H1 is displayed at position p1, which is a position according to the analysis result of the facial features of user H1. Also, a speech bubble #1 indicates that user H1's face type is a "kitten face."

[0159] In this example, the user's face is classified into four types of face types represented by animal faces. The upper left area of ​​the face feature map 522, i.e., the area of ​​"baby face" and "wide-set eyes", is assigned the face type of "kitten face". Similarly, the upper right, lower right, and lower left areas are assigned the face types of "puppy face", "squirrel face", and "bunny face (rabbit face)", respectively.

[0160] On the other hand, the face image of user H2 is displayed at position p2, which is a position according to the analysis result of the facial features of user H2. Also, speech bubble #2 indicates that user H2's face type is a "bunny face."

[0161] From such a display of facial feature map 522, two users can enjoy comparing the analysis results of their respective facial features.

[0162] After the screen of Fig. 16 is displayed for a predetermined time, a guide image P11 is displayed around the face image of user H1 as shown in Fig. 17. As described above, after checking the analysis results of the facial features, the user selects the face they would like to have. The guide image P11 is an image that guides the user as to what range of faces they can select as the face they would like to have, based on the position on the facial feature map 522 according to the facial features of the user.

[0163] 17, an arrow A1 pointing to the upper right, an arrow A2 pointing to the upper left, an arrow A3 pointing to the lower left, and an arrow A4 pointing to the lower right are displayed with the face image of user H1 at the center. At the ends of the arrows A1 to A4, characters and illustrations representing "puppy face," "kitten face," "rabbit face," and "squirrel face," respectively, assigned to the respective regions of the facial feature map 522, are displayed.

[0164] For example, the display of the arrow A1 indicates that by selecting a "puppy face" as the desired face, the appearance of the user H1's face will be adjusted to approximate the "puppy face." Since the "puppy face" is characterized by a "baby face" and "close-set eyes," selecting the "puppy face" selects face parameters that shorten the length of the user H1's face by a predetermined amount, widen the face by a predetermined amount, and reduce the distance between the eyes by a predetermined amount.

[0165] The circle displayed as the guide image P11 represents the range of the amount by which the length of the user H1's face is shortened and the width of the face is widened, and the distance between the eyes is shortened. Even if the "puppy face" is selected as the desired face, the face parameters are selected to make the user H1's face closer to the "puppy face" within a certain range based on the facial features of the user H1, and are used in image processing, rather than making the user H1's face closer to the "puppy face" unnecessarily.

[0166] The certain range based on the facial features of the user H1 is preset to be an optimal range that allows the user H1 to approach each face type while making the most of the facial features of the user H1. The range that allows the user H1 to approach the desired face may be changed according to the facial features.

[0167] Similarly, the arrows A2 to A4 each indicate the direction of adjustment of the image quality.

[0168] 17 is displayed for user H1, and then guidance for user H2 is displayed. A guidance image P11 is displayed around the face image of user H2 displayed at position p2, thereby guiding the user H2 to a range that can be selected as a desired face based on a position according to the facial features of user H2.

[0169] In this way, the features of each face type are displayed together with the user's own facial features, so the user can easily imagine the content of image processing that will be performed when each face type is selected. The user can select the face type that suits their preferred features from among the face type that matches their own features and other face types as the face they want to have, while checking the objective features of each face type.

[0170] Returning to the explanation of FIG. 12, in step S15, display control section 414 displays a selection screen for a desired face.

[0171] In step S16, the image processing unit 413 receives the user's selection on the selection screen. The selection of the desired face is performed by each user.

[0172] FIG. 18 is a diagram showing an example of a display of a selection screen for selecting a desired face.

[0173] The left side of the selection screen displays a facial image of user H1, and the right side displays buttons 531-1 to 531-4 that can be pressed to select each of the faces: "Puppy face," "Kitten face," "Rabbit face," and "Squirrel face."

[0174] The selection screen shown in Fig. 18 is a selection screen for the first user, user H1. In this example, each user can select the face they want to have. Two users may simultaneously select the face they want to have using the same screen.

[0175] When the button 531-1 is pressed, the face image of the user H1 is switched to a face image resembling a "puppy face." Similarly, when the buttons 531-2 to 531-4 are pressed, the face image of the user H1 is switched to a face image resembling a "kitten face," a "rabbit face," and a "squirrel face," respectively.

[0176] That is, image processing for adjusting the appearance of the face image is performed by the image processing unit 413 using face parameters corresponding to each face type of "puppy face", "kitten face", "rabbit face" and "squirrel face". The display of the face image is switched using the image generated by the image processing unit 413.

[0177] When each user selects the face they want to have using such a selection screen display, the selection screen of FIG. 18 is switched to the screen shown in FIG.

[0178] The screen shown in Fig. 19 displays the results of two users' selections of the faces they would like to have. In the example of Fig. 19, user H1, whose face is classified as a "kitten face," has selected a "bunny face" ("fashionable"), and user H2, whose face is classified as a "bunny face," has selected a "puppy face" ("baby"), as shown. After these selection results are displayed, normal shooting begins.

[0179] Here, the face analysis by the face analysis unit 412 will be described.

[0180] As described above, the face analysis unit 412 analyzes two types of features, the shape of the facial outline and the distance between the eyes, and maps the face of each user to a position on the facial feature map according to the analysis results. The shape of the facial outline and the distance between the eyes are each evaluated on a five-point scale, for example, and the position of the user's face on the vertical axis and the position of the user's face on the horizontal axis are determined according to the evaluation value from 1 to 5.

[0181] FIG. 20 is a diagram showing an example of mapping according to eye distance.

[0182] As shown by the horizontal ovals, a value from 1 to 5 is assigned to each position on the horizontal axis. Smaller values ​​are assigned to positions closer to the "separate eye", and larger values ​​are assigned to positions closer to the "knot".

[0183] When the evaluation value of the distance between the eyes is 1, 2, 4, or 5, the position of the user's face on the horizontal axis will be a position corresponding to the evaluation value. For example, when the evaluation value of the distance between the eyes is 2, the position of the value 2 will be the position of the user's face on the horizontal axis, as shown in Figure 20.

[0184] When the evaluation value of the distance between the eyes is 3, the position of the user's face on the horizontal axis is not the center of the horizontal axis (the position of the value 3), but is a position shifted a predetermined amount to the left or right from the center. The position shifted to the left or right is selected randomly, for example.

[0185] By mapping the data at a position on the horizontal axis that is shifted from the center, it is possible to improve the visibility of the analysis results.

[0186] FIG. 21 is a diagram showing an example of mapping according to the shape of the face outline.

[0187] As shown by the vertical ovals, a value from 1 to 5 is assigned to each position on the vertical axis. Small values ​​are assigned to positions closer to a "baby face," and large values ​​are assigned to positions closer to an "adult face."

[0188] As described above, the evaluation value of the shape of the facial contour includes an evaluation value of thinness and an evaluation value of length. As shown in the lower part of FIG. 21, the shape of the facial contour is evaluated on two axes: "thin-round" according to the evaluation value of thinness, and "long-short" according to the evaluation value of length. Of the evaluation value of thinness and the evaluation value of length, for example, the former evaluation value of thinness is given priority. Each evaluation value also takes a value from 1 to 5.

[0189] If the evaluation value of thinness is 1, 2, 4, or 5, the evaluation value of thinness is used as the evaluation value of the shape of the facial outline, and the position of the user's face on the vertical axis (on the vertical axis of the facial feature map) will be a position according to the evaluation value. For example, if the evaluation value of thinness is 2, the position of the value 2 will be the position on the vertical axis of the user's face.

[0190] When the evaluation value of thinness is 3, the evaluation value of length is used as the evaluation value of the shape of the facial contour, and the position of the user's face on the vertical axis is determined according to the evaluation value of length.

[0191] If the evaluation value of the length is one of the values ​​1, 2, 4, or 5, the position of the user's face on the vertical axis will be a position corresponding to the evaluation value. For example, if the evaluation value of the length is 2, the position of the value 2 will be the position on the vertical axis of the user's face.

[0192] On the other hand, if the evaluation value of the length is 3, the position of the user's face on the vertical axis is shifted a predetermined amount to the upper or lower side from the center. Whether the position is shifted to the upper side or the lower side is selected randomly, for example.

[0193] By mapping the face at a position on the vertical axis that is shifted from the center, it is possible to improve the visibility of the analysis results. As a method for determining the position on the vertical axis based on the evaluation value of the face thinness and the evaluation value of the face length, any method other than the above may be adopted.

[0194] FIG. 22 is a diagram showing an example of how to calculate the evaluation value.

[0195] A face analysis model used to obtain an evaluation value of a face feature is prepared in the face analysis unit 412. The face analysis model is, for example, an inference device configured with a neural network or the like that receives a face image as input and outputs an evaluation value of the shape of the face outline and an evaluation value of the distance between the eyes.

[0196] For example, the face analysis model is generated by performing machine learning using various face images and evaluation values ​​of the shape of the face outline and the distance between the eyes that appear in each face image, and is set in advance in the face analysis unit 412. Supervised learning may be performed using each evaluation value as teacher data, or unsupervised learning may be performed.

[0197] When analyzing the facial features of a user, the face analysis unit 412 inputs the user's facial image into a face analysis model and obtains an evaluation value of the shape of the facial outline and an evaluation value of the distance between the eyes, which are the inference results. Based on the obtained evaluation values, the face analysis unit 412 performs analysis of the facial features of the user by mapping the user's face as described above.

[0198] Instead of using a face analysis model generated by machine learning, mapping may be performed based on facial features determined by analyzing a face image.

[0199] Returning to the explanation of FIG. 12, in step S17, the display control unit 414 displays a live view screen for normal shooting.

[0200] FIG. 23 is a diagram showing a display example of a live view screen for normal shooting.

[0201] A display area 551 for displaying a live view image is formed in the center of the live view screen. On the left side of the display area 551, face type icons 552 representing four types of face types are displayed.

[0202] Among the face type icons 552, the icons representing "kitten face" and "bunny face" are highlighted, thereby presenting each face type to the users H1 and H2. Only the icons representing "kitten face" and "bunny face" classified according to the facial features of the users H1 and H2 may be displayed, or the icons representing the face types selected by each user as the face they would like to have may be highlighted.

[0203] Display areas 553-1 to 553-5 are displayed side by side below display area 551. Display areas 553-1 to 553-5 are display areas in which images of the shooting results are displayed in shooting order. In the example of Fig. 23, since shooting has not been performed, shot images are not yet displayed in display areas 553-1 to 553-5. As shooting progresses, shot images are displayed as the shooting results.

[0204] After a predetermined time has elapsed since the display of such a live view screen started, in step S18, the shooting control unit 411 controls the camera 91 to perform normal shooting. A still image obtained by normal shooting is supplied to the image processing unit 413.

[0205] In step S19, image processing unit 413 adjusts the appearance of each user's face in the captured image using face parameters according to the desired face selected by each user.

[0206] 19, for example, if user H1 selects a "bunny face" as the face he or she wants to have, image processor 413 adjusts the shape of the facial outline and the distance between the eyes so that the face of user H1, which has been classified as a "kitten face," approaches a "bunny face." Specifically, image processor 413 adjusts the shape of the facial outline so that it is thinner and longer by a predetermined amount, and adjusts the distance between the eyes so that the eyes are wider apart by a predetermined amount.

[0207] Also, if user H2 selects a "puppy face" as the face he or she would like to have, image processor 413 adjusts the shape of the facial outline and the distance between the eyes so that the face of user H2, which has been classified as a "bunny face," approaches a "puppy face." Specifically, image processor 413 adjusts the shape of the facial outline so that it is rounder and shorter by a predetermined amount, and adjusts the distance between the eyes so that the eyes become wider by a predetermined amount.

[0208] The captured images in which image processing has been performed on the facial portions of the users H1 and H2 in this manner are supplied to the display control unit 414.

[0209] In step S20, the display control unit 414 displays the photographing results.

[0210] FIG. 24 is a diagram showing an example of a display screen for the photographing result.

[0211] The images displayed in the display area 551 in Fig. 24 are photographed images that have been subjected to image processing on the facial portion to make each user's face look like the face they want. The face of user H1 has been made to look like a "bunny face," and the face of user H2 has been made to look like a "puppy face."

[0212] Each user can check on the screen in FIG. 24 whether the face they see in the captured image matches the face they selected as the face they would like to have.

[0213] In step S21, the photographing control unit 411 determines whether or not photographing has ended. If it is determined in step S21 that photographing has not ended, the process returns to step S17, and normal photographing is repeated.

[0214] For example, if it is determined in step S21 that the shooting has ended because five images have been shot, the process returns to step S2 in FIG. 11, and the subsequent processes are carried out.

[0215] Example of a course for 3 people When the 3-person course is selected, as well as when the 2-person course is selected, face analysis photography is performed before normal photography, and the facial features of each user are analyzed. Descriptions that overlap with those described above will be omitted as appropriate.

[0216] FIG. 25 is a diagram showing an example of a live view screen for face analysis photography when the three-person course is selected.

[0217] A horizontally long rectangular display area 501 formed in the upper center of the live view screen displays a live view image showing three users, user H1, user H2, and user H3.

[0218] Frames F1, F2, and F3 are displayed overlapping on the live view image. Each user adjusts his / her standing position so that his / her face fits within frames F1 to F3.

[0219] FIG. 26 is a diagram showing an example of a display screen of the analysis results when a course for three people is selected.

[0220] On the analysis result display screen, areas 511L, 511C, and 511R are formed side by side. Area 511L is an area for the analysis results of user H1 shown on the left. Area 511C is an area for the analysis results of user H2 shown in the center. Area 511R is an area for the analysis results of user H3 shown on the right.

[0221] From such a display, each user can simultaneously check the analysis results of the facial features. After checking the analysis results of the facial features, the user selects the face they would like to have. The user selects the preferred face type to be the face they would like to have, respectively.

[0222] FIG. 27 is a diagram showing an example of a display screen of the selection results.

[0223] The screen shown in Fig. 27 displays the results of three users' selections of the face they would like to have. In the example of Fig. 27, user H1, whose face was classified as a "kitten face," has selected a "bunny face" ("fashionable"), user H2, whose face was classified as a "bunny face," has selected a "puppy face" ("baby"), and user H3, whose face was classified as a "kitten face," has selected a "squirrel face" ("adult"), and so on.

[0224] In this way, when the three-person course is selected, the facial features of each of the three users are analyzed and the analysis results are presented to the users. The three users can then view the facial feature analysis results, select the face they want to have, and take a photo with image processing applied according to the face they want to have.

[0225] · Course for 4 or more people FIG. 28 is a diagram showing an example of a live view screen for face analysis photography when the four or more people course is selected.

[0226] When the four or more people course is selected, the live view screen does not display a frame that serves as a reference for determining the position of the face. Analysis of individual facial features, which is performed when the two or three people course is selected, is not performed when the four or more people course is selected. Although face analysis shooting is performed, the features of each face are not analyzed and the analysis results are not presented. It is also possible to analyze the features of each face and display, for example, the average of the features as the analysis result.

[0227] FIG. 29 is a diagram showing an example of a desired face selection screen when the four or more people course is selected.

[0228] When the four or more people course is selected, the facial features of each person are not analyzed, but one desired face common to all four people can be selected. Image processing is applied to the facial parts of each of the four users that appear in the captured image so as to make them closer to the face selected as the desired face. For example, if a "puppy face" is selected by pressing button 531-1, the same image processing is applied to the facial parts of each of the four people that appear in the captured image so as to make them closer to the "puppy face."

[0229] If the facial features of each of the four users were analyzed and each user was asked to select the face they would like to have, this would take time and would reduce the time available for other processing. By omitting the selection of the face that the user would like to have when selecting a course for four or more people, it is possible to secure more time for other processing.

[0230] Of course, it is possible to analyze the faces of each of the four users and have each user select the face they want to have. Also, when selecting a course for three people, it is possible not to analyze each user's face.

[0231] In addition, if the number of people (course) selected in the pre-selection space A0 does not match the number of faces recognized based on the images obtained by shooting for analysis, the number of people selected in the former selection space A0 may be given priority, or the number of faces recognized based on the latter images obtained by shooting for analysis may be given priority.

[0232] By giving priority to the number of faces recognized based on the image obtained by photographing for analysis, it is possible to provide a game appropriate to the number of users even if a wrong course is selected when selecting a course in the pre-selection space A0 or a timeout occurs.

[0233] Course selection may be omitted in the pre-selection space A0. This makes it possible to shorten the time required for pre-selection. Course selection may be omitted if there is no other user who has started the game earlier in the shooting space A1.

[0234] <Editing screen example> Here, an editing screen used for editing a captured image will be described. During the editing process performed in step S3 in Fig. 11, an editing screen is displayed on the tablet built-in monitor 131 by the display control unit 432 (Fig. 10) of the editing processing unit 403.

[0235] FIG. 30 is a diagram showing an example of the editing screen.

[0236] On the editing screen, editing areas 601L and 601R are provided side by side on the left and right. Between the editing areas 601L and 601R, a thumbnail display area 602 is provided in which thumbnails of captured images are displayed.

[0237] In addition, below the editing areas 601L and 601R, editing palettes 603L and 603R are provided, respectively.

[0238] By providing screens with the same configuration on the left and right, editing of captured images can be accepted individually by two users on the editing screen. The left screen including editing area 601L and editing palette 603L is an area for a user standing on the left side facing tablet built-in monitor 131, and the right screen including editing area 601R and editing palette 603R is an area for a user standing on the right side facing tablet built-in monitor 131.

[0239] The following mainly describes the configuration of the left screen including the editing area 601L. The right screen has a similar configuration to the left screen.

[0240] The editing area 601L is a display area for a captured image selected as an editing target. The user selects an editing tool using the touch pen 132A and edits the captured image displayed in the editing area 601L. A tool area 611L in which buttons used to select various tools are arranged is provided along the lower edge of the editing area 601L.

[0241] As shown in Fig. 30, a frame is displayed superimposed on the face of each user appearing in the photographed image to be edited. For example, a display switching button 612L is displayed in the upper right of the frame surrounding the face of user H1 appearing on the left side. The display switching button 612L is a button that is pressed when enlarging and displaying the face surrounded by the frame. Note that in the example of Fig. 30, the display switching button 612L is displayed in the upper right of the frame surrounding the face, but if the frame of each user's face does not fit in the editing area 601L, the display switching button 612L is displayed in another position, such as the upper left, lower right, or lower left of the editing area 601L.

[0242] For example, when the display switching button 612L in Fig. 30 is pressed, the face of user H1 shown on the left side is enlarged and displayed in the entire editing area 601L, as shown in Fig. 31. By using the function of the display switching button 612L, the user can perform editing using a makeup tool or the like while keeping the face portion enlarged, and check the results in detail.

[0243] When the display switching button 612L is pressed in the state of FIG. 31 in which the face is displayed in an enlarged manner, the display in the editing area 601L returns to the display of FIG. 30 in which the two users are shown.

[0244] When a face is enlarged, a face switching button 613L is displayed in the tool area 611L. The face switching button 613L is a button that is pressed when switching the face to be enlarged to the face of a different user. When the face switching button 613L is pressed in the state of FIG. 31 in which the face of user H1 shown on the left side is enlarged, the face of user H2 shown on the right side is enlarged and displayed in place of the face of user H1 in the editing area 601L.

[0245] Returning to the explanation of Fig. 30, the thumbnail display area 602 is a display area for thumbnails representing captured images. In the example of Fig. 30, thumbnails representing five captured images obtained by normal shooting are displayed. The user selects a thumbnail displayed in the thumbnail display area 602 to select the captured image to be edited.

[0246] Editing palette 603L is an area that displays icons (buttons) used to select various editing tools such as a makeup tool, a doodle tool, a pen tool, etc. Tabs used to select the makeup tool, the doodle tool, and the pen tool are displayed above editing palette 603L. The category of editing tools displayed in editing palette 603L is switched in response to switching of the tabs.

[0247] The makeup tool is an editing tool used when applying image processing imitating makeup to the eyes, lips, cheeks, etc. The doodle tool is an editing tool used when compositing an image for synthesis such as a stamp image with a photographed image. The pen tool is an editing tool used when compositing a pen image input according to the trajectory of the touch pen 132A with a photographed image as an image for synthesis.

[0248] For example, when a predetermined icon displayed on the editing palette 603L is pressed while the makeup tool category is selected, image processing of makeup corresponding to the pressed icon is applied to the photographed image to be edited.

[0249] In this way, various editing tools are prepared as editing content on the editing screen, such as editing tools used when applying image processing for makeup, editing tools used when compositing images for compositing, etc. The content includes at least one of editing tools used when applying image processing for makeup and editing tools used when compositing images for compositing, such as stamp images and pen images.

[0250] For example, the editing content prepared on the editing screen is changed according to the face type classified according to the facial features of the user. For each face type, editing content appropriate to the features of the face type is linked. By changing the available editing content according to the facial features of the user, the user can use editing content appropriate to his or her facial features.

[0251] For example, if user H1, who appears on the left side of each captured image, is standing on the left side, editing content according to the face type of user H1 is displayed in the editing palette 603L on the left screen. Also, if user H2, who appears on the right side of each captured image, is standing on the right side, editing content according to the face type of user H2 is displayed in the editing palette 603R on the right screen.

[0252] Each user may select whether to use the left screen or the right screen. Also, available editing content may be changed according to the combination of the face type of user H1 and the face type of user H2.

[0253] The following specifications can be adopted for displaying the edit screen.

[0254] (1) Enlarged face display When the user's face is displayed enlarged in the editing area 601L (FIG. 31), a display switching button 612L is displayed outside the editing area 601L.

[0255] The display switching button 612L may be displayed so as to fit within the editing area 601L.

[0256] When the display switching button 612L is pressed while the user's face is being enlarged and displayed in the editing area 601L, the original image before enlargement is displayed in the editing area 601L.

[0257] When the display switching button 612L is pressed while the user's face is being enlarged and displayed in the editing area 601L, a further enlarged version of the face is displayed in the editing area 601L.

[0258] In response to the selection of the "Makeup Tool" tab, the user's face is displayed in an enlarged scale in the editing area 601L.

[0259] When the "Makeup tool" tab is selected and a desired makeup tool is selected from those prepared in editing palette 603L, the user's face is enlarged and displayed in editing area 601L.

[0260] When the "doodle tool" or "pen tool" tab is selected while the user's face is being enlarged and displayed in editing area 601L, the original image before enlargement is displayed in editing area 601L.

[0261] In response to pressing a frame (a frame surrounding a face) displayed in editing area 601L, the user's face is enlarged and displayed in editing area 601L.

[0262] When the face switching button 613L is pressed while the user's face is enlarged and displayed in the editing area 601L, if the face switching button 613L is pressed and then the image is returned to the original image before enlargement, when the display switching button 612L is pressed again, the face of the user selected by the face switching button 613L is enlarged and displayed in the editing area 601L.

[0263] When the face switching button 613L is pressed while the user's face is being enlarged and displayed in the editing area 601L, if the face switching button 613L is pressed and then the image is returned to the original image before enlargement, when the display switching button 612L is pressed again, the same face that was enlarged and displayed before the face switching button 613L was pressed is enlarged and displayed in the editing area 601L.

[0264] (2) Adjusting the shade of makeup When the tab for adjusting the shading of makeup is pressed, a button for adjusting the shading is displayed in the editing palette 603L.

[0265] When content that allows the adjustment of the shading of makeup is selected, a button for adjusting the shading is displayed in the editing palette 603L.

[0266] <<2. Details of changes to game content>> The game content, including the change of the editing tool, is changed by the game progress control unit 405 (FIG. 8). The details of the change of the game content will be described below.

[0267] 1. Images to be analyzed The following images can be used for feature analysis:

[0268] (1) Images obtained by face recognition photography (face analysis photography) (2) Image obtained in the first normal shot (3) Images where face recognition was successful (4) Images in which the user's face is facing forward (5) Images that do not show hands near the user's face (6) Close-up image (7) Whole-body images (8) Images selected by the user (9) Images that meet any of the above conditions (3) to (5) and also the above condition (6) or (7). (10) In the case where multiple images satisfy the above conditions (3) to (5) and the above conditions (6) or (7), the image selected by the user from among the candidates. (11) In the case where multiple images satisfy both the above conditions (3) to (5) and (6) or (7), the image that satisfies the most items in (3) to (5) above will be selected. (12) Different images for each user (13) Not limited to still images, but also moving images (14) Multiple Images

[0269] Using the images (4) and (5) above makes it easier to detect facial feature points.

[0270] An example of the above (9) image is a close-up image in which face recognition has been successful. Note that a close-up image is an image that shows the subject's face and the chest area in a large size. A full-body image is an image that shows the subject's entire body.

[0271] When using the image (12) above, for example, if user A's face is facing forward and user B's face is facing sideways when the first image is captured, and user A's face is facing sideways and user B's face is facing forward when the second image is captured, the image obtained from the first image capture is used to analyze user A's facial features, and the image obtained from the second image capture is used to analyze user B's facial features.

[0272] When using the above image (14), for example, facial feature analysis is performed using multiple images with the same angle of view. Features of the same part are detected and their average is calculated.

[0273] 2. Number of users to analyze characteristics The number of users whose features are to be analyzed can be as follows:

[0274] (1) When analyzing individual features ·1 person ·2 people ·3 people ·4 or more people

[0275] (2) When analyzing characteristics by group (multiple people) ·2 people ·3 people 1 person and 2 people All three together ·4 or more people 1 person and 3 people Two and two All four of us

[0276] (3) It is possible to not analyze the features when there are a certain number of people or more. In the above example, when there are four or more people, the analysis of the facial features of each user is not performed.

[0277] 3. Areas to analyze features Although the areas targeted for feature analysis are assumed to be the facial contour and eyes, it is also possible to perform feature analysis on the following areas:

[0278] (1) Specific parts of the face Contour ·amount ·eyebrow ·eye Inner corner of the eye · Corners of the eyes ·Iris ·nose Nose bridge ·Nose ·mouth ·ear Hairstyle

[0279] (2) A specific part of the body ·neck ·arm Upper arms Forearm (below the elbow or the part of the arm just past the elbow) Back of the hand ·palm ·finger ·abdomen ·leg ·feet

[0280] (3) Combination of the parts listed in (1) above As described above, the contour and eyes can be the target of feature analysis. Instead of two parts, feature analysis may be performed on three or more parts.

[0281] (4) Combination of the parts listed in (2) above For example, the arms and legs may be the target of feature analysis. Instead of two parts, three or more parts may be targeted for feature analysis.

[0282] (5) Combination of parts (1) and (2) above For example, the contour and the neck can be the target of feature analysis. Instead of two regions, three or more regions may be targeted for feature analysis.

[0283] (6) Location and distance of parts -Distance between eyebrows Distance between eyebrows and eyes ·Distance between eyes (eye spacing) Cross-eyed Wide-set eyes Distance between the inner corners of the eyes Distance between the corners of the eyes - Philtrum (nose to mouth) distance

[0284] The distances of the above-mentioned parts are analyzed as facial features. Regardless of the above-mentioned distances, the distances of two or more other parts may be analyzed.

[0285] (7) Shape of the part Contour shape round face Long face egg-shaped inverted triangle Diamond-shaped (narrow width from temples to forehead, etc.) Base Type square Forehead shape Eyebrow shape Eye shape Round eyes Cat Eyes Droopy eyes

[0286] The shapes of each of these parts are analyzed as facial features.

[0287] (8) Size and length of the part ·eye ·nose ·neck

[0288] It is possible for the size and length of the parts to be analysed as features rather than the shapes of the parts or the distances between the parts.

[0289] 4. Where the analysis results will be presented (1) Pre-selected space A0 The analysis results may be presented in the pre-selection space A0, for example, by using the touch panel monitor 71. In this case, a camera for face analysis photography is installed at a predetermined position in the pre-selection space A0. An image taken by a user using a mobile terminal such as a smartphone to take a selfie may be captured and used to analyze the features.

[0290] (2) Shooting space A1 As described above, it is possible to present the analysis result in the shooting space A1. In this case, the analysis result is presented at the following timing. Timing after facial analysis shooting Timing between normal shots Timing after all shooting is finished

[0291] (3) Editing space A2 It is possible to have the analysis results presented in the editing space A2. In this case, the analysis results are presented at the following times: -Timing before graffiti (before the editing screen is displayed) Timing while scribbling (while the editing screen is displayed) - Print waiting time

[0292] 5. Presentation of analysis results The results of the facial feature analysis are presented as explained with reference to Figure 14. The following presentations can be used:

[0293] (1) Displaying feature points one by one (2) For faces where feature points have been detected, a display indicating that feature points have been detected (3) When features of a face are detected, the color of the frame surrounding the face is changed. (4) When feature points of a face are detected, the frame surrounding the face is removed. (5) When features of a face have been detected, a frame appears around the face. (6) When features of a face have been detected, a frame surrounding the face will blink.

[0294] The effect of (1) above makes it possible to make the processing appear to be in real time. The effect of (2) above is to display the word "COMPLETE" or the like.

[0295] 6. Subjects to change according to characteristics As described above, the content of the game is changed according to the face type classified by analyzing the user's facial features. The following content can be changed as the content of the game.

[0296] (1) Changes to screen display -Change the color of the Live View screen Change the display of face type icon 552 displayed on the live view screen (number, color, shape, etc.) -Change the display of the editing screen (color, shape, timer display, etc.) -Changes to the display of the screen for creating bonus images to send to mobile phones (color, shape, timer display, etc.) - Change the display of the email address input screen (color, shape, timer display, etc.) - Change the display of the print waiting screen (color, shape, timer display, etc.)

[0297] The bonus image for mobile transmission is an image provided to the user's mobile terminal via an image management server, and is not printed on the sticker paper 363. The bonus image for mobile transmission is created based on the captured image, for example, by using a creation screen that is displayed when a specific operation is performed on the editing screen.

[0298] Furthermore, the email address is input by the user who has finished editing using the input screen displayed on the tablet built-in monitor 131. The email address input using the input screen is added to the edited image or the like sent to the image management server as information used to identify the user.

[0299] (2) Changes to the Makeup tool The following makeup tools are provided according to the user's face type. When the user selects the makeup tool tab, the user can use the makeup tools provided in the editing palette to apply makeup suitable for their face type.

[0300] Eyebrows (change the shape of your eyebrows) Eyebrow Mascara (get the eyebrow mask and change the color) Eyeshadow Eyeliner Double Synthesizing double eyelids according to eye shape Color contact lenses -Fake catchlight False eyelashes ·Tear bags Lower eyelashes Cheek Highlights Forehead highlight Nose highlight Cheek highlight Chin highlight ·shading Blending shading according to contours Cheek ·lip Skin color (whitening, darkening, toning up, blue-based, yellow-based) · Skin type (matte, glossy)

[0301] Regarding shading, for example, if the face has a long face shape, an image with shading on the forehead and chin is synthesized, and regarding blush, for example, if the face has a round face shape, an image with blush diagonally applied from the center of the cheek to the temple is synthesized.

[0302] (3) Changes to other editing tools Depending on the user's face type, the editing tool is changed as follows: When the user selects the scribble tool or pen tool tab, the user can use the editing tools provided in the editing palette to perform processing suited to his or her face type.

[0303] - Change the type and color of the pen image - Change the type and color of the stamp image -Change background image type and color - Change the type and color of the veil image -Change the type and color of the aura image -Change filter type and color - Change the type and color of the sticky stamp image

[0304] The sticky stamp image is a stamp image that recognizes a specific part of the user's face in the captured image and synthesizes it into that specific part. For example, if the user's face type is a "puppy face," a sticky stamp image that synthesizes images of a dog's ears and nose is prepared.

[0305] (4) Changes regarding printing The following printing-related content will change depending on the user's face type.

[0306] Number of divisions (layout) ·design

[0307] If the user's face type is a "bunny face," a layout with a rabbit pattern design is prepared and printed together with the edited image on sticker paper 363. In this way, the design of the content to be printed together with the edited image on sticker paper 363, which is the print medium, is changed according to the user's face type.

[0308] (5) Changes in time The following game-related times will change depending on the user's face type.

[0309] -Display time of sample poses when taking photos Shooting time Shooting time per shot Total shooting time Preview (shooting result) display time Edit image selection time - Image quality adjustment selection time Time for retouching selection Editing time Time required to create bonus images for mobile phone transmission Time to enter email address Print waiting time

[0310] (6) Changes in quantity The following game-related numbers will change depending on the user's face type.

[0311] Number of shots Number of retakes you can take Number of images that can be edited Number of bonus images to be sent to mobile phones Number of bonus images to send to mobile phone Number of sticker sheets printed Number of bonus stickers

[0312] In order to obtain images created in the game via the image management server, the user must register as a member. Membership attributes include, for example, free membership, paid membership, and plus membership. The change in the number of bonus images to be acquired for sending to mobile phones means that the number of bonus images to be acquired for sending to mobile phones changes depending on the combination of face type and the user's membership attributes.

[0313] (7) Changes regarding transformation When image processing for deforming a specific part is applied to a captured image, the content of the following processing is changed according to the face type of the user. In the following, deformation for a close-up image and deformation for a whole-body image are explained, but the deformation for a close-up image may be used for a whole-body image, and the deformation for a whole-body image may be used for a close-up image.

[0314] (i) Transformation for close-up images Contour processing Transform into an egg shape Transform into a round face Transform into an oblong face Transform into an inverted triangle Deformed into a diamond shape (narrow width from temples to forehead) Transform into base shape Transform into a rectangle ·Face slimming treatment - Forehead enlargement, deformation - Change eye size -Eye shape modification Transform into a round eye Transform into cat eyes Transformation into droopy eyes -Nose bridge stretching - Nostril enlargement / reduction · Lip corner lifting treatment Mouth size processing -Stretching of the philtrum (distance between nose and mouth) - Part movement (moving eyes up, down, left, right, etc., moving the mouth, moving the nose, moving eyebrows, etc.)

[0315] (ii) Transformation for whole body images ·Leg length treatment - Slimming legs Height processing ·Upper arm treatment - Waist treatment ·Neck length processing Waist raising treatment Finger slimming treatment - Nail shape modification - Nail length changing process

[0316] (8)Other The following will change depending on the user's face type. -Change game sequence - Change background music Change the narration -Changes to sample poses Mini-game changes Optical changes -Facial deformation change amount

[0317] Mini-games include special photography, creating 3D models, and creating illustrations based on characteristics.

[0318] In the above (1) to (8), the contents that can be changed for each user may be changed for each user.

[0319] In the above (1) to (8), the change may be made on a user group basis.

[0320] <<3.Other>> The face type selected as the desired face during the photographing process may be changed during the editing process. In this case, when the face type is changed, image processing according to the changed face type is applied to the face part of the user appearing in the photographed image.

[0321] Although the feature analysis is performed for specific parts of the face or body, the feature analysis may be performed for hairstyle or clothing.Face orientation, face size, distance between two faces, etc. may also be analyzed.

[0322] If the face appearing in the image for face analysis cannot be recognized by the face analysis unit 412, the image for face analysis may be retaken. If the user is not facing the camera 91, the face may not be recognized from the image for face analysis.

[0323] For example, when a course for two people is selected, if three faces appear in the face analysis image, the two faces closest to the two frames (frame F1, frame F2) may be selected as the analysis targets.

[0324] The above processing may be performed on a mobile terminal such as a smartphone, instead of being performed on the photo sticker creating device 1. In this case, an application (program) that realizes the above processing including the analysis of facial features is installed on the mobile terminal.

[0325] The program to be installed is provided to the mobile terminal from a predetermined server via the Internet, and may be pre-installed in the memory of the mobile terminal.

[0326] The program executed by the CPU of the mobile terminal may be a program that is processed chronologically in the order described in this specification, or it may be a program that is processed in parallel or at the required timing, such as when called.

[0327] The effects described herein are merely examples and are not limiting, and other effects may also be present.

[0328] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology. [Explanation of symbols]

[0329] 1 Photo sticker creation device, 91 camera, 92 touch panel monitor, 131 tablet built-in monitor, 301 PC section, 302 pre-selection section, 304 photography section, 305 editing section, 306 printing section, 401 pre-selection processing section, 402 photography processing section, 403 editing processing section, 404 printing processing section, 405 game progress control section, 411 photography control section, 412 face analysis section, 413 image processing section, 414 display control section, 415 image storage section, 431 editing control section, 432 display control section, 433 analysis result image generation section, 434 print data generation section, 435 communication control section

Claims

1. an analysis unit that acquires a first captured image in which a plurality of persons are photographed as subjects and analyzes facial features of each of the persons photographed in the first captured image; a display control unit that displays the first captured image and an analysis result of the facial features of each of the persons; an image processing unit that performs image processing on the first captured image, including deformation of a specific part of the face of each of the persons captured in the first captured image, based on an analysis result of the facial features of each of the persons and a selection operation by a user regarding the appearance of the face of each of the persons, and performs image processing with the same content as the image processing on the first captured image on each of a plurality of second captured images; Equipped with The display control unit displays the photographed image after image processing each time image processing including deformation of a specific part of the face of the person is performed on each of the photographed images including the first photographed image and the plurality of second photographed images. Image processing device.

2. The display control unit simultaneously displays the analysis result of the facial features of each of the persons together with the first photographed image. The image processing device according to claim 1 .

3. The image capturing apparatus further includes an image capturing unit configured to capture a plurality of the first captured images.

3. The image processing device according to claim 1 or 2.

4. The analysis unit analyzes two types of facial features of the person, The display control unit displays, as a result of the analysis, an image of the face of the person at a position according to the facial features of the person in a space having axes of the two types of features. The image processing device according to claim 2 .

5. The analysis unit analyzes facial features of the person using an inference unit generated by machine learning.

5. The image processing device according to claim 1.

6. The image processing device A first captured image is obtained in which a plurality of people are photographed as subjects; analyzing facial features of each of the persons depicted in the first captured image; displaying the first captured image and an analysis of the facial features of each of the persons; performing image processing on the first photographed image, including deformation of specific parts of the face of each of the persons appearing in the first photographed image, based on the analysis result of the facial features of each of the persons and a selection operation by a user regarding the appearance of the face of each of the persons; performing image processing on each of the plurality of second captured images that is the same as the image processing on the first captured images; Each time image processing including deformation of a specific part of the face of the person is performed on each of the photographed images including the first photographed image and the plurality of second photographed images, the photographed image after image processing is displayed. Image processing methods.

7. On the computer, Obtaining a first captured image in which a plurality of people are photographed as subjects; analyzing facial features of each of the persons depicted in the first captured image; displaying the first captured image and an analysis of the facial features of each of the persons; performing image processing on the first captured image, including deformation of specific portions of the faces of the respective persons appearing in the first captured image, based on the analysis results of the facial features of the respective persons and a selection operation by a user regarding the appearance of the faces of the respective persons; performing image processing on each of the plurality of second captured images that is the same as the image processing on the first captured image; displaying the photographed image after image processing each time image processing including deformation of a specific part of the face of the person is performed on each of the photographed images including the first photographed image and the plurality of second photographed images; A program for executing the above.

8. The display of the first photographed image is performed by simultaneously displaying the analysis results of the facial features of each of the persons together with the first photographed image. The program according to claim 7.

9. The analysis of the person's face is performed by analyzing two types of features, The analysis result is displayed by displaying an image of the face of the person at a position according to the facial features of the person on a space having axes of the two types of features. The program according to claim 8.

10. The analysis of the person's facial features is performed using a reasoner generated by machine learning.

10. The program according to claim 7.

Images