Image processing system and image processing method

Abstract

To efficiently adjust the appearance of a subject in a photographed image.SOLUTION: In an image processing system according to an embodiment of the present technology, a terminal takes a photograph of a user as a subject, and after the photographed image is uploaded to a server, the terminal communicates with the server in response to a user's operation, displays an adjustment screen used to adjust the appearance of some of the photographed images, and displays the photographed images after the appearance adjustment in response to the user's operation on the adjustment screen. After the adjustment of the appearance of the uploaded photographed images using the adjustment screen is completed, the server reflects the adjustment of the appearance of the some of the photographed images to each of the other photographed images that have not been adjusted. The present technology can be applied to a computer.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present technology relates to an image processing system and an image processing method, and more particularly to an image processing system and an image processing method that enable efficient adjustment of the appearance of a subject appearing in a captured image. [Background technology]

[0002] In photo taking devices such as ID photo machines and print sticker machines installed in game arcades, the user can adjust the appearance of the captured image by himself / herself. The user can adjust the appearance of the image that is printed on sticker paper or the like and receive it according to his / her preference for each piece. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-118329 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is tedious to adjust the quality of a large number of images, such as 50 or 100, one by one.

[0005] The present technology has been made in consideration of such circumstances, and makes it possible to efficiently adjust the appearance of a subject in a captured image. [Means for solving the problem]

[0006] An image processing system according to one aspect of the present technology includes a terminal used by a user who is a subject of a captured image, and a server to which the captured image is uploaded. The terminal includes a communication unit that communicates with the server in response to an operation by the user after a photograph is taken of the user as a subject and the captured image is uploaded to the server, and a display unit that displays an adjustment screen used to adjust the appearance of some of the captured images among the captured images and displays the some of the captured images after the appearance adjustment in response to an operation by the user on the adjustment screen. The server includes an image processing unit that reflects the content of the adjustment of the appearance of the some of the captured images that are similar in at least one of the size, position, orientation, brightness, and number of faces captured in the captured images to each of the other captured images that have not been adjusted after the adjustment of the appearance using the adjustment screen for the uploaded captured image is completed.

[0007] In one aspect of the present technology, the terminal takes a photograph of the user as a subject, and after the photographed image is uploaded to the server, communication with the server is performed in response to an operation by the user, an adjustment screen used to adjust the appearance of a portion of the photographed images among the plurality of photographed images is displayed, and the portion of the photographed images after the appearance adjustment is displayed in response to an operation by the user on the adjustment screen. After the server finishes adjusting the appearance of the uploaded photographed images using the adjustment screen, the content of the adjustment of the appearance of the portion of the photographed images that are similar in at least one of the size, position, orientation, brightness, and number of faces captured in the photographed images is reflected in each of the other photographed images that have not been adjusted in appearance. Effect of the Invention

[0008] According to the present technology, it is possible to efficiently adjust the appearance of a subject in a captured image. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram showing a workflow in a photography studio. [Diagram 2] FIG. 13 is a diagram showing an example of providing a captured image. [Diagram 3] FIG. 13 is a diagram showing an example of file management in a retouching PC. [Figure 4] FIG. 13 is a diagram showing an example of an image selection screen. [Diagram 5] FIG. 13 is a diagram showing an example of a retouch screen. [Figure 6] FIG. 13 is an enlarged view showing a display of a retouch selection area. [Figure 7] FIG. 13 is a diagram illustrating an example of application of retouching parameters. [Figure 8] FIG. 13 is a diagram illustrating another example of application of retouching parameters. [Figure 9] FIG. 13 is a diagram illustrating an example of application of retouching parameters. [Figure 10] FIG. 13 is a diagram illustrating an example of calculation of a face ratio. [Figure 11] FIG. 2 is a block diagram showing an example of the configuration of a retouching PC. [Figure 12] FIG. 2 is a block diagram showing an example of the functional configuration of a retouching PC. [Figure 13] 11 is a flowchart illustrating a retouching process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Work flow in the photography studio> FIG. 1 is a diagram showing the flow of work in a photography studio.

[0011] This technology is used in a photography studio where commemorative photos are taken for coming-of-age ceremonies, for example. This technology can also be applied to cases where commemorative photos are taken in a photography studio for various events, such as entrance / graduation ceremonies, wedding photos, and Shichi-Go-San ceremonies.

[0012] As shown on the left side of Fig. 1, a photographer takes a picture in a photography studio. In the example of Fig. 1, a user wearing a long-sleeved kimono stands in front of a background screen and is photographed as a subject. Strobe devices that emit light in accordance with the photographer's shooting are placed around the user.

[0013] The photographer takes pictures by changing the composition and taking many still images (photographs), such as 50 or 100. The composition of the still image is determined by the user's pose, the positional relationship between the user and the photographer, and camera parameters such as focal length. The still images taken by the photographer are transferred to and saved in a PC (not shown) connected to the camera.

[0014] After the shooting is completed, the user moves to the retouching space prepared in the shooting studio and, as shown on the right side of FIG. 1, uses the retouching PC 1 to check the images taken by the photographer.

[0015] Furthermore, the user selects the number of photographed images to be ultimately printed, for example, and adjusts the appearance of each photographed image by retouching them himself. Retouching is performed on a portion of photographed images, such as five or ten images, selected from the photographed images taken by the photographer. Retouching (processing) includes various types of image processing to change the appearance of photographed images.

[0016] In the retouching space, one or more retouching PCs 1 are prepared. Photographed images stored in a storage PC are transferred to the retouching PC 1 and displayed on a display 11 of the retouching PC 1. A retouching application 1A, which is an application used to retouch photographed images taken by a photographer, is installed in the retouching PC 1.

[0017] In this way, in a photography studio where a retouching PC 1 to which the present technology is applied is installed, a user himself / herself retouches a photographed image taken by a photographer. The user can use the functions of the retouching application 1A to retouch the photographed image so that it looks the way the user likes. The retouching application 1A may be installed on a terminal other than a PC, such as a tablet terminal, and used for retouching.

[0018] FIG. 2 is a diagram showing an example of providing a captured image.

[0019] The photographed images retouched by the retouching application 1A are printed by the photography studio's management company and later handed over to the user in the form of, for example, an album, as indicated by arrow #1.

[0020] Furthermore, as shown by arrow #2, the retouched photographed image is uploaded from the retouching PC 1 to the image management server 2 and managed in the image management server 2. The image management server 2 is, for example, a server on the Internet. The image management server 2 is managed by the management company of the photography studio, or by a business entity different from the management company of the photography studio.

[0021] A user can view and download images that have been taken of him or her as a subject by operating the image management app 3A on his or her mobile terminal 3 such as a smartphone, as shown by arrow #3. The image management app 3A is an application for viewing and so on the captured images managed in the image management server 2. The image management app 3A is provided, for example, by a business entity that manages the image management server 2.

[0022] The images taken in the photography studio are provided in the form of an album and, if necessary, in the form of data. The provision of the images in the form of data may be made possible for a fee separate from the fee for the album.

[0023] The provision of the photographed images in the form of data is performed so that not only the photographed images that have been retouched by the user but also the photographed images that have not been selected by the user as the images to be retouched are provided. As for the photographed images that have not been selected by the user as the images to be retouched, they are automatically retouched by the retouching application 1A without any user operation, as described later.

[0024] The user can also receive, in the form of data, photographed images that were not selected as targets for retouching. Instead of using the image management application 3A, a browser may be used to access a specific site and receive photographed images managed in the image management server 2. For example, by registering as a member of an image providing service provided by the administrator of the image management server 2, the user can receive photographed images managed in the image management server 2 using the image management application 3A or a browser.

[0025] Instead of the mobile terminal 3, other terminals such as a tablet terminal or a PC may be used to receive the captured images.

[0026] <About the retouching app> File Management FIG. 3 is a diagram showing an example of file management in the retouching PC 1. As shown in FIG.

[0027] As shown in the speech bubble in Figure 3, two folders, the all folder and the album folder, are prepared and linked to the user ID, which is the user's identification information. The all folder is a folder named "all," and the album folder is a folder named "album."

[0028] The captured image files transferred from the storage PC are stored in the all folder. The captured image files that were selected for retouching and have been retouched are stored in the album folder. The files stored in the album folder are the files used for printing.

[0029] When a user who has finished taking a photograph starts retouching, a user ID is input on the screen of the retouching application 1A. Based on the user ID, the captured image is read by the retouching application 1A, and processing is performed using the captured image. For example, the retouching application 1A reads all captured image files stored in an all folder linked to the user ID. After the captured images are read, the captured images to be retouched are selected.

[0030] ·Screen display FIG. 4 is a diagram showing an example of the image selection screen.

[0031] A message is displayed at the top of the image selection screen to guide the user to select five captured images. The number of selected captured images is set, for example, by a staff member of the photography studio before image selection. If the user applies for a plan to print five captured images and receive them as an album, five is set as the number of selected images.

[0032] A thumbnail display area 21 is formed on the right side of the image selection screen. The thumbnail display area 21 is an area in which thumbnail images of the captured images are displayed. For example, when 50 captured images have been captured, thumbnail images of each of the 50 captured images are displayed. Various operations using the screen of the retouching application 1A are performed using a mouse, a keyboard, etc.

[0033] A photographed image selected using a thumbnail image is enlarged and displayed in a display area 22 formed on the left side of the thumbnail display area 21. In the example of Fig. 4, the thumbnail image of the photographed image P1 displayed at the upper left end of the thumbnail display area 21 is selected, and the photographed image P1 is enlarged and displayed in the display area 22.

[0034] A confirm button 23 is displayed below the display area 22. The confirm button 23 is a button that is pressed when deciding on a photographed image displayed in the display area 22 as a subject for retouching. The user selects thumbnail images to sequentially enlarge and display each photographed image in the display area 22, and when the user finds a photographed image that the user likes, the user presses the confirm button 23 to select it as a subject for retouching.

[0035] The number of remaining selected images is displayed above the thumbnail display area 21. Each time a photographed image to be retouched is selected, the display of the number of selected images is updated.

[0036] When five photographed images have been selected and the end button 24 has been pressed, the selection of the photographed images to be retouched is completed. After the selection of the photographed images to be retouched is completed, retouching is performed on each of the photographed images selected as the retouched images.

[0037] FIG. 5 is a diagram showing an example of a retouch screen.

[0038] A display area 31 is formed at the upper center of the retouch screen. The photographed image currently being retouched is displayed in an enlarged form in the display area 31. A photographed image selected in a thumbnail display area 32 using a thumbnail image is displayed in an enlarged form in the display area 31.

[0039] A thumbnail display area 32 is formed in the lower left of the retouch screen. The thumbnail display area 32 is an area in which thumbnail images of photographed images selected on the image selection screen are displayed. In the example of Fig. 5, thumbnail images of five photographed images P1 to P5 are displayed in the thumbnail display area 32. The photographed image P1 selected using the thumbnail image is enlarged and displayed in the display area 31.

[0040] A retouching selection area 33 is formed below the retouching screen. The buttons displayed in the retouching selection area 33 are used to select the type of retouching. Each time a button displayed in the retouching selection area 33 is used to select the type of retouching, the selected type is reflected in the display in the display area 31. While viewing the display in the display area 31, the user uses the display in the retouching selection area 33 to select the type of retouching to be applied to the captured image currently being retouched.

[0041] FIG. 6 is an enlarged view of the display of the retouch selection area 33. As shown in FIG.

[0042] In the retouch selection area 33, buttons are displayed for selecting eye size, eye shape, makeup intensity, nose bridge on / off, skin type, face shape, face slimming, and style up on / off.

[0043] It is possible to select from five different eye sizes. Eye size "1" indicates no processing, i.e., the eye size is left unchanged at its original size. By selecting one of the sizes, the user can adjust the size of their own eyes in the captured image displayed in the display area 31.

[0044] The user can select one of the following eye shapes: "round eyes," "downturned eyes," and "cat eyes." By selecting one of the shapes, the user can adjust the shape of their own eyes that will appear in the photographed image displayed in the display area 31. By selecting "no processing," it is also possible to leave the shape of the eyes unchanged.

[0045] The intensity of makeup can be selected from three levels. The level of makeup "1" indicates that the intensity of makeup is not changed and remains the original intensity. By selecting one of the levels, the user can adjust the intensity of their makeup in the photographed image displayed in the display area 31.

[0046] It is possible to select ON / OFF for the nose bridge. By selecting ON or OFF, the user can select whether to emphasize the highlight part of the bridge of the nose that appears in the photographed image displayed in the display area 31, or to leave it in its original state.

[0047] It is possible to select one of five levels of strength for the skin texture. Skin texture "1" indicates that the skin texture is not changed and remains the original skin texture. By selecting one of the strengths, the user can adjust the degree of smoothness of the user's skin texture in the captured image displayed in the display area 31.

[0048] The face shape can be selected from among "round," "egg," and "slim." By selecting one of the shapes, the user can adjust the shape of his or her face that appears in the photographed image displayed in the display area 31. By selecting "no processing," it is also possible to leave the shape of the face unchanged.

[0049] It is possible to select five different sizes for the small face effect. Small face effect "1" indicates that the size of the face is not changed and remains the original size. The user can adjust the size of their face in the captured image displayed in the display area 31 by selecting one of the sizes "2" to "4."

[0050] It is possible to select ON / OFF as the style enhancement. By selecting ON or OFF, the user can select whether to increase his / her own height in the photographed image displayed in the display area 31 or to keep his / her original height.

[0051] Using these various functions, the user can select the content of retouching to be applied to the captured image displayed in the display area 31. In this way, retouching that can be performed using the retouching application 1A includes at least adjustment of the size and shape of the eyes and adjustment of the size and shape of the face.

[0052] The photographed image to be retouched is switched using the display in the thumbnail display area 32, and retouching is performed in sequence on each of the photographed images P1 to P5. The retouched files of the photographed images P1 to P5 are stored in the album folder.

[0053] Returning to the explanation of Fig. 5, a zoom in / out button 34 is provided on the right side of the display area 31. The zoom in / out button 34 is a button that is operated when zooming in on a part of the captured image displayed in the display area 31 or when zooming in on the zoomed in part.

[0054] Provided below the zoom in / out button 34 is a before-edit button 35. The before-edit button 35 is a button that is operated when switching the display in the display area 31 to the state before retouching. While the before-edit button 35 is pressed, the captured image in the state before retouching is displayed in the display area 31. By using the before-edit button 35, the user can compare the state after retouching with the state before retouching and check whether the degree of retouching is too strong, for example.

[0055] A back button 36, which is operated to return to the image selection screen, and an end button 37, which is operated to end retouching, are displayed at the top right of the retouching screen. When the retouching content for each of the captured images P1 to P5 has been selected and the end button 37 is pressed, the retouching ends.

[0056] -Transfer of retouching parameters The retouching parameters are parameters that represent the content of retouching selected using the display of the retouching selection area 33. When retouching is performed on a certain photographed image, the retouching parameters of the photographed image to be retouched are changed. The retouching parameters for a certain photographed image are applied as retouching parameters for other photographed images.

[0057] 7 and 8 are diagrams showing examples of application of retouching parameters.

[0058] As shown by arrow #11 in Fig. 7, when the captured image P1 is the first image to be retouched, the following retouching parameters are set for the captured image P1 by user operations: "eye size" to "3," "eye shape" to "cat eyes," "makeup intensity" to "2," ... The appearance of the retouched captured image P1 shown in the lower part of Fig. 7 is different from the appearance of the unretouched captured image P1 shown in the upper part.

[0059] When retouching of the photographed image P1 is completed and the retouching target is switched from the photographed image P1 to the photographed image P2, the retouching parameters of the retouched photographed image P1 are inherited as the default retouching parameters of the photographed image P2, which is the second photographed image to be retouched, as shown at the end of the arrow #12 in Fig. 8. The photographed image P2 is displayed in the display area 31 in a state in which it has been retouched to the same content as the retouched photographed image P1. The display of the retouching selection area 33 also changes depending on the content of the retouching applied to the photographed image P2.

[0060] The user can start retouching the photographed image P2 by using the photographed image P2, which has been retouched in the same manner as the photographed image P1, as a reference image. In the example of Fig. 8, the images have been adjusted to the contents represented by the retouching parameters: "eye size" is "4", "eye shape" is "cat eyes", "makeup intensity" is "1", ...

[0061] Similarly, when retouching of the photographed image P2 is completed and a third photographed image to be retouched is selected, the retouching parameters of the photographed image P2 are inherited as the default retouching parameters of the third photographed image to be retouched, as shown by arrow #14.

[0062] In this way, when a new photographed image that has not yet been retouched is selected as the photographed image to be retouched, the retouching contents of the photographed image that was previously the subject of retouching are inherited, thereby allowing the user to efficiently retouch each photographed image.

[0063] Each photographed image to be retouched may be classified according to the composition of the subject, and retouching parameters of a photographed image with the same composition may be applied. In this case, the composition of each photographed image is analyzed by the retouching application 1A, and each photographed image is classified as a photographed image with one of a plurality of compositions such as "close-up," "bust-up," and "full body."

[0064] A "close-up" is a composition in which the subject's face is shown prominently, and a "bust-up" is a composition in which the subject's upper body is shown prominently. A "full-body" is a composition in which the subject's entire body is shown. For example, the size of the face is analyzed, and the composition is classified according to the size of the face. The composition may also be classified according to not only the size of the face, but also the direction of the face, the position of the face, and the like.

[0065] For example, if a close-up photographed image is retouched and then the same close-up photographed image is selected as the subject of retouching, the retouching parameters of the retouched image are set as the retouching parameters of the close-up photographed image newly selected as the subject of retouching.

[0066] Appropriate retouching parameters may be prepared in advance for each composition. When a photographed image is selected as the subject of retouching, the retouching parameters prepared in advance for the composition of the photographed image are set as the retouching parameters of the photographed image newly selected as the subject of retouching.

[0067] A recommendation button that is operated when applying recommended retouching parameters may be provided on the retouching screen. When the recommendation button is pressed, retouching parameters that are prepared in advance for the composition of the photographed image selected as the subject of retouching are set to the photographed image.

[0068] A recommendation button for each composition may be provided on the retouching screen. In the above example, recommendation buttons are provided for each of the compositions, "close-up," "bust-up," and "full body." For example, information indicating the type of composition of the photographed image selected as the subject of retouching is displayed near the display area 31. The user can set appropriate retouching parameters by checking the composition of the photographed image selected as the subject of retouching and pressing the recommendation button for the same composition.

[0069] Retouching parameters may be recorded. For example, retouching parameters set by a user are recorded in association with a user ID together with information on the type of composition of a photographed image to be retouched.

[0070] Appropriate retouching parameters may be proposed using an inference model generated by machine learning. In this case, the retouching application 1A is provided with an inference function that uses an inference model that receives a captured image as input and outputs retouching parameters.

[0071] <Applying retouching parameters to unselected images> The five photographed images, for example, that have been retouched in this manner in response to user operations are printed on photographic paper and provided to the user in the form of an album. The photographed images that have been retouched in response to user operations are also provided in the form of data together with the photographed images that were not selected as targets for retouching on the image selection screen (FIG. 4).

[0072] Hereinafter, a photographed image that has not been selected as a target for retouching will be referred to as an unselected image, and a photographed image that has been selected as a target for retouching will be referred to as a selected image.

[0073] Unselected images are automatically retouched by the retouching application 1A without any operation by the user. Unselected images provided in the form of data are also retouched photographed images.

[0074] FIG. 9 is a diagram showing an example of application of retouching parameters.

[0075] 9, the retouching contents applied to each of the photographed images P1 to P5 by a user operation are reflected in each unselected image stored in the all folder. The retouching contents applied to each of the photographed images P1 to P5 may be reflected only to the unselected images stored in the all folder, or may be reflected to all photographed images stored in the all folder.

[0076] The retouching contents of which of the retouched photographed images P1 to P5 should be reflected in each unselected image is selected based on the state of the face appearing in the selected image and the state of the face appearing in the unselected image. A selected image in which a face appears in a state similar to the state of the face appearing in each unselected image is selected, and the retouching contents applied to the selected selected image are reflected in the unselected image.

[0077] For example, the size of the face can be used as the state of the face in the selected image and the unselected image. Specifically, the face ratio, which is the ratio of the face size to the size of the captured image, is calculated for each captured image (selected image and unselected image), and retouching is performed on each unselected image so as to reflect the retouching content of the selected image with a similar face ratio.

[0078] FIG. 10 is a diagram showing an example of calculation of the face ratio.

[0079] Fig. 10A shows an example of calculation of the face ratio for a photographed image P1 which is a horizontally long image, and Fig. 10B shows an example of calculation of the face ratio for a photographed image P4 which is a vertically long image.

[0080] Face recognition is performed, and the area enclosed by a frame F is recognized as the area of ​​the face. In addition, a face ratio Z is calculated based on the width X of the frame F and the length Y of the short side of the captured image, for example, by the following formula (1). Face ratio Z = width of frame F X ÷ length of short side Y (1)

[0081] For the horizontally long captured image P1 shown in Fig. 10A, the face ratio is calculated based on the width of the frame F and the height of the captured image indicated by the dashed-dotted arrow. On the other hand, for the vertically long image shown in Fig. 10B, the face ratio is calculated based on the width of the frame F and the width of the image indicated by the dashed-dotted arrow.

[0082] The face ratio is calculated for each of the photographed images P1 to P5, and also for each of the unselected images.

[0083] For a certain unselected image, the same retouching content as that applied to the retouched photographed image for which the face ratio, for example, closest to the face ratio of the unselected image is calculated is reflected.

[0084] For example, when the face ratios of the photographed images P1 to P5 are calculated as 0.30, 0.40, 0.50, 0.60, and 0.70, the unselected image with a calculated face ratio of 0.32 is retouched in the same manner as the photographed image P1, and the unselected image with a calculated face ratio of 0.80 is retouched in the same manner as the photographed image P5.

[0085] Usually, in studio photography, multiple images are taken with the same composition. When a user selects an image with a certain composition as a retouching target and performs retouching, the unselected images include an image with a similar composition to the retouched image. By selecting which retouched image to reflect the retouching content based on the face condition such as face ratio, it becomes possible to apply retouching to the unselected images according to the user's intention.

[0086] The user can receive unselected images that have been retouched in accordance with his / her intentions without having to select images one by one and manually retouch them. The user can efficiently adjust the appearance of the captured images.

[0087] The face ratio may be calculated by a method other than the method using the above formula (1). For example, the face ratio may be calculated based on the width of the frame F and the length of the long side of the image, or the face ratio may be calculated based on the area of ​​the face range relative to the area of ​​the image.

[0088] When there are multiple users as subjects and therefore multiple faces are captured in the image, the average face ratio of each person may be used as the face ratio for the entire image.

[0089] The face state in the photographed image may be represented by information other than the face ratio and used as a criterion for determining which photographed image should have its retouching contents reflected. It is possible to use face states such as face position, face direction, face brightness, and number of faces. It is also possible to use a combination of a plurality of pieces of information related to face states. That is, it is possible to use at least one of face size including face ratio, face position, face direction, face brightness, and number of faces as the face state.

[0090] The retouching contents of the photographed image to be reflected in the unselected image may be selected based on a criterion different from the face condition, such as face ratio. For example, the selection may be based on the eye condition, or the gender of the subject. As with the face condition, at least one of the eye size, eye position, and eye direction can be used as the eye condition.

[0091] It is also possible to select which captured image's retouching content should be reflected in the unselected image based on the type of composition described above.

[0092] <Configuration and operation of retouching PC1> Retouching PC1 configuration FIG. 11 is a block diagram showing an example of the configuration of the retouching PC 1. As shown in FIG.

[0093] A central processing unit (CPU) 101, a read only memory (ROM) 102, and a random access memory (RAM) 103 are interconnected by a bus 104.

[0094] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0095] The input unit 106 includes a keyboard, a mouse, and the like.

[0096] The output unit 107 is composed of a display 11 and the like.

[0097] The storage unit 108 is configured with a hard disk, a non-volatile memory, etc. The storage unit 108 stores programs such as the retouch application 1A executed by the CPU 101, and various information such as captured images.

[0098] The communication unit 109 is an interface to a network such as the Internet or a LAN. For example, the communication unit 109 communicates with a storage PC connected to a camera in a photography studio, and receives a captured image transmitted from the storage PC, in which the user about to start retouching appears as a subject. The communication unit 109 also communicates with the image management server 2, and transmits the retouched captured image to the image management server 2.

[0099] The drive 110 controls writing of data to the removable medium 111 and reading of data from the removable medium 111. Import of captured images from a PC for storage may be performed via the removable medium 111.

[0100] FIG. 12 is a block diagram showing an example of the functional configuration of the retouching PC 1. As shown in FIG.

[0101] 12, in the retouching PC 1, a retouching application 1A is executed to implement an image processing unit 121. The retouching PC 1 is an image processing device having the image processing unit 121.

[0102] The image processing unit 121 is made up of an image acquisition unit 131, a retouch processing unit 132, an output control unit 133, and a display control unit .

[0103] The image acquisition unit 131 accepts a user's operation on the image selection screen, and acquires the captured image selected by the user as the target for retouching from, for example, the storage unit 108. The captured image to be retouched acquired by the image acquisition unit 131 is supplied to the retouching processing unit 132 and the display control unit 134.

[0104] Furthermore, when retouching of each photographed image to be retouched is completed, the image acquisition unit 131 acquires an unselected image, for example, from the storage unit 108 , and outputs the image to the retouch processing unit 132 .

[0105] The retouching processor 132 performs face recognition to recognize a face appearing in a captured image to be retouched. The retouching processor 132 analyzes the features of each part of the recognized face and performs various image processing related to retouching in response to a user's operation on the retouching screen. The analysis of the features of each part may be performed using an inference model generated by machine learning.

[0106] For example, when an operation for adjusting the size or shape of the eyes is performed, the retouching processor 132 performs image processing such as shifting the position of the eye outline in accordance with the content selected by the user. When an operation for adjusting the intensity of makeup is performed, the retouching processor 132 performs image processing such as compositing an image of a blush of a predetermined color on the cheeks or compositing an image of a lipstick of a predetermined color on the lips.

[0107] When an operation related to adjusting the skin quality is performed, the retouching processor 132 performs image processing such as adjusting the brightness of the skin. When an operation related to adjusting the face size (feeling of a small face) or face shape is performed, the retouching processor 132 performs image processing such as shifting the position of the face outline according to the content selected by the user.

[0108] Information representing the results of the image processing by the retouching processor 132 is supplied to the display controller 134 and used to display the retouched screen.

[0109] Furthermore, when retouching of each photographed image to be retouched is completed, the retouching processor 132 analyzes each photographed image including an unselected image as described above, and reflects the retouching content applied to the retouched photographed image to the unselected image. The photographed images retouched by the retouching processor 132 are supplied to the output controller 133.

[0110] The output control unit 133 saves the captured image that has been retouched in response to a user operation in an album folder.

[0111] Furthermore, the output control unit 133 outputs the retouched photographed images (selected image and unselected image) to the communication unit 109, and causes it to be transmitted to the image management server 2.

[0112] The display control unit 134 controls the display of various screens of the retouching application 1A, such as the image selection screen and the retouching screen.

[0113] · Operation of retouching PC1 The retouching process performed by the retouching application 1A will be described with reference to the flowchart of FIG.

[0114] In step S1, the display control unit 134 of the image processing unit 121 displays an image selection screen (FIG. 4).

[0115] In step S2, the image acquisition unit 131 accepts the selection of a photographed image to be retouched in response to a user's operation on the image selection screen. The image acquisition unit 131 acquires the photographed image to be retouched.

[0116] In step S3, the image acquisition unit 131 determines whether or not the selection of the captured images to be retouched has been completed. If it is determined in step S3 that the selection of the captured images to be retouched has not been completed, the process returns to step S1, and image selection continues.

[0117] For example, if five photographed images are selected and the end button 24 is pressed, it is determined in step S3 that the selection of photographed images to be retouched is to be ended.

[0118] In step S4, the display control unit 134 displays a retouch screen (FIG. 5). The user retouches the captured image using the retouch screen.

[0119] In step S5, the retouching processor 132 performs image processing in response to a user's operation, and retouches the photographed image selected as the target for retouching.

[0120] In step S6, the retouching processor 132 determines whether or not the retouching is complete. If it is determined in step S6 that the retouching is not complete, the process returns to step S4, and retouching continues for each captured image selected as the retouching target.

[0121] For example, when all the photographed images selected as the retouching targets have been retouched and the end button 37 is pressed, it is determined in step S6 that the retouching is to be ended.

[0122] In step S7, the retouch processing unit 132 calculates the face ratio of each of the selected image and the unselected image.

[0123] In step S8, the retouching processor 132 applies to each unselected image the same retouching content as that applied to the selected image for which the face ratio closest to the face ratio of the unselected image has been calculated.

[0124] In step S9, the output control unit 133 stores the retouched unselected image.

[0125] In step S10, the communication unit 109 transmits the retouched selected images and the unselected images to the image management server 2. Thereafter, the processing of the retouching application 1A ends.

[0126] Through the above process, the user can adjust the appearance of the photographed image in which he or she is the subject, and can receive an album of the photographed images that are in the user's preferred appearance.

[0127] Furthermore, by using the mobile terminal 3, the user can receive, in the form of data, photographed images whose appearance has been manually adjusted and photographed images (unselected images) whose appearance has been automatically retouched in the same manner as the manual adjustments, which are managed in the image management server 2.

[0128] <Modification> Examples of retouching locations Although retouching is performed using a PC installed in a photography studio, retouching may also be performed using other devices, such as a mobile terminal 3 or a PC installed in the user's home.

[0129] When retouching using the mobile terminal 3 is possible, a user who has finished shooting in a photography studio operates the mobile terminal 3 to start up the image management application 3A and perform retouching on the screen of the image management application 3A. The image management application 3A has the same functions as the retouching application 1A.

[0130] The above-mentioned screen is displayed on the display of the mobile terminal 3 by the image management application 3A, and the photographed images to be retouched are selected and retouched from among the photographed images uploaded to the image management server 2. The photographed images that have been retouched using the functions of the image management application 3A are printed and provided to the user in the form of an album, or the retouched photographed images are provided to the user as they are in the form of data.

[0131] This allows the user to carry out retouching in various locations, such as at home, and allows the user to carry out retouching to the best of their ability without worrying about time.

[0132] -Screen display switching The above-mentioned retouching function can also be applied to retouching other captured images, such as commemorative wedding photos, etc. In a captured wedding image, for example, two users, the bride and groom, may appear.

[0133] In this case, it may be possible to select on the retouching screen whether to retouch the person on the left side or the person on the right side. This makes it possible to apply different retouching parameters to the groom and the bride, respectively. It may also be possible to retouch the groom and the bride simultaneously with a single operation.

[0134] In this way, the display of the retouching screen and the type of retouching available are switched depending on the type of commemorative photo. The type of commemorative photo, such as a coming-of-age ceremony photo or a wedding photo, may be selected before the start of retouching.

[0135] Warning: Be careful not to overfill As described above, a user can adjust the appearance of his / her face in a photographed image according to his / her preference, and enhance his / her face in the photographed image. Commemorative photographs such as coming-of-age ceremonies and wedding photographs are important photographs of life events that will remain for a long time.

[0136] In order to prevent such important photographs from being overexaggerated (too much retouching), a warning to that effect may be displayed when the degree of retouching selected by the user is greater than a threshold degree. The warning to be careful not to overexaggerate may be displayed, for example, on the retouching screen while the user is retouching.

[0137] A combination of retouching that results in excessive overexposure may be set, and a warning may be displayed when the user attempts to select such a retouching.

[0138] -Uniformity of the image As described above, a predetermined number of captured images (e.g., 5 or 10 images) that have been adjusted by the user are printed on photographic paper and provided to the user in the form of an album. If there is variation in the quality of each captured image, the album as a whole will lack a sense of unity.

[0139] In order to prevent the album from losing its sense of unity as a whole, the degree of retouching that can be selected may be limited. For example, one photographic image is selected as a reference, and the degree of retouching that can be selected is limited based on the content of the retouching applied to that photographic image.

[0140] It may be possible to make it possible to compare retouched photographed images while retouching. For example, when a predetermined operation is performed on the retouching screen, the retouched photographed images are displayed side by side. This makes it possible to ensure a sense of uniformity in the appearance of each photographed image.

[0141] ·others Although the number of photographed images to be retouched can be selected to be the same as the number of copies to be printed, a number greater than the number of copies to be printed may be selected. In this case, the user selects from the retouched photographed images the photographed images to be actually printed and received in the form of an album.

[0142] Although still images are taken in the photography studio, moving images may also be taken. In this case, the moving images are taken as the photographed images, and the photographed images can be retouched by the user.

[0143] About the Program The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0144] The program to be installed is provided by being recorded on removable medium 111 shown in FIG. 11, which is made up of an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. Alternatively, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be installed in advance in ROM 102 or storage unit 108.

[0145] The program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0146] The effects described herein are merely examples and are not limiting, and other effects may also be present.

[0147] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0148] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0149] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0150] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices. [Explanation of symbols]

[0151] 1 Retouching PC, 2 Image management server, 3 Mobile terminal, 11 Display, 121 Image processing section, 131 Image acquisition section, 132 Retouching processing section, 133 Output control section, 134 Display control section

Claims

1. In an image processing system including a terminal used by a user who is a subject of a photographed image and a server to which the photographed image is uploaded, The terminal includes: a communication unit that communicates with the server in response to an operation by the user after a photograph of the user is taken and the photographed image is uploaded to the server; a display unit that displays an adjustment screen used for adjusting the appearance of a portion of the plurality of captured images among the plurality of captured images, and displays the portion of the plurality of captured images after the appearance adjustment in response to an operation of the user on the adjustment screen; Equipped with The server, and an image processing unit that, after adjustment of appearance using the adjustment screen for the uploaded photographed image is completed, reflects the adjustment contents of the appearance of the part of the plurality of photographed images that are similar in at least one of the size, position, orientation, brightness, and number of faces included in the photographed image to each of the other photographed images that have not been adjusted in appearance. Image processing system.

2. The image processing unit of the server performs image processing to adjust the appearance of each of the plurality of captured images in response to an operation of the user on the adjustment screen. The image processing system according to claim 1 .

3. The display unit of the terminal displays the adjustment screen on which thumbnail images of the plurality of captured images are displayed side by side, and a captured image to be adjusted selected using the thumbnail images is displayed in an enlarged manner.

3. The image processing system according to claim 1.

4. The image processing device further includes: an image processing device that captures an image obtained by photographing the user as a subject and uploads the image to the server; 4. An image processing system according to claim 1, 2 or 3.

5. In an image processing system including a terminal used by a user who is a subject of a photographed image and a server to which the photographed image is uploaded, The terminal, a photographing device that photographs the user as a subject, and after the photographed image is uploaded to the server, the device communicates with the server in response to an operation by the user; displaying an adjustment screen used for adjusting the appearance of a portion of the plurality of captured images among the plurality of captured images, and displaying the portion of the plurality of captured images after the appearance adjustment in response to an operation of the user on the adjustment screen; The server, After the adjustment of appearance using the adjustment screen for the uploaded photographed images is completed, the adjustment of appearance of the part of the photographed images that are similar in at least one of the size, position, orientation, brightness, and number of faces in the photographed images is reflected in each of the other photographed images that have not been adjusted in appearance. Image processing methods.

Images