Image processing systems, image processing methods, servers

Abstract

To efficiently adjust the appearance of a subject in a photographed image.SOLUTION: In an image processing system according to an embodiment of the present technology, a terminal takes a photograph of a user as a subject, and after the photographed image is uploaded to a server, the terminal communicates with the server in response to a user's operation, displays an adjustment screen used to adjust the appearance of some of the photographed images, and displays the photographed images after the appearance adjustment in response to the user's operation on the adjustment screen. After the adjustment of the appearance of the uploaded photographed images using the adjustment screen is completed, the server reflects the adjustment of the appearance of the some of the photographed images to each of the other photographed images that have not been adjusted. The present technology can be applied to a computer.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This technology relates to an image processing system and an image processing method ,server and, in particular, to an image processing system and an image processing method that enable efficient adjustment of the appearance of a subject captured in a photographed image ,server relates to.

Background Art

[0002] In photographing devices such as certificate photo machines and print sticker machines installed in game centers, users can adjust the appearance of the captured image by themselves. The user can adjust the appearance of the image to be received in the state printed on sticker paper, etc., one by one according to their preference.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Adjusting the appearance of a large number of images, such as 50 or 100 images, one by one is cumbersome.

[0005] This technology has been made in view of such a situation, and enables efficient adjustment of the appearance of a subject captured in a photographed image.

Means for Solving the Problems

[0006] One aspect of this technology is an image processing system that includes a terminal used by a user who is the subject of a captured image, and a server to which the captured image is uploaded. The terminal includes a communication unit that communicates with the server in response to user operations after the user has been photographed and the captured image has been uploaded to the server, and a display unit that displays an adjustment screen used to adjust the appearance of some of the multiple captured images, and displays the adjusted version of the multiple captured images in response to user operations on the adjustment screen. The server includes an image processing unit that, after the adjustment of the uploaded captured image using the adjustment screen is completed, reflects the adjustments made to the appearance of the multiple captured images that are closest in at least one of the following aspects: face size, face position, face orientation, face brightness, and number of faces, to each of the other multiple captured images that have not been adjusted.

[0007] In one aspect of this technology, the terminal takes a photograph of the user, and after the captured image is uploaded to the server, communication is performed with the server in response to the user's operation, and an adjustment screen is displayed which is used to adjust the appearance of some of the multiple captured images, and in response to the user's operation on the adjustment screen, the adjusted version of the selected multiple captured images is displayed. After the adjustment of the uploaded captured images using the adjustment screen is completed on the server, the adjustments made to the appearance of the remaining multiple captured images that have not been adjusted are applied to each of the remaining multiple captured images that are closest in at least one of the following aspects: face size, face position, face orientation, face brightness, or number of faces. [Effects of the Invention]

[0008] This technology allows for efficient adjustment of how subjects appear in captured images. [Brief explanation of the drawing]

[0009] [Figure 1]This diagram shows the workflow in a photography studio. [Figure 2] This figure shows an example of how to provide captured images. [Figure 3] This figure shows an example of file management on a PC used for photo retouching. [Figure 4] This figure shows an example of an image selection screen. [Figure 5] This figure shows an example of a retouching screen. [Figure 6] This diagram shows a magnified view of the retouching selection area. [Figure 7] This figure shows an example of applying retouching parameters. [Figure 8] This figure shows another example of applying retouching parameters. [Figure 9] This figure shows an example of applying retouching parameters. [Figure 10] This figure shows an example of how to calculate facial proportions. [Figure 11] This block shows an example of a PC configuration for photo retouching. [Figure 12] This block diagram shows an example of the functional configuration of a PC for photo retouching. [Figure 13] This is a flowchart explaining the retouching process. [Modes for carrying out the invention]

[0010] <Workflow in the photo studio> Figure 1 shows the workflow in a photography studio.

[0011] This technology can be used, for example, in a photography studio for commemorative photos taken at coming-of-age ceremonies. It can also be applied to photography studios for commemorative photos of various events, such as school entrance / graduation ceremonies, weddings, and Shichi-Go-San celebrations.

[0012] As shown on the left side of FIG. 1, shooting in the photo studio is carried out by a cameraman. In the example of FIG. 1, shooting is being carried out with a user in a kimono pose standing in front of a background screen as the subject. Around the user, strobe devices that emit light in accordance with the shooting by the cameraman are arranged.

[0013] Shooting by the cameraman is carried out by changing the composition to shoot a large number of still images (photos) such as 50 or 100. The composition of the still image is determined by camera parameters such as the user's pose, the positional relationship between the user and the cameraman, and the focal length. The captured image, which is a still image captured by the cameraman, is transferred to a PC (not shown) connected to the camera and saved.

[0014] After the shooting is completed, the user moves to the retouch space prepared in the photo studio and, as shown on the right side of FIG. 1, uses the retouch PC 1 to check the captured images taken by the cameraman.

[0015] Also, the user selects, for example, the number of captured images to be finally printed and adjusts the appearance of each captured image by retouching them himself. Retouching is performed on some captured images such as 5 or 10 selected from the captured images taken by the cameraman. Retouching (processing) includes various image processes for changing the appearance of the captured image.

[0016] One or more retouch PCs 1 are prepared in the retouch space. The captured images saved in the storage PC are transferred to the retouch PC 1 and displayed on the display 11 of the retouch PC 1. An application for retouching the captured images taken by the cameraman, the retouch app 1A, is installed on the retouch PC 1.

[0017] Thus, in a photography studio equipped with a retouching PC 1 that utilizes this technology, the user can retouch the images taken by the photographer themselves. The user can use the functions of the retouching application 1A to retouch the images to their liking. The retouching application 1A may also be installed on a device other than the PC, such as a tablet, and used for retouching.

[0018] Figure 2 shows an example of how captured images are provided.

[0019] The retouched images taken using retouching app 1A are printed by the photography studio operator and later delivered to the user in the form of an album, for example, as shown by arrow #1.

[0020] Furthermore, as shown by arrow #2, the retouched images are uploaded from the retouching PC 1 to the image management server 2 and managed by the image management server 2. The image management server 2 is, for example, a server on the internet. The image management server 2 is managed by the photography studio's operating company, or by a different company.

[0021] The user can operate their mobile device 3, such as a smartphone, and use the image management application 3A to view and download images in which they are the subject, as shown by arrow #3. The image management application 3A is an application for viewing images managed on the image management server 2. The image management application 3A is provided, for example, by the business operator that manages the image management server 2.

[0022] The photographs taken at the photography studio will be provided in album form, and, as appropriate, in digital format. Providing the photographs digitally may be offered at a separate charge from the album fee.

[0023] The data provided includes not only images retouched by the user, but also images that were not selected by the user for retouching. As described later, the retouching of images not selected by the user is performed automatically by the retouching application 1A without any user intervention.

[0024] Users can also receive images in data format that they did not select for retouching. Instead of using the image management application 3A, users can access a designated website using a browser and receive the images managed on the image management server 2. For example, by registering as a member of the image provision service provided by the administrator of the image management server 2, users can receive the images managed on the image management server 2 using the image management application 3A or by using a browser.

[0025] Instead of using mobile device 3, it may be possible to receive the captured images using other devices such as tablet devices or PCs.

[0026] <About retouching apps> File management Figure 3 shows an example of file management on PC1 used for retouching.

[0027] As shown in the callout in Figure 3, two folders, the "all" folder and the "album" folder, are created and linked to the user's identification information, the user ID. The "all" folder is a folder named "all", and the "album" folder is a folder named "album".

[0028] Images transferred from the storage PC are stored in the "all" folder. Images selected for retouching and retouched are stored in the "album" folder. The files stored in the "album" folder are the ones used for printing.

[0029] When a user finishes shooting and begins retouching, they enter their user ID on the retouching application 1A screen. Based on the user ID, the retouching application 1A loads the captured images and processes them. For example, all the captured image files stored in the "all" folder associated with the user ID are loaded by the retouching application 1A. After the captured images are loaded, the user selects which images to retouch.

[0030] ·Screen display Figure 4 shows an example of an image selection screen.

[0031] At the top of the image selection screen, a message appears guiding you to select five photos. The number of photos to select is set before the selection process, for example, by the staff at the photography studio. If the user has subscribed to a plan to receive five printed photos as an album, the number of photos to select will be set to five.

[0032] A thumbnail display area 21 is formed on the right side of the image selection screen. The thumbnail display area 21 is the area where thumbnail images of the captured images are displayed. For example, if 50 images have been taken, a thumbnail image of each of the 50 captured images will be displayed. Various operations using the screen of the retouching application 1A are performed using a mouse, keyboard, etc.

[0033] The captured image selected using the thumbnail image is displayed enlarged in the display area 22 formed to the left of the thumbnail display area 21. In the example in Figure 4, the thumbnail image of captured image P1 displayed at the upper left corner of the thumbnail display area 21 is selected, and captured image P1 is displayed enlarged in the display area 22.

[0034] A confirmation button 23 is displayed below the display area 22. The confirmation button 23 is pressed when the user decides to select a photograph displayed in the display area 22 as the subject for retouching. The user sequentially enlarges and displays each photograph in the display area 22 by selecting a thumbnail image, and when they find a photograph they like, they press the confirmation button 23 to select it as the subject for retouching.

[0035] The remaining number of selected images is displayed above the thumbnail display area 21. The number of selected images is updated each time an image to be retouched is selected.

[0036] Once five images have been selected and the "End" button 24 is pressed, the selection of images to be retouched is complete. After the selection of images to be retouched is complete, retouching will be performed on each of the selected images.

[0037] Figure 5 shows an example of a retouching screen.

[0038] A display area 31 is formed in the upper center of the retouching screen. The display area 31 is the area where the currently retouched image is displayed in an enlarged view. The image selected in the thumbnail display area 32 using thumbnail images is displayed in an enlarged view in the display area 31.

[0039] A thumbnail display area 32 is formed in the lower left of the retouching screen. The thumbnail display area 32 is the area where thumbnail images of the captured images selected on the image selection screen are displayed. In the example in Figure 5, thumbnail images of five captured images, P1 to P5, are displayed in the thumbnail display area 32. Captured image P1, selected using the thumbnail images, is displayed enlarged in the display area 31.

[0040] A retouching selection area 33 is formed below the retouching screen. The user selects the retouching content using the buttons displayed in the retouching selection area 33. Each time a retouching content is selected using the buttons in the retouching selection area 33, the selected content is reflected in the display area 31. The user selects the retouching content to be applied to the currently selected image using the display in the retouching selection area 33 while looking at the display in the display area 31.

[0041] Figure 6 shows a magnified view of the retouching selection area 33.

[0042] The retouching selection area 33 displays buttons used to select eye size, eye shape, makeup intensity, nose bridge enhancement (on / off), skin texture, face shape, face slimming effect, and style enhancement (on / off).

[0043] Five different eye sizes are available to choose from. An eye size of "1" represents no processing, meaning the eye size remains unchanged and is kept at its original size. By selecting one of these sizes, the user can adjust the size of their own eyes as they appear in the captured image displayed in the display area 31.

[0044] Users can choose from three eye shapes: "round eyes," "droopy eyes," and "cat eyes." By selecting one of these shapes, users can adjust the shape of their eyes as they appear in the captured image displayed in the display area 31. Alternatively, users can choose "no processing" to leave their eye shape unchanged.

[0045] It is possible to select from three levels of makeup intensity. "1" represents the original level of makeup intensity, without any change. By selecting one of these intensity levels, the user can adjust the intensity of their makeup as it appears in the captured image displayed in the display area 31.

[0046] The nose bridge feature can be turned on or off. By selecting on or off, the user can choose whether to emphasize the highlight of their nose bridge in the captured image displayed in the display area 31, or to leave it in its original state.

[0047] It is possible to select from five levels of intensity for skin texture. Skin texture "1" represents keeping the original skin texture unchanged. By selecting one of the intensity levels, the user can adjust the degree of smoothness of their skin texture as it appears in the captured image displayed in the display area 31.

[0048] Users can select one of three face shapes: "round," "egg," or "slender." By selecting a shape, the user can adjust the shape of their face as it appears in the captured image displayed in the display area 31. Alternatively, selecting "no processing" allows the user to keep their face shape unchanged.

[0049] Five levels of face-slimming adjustment are available. Level "1" indicates that the face size remains unchanged, at its original size. By selecting one of levels "2" through "4," the user can adjust the size of their face as it appears in the captured image displayed in the display area 31.

[0050] As a style enhancement feature, it is possible to select whether to turn it on or off. By selecting on or off, the user can choose whether to increase their height in the captured image displayed in the display area 31, or to leave it as it is.

[0051] The user can use these various functions to select the retouching to be applied to the captured image displayed in the display area 31. Thus, the retouching that can be performed using the retouching application 1A includes at least adjustments to the size and shape of the eyes and the size and shape of the face.

[0052] The thumbnail display area 32 is used to switch between the images to be retouched, and retouching is performed sequentially on each of the images P1 through P5. The retouched files of each of the images P1 through P5 are stored in the album folder.

[0053] Returning to the explanation of Figure 5, a zoom in / out button 34 is provided on the right side of the display area 31. The zoom in / out button 34 is used to enlarge a portion of the captured image displayed in the display area 31, or to reduce the size of the enlarged portion.

[0054] Below the zoom in / out button 34, there is a pre-processing button 35. The pre-processing button 35 is operated to switch the display area 31 to the pre-retouching state. While the pre-processing button 35 is pressed, the pre-retouching image is displayed in the display area 31. By using the pre-processing button 35, the user can compare the retouched state with the pre-retouching state to check whether the retouching is too strong, etc.

[0055] In the upper right corner of the retouching screen, there is a back button 36 used to return to the image selection screen, and an exit button 37 used to finish retouching. Retouching is completed when the retouching content for each of the captured images P1 to P5 is selected and the exit button 37 is pressed.

[0056] • Transfer of retouching parameters Retouch parameters are parameters that represent the content of the retouching selected using the display of the retouching selection area 33. When retouching is performed on a particular image, the retouch parameters of the image being retouched are changed. The retouch parameters for one image are applied as retouching parameters to other images.

[0057] Figures 7 and 8 show examples of applying retouching parameters.

[0058] As shown by arrow #11 in Figure 7, when the captured image P1 is selected as the first image to be retouched, the following retouch parameters are set for the captured image P1 through user actions: "Eye size" to "3", "Eye shape" to "Cat eyes", "Makeup intensity" to "2", etc. The retouched captured image P1 shown in the lower part of Figure 7 looks different from the unretouched captured image P1 shown in the upper part.

[0059] When retouching of captured image P1 is complete and the retouching target is switched from captured image P1 to captured image P2, the retouching parameters of the retouched captured image P1 are inherited as the default retouching parameters for captured image P2, which is the second captured image to be retouched, as shown at the end of arrow #12 in Figure 8. In the display area 31, captured image P2 is displayed with the same retouching applied as the retouched captured image P1. The display in the retouching selection area 33 also changes according to the retouching applied to captured image P2.

[0060] The user can start retouching image P2, which has been retouched to the same extent as image P1, as a baseline. In the example in Figure 8, the adjustments are made to the content represented by the retouching parameters: "Eye size" is "4", "Eye shape" is "Cat eyes", "Makeup intensity" is "1", etc.

[0061] Similarly, when retouching of image P2 is completed and a third image to be retouched is selected, the retouching parameters of image P2 will be inherited as the default retouching parameters for the third image to be retouched, as shown by arrow #14.

[0062] In this way, when a new image that has not yet been retouched is selected as the image to be retouched, the retouching settings of the previous image to be retouched are carried over. This allows the user to efficiently retouch each image.

[0063] Each image to be retouched may be classified according to the composition of the subject, and retouching parameters may be applied to images with the same composition. In this case, the composition of each image is analyzed by the retouching application 1A, and each image is classified as an image with one of several types of compositions, such as "close-up," "bust-up," or "full body."

[0064] A "close-up" is a composition in which the subject's face is prominently featured, while a "bust-up" is a composition in which the subject's upper body is prominently featured. A "full-body" is a composition in which the subject's entire body is featured. For example, the size of the face is analyzed, and the composition is classified according to the size of the face. In addition to the size of the face, the composition may also be classified according to the direction and position of the face.

[0065] For example, if a close-up shot is retouched and then another close-up shot is selected for retouching, the retouching parameters of the already retouched shot will be set as the retouching parameters for the newly selected close-up shot.

[0066] Appropriate retouching parameters may be pre-configured for each composition. When a photographic image is selected for retouching, the retouching parameters pre-configured for the composition of that photographic image are set as the retouching parameters for the newly selected photographic image.

[0067] A "Recommended" button may be provided on the retouching screen to be used when applying recommended retouching parameters. When the Recommended button is pressed, the retouching parameters pre-configured for the composition of the selected image are applied to the image being retouched.

[0068] The retouching screen may also include recommendation buttons for each composition. In the example above, recommendation buttons would be provided for each composition: "close-up," "bust-up," and "full body." For example, information indicating the type of composition of the selected image for retouching would be displayed near the display area 31. The user can check the composition of the selected image for retouching and set appropriate retouching parameters by pressing the recommendation button for the same composition.

[0069] The system may also be configured to record retouching parameters. For example, retouching parameters set by the user may be recorded along with information about the composition of the image being retouched, linked to the user ID.

[0070] The system may also use an inference model generated by machine learning to suggest appropriate retouching parameters. In this case, the retouching application 1A will be equipped with an inference function that uses an inference model that takes a captured image as input and outputs retouching parameters.

[0071] <Applying retouching parameters to an unselected image> As described above, five retouched images, in accordance with the user's actions, are printed on photographic paper and provided to the user in the form of an album. In addition, the retouched images, along with the images not selected for retouching on the image selection screen (Figure 4), are provided as data.

[0072] Hereafter, images that were not selected for retouching will be referred to as "unselected images." Images that were selected for retouching will be referred to as "selected images."

[0073] For unselected images, retouching is automatically applied by the retouching application 1A without any user intervention. Unselected images provided as data will also become retouched images.

[0074] Figure 9 shows an example of applying retouching parameters.

[0075] As shown in Figure 9, the retouching applied to each of the captured images P1 through P5 by user operation is reflected in the respective unselected images stored in the "all" folder. The retouching applied to each of the captured images P1 through P5 may be reflected only in the unselected images stored in the "all" folder, or it may be reflected in all captured images stored in the "all" folder.

[0076] The selection of which of the retouched images P1 through P5 will have its retouching applied to each unselected image is based on the facial appearance in the selected image and the facial appearance in the unselected image. The selected image that shows a facial appearance closest to that of the unselected image is chosen, and the retouching applied to the selected image is applied to the unselected image.

[0077] For example, the size of the face can be used as a measure of the facial features in the selected and unselected images. Specifically, the face ratio, which is the ratio of the face size to the size of the image, is calculated for each captured image (selected and unselected image). The retouching applied to each unselected image is then adjusted to reflect the retouching applied to the selected image, which has a similar face ratio.

[0078] Figure 10 shows an example of how to calculate facial proportions.

[0079] Figure 10A shows an example of calculating the face ratio of captured image P1, which is a landscape image. Figure 10B shows an example of calculating the face ratio of captured image P4, which is a portrait image.

[0080] Face recognition is performed, and the area enclosed by frame F is recognized as the face. Furthermore, based on the width X of frame F and the length Y of the shorter side of the captured image, the face ratio Z is calculated, for example, using equation (1) below. Face ratio Z = Width X of frame F ÷ Length of shorter side Y ... (1)

[0081] For the horizontally oriented image P1 shown in Figure 10A, the face ratio is calculated based on the width of frame F and the height of the image indicated by the dashed-dotted arrow. On the other hand, for the vertically oriented image shown in Figure 10B, the face ratio is calculated based on the width of frame F and the width of the image indicated by the dashed-dotted arrow.

[0082] The face ratio is calculated for each of the captured images P1 through P5. Additionally, the face ratio is calculated for each of the unselected images.

[0083] For any unselected image, the same retouching applied to a retouched photograph whose face ratio is closest to that of the unselected image will be applied.

[0084] For example, if the face ratios of captured images P1 through P5 are calculated to be 0.30, 0.40, 0.50, 0.60, and 0.70 respectively, then the unselected image with a calculated face ratio of 0.32 will be retouched in the same way as the retouching applied to captured image P1. Similarly, the unselected image with a calculated face ratio of 0.80 will be retouched in the same way as the retouching applied to captured image P5.

[0085] Typically, in studio photography, multiple images are taken with similar compositions. When a user selects an image with a particular composition for retouching and performs the retouching, some unselected images will have a similar composition to the retouched image. By selecting which retouched image to apply the effects of based on facial features such as face ratio, it becomes possible to apply retouching to the unselected images in accordance with the user's intentions.

[0086] Users can receive unselected images that have been retouched according to their intentions, without having to manually select and retouch each image one by one. This allows users to efficiently adjust the appearance of their captured images.

[0087] The face ratio may be determined by a method other than the one using equation (1) above. For example, the face ratio may be determined based on the width of frame F and the length of the long side of the image, or it may be determined based on the area of ​​the face relative to the area of ​​the image.

[0088] If there are multiple users as subjects and therefore multiple faces are visible, the average of the face ratios of each person may be used as the overall face ratio of the image.

[0089] The condition of the faces in the captured images may be represented by information other than face ratio, and this information may be used as a criterion for determining which captured images should be retouched. It is possible to use face condition information such as face position, face orientation, face brightness, and number of faces. Multiple pieces of information regarding face condition may be used in combination. That is, at least one of the following can be used as the face condition: face size (including face ratio), face position, face orientation, face brightness, and number of faces.

[0090] The criteria for which retouching effects from a captured image are applied to an unselected image may be based on criteria other than facial features such as face ratio. For example, the selection may be based on the appearance of the eyes, or on the gender of the subject. Similar to facial features, at least one of the following can be used to define the appearance of the eyes: eye size, eye position, or eye orientation.

[0091] It is also possible to configure the system so that the retouching details of each image are applied to the unselected images based on the type of composition described above.

[0092] <Configuration and operation of PC1 for retouching> • Configuration of PC1 for retouching Figure 11 is a block diagram showing an example configuration of the PC1 used for retouching.

[0093] The CPU (Central Processing Unit) 101, ROM (Read Only Memory) 102, and RAM (Random Access Memory) 103 are interconnected by a bus 104.

[0094] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0095] The input unit 106 consists of a keyboard, mouse, and the like.

[0096] The output unit 107 consists of a display 11 and the like.

[0097] The memory unit 108 is composed of a hard disk, non-volatile memory, and the like. The memory unit 108 stores various information such as programs such as the retouching application 1A executed by the CPU 101, and captured images.

[0098] The communication unit 109 is an interface to networks such as the Internet and LAN. For example, the communication unit 109 communicates with a storage PC connected to a camera in the shooting studio and receives captured images of a user who is about to begin retouching, which have been sent from the storage PC. The communication unit 109 also communicates with the image management server 2 and sends the retouched captured images to the image management server 2.

[0099] Drive 110 controls the writing of data to and reading of data from the removable media 111. The capture images may also be imported from a PC for storage via the removable media 111.

[0100] Figure 12 is a block diagram showing an example of the functional configuration of PC1 for retouching.

[0101] As shown in Figure 12, in the retouching PC1, the image processing unit 121 is realized when the retouching application 1A is executed. The retouching PC1 is an image processing device having the image processing unit 121.

[0102] The image processing unit 121 consists of an image acquisition unit 131, a retouching processing unit 132, an output control unit 133, and a display control unit 134.

[0103] The image acquisition unit 131 receives user input on the image selection screen and acquires the captured image selected by the user as the retouching target from, for example, the storage unit 108. The captured image to be retouched acquired by the image acquisition unit 131 is supplied to the retouching processing unit 132 and the display control unit 134.

[0104] Furthermore, when retouching is completed for each of the captured images to be retouched, the image acquisition unit 131 acquires the unselected images, for example, from the storage unit 108, and outputs them to the retouching processing unit 132.

[0105] The retouching processing unit 132 performs face recognition and identifies faces in the captured image to be retouched. The retouching processing unit 132 analyzes the features of each part of the recognized face and performs various image processing related to retouching in response to user operations on the retouching screen. The analysis of the features of each part may be performed using an inference model generated by machine learning.

[0106] For example, when an operation related to adjusting the size or shape of the eyes is performed, the retouching processing unit 132 performs image processing such as shifting the position of the eye outline according to the user's selection. Also, when an operation related to adjusting the intensity of makeup is performed, the retouching processing unit 132 performs image processing such as compositing an image of blush of a predetermined color onto the cheek area or compositing an image of lipstick of a predetermined color onto the lips.

[0107] The retouching processing unit 132 performs image processing such as adjusting the brightness of the skin when an operation related to adjusting skin texture is performed. The retouching processing unit 132 also performs image processing such as shifting the position of the facial contour according to the user's selection when an operation related to adjusting the size (smaller face appearance) or shape of the face is performed.

[0108] Information representing the results of image processing by the retouching processing unit 132 is supplied to the display control unit 134 and used for displaying the retouched screen.

[0109] Furthermore, once retouching is complete for each of the images to be retouched, the retouching processing unit 132 analyzes each of the images, including the unselected images, as described above, and reflects the retouching applied to the retouched images onto the unselected images. The retouched images from the retouching processing unit 132 are then supplied to the output control unit 133.

[0110] The output control unit 133 saves the retouched captured image to the album folder according to the user's operation.

[0111] Furthermore, the output control unit 133 outputs the retouched captured images (selected and unselected images) to the communication unit 109 and sends them to the image management server 2.

[0112] The display control unit 134 controls the display of various screens of the retouching application 1A, such as the image selection screen and the retouching screen.

[0113] • Operation of PC1 for retouching Refer to the flowchart in Figure 13 to explain the retouching process performed by retouching application 1A.

[0114] In step S1, the display control unit 134 of the image processing unit 121 displays an image selection screen (Figure 4).

[0115] In step S2, the image acquisition unit 131 accepts the user's operation on the image selection screen and accepts the selection of the image to be retouched. The image acquisition unit 131 acquires the image to be retouched.

[0116] In step S3, the image acquisition unit 131 determines whether the selection of images to be retouched is complete. If it is determined in step S3 that the selection of images to be retouched is not complete, the process returns to step S1 and the image selection continues.

[0117] For example, if five images are selected and the exit button 24 is pressed, it is determined in step S3 that the selection of images to be retouched is complete.

[0118] In step S4, the display control unit 134 displays the retouching screen (Figure 5). The user uses the retouching screen to retouch the captured image.

[0119] In step S5, the retouching processing unit 132 performs image processing in response to user operations and applies retouching to the captured image selected as the retouching target.

[0120] In step S6, the retouching processing unit 132 determines whether the retouching is complete or not. If it is determined in step S6 that the retouching is not complete, the process returns to step S4, and retouching continues for each of the images selected as retouching targets.

[0121] For example, if all the images selected for retouching have been retouched and the finish button 37 is pressed, it is determined in step S6 that the retouching process is complete.

[0122] In step S7, the retouching processing unit 132 calculates the face ratio of the selected image and the unselected image, respectively.

[0123] In step S8, the retouching processing unit 132 applies the same retouching to each of the unselected images as the retouching applied to the selected image whose face ratio is closest to that of the unselected image.

[0124] In step S9, the output control unit 133 saves the retouched unselected image.

[0125] In step S10, the communication unit 109 sends the retouched selected image and the unselected image to the image management server 2. After that, the processing of the retouching application 1A is completed.

[0126] Through the above process, users can adjust the appearance of images taken of themselves. Users can then receive an album of images that match their preferred appearance.

[0127] Furthermore, by using the mobile device 3, users can receive, in data form, both the manually adjusted captured image and the automatically retouched image (unselected image) that has been automatically processed with the same content as the manually adjusted image, all managed by the image management server 2.

[0128] <Variation> • Examples of retouching locations While retouching is assumed to be performed using a PC installed in the shooting studio, it may also be possible to perform retouching using other devices such as a mobile device 3 or a PC installed in the user's home.

[0129] If retouching using the mobile device 3 is possible, after finishing a photoshoot in the studio, the user will operate the mobile device 3 to launch the image management app 3A and perform retouching on the screen of the image management app 3A. The image management app 3A will have the same functions as the retouching app 1A.

[0130] The display of the mobile device 3 shows the screen described above, and the image management application 3A selects and performs retouching on the captured images uploaded to the image management server 2. The retouched images are then printed and provided to the user in the form of an album, or the retouched images are provided to the user as data.

[0131] This allows users to perform retouching in various locations, such as their homes. Furthermore, users can meticulously retouch their photos without worrying about time constraints.

[0132] • Switching screen displays The retouching functions described above can also be applied to retouching other types of photographs, such as wedding photos. Wedding photos, for example, may include both the bride and groom.

[0133] In this case, the retouching screen may allow the user to select whether to retouch the person on the left or the person on the right. This makes it possible to apply different retouching parameters to the bride and groom. It may also be possible to retouch both the bride and groom simultaneously in a single operation.

[0134] In this way, the retouching screen display and the types of retouching available can be switched depending on the type of commemorative photo. It would also be possible to have the type of commemorative photo, such as coming-of-age ceremony photos or wedding photos, selected before starting the retouching process.

[0135] • Warning: Be careful not to overdo it. As mentioned above, users can adjust their facial features to their liking, essentially enhancing their appearance in the captured images. Commemorative photos such as coming-of-age ceremony photos and wedding photos are important pictures of life events that will be kept for a long time.

[0136] To prevent such important photos from being over-retouched (too much retouching), a warning could be displayed if the user's selected level of retouching exceeds a threshold. This warning could appear, for example, on the retouching screen while the user is performing retouching.

[0137] There may be predefined combinations of retouching that result in excessive effects, and a warning could be displayed when a user attempts to select such a combination.

[0138] • Consistency in the image As mentioned above, a predetermined number of photos, such as 5 or 10, that the user has adjusted are printed on photographic paper and provided to the user in the form of an album. If there is variation in the quality of each photo, the overall consistency of the album will be lost.

[0139] To prevent the overall coherence of the album from being compromised, the degree of retouching that can be selected may be limited. For example, one reference image could be selected, and the degree of retouching that can be selected could be limited based on the level of retouching applied to that image.

[0140] It would be helpful to allow users to compare retouched images with their original photos while retouching. For example, when a specific operation is performed on the retouching screen, the retouched images could be displayed side-by-side. This would help ensure consistency in the image quality of each photograph.

[0141] ·others While the system allows users to select the same number of images to be retouched as the number of prints, it may also be possible to select a larger number of images. In this case, the user will select the images from the retouched images that will actually be printed and received in album form.

[0142] While the scenario assumes that still images are captured in the studio, it is also possible to capture moving images. In this case, the moving images can be used as the captured images, and the user can perform retouching on the captured images.

[0143] • About the program The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed on a computer that is built into dedicated hardware, or on a general-purpose personal computer.

[0144] The program to be installed is provided by recording it on removable media 111, as shown in Figure 11, which consists of optical discs (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or semiconductor memory. Alternatively, it may be provided via wired or wireless transmission media such as a local area network, the internet, or digital broadcasting. The program can be pre-installed in ROM 102 or storage unit 108.

[0145] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when they are called.

[0146] The effects described herein are illustrative and not limited to those described herein, and other effects may also occur.

[0147] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0148] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0149] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0150] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices. [Explanation of Symbols]

[0151] 1 PC for retouching, 2 Image management server, 3 Mobile terminal, 11 Display, 121 Image processing unit, 131 Image acquisition unit, 132 Retouching processing unit, 133 Output control unit, 134 Display control unit

Claims

1. In an image processing system that includes the terminal used by the user who is the subject of the captured image, and the server to which the captured image is uploaded, The aforementioned terminal is A communication unit communicates with the server after the user has been photographed and the captured image has been uploaded to the server, in response to the user's actions. A display unit that displays an adjustment screen used to adjust the image quality of some of the multiple captured images, and displays the adjusted images of the multiple captured images in response to the user's operation on the adjustment screen. Equipped with, The aforementioned server, After adjusting the appearance of uploaded captured images using the adjustment screen, the system includes an image processing unit that, for each of the remaining captured images that have not undergone any adjustments, reflects the adjustments made to the image of the selected image that is closest in at least one of the following aspects: face size, face position, face orientation, face brightness, and number of faces. Image processing system.

2. The image processing unit of the server performs image processing to adjust the appearance of each of the multiple captured images in response to the user's operation on the adjustment screen. The image processing system according to claim 1.

3. The display unit of the terminal displays an adjustment screen in which thumbnail images of some of the multiple captured images are displayed side by side, and the captured image to be adjusted, selected using the thumbnail images, is displayed in an enlarged view. The image processing system according to claim 1 or 2.

4. In an image processing system that includes the terminal used by the user who is the subject of the captured image, and the server to which the captured image is uploaded, The aforementioned terminal, After the user is photographed and the captured image is uploaded to the server, communication is performed with the server in response to the user's actions. An adjustment screen is displayed for adjusting the appearance of some of the multiple captured images, and in response to the user's operation on the adjustment screen, the adjusted versions of the multiple captured images are displayed. The aforementioned server, After adjusting the appearance of the uploaded captured images using the adjustment screen, the adjustments made to the appearance of the remaining multiple captured images that have not undergone any adjustments are applied to each of the other multiple captured images that have the closest match in at least one of the following aspects: face size, face position, face orientation, face brightness, or number of faces. Image processing methods.

5. A communication unit that communicates with a terminal used by the user after an image of the user has been uploaded, A display unit that displays an adjustment screen on the terminal's display for adjusting the image quality of some of the multiple captured images uploaded, and displays the adjusted images of the multiple captured images in response to user operations on the adjustment screen, After the adjustment of the image using the adjustment screen is completed, an image processing unit applies the image adjustment settings of the selected image to each of the other images that have not been adjusted, based on at least one of the following: the size of the face, the position of the face, the orientation of the face, the brightness of the face, or the number of faces. A server equipped with the following features.

Images