Image processing device, image processing method, and program

Abstract

To efficiently adjust the appearance of a subject shown in a photographed image.SOLUTION: An image processing device displays an adjustment screen used for adjusting the appearance of partial multiple photographed images from among multiple photographed images, and adjusts the appearance of each of the partial multiple photographed images according to a user's operation to the adjustment screen. The image processing device also allows each of other multiple photographed images for which the adjustment of appearance is not performed yet to reflect the contents of adjustment of appearance of partial multiple photographed images in which at least one of the size of included faces, the positions of the faces, the directions of the faces, the brightness of the faces, and the number of the faces is the closest after operation for adjusting appearance using the adjustment screen is performed. This technique can be applied to a computer installed in a photographing studio.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present technology relates to an image processing device, an image processing method, and a program, and more particularly to an image processing device, an image processing method, and a program that enable efficient adjustment of the appearance of a subject appearing in a captured image. [Background technology]

[0002] In photo taking devices such as ID photo booths and print sticker machines installed in game arcades, users can adjust the appearance of the captured image themselves. Users can adjust the appearance of the image that they receive printed on sticker paper or the like one by one to their liking. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-118329 Summary of the Invention [Problem to be solved by the invention]

[0004] It is cumbersome to adjust the appearance of a large number of images, such as 50 or 100, one by one.

[0005] The present technology has been made in consideration of such circumstances, and makes it possible to efficiently adjust the appearance of a subject in a captured image. [Means for solving the problem]

[0006] An image processing device according to one aspect of the present technology includes a display control unit that displays an adjustment screen used to adjust the appearance of a portion of a plurality of captured images among a plurality of captured images, and an image processing unit that adjusts the appearance of each of the portion of the captured images in accordance with a user's operation on the adjustment screen, and, after an operation to complete the appearance adjustment using the adjustment screen has been performed, reflects the details of the appearance adjustment of the portion of the captured images that are closest in at least one of the size, position, orientation, brightness, and number of faces included in the captured images to each of the other captured images for which appearance adjustment has not been performed.

[0007] In one aspect of the present technology, an adjustment screen used to adjust the appearance of a portion of a plurality of captured images among a plurality of captured images is displayed, and the appearance of each of the portion of the captured images is adjusted in accordance with a user's operation on the adjustment screen. Furthermore, after an operation to end the appearance adjustment using the adjustment screen is performed, the content of the appearance adjustment of the portion of the captured images that is closest in at least one of the size, position, orientation, brightness, and number of faces included in the captured images is reflected in each of the other captured images that have not been adjusted in appearance. [Effects of the Invention]

[0008] According to the present technology, it is possible to efficiently adjust the appearance of a subject in a captured image. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the workflow in a photography studio. [Figure 2] FIG. 10 is a diagram illustrating an example of providing a captured image. [Figure 3] FIG. 10 is a diagram illustrating an example of file management in a retouching PC. [Figure 4] FIG. 10 is a diagram illustrating an example of an image selection screen. [Figure 5] FIG. 10 is a diagram illustrating an example of a retouch screen. [Figure 6] FIG. 10 is an enlarged view showing a display of a retouch selection area. [Figure 7] FIG. 10 is a diagram illustrating an example of application of retouching parameters. [Figure 8] FIG. 10 is a diagram illustrating another example of application of retouching parameters. [Figure 9] FIG. 10 is a diagram illustrating an example of application of retouching parameters. [Figure 10] FIG. 10 is a diagram illustrating an example of calculation of a face ratio. [Figure 11] FIG. 1 is a block diagram showing an example of the configuration of a retouching PC. [Figure 12] FIG. 2 is a block diagram showing an example of the functional configuration of a retouching PC. [Figure 13] 10 is a flowchart illustrating a retouching process. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Work flow in the photography studio> FIG. 1 is a diagram showing the workflow in a photography studio.

[0011] This technology is used in a photography studio for taking commemorative photos for coming-of-age ceremonies, for example. This technology can also be applied to photography studios taking commemorative photos for various events, such as entrance / graduation ceremonies, weddings, and Shichi-Go-San celebrations.

[0012] As shown on the left side of Figure 1, photography in a photography studio is performed by a photographer. In the example of Figure 1, a user wearing a furisode kimono is standing in front of a background screen and photographed as the subject. Strobe devices that emit light in time with the photographer's photography are placed around the user.

[0013] The photographer takes pictures by changing the composition and taking as many as 50, 100, or more still images (photographs). The composition of the still image is determined by the user's pose, the positional relationship between the user and the photographer, and camera parameters such as focal length. The still images taken by the photographer are transferred to and saved on a PC (not shown) connected to the camera.

[0014] After the photo shoot is completed, the user moves to the retouching space provided in the photo studio and, as shown on the right side of FIG. 1, uses the retouching PC 1 to check the images taken by the photographer.

[0015] Furthermore, the user may select the number of photographed images to be printed, for example, and adjust the appearance of each photographed image by retouching them themselves. Retouching is performed on a selection of photographed images, such as five or ten, selected from the photographed images taken by the photographer. Retouching (processing) includes various image processing methods for changing the appearance of photographed images.

[0016] The retouching space is equipped with one or more retouching PCs 1. Photographed images stored on a storage PC are transferred to the retouching PC 1 and displayed on the display 11 of the retouching PC 1. A retouching application 1A, which is an application used to retouch photographed images taken by a photographer, is installed on the retouching PC 1.

[0017] In this way, in a photography studio where a retouching PC 1 to which the present technology is applied is installed, a user can retouch the photographed image taken by a photographer. The user can use the functions of the retouching application 1A to retouch the photographed image to achieve the desired look. The retouching application 1A may be installed on a device other than a PC, such as a tablet device, and used for retouching.

[0018] FIG. 2 is a diagram showing an example of providing a captured image.

[0019] The photographed images retouched by the retouching application 1A are printed by the photography studio management company and later handed over to the user in the form of, for example, an album, as indicated by arrow #1.

[0020] Furthermore, as indicated by arrow #2, the retouched photographed images are uploaded from the retouching PC 1 to the image management server 2, where they are managed. The image management server 2 is, for example, a server on the Internet. The image management server 2 is managed by the photography studio's operating company or by a business entity other than the photography studio's operating company.

[0021] By operating the image management app 3A on a user's mobile device 3 such as a smartphone, the user can view and download images that have been taken of the user as the subject, as shown by arrow #3. The image management app 3A is an application for viewing the images that are managed in the image management server 2. The image management app 3A is provided, for example, by a business operator that manages the image management server 2.

[0022] The images taken in the photography studio will be provided in the form of an album and, if appropriate, in the form of data. The provision of the images in the form of data may be made possible for a separate fee from the fee for the album.

[0023] The provision of photographed images as data is performed so that not only photographed images that have been retouched by the user but also photographed images that have not been selected by the user as retouching targets are provided. As for photographed images that have not been selected by the user as retouching targets, as will be described later, retouching is automatically performed by the retouching application 1A without any user operation.

[0024] The user can also receive, in the form of data, photographed images that were not selected for retouching. Instead of using the image management app 3A, the user may access a predetermined site using a browser and receive photographed images managed in the image management server 2. For example, by registering as a member of an image providing service provided by the administrator of the image management server 2, the user can receive photographed images managed in the image management server 2 using the image management app 3A or a browser.

[0025] Instead of the mobile terminal 3, other terminals such as a tablet terminal or a PC may be used to receive the captured images.

[0026] <About the retouching app> File Management FIG. 3 is a diagram showing an example of file management in the retouching PC 1. As shown in FIG.

[0027] As shown in the speech bubble in Figure 3, two folders, the all folder and the album folder, are prepared and linked to the user ID, which is the user's identification information. The all folder is a folder with the folder name "all," and the album folder is a folder with the folder name "album."

[0028] The image files transferred from the storage PC are stored in the all folder. The image files selected for retouching and retouched are stored in the album folder. The files stored in the album folder are the files used for printing.

[0029] When a user who has finished taking a photo starts retouching, the user inputs a user ID on the screen of retouching application 1A. Based on the user ID, the captured image is read by retouching application 1A, and processing using the captured image is performed. For example, all captured image files stored in an all folder linked to the user ID are read by retouching application 1A. After the captured images are read, the captured images to be retouched are selected.

[0030] ·Screen display FIG. 4 is a diagram showing an example of the image selection screen.

[0031] A message instructing the user to select five captured images is displayed at the top of the image selection screen. The number of captured images to be selected is set, for example, by a staff member at the photography studio before the image selection. If the user has signed up for a plan to print five captured images and receive them as an album, five is set as the number of images to be selected.

[0032] A thumbnail display area 21 is formed on the right side of the image selection screen. The thumbnail display area 21 is an area where thumbnail images of the captured images are displayed. For example, if 50 captured images have been taken, thumbnail images of each of the 50 captured images are displayed. Various operations using the screen of the retouching application 1A are performed using a mouse, keyboard, etc.

[0033] A photographed image selected using a thumbnail image is enlarged and displayed in a display area 22 formed on the left side of the thumbnail display area 21. In the example of Fig. 4, the thumbnail image of photographed image P1 displayed in the upper left corner of the thumbnail display area 21 is selected, and photographed image P1 is enlarged and displayed in the display area 22.

[0034] A confirm button 23 is displayed below the display area 22. The confirm button 23 is a button that is pressed when deciding on a photographed image displayed in the display area 22 as the subject of retouching. The user selects thumbnail images to sequentially enlarge and display each photographed image in the display area 22, and when the user finds a photographed image that he or she likes, he or she presses the confirm button 23 to select it as the subject of retouching.

[0035] The number of remaining selected images is displayed above the thumbnail display area 21. Each time a photographed image to be retouched is selected, the display of the number of selected images is updated.

[0036] The selection of the photographed images to be retouched is completed when five photographed images have been selected and the end button 24 has been pressed. After the selection of the photographed images to be retouched is completed, retouching is performed on each of the photographed images selected as the retouched images.

[0037] FIG. 5 is a diagram showing an example of a retouch screen.

[0038] A display area 31 is formed in the upper center of the retouch screen. The captured image currently being retouched is displayed in an enlarged form in the display area 31. A captured image selected in a thumbnail display area 32 using a thumbnail image is displayed in an enlarged form in the display area 31.

[0039] A thumbnail display area 32 is formed in the lower left of the retouch screen. The thumbnail display area 32 is an area where thumbnail images of photographed images selected on the image selection screen are displayed. In the example of Fig. 5, thumbnail images of five photographed images P1 to P5 are displayed in the thumbnail display area 32. The photographed image P1 selected using the thumbnail images is enlarged and displayed in the display area 31.

[0040] A retouching selection area 33 is formed below the retouching screen. The retouching content is selected using the buttons displayed in the retouching selection area 33. Each time a retouching content is selected using the buttons displayed in the retouching selection area 33, the selected content is reflected in the display in the display area 31. While viewing the display in the display area 31, the user uses the display in the retouching selection area 33 to select the retouching content to be applied to the captured image currently being retouched.

[0041] FIG. 6 is an enlarged view of the display of the retouch selection area 33. As shown in FIG.

[0042] The retouch selection area 33 displays buttons used to select eye size, eye shape, makeup intensity, nose bridge on / off, skin type, face shape, face slimming effect, and style up on / off.

[0043] The eye size can be selected from five sizes. Eye size "1" indicates no processing, i.e., the eye size is left unchanged at its original size. By selecting one of the sizes, the user can adjust the size of their own eyes that appear in the captured image displayed in the display area 31.

[0044] The user can select one of the following eye shapes: "round eyes," "droopy eyes," and "cat eyes." By selecting one of these shapes, the user can adjust the shape of their own eyes that will appear in the photographed image displayed in the display area 31. By selecting "no processing," the user can leave the shape of their eyes unchanged.

[0045] The user can select from three levels of makeup intensity. The makeup intensity "1" indicates that the makeup intensity will remain unchanged at the original intensity. By selecting one of the levels, the user can adjust the intensity of their makeup in the captured image displayed in the display area 31.

[0046] The nose bridge can be selected as on or off. By selecting on or off, the user can choose whether to emphasize the highlights of the bridge of their nose in the photographed image displayed in the display area 31, or to leave it in its original state.

[0047] The user can select one of five levels of strength for the skin texture. Skin texture "1" indicates that the original skin texture will remain unchanged. By selecting one of the levels, the user can adjust the smoothness of their skin texture in the captured image displayed in the display area 31.

[0048] The face shape can be selected from among "round," "egg," and "slim." By selecting one of these shapes, the user can adjust the shape of their face that appears in the captured image displayed in the display area 31. By selecting "no processing," it is also possible to leave the shape of the face unchanged.

[0049] The user can select five different sizes to achieve a small face effect. The small face effect "1" indicates that the face size is not changed and remains at its original size. The user can adjust the size of their face in the captured image displayed in the display area 31 by selecting one of the sizes "2" to "4."

[0050] The style enhancement can be selected to be on or off. By selecting on or off, the user can choose whether to increase their own height in the photographed image displayed in the display area 31 or to keep their original height.

[0051] Using these various functions, the user can select the retouching content to be applied to the captured image displayed in the display area 31. In this way, the retouching that can be performed using the retouching application 1A includes at least adjusting the size and shape of the eyes and the size and shape of the face.

[0052] The photographed image to be retouched is switched using the display in the thumbnail display area 32, and retouching is performed sequentially on each of the photographed images P1 to P5. The retouched files for each of the photographed images P1 to P5 are stored in the album folder.

[0053] Returning to the explanation of Fig. 5, a zoom in / out button 34 is provided on the right side of the display area 31. The zoom in / out button 34 is a button that is operated to zoom in on a portion of the captured image displayed in the display area 31 or to zoom out on the zoomed in portion.

[0054] Below the zoom in / out button 34, there is provided a before-retouch button 35. The before-retouch button 35 is operated to switch the display in the display area 31 to the state before retouching. While the before-retouch button 35 is pressed, the captured image in the state before retouching is displayed in the display area 31. By using the before-retouch button 35, the user can compare the state after retouching with the state before retouching and check whether the degree of retouching is too strong, for example.

[0055] At the top right of the retouching screen, there are displayed a back button 36, which is operated to return to the image selection screen, and an end button 37, which is operated to end retouching. When the retouching content for each of the captured images P1 to P5 has been selected and the end button 37 is pressed, the retouching ends.

[0056] -Transfer of retouching parameters The retouching parameters are parameters that represent the content of retouching selected using the display in the retouching selection area 33. When retouching is performed on a certain photographed image, the retouching parameters of the photographed image to be retouched are changed. The retouching parameters for one photographed image are applied as retouching parameters for other photographed images.

[0057] 7 and 8 are diagrams showing examples of application of retouching parameters.

[0058] As indicated by arrow #11 in Fig. 7, when photographed image P1 is the first image to be retouched, the following retouching parameters are set for photographed image P1 by user operations: "eye size" to "3," "eye shape" to "cat eyes," "makeup intensity" to "2," etc. The appearance of retouched photographed image P1 shown in the bottom row of Fig. 7 is different from the appearance of photographed image P1 before retouching shown in the top row.

[0059] When retouching of photographed image P1 is completed and the retouching target is switched from photographed image P1 to photographed image P2, the retouching parameters of the retouched photographed image P1 are inherited as the default retouching parameters for photographed image P2, the second photographed image to be retouched, as indicated by arrow #12 in Fig. 8. Photographed image P2 is displayed in display area 31 with the same retouching applied as retouched photographed image P1. The display of retouching selection area 33 also changes depending on the retouching applied to photographed image P2.

[0060] The user can start retouching the photographed image P2 by using the photographed image P2, which has been retouched in the same manner as the photographed image P1, as a reference. In the example of Fig. 8, the "eye size" has been adjusted to "4," the "eye shape" to "cat eyes," and the "makeup intensity" to "1," etc., as indicated by the retouching parameters.

[0061] Similarly, when retouching of photographed image P2 is completed and a third photographed image to be retouched is selected, the retouching parameters of photographed image P2 are inherited as the default retouching parameters for the third photographed image to be retouched, as shown by arrow #14.

[0062] In this way, when a new photographed image that has not yet been retouched is selected as the photographed image to be retouched, the retouching details of the photographed image that was previously selected as the photographed image to be retouched are inherited, thereby allowing the user to efficiently retouch each photographed image.

[0063] Each photographed image to be retouched may be classified according to the composition of the subject, and retouching parameters of a photographed image with the same composition may be applied. In this case, the composition of each photographed image is analyzed by the retouching application 1A, and each photographed image is classified as a photographed image with one of multiple compositions, such as "close-up," "bust-up," and "full body."

[0064] "Close-up" is a composition in which the subject's face is shown prominently, and "bust-up" is a composition in which the subject's upper body is shown prominently. "Full body" is a composition in which the subject's entire body is shown. For example, the size of the face is analyzed, and the composition is classified according to the size of the face. The composition may also be classified according to not only the size of the face, but also the direction of the face, the position of the face, etc.

[0065] For example, if a close-up photographed image is retouched and then the same close-up photographed image is selected as the subject of retouching, the retouching parameters of the retouched photographed image are set as the retouching parameters of the close-up photographed image newly selected as the subject of retouching.

[0066] Appropriate retouching parameters may be prepared in advance for each composition. When a photographed image is selected as the subject of retouching, the retouching parameters prepared in advance for the composition of that photographed image are set as the retouching parameters for the photographed image newly selected as the subject of retouching.

[0067] A recommendation button that is operated when applying recommended retouching parameters may be provided on the retouching screen. When the recommendation button is pressed, retouching parameters that are prepared in advance for the composition of the photographed image selected as the subject of retouching are applied to the photographed image.

[0068] A recommendation button for each composition may be provided on the retouching screen. In the example described above, recommendation buttons are provided for each of the compositions: "close-up," "bust-up," and "full body." For example, information indicating the type of composition of the photographed image selected for retouching is displayed near the display area 31. The user can check the composition of the photographed image selected for retouching and set appropriate retouching parameters by pressing the recommendation button for the same composition.

[0069] Retouching parameters may be recorded. For example, retouching parameters set by a user are recorded in association with a user ID along with information on the type of composition of the photographed image to be retouched.

[0070] Appropriate retouching parameters may be proposed using an inference model generated by machine learning. In this case, the retouching application 1A is equipped with an inference function that uses the inference model, which takes a captured image as input and outputs retouching parameters.

[0071] <Applying retouching parameters to unselected images> The five captured images, for example, that have been retouched in accordance with the user's operations as described above are printed on photographic paper and provided to the user in the form of an album. The captured images that have been retouched in accordance with the user's operations are also provided in the form of data together with the captured images that were not selected for retouching on the image selection screen (Figure 4).

[0072] Hereinafter, a photographed image that has not been selected as a target for retouching will be referred to as an unselected image, and a photographed image that has been selected as a target for retouching will be referred to as a selected image.

[0073] Unselected images are automatically retouched by the retouching application 1A without any user operation. Unselected images provided in the form of data are also retouched photographed images.

[0074] FIG. 9 is a diagram showing an example of application of retouching parameters.

[0075] 9, the retouching content applied to each of the photographed images P1 to P5 by a user operation is reflected in each of the unselected images stored in the all folder. The retouching content applied to each of the photographed images P1 to P5 may be reflected only in the unselected images stored in the all folder, or may be reflected in all of the photographed images stored in the all folder.

[0076] The retouching content of which of the retouched photographed images P1 to P5 should be reflected in each unselected image is selected based on the state of the face appearing in the selected image and the state of the face appearing in the unselected image. A selected image in which a face appears in a state similar to the state of the face appearing in each unselected image is selected, and the retouching content applied to the selected selected image is reflected in the unselected image.

[0077] For example, the size of the face can be used as the state of the face in the selected image and the unselected image. Specifically, for each captured image (selected image and unselected image), a face ratio, which is the ratio of the face size to the captured image size, is calculated, and each unselected image is retouched so as to reflect the retouching content of the selected image with a similar face ratio.

[0078] FIG. 10 is a diagram showing an example of calculation of the face ratio.

[0079] Fig. 10A shows an example of calculation of the face ratio for a photographed image P1 which is a horizontally long image, and Fig. 10B shows an example of calculation of the face ratio for a photographed image P4 which is a vertically long image.

[0080] Face recognition is performed, and the area enclosed by a frame F is recognized as the area of ​​the face. Furthermore, based on the width X of the frame F and the length Y of the short side of the captured image, a face ratio Z is calculated, for example, using the following formula (1): Face ratio Z = width of frame F X ÷ length of short side Y (1)

[0081] For the horizontally long captured image P1 shown in Fig. 10A, the face ratio is calculated based on the width of the frame F and the height of the captured image indicated by the dashed-dotted arrow. On the other hand, for the vertically long image shown in Fig. 10B, the face ratio is calculated based on the width of the frame F and the width of the image indicated by the dashed-dotted arrow.

[0082] The face ratio is calculated for each of the photographed images P1 to P5, and also for each of the unselected images.

[0083] For a certain unselected image, the same retouching content as that applied to the retouched photographed image for which the face ratio closest to that of the unselected image has been calculated is reflected.

[0084] For example, if the face ratios of the photographed images P1 to P5 are calculated as 0.30, 0.40, 0.50, 0.60, and 0.70, the unselected image with a calculated face ratio of 0.32 will be retouched in the same manner as the photographed image P1, and the unselected image with a calculated face ratio of 0.80 will be retouched in the same manner as the photographed image P5.

[0085] Typically, in studio photography, multiple images are taken with similar compositions. When a user selects a photographed image with a certain composition as a target for retouching and performs retouching, the unselected images include an image with a similar composition to the retouched photographed image. By selecting which of the retouched photographed images the retouching content should be reflected on the basis of the face condition such as the face ratio, it becomes possible to apply retouching in line with the user's intention to unselected images as well.

[0086] Users can receive unselected images that have been retouched in accordance with their intentions, without having to select each image one by one and manually retouch them. Users can efficiently adjust the appearance of captured images.

[0087] The face ratio may be calculated by a method other than the method using the above formula (1). For example, the face ratio may be calculated based on the width of the frame F and the length of the long side of the image, or the face ratio may be calculated based on the area of ​​the face range relative to the area of ​​the image.

[0088] When there are multiple users as subjects and therefore multiple faces are captured in the image, the average value of the face ratios of the respective people may be used as the face ratio for the entire image.

[0089] The state of the face in the photographed image may be represented by information other than the face ratio and used as a criterion for determining which photographed image to apply the retouching content to. It is possible to use face states such as face position, face direction, face brightness, and number of faces. It is also possible to use a combination of multiple pieces of information related to face states. That is, it is possible to use at least one of face size including face ratio, face position, face direction, face brightness, and number of faces as the face state.

[0090] The retouching content of the captured image to be reflected in the unselected image may be selected based on criteria other than the facial condition, such as face ratio. For example, the selection may be based on the eye condition or the gender of the subject. As with the facial condition, at least one of the eye size, eye position, and eye direction can be used as the eye condition.

[0091] It is also possible to select which photographed image's retouching content should be reflected in the unselected image based on the type of composition described above.

[0092] <Configuration and operation of retouching PC1> Retouching PC1 configuration FIG. 11 is a block diagram showing an example of the configuration of the retouching PC 1. As shown in FIG.

[0093] A CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103 are interconnected by a bus 104.

[0094] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0095] The input unit 106 is composed of a keyboard, a mouse, and the like.

[0096] The output unit 107 is configured by a display 11 and the like.

[0097] The storage unit 108 is configured by a hard disk, a non-volatile memory, etc. The storage unit 108 stores programs such as the retouch application 1A executed by the CPU 101, and various information such as captured images.

[0098] The communication unit 109 is an interface to a network such as the Internet or a LAN. For example, the communication unit 109 communicates with a storage PC connected to a camera in a photography studio, and receives a photographed image transmitted from the storage PC, in which the user about to begin retouching appears as a subject. The communication unit 109 also communicates with the image management server 2, and transmits the retouched photographed image to the image management server 2.

[0099] The drive 110 controls writing of data to and reading of data from the removable medium 111. Captured images may be imported from a PC via the removable medium 111 for storage.

[0100] FIG. 12 is a block diagram showing an example of the functional configuration of the retouching PC 1. As shown in FIG.

[0101] 12, in the retouching PC 1, an image processing unit 121 is implemented by executing a retouching application 1A. The retouching PC 1 is an image processing device that includes the image processing unit 121.

[0102] The image processing unit 121 is made up of an image acquisition unit 131, a retouch processing unit 132, an output control unit 133, and a display control unit .

[0103] The image acquisition unit 131 accepts a user's operation on the image selection screen and acquires the captured image selected by the user as the target for retouching from, for example, the storage unit 108. The captured image to be retouched acquired by the image acquisition unit 131 is supplied to the retouching processing unit 132 and the display control unit 134.

[0104] Furthermore, when retouching of each photographed image to be retouched is completed, the image acquisition unit 131 acquires an unselected image from, for example, the storage unit 108 and outputs it to the retouching processing unit 132 .

[0105] The retouching processor 132 performs face recognition to recognize faces appearing in captured images to be retouched. The retouching processor 132 analyzes the features of each part of the recognized face and performs various image processing related to retouching in response to user operations on the retouching screen. The analysis of the features of each part may be performed using an inference model generated by machine learning.

[0106] For example, when an operation related to adjusting the size or shape of the eyes is performed, the retouching processor 132 performs image processing such as shifting the position of the eye outline in accordance with the user's selection. Also, when an operation related to adjusting the intensity of makeup is performed, the retouching processor 132 performs image processing such as combining an image of blush of a predetermined color on the cheeks or combining an image of lipstick of a predetermined color on the lips.

[0107] When an operation related to adjusting the skin quality is performed, the retouching processing unit 132 performs image processing such as adjusting the brightness of the skin. When an operation related to adjusting the face size (feeling of a small face) or face shape is performed, the retouching processing unit 132 performs image processing such as shifting the position of the face outline in accordance with the selection made by the user.

[0108] Information representing the results of image processing by the retouching processor 132 is supplied to the display controller 134 and used to display the retouched screen.

[0109] Furthermore, when retouching of each photographed image to be retouched is completed, the retouching processing unit 132 analyzes each photographed image, including unselected images, as described above, and reflects the retouching content applied to the retouched photographed images in the unselected images. The photographed images retouched by the retouching processing unit 132 are supplied to the output control unit 133.

[0110] The output control unit 133 saves the captured image that has been retouched in response to the user's operation in the album folder.

[0111] Furthermore, the output control unit 133 outputs the retouched photographed images (selected image and unselected image) to the communication unit 109, and causes them to be transmitted to the image management server 2.

[0112] The display control unit 134 controls the display of various screens of the retouching application 1A, such as the image selection screen and the retouching screen.

[0113] ·Retouching PC1 operation The retouching process performed by the retouching application 1A will be described with reference to the flowchart of FIG.

[0114] In step S1, the display control unit 134 of the image processing unit 121 displays the image selection screen (FIG. 4).

[0115] In step S2, the image acquisition unit 131 receives a selection of a photographed image to be retouched in response to a user's operation on the image selection screen. The image acquisition unit 131 acquires the photographed image to be retouched.

[0116] In step S3, the image acquisition unit 131 determines whether or not the selection of the captured images to be retouched has been completed. If it is determined in step S3 that the selection of the captured images to be retouched has not been completed, the process returns to step S1, and image selection continues.

[0117] For example, if five photographed images are selected and the end button 24 is pressed, it is determined in step S3 that the selection of photographed images to be retouched has ended.

[0118] In step S4, the display control unit 134 displays the retouch screen (FIG. 5). The user retouches the captured image using the retouch screen.

[0119] In step S5, the retouching processing unit 132 performs image processing in response to a user operation, and retouches the photographed image selected as the target for retouching.

[0120] In step S6, the retouching processor 132 determines whether or not the retouching is completed. If it is determined in step S6 that the retouching has not ended, the process returns to step S4, and retouching continues for each photographed image selected as the retouching target.

[0121] For example, when all the photographed images selected as retouching targets have been retouched and the end button 37 is pressed, it is determined in step S6 that the retouching is to be ended.

[0122] In step S7, the retouching processor 132 calculates the face ratio of each of the selected image and the unselected image.

[0123] In step S8, the retouching processor 132 applies to each unselected image the same retouching content as that applied to the selected image whose face ratio has been calculated to be closest to the face ratio of the unselected image.

[0124] In step S9, the output control unit 133 saves the retouched unselected image.

[0125] In step S10, the communication unit 109 transmits the retouched selected images and the unselected images to the image management server 2. After that, the processing of the retouching application 1A ends.

[0126] Through the above process, the user can adjust the appearance of the photographed image in which the user is the subject, and can receive an album of the photographed images that are in the user's preferred appearance.

[0127] Furthermore, by using the mobile terminal 3, the user can receive, in the form of data, photographed images whose appearance has been manually adjusted and photographed images (unselected images) whose appearance has been automatically retouched in the same manner as the manual adjustments, which are managed in the image management server 2.

[0128] <Modification> Examples of retouching locations Although retouching is performed using a PC installed in the photography studio, retouching may also be performed using other devices, such as a mobile terminal 3 or a PC installed in the user's home.

[0129] If retouching using the mobile terminal 3 is possible, after a user has finished taking photos in a photography studio, the user operates the mobile terminal 3 to launch the image management application 3A and performs retouching on the screen of the image management application 3A. The image management application 3A has the same functions as the retouching application 1A.

[0130] The image management application 3A displays the above-described screen on the display of the mobile terminal 3, and the photographed images to be retouched are selected and retouched from among the photographed images uploaded to the image management server 2. The photographed images that have been retouched using the functions of the image management application 3A are printed and provided to the user in the form of an album, or the retouched photographed images are provided to the user as digital data.

[0131] This allows the user to carry out retouching in various places such as at home, and the user can carry out retouching to their heart's content without worrying about time.

[0132] -Switching screen display The above-described retouching function can also be applied to retouching other captured images, such as commemorative wedding photos, etc. A captured wedding image may include, for example, two users, the bride and groom.

[0133] In this case, it may be possible to select on the retouching screen whether to retouch the person on the left side or the person on the right side. This makes it possible to apply different retouching parameters to the bride and groom. It may also be possible to retouch the bride and groom simultaneously with a single operation.

[0134] In this way, the display of the retouching screen and the types of retouching available are switched depending on the type of commemorative photo. The type of commemorative photo, such as a coming-of-age ceremony photo or a wedding photo, may be selected before retouching begins.

[0135] Warning: Be careful not to overfill As described above, users can adjust the appearance of their face in photographs to suit their preferences, so to speak, enhancing their face in photographs. Commemorative photographs such as coming-of-age ceremony photographs and wedding photographs are important photographs of life events that will remain for a long time.

[0136] To prevent such important photos from being overexaggerated (too much retouching), a warning to that effect may be displayed when the degree of retouching selected by the user is greater than a threshold degree. The warning to be careful about overexaggeration may be displayed, for example, on the retouching screen while the user is retouching.

[0137] A combination of retouching that results in excessive enhancement may be set, and a warning may be displayed when the user attempts to select such a retouching.

[0138] ·Uniformity of the image As mentioned above, a predetermined number of captured images, such as five or ten, that have been adjusted by the user are printed on photographic paper and provided to the user in the form of an album. If there is variation in the quality of each captured image, the sense of unity in the album as a whole will be lost.

[0139] To prevent the album from losing its overall sense of unity, the degree of retouching that can be selected may be limited. For example, one photographed image is selected as a reference, and the degree of retouching that can be selected is limited based on the retouching content applied to that photographed image.

[0140] It may also be possible to compare retouched photographed images while retouching. For example, when a predetermined operation is performed on the retouching screen, the retouched photographed images are displayed side by side. This makes it possible to ensure a sense of uniformity in the appearance of each photographed image.

[0141] ·others Although the number of photographed images to be retouched can be selected to be the same as the number of prints, it is also possible to select a number greater than the number of prints. In this case, the user selects from the retouched photographed images which he or she will actually print and receive in the form of an album.

[0142] Although still images are taken in the photography studio, moving images may also be taken. In this case, the moving images are taken as photographed images, and the photographed images can be retouched by the user.

[0143] About the program The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0144] The program to be installed is provided by being recorded on removable media 111 shown in FIG. 11, which may be an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. Alternatively, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be installed in advance in ROM 102 or storage unit 108.

[0145] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0146] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0147] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0148] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0149] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0150] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. [Explanation of symbols]

[0151] 1 Retouching PC, 2 Image management server, 3 Mobile terminal, 11 Display, 121 Image processing unit, 131 Image acquisition unit, 132 Retouching processing unit, 133 Output control unit, 134 Display control unit

Claims

1. a display control unit that displays an adjustment screen used for adjusting the appearance of some of the plurality of captured images; an image processing unit that adjusts the appearance of each of the plurality of captured images in response to a user's operation on the adjustment screen, and, after an operation to end the adjustment of the appearance using the adjustment screen is performed, reflects the adjustment content of the appearance of the portion of the plurality of captured images that is closest in at least any of the size, position, orientation, brightness, and number of faces included in the captured images to each of the remaining plurality of captured images that have not been adjusted in appearance; An image processing device comprising:

2. a display unit that displays the adjustment screen according to the control of the display control unit The image processing device according to claim 1 .

3. The adjustment of the appearance of the part of the plurality of captured images includes at least adjustment of the size and shape of the person's eyes and adjustment of the size and shape of the face.

3. The image processing device according to claim 1 or 2.

4. the captured image used for processing by the image processing unit is an image uploaded from a device connected via the Internet, The display control unit causes another device operated by the user to display the adjustment screen.

4. An image processing device according to claim 1, 2 or 3.

5. The image processing device displaying an adjustment screen used for adjusting the appearance of some of the plurality of photographed images; adjusting the appearance of each of the plurality of captured images in response to a user's operation on the adjustment screen; After an operation for completing the adjustment of appearance using the adjustment screen is performed, the adjustment of appearance of the part of the plurality of photographed images that is closest in at least one of the size, position, orientation, brightness, and number of faces in the photographed images is reflected in each of the other plurality of photographed images that have not been subjected to adjustment of appearance. Image processing methods.

6. displaying an adjustment screen used for adjusting the appearance of some of the plurality of photographed images; adjusting the appearance of each of the plurality of captured images in response to a user's operation on the adjustment screen; After an operation for completing the adjustment of appearance using the adjustment screen is performed, the adjustment of appearance of the part of the plurality of photographed images that is closest in at least one of the size, position, orientation, brightness, and number of faces in the photographed images is reflected in each of the other plurality of photographed images that have not been subjected to adjustment of appearance. A program that causes a computer to execute a process.

Images