Information processing system, information processing device, image processing device, information processing method, and program

Abstract

To enable selection of a device to be used for retouching a photographed image. [Solution] An information processing device according to the present technology acquires multiple captured images and displays a setting screen used to set whether retouching work should be performed using a first device that uploads the captured images or a second device that is accessed based on access destination information notified by a server after the captured images are uploaded. The information processing device also uploads multiple captured images by transmitting them to an upload destination server, and when the retouching work is set to be performed using the first device, displays a retouching screen used for retouching work after uploading some of the captured images. The present technology can be applied to a system that provides retouched captured images to a computer installed in a photo studio.

Description

[Technical Field]

[0001] In particular, the present technology relates to an information processing system, an information processing device, an image processing device, an information processing method, and a program that enable selection of an apparatus to be used for retouching a captured image. [Background technology]

[0002] As a service for commemorative photography at events such as coming-of-age ceremonies and weddings, there is a service that allows customers to retouch the images taken by a photographer at a photo studio (Patent Document 1). After the photo shoot, customers can retouch the images using a PC or other device provided in a separate space in the photo studio. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-13621 [Patent Document 2] Japanese Patent Application Publication No. 2020-154639 Summary of the Invention [Problem to be solved by the invention]

[0004] It would be convenient if retouching work could be performed using not only equipment provided in a photo studio, but also other equipment. For example, if one's own equipment could be used for retouching work, one could choose whether to perform retouching work using equipment provided in a photo studio or one's own equipment.

[0005] The present technology has been made in view of such circumstances, and makes it possible to select a device to be used for retouching a photographed image. [Means for solving the problem]

[0006] According to a first aspect of the present technology, an information processing system includes an information processing device and an image processing device connected via a network. The information processing device includes an image acquisition unit that acquires multiple captured images, a display control unit that displays a setting screen used to set whether retouching work will be performed using the information processing device that uploads the captured images or using another device that accesses the captured images based on access destination information notified by the image processing device after uploading the captured images, and a transmission control unit that uploads the multiple captured images by transmitting them to the image processing device. If the retouching work is set to be performed using the information processing device, the display control unit displays a retouching screen used for the retouching work after uploading some of the captured images. The image processing device also includes a reception control unit that receives the multiple captured images uploaded from the information processing device, an image management unit that manages the received captured images, and a transmission control unit that, if the retouching work is set to be performed using the other device, transmits the access destination information after uploading some of the captured images based on customer information that is information about a customer who will receive the retouched captured images.

[0007] An information processing device according to a second aspect of the present technology includes an image acquisition unit that acquires a plurality of captured images, a display control unit that displays a setting screen used to set whether retouching work will be performed using a first device that uploads the captured images, or using a second device that accesses the captured images based on access destination information notified by a server after the captured images have been uploaded, and a transmission control unit that uploads the captured images by sending the plurality of captured images to the server as the upload destination, and when the retouching work is set to be performed using the first device, the display control unit displays a retouching screen used for the retouching work after some of the captured images have been uploaded.

[0008] An image processing device according to a third aspect of the present technology includes a reception control unit that receives a plurality of photographed images uploaded from an information processing device that is a first device that acquires the photographed images and sets whether retouching work will be performed using a first device that uploads the photographed images or using a second device that accesses the photographed images based on access destination information notified by a server after the photographed images are uploaded, an image management unit that manages the received photographed images, and a transmission control unit that, if the retouching work is set to be performed using the second device, transmits information about the access destination after uploading some of the photographed images based on customer information that is information about a customer who will be provided with the retouched photographed images.

[0009] In the information processing device of the present technology, a plurality of captured images are acquired, and a setting screen is displayed for setting whether retouching work should be performed using the information processing device that uploads the captured images or using another device that is accessed based on access destination information notified by the image processing device after the captured images are uploaded. When uploading is performed by sending the plurality of captured images to the image processing device and it is set that retouching work should be performed using the information processing device, a process is performed to display a retouching screen used for the retouching work after uploading some of the captured images. Furthermore, the image processing device receives the plurality of captured images uploaded from the information processing device and manages the received captured images. When it is set that retouching work should be performed using the other device, the access destination information is sent after uploading some of the captured images based on customer information, which is information about the customer who will receive the retouched captured images. [Effects of the Invention]

[0010] According to the present technology, it is possible to select a device to be used for retouching a captured image. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present technology. [Figure 2] FIG. 1 is a diagram illustrating an example of relationships between users of an information processing system. [Figure 3] FIG. 1 is a diagram showing the flow of work and processing. [Figure 4] FIG. 4 is a diagram showing the flow following FIG. 3. [Figure 5] 1A to 1C are diagrams illustrating examples of retouching work forms. [Figure 6] FIG. 1 illustrates the main differences between the functions of off-site and in-site retouching. [Figure 7] FIG. 10 is a diagram showing a sequence when off-site retouching is used. [Figure 8] FIG. 10 is a diagram showing a sequence when in-store retouching is used. [Figure 9] 10 is a flowchart illustrating processing of an in-store terminal during uploading. [Figure 10] 10 is a flowchart continuing from FIG. 9. [Figure 11] FIG. 10 is a diagram illustrating an example of a customer list screen. [Figure 12] FIG. 10 is a diagram illustrating an example of a setting screen. [Figure 13] FIG. 10 is a diagram illustrating an example of an image selection screen. [Figure 14] FIG. 10 is a diagram illustrating an example of an upload screen. [Figure 15] FIG. 10 is a diagram illustrating an example of an upload screen. [Figure 16] FIG. 10 is a diagram illustrating an example of an upload screen. [Figure 17] 10 is a flowchart illustrating a process of the image processing server during uploading. [Figure 18] 10 is a flowchart illustrating a retouching process of an in-store terminal. [Figure 19] FIG. 10 is a diagram showing an example of an in-store retouch screen. [Figure 20] FIG. 10 is an enlarged view showing a portion of the in-store retouch screen. [Figure 21]10 is a flowchart illustrating a retouching process of a customer terminal. [Figure 22] FIG. 10 is a diagram showing an example of an identity verification screen. [Figure 23] FIG. 10 is a diagram showing an example of an outside-store retouch screen. [Figure 24] 10 is a flowchart illustrating a retouching process of the image processing server. [Figure 25] FIG. 2 is a block diagram illustrating an example of the hardware configuration of an image processing server. [Figure 26] FIG. 2 is a block diagram illustrating an example of a functional configuration of an image processing server. [Figure 27] 10A and 10B are diagrams illustrating examples of image management information and store setting information. [Figure 28] FIG. 2 is a block diagram illustrating an example of a functional configuration of an in-store terminal. [Figure 29] FIG. 10 is a diagram showing an example of a screen displayed when a retouching operation is interrupted. [Figure 30] FIG. 10 is a diagram showing an example of a screen displayed when retouching work is resumed. [Figure 31] FIG. 10 is a diagram showing an example of a screen displayed before a select image is selected. [Figure 32] FIG. 10 is a diagram illustrating an example of calculation of a face ratio. [Figure 33] FIG. 10 is a diagram illustrating an example of complete reflection. [Figure 34] 10A and 10B are diagrams illustrating an example of taking over retouching parameters. [Figure 35] FIG. 10 is a diagram showing an example of a screen when retouching parameters are taken over. [Figure 36] FIG. 10 is a diagram showing another example of an outside-store retouch screen. [Figure 37] FIG. 10 is a diagram showing another example of an outside-store retouch screen. [Figure 38] FIG. 10 is a diagram showing an example of a display of a sheet. [Figure 39] FIG. 10 is a diagram showing another display example of the sheet. DETAILED DESCRIPTION OF THE INVENTION

[0012] <<Photo retouching service>> <System configuration> FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present technology.

[0013] 1, for example, a function is realized to retouch a captured image, which is a still image (photograph) taken in a photo studio, and to provide the retouched captured image to the photo studio. Retouching (processing) a captured image includes various image processing steps for adjusting the appearance of the captured image.

[0014] Photo studios offer photography services for a variety of events, such as wedding commemorative photos, entrance / graduation photos, coming-of-age ceremonies, and Shichi-Go-San (7-5-3) ceremonies. They also offer a variety of photography services, such as ID photos and family photos. In all photography services, the client is the subject of the photo. The subject may be one person or multiple people.

[0015] 1, the information processing system 1 includes an image processing server 11, an in-store terminal 12, and a customer terminal 13. Each device is connected via a network such as the Internet.

[0016] The image processing server 11 is an image processing device managed by a retouching service provider. The retouching service is a service that retouches photographed images and provides the retouched photographed images. The image processing server 11 is composed of one computer or multiple computers. The image processing server 11 transmits and receives various data to and from the in-store terminal 12 and the customer terminal 13.

[0017] The in-store terminal 12 is an information processing device installed in a photo studio. The in-store terminal 12 is configured by a computer such as a PC on which various applications such as a browser are installed. The in-store terminal 12 is used to manage information about customers who visit the photo studio to have their photos taken, and to manage the images taken of the customers after their photos have been taken. A user who operates the in-store terminal 12 is, for example, a staff member of the photo studio. As will be described later, the in-store terminal 12 may also be used by a customer. The in-store terminal 12 transmits and receives various data to and from the image processing server 11.

[0018] When photography is performed outside a photo studio, the in-store terminal 12 may be configured by a computer different from the computer installed in the photo studio, such as a PC used by the photographer.

[0019] The customer terminal 13 is a device used by customers who visit the photo studio and have their photographs taken. The customer terminal 13 is composed of an information processing terminal such as a PC, smartphone, or tablet terminal on which various applications such as a browser are installed. The customer terminal 13 transmits and receives various data to and from the image processing server 11.

[0020] FIG. 2 is a diagram showing an example of relationships between users of the information processing system 1. As shown in FIG.

[0021] As indicated by arrow A1, a customer who wishes to have commemorative photos taken applies to the photo studio management company for the photo shoot and pays the specified fee. As indicated by arrow A2, the photo studio management company provides the customer with photo studio photography services and also provides the captured images. The captured images provided to the customer are, for example, retouched images. The retouched captured images are provided to the customer either printed on a print medium in the form of an album, or in the form of data. Depending on the plan the customer subscribes to, both the album and the data may be provided.

[0022] As indicated by arrow A3, after the photo studio management company takes a photo of a customer, it provides the photographed image to the retouching service provider. The photo studio management company also pays the retouching service fee to the retouching service provider. When the full-count update process (described later) is performed in image processing server 11, the retouching service is counted as having been used, and the retouching service fee is incurred.

[0023] The use of the retouching service may be counted at other times, such as when the customer finishes the retouching work or when the photo studio operating company receives the retouched photographed image from the retouching service provider. The time when the in-store terminal 12 downloads the retouched photographed image from the image processing server 11 becomes the time when the photo studio operating company receives the retouched photographed image from the retouching service provider. The usage fee for the retouching service may be changed depending on the number of photographed images received by the photo studio operating company, such as the number of photographed images that have been retouched by full reflection or the number of retouched photographed images downloaded from the image processing server 11.

[0024] As indicated by arrow A4, the retouching service provider retouches the photographed images provided by the photo studio operating company, and provides the retouched photographed images to the photo studio operating company.

[0025] As indicated by arrow A5, a retouching service provider provides a customer with a function for retouching photographed images. In the information processing system 1, for example, a customer who has had their photograph taken at a photo studio performs the retouching work themselves. The customer can use the retouching function to retouch the image to their liking. Retouching work may also be performed by someone other than the customer, such as a photo studio staff member. Retouching work refers to manual operations by a person for retouching as image processing in the image processing server 11.

[0026] <Work and processing flow> 3 and 4 are diagrams showing a series of steps from when a photo is taken in a photo studio until the customer receives the retouched photographed image.

[0027] As shown in the upper part of Figure 3, photography in a photo studio is performed by a photographer. In the example of Figure 3, a commemorative wedding photo is being taken with the bride and groom standing in front of a background screen as the subjects. The bride and groom are the customers. Around the bride and groom, equipment such as strobe devices that emit light in time with the photographer's photography is placed.

[0028] The photographer takes a large number of images, such as 100 or 200, by changing the composition. Here, it is assumed that 100 images are taken. The composition of the captured images is determined by the poses of the bride and groom, the relative positions of the bride and groom and the photographer, and camera parameters such as focal length. The images taken by the photographer are transferred to the in-store terminal 12 via another computer connected to the camera, and are saved in association with customer information.

[0029] After the images are taken, as indicated by the tip of arrow A11, 100 taken images are uploaded from in-store terminal 12 to image processing server 11. The uploaded taken images are saved in the storage of image processing server 11.

[0030] After uploading the captured image, as indicated by the arrow A12, the customer operates the customer terminal 13 to access the image processing server 11 and perform the retouching work. When the upload of the captured image is complete, the image processing server 11 sends the customer a notification including information such as a URL to be accessed when performing the retouching work. When the customer performs an operation such as selecting a URL on the customer terminal 13 and accesses the image processing server 11, a retouching screen to be used for the retouching work is displayed on the display of the customer terminal 13.

[0031] The customer's retouching work is performed on a portion of the total number of uploaded images, such as five. For example, before starting the upload, the customer selects the images to be retouched. Using the functions available on the retouching screen, the customer retouches the selected images one by one. Each time a retouching operation is performed, the details of the customer's operation are reflected on the display on the retouching screen, and retouching parameters, which are parameters indicating the details of the customer's operation, are set for the image being retouched.

[0032] After the photographed images have been retouched, the image processing server 11 performs a full reflection, as shown in the upper part of Fig. 4. Full reflection is a process in which the same retouching as that performed on some of the photographed images selected for retouching is reflected on unretouched photographed images that have not been selected for retouching. If five photographed images have been selected for retouching, full reflection is performed on the remaining 95 photographed images.

[0033] As will be described in detail later, in full reflection, for each unretouched photographed image that serves as the reflection destination, a retouched photographed image that contains the same person as the person appearing in the reflection destination photographed image is selected as the reflection source photographed image. The reflection source photographed image is selected for each reflection destination photographed image based on various characteristics of the photographed image, such as the size and orientation of the face, the number of people in the subject, etc. The full reflection is performed on each unretouched photographed image by applying the retouching parameters set for the reflection source photographed image to the reflection destination photographed image (by performing image processing defined by the retouching parameters). The photographed image that has undergone full reflection becomes a retouched photographed image.

[0034] After all the images have been reflected in the image processing server 11, the retouched 100 photographed images are downloaded to the in-store terminal 12, as indicated by the tip of arrow A13.

[0035] The downloaded photographed images are provided to the customer as an album or data, as indicated by the arrow A14. Before providing the photographed images, the photo studio staff visually checks the appearance of the retouched images.

[0036] In this way, in the information processing system 1, retouching work is performed on the customer's device, the customer terminal 13. The customer can perform retouching work using a device different from the device used to capture the photographed image. After finishing photography at the photo studio, the customer can leave the studio without retouching work and perform retouching work at their own convenience, such as at home.

[0037] Since all images are reflected in the image processing server 11, the customer can obtain retouched images for all the photographed images by performing retouching work on only some of the photographed images.

[0038] <Out-of-store retouching and in-store retouching> FIG. 5 is a diagram showing an example of a form of retouching work.

[0039] While retouching work is performed using a customer terminal 13 as shown in FIG. 5A, it can also be performed using an in-store terminal 12 as shown in FIG. 5B. When retouching work is performed using an in-store terminal 12, a customer who has finished taking photos moves to a retouching space in a photo studio, for example, and performs the retouching work by operating an in-store terminal 12 provided in the retouching space. In the retouching space, for example, multiple in-store terminals 12 are provided. A retouching screen similar to the retouching screen displayed on the customer terminal 13 is displayed on the in-store terminal 12 after the photographed image is uploaded, and is used for the retouching work.

[0040] In this way, there are two types of retouching work available: one in which the retouching work is performed using a customer terminal 13, which is a device (another device) different from the device used to upload the photographed image, and one in which the retouching work is performed using an in-store terminal 12, which is the same device as the device used to upload the photographed image.

[0041] As described above, retouching using the customer terminal 13 is performed outside the store, such as at the customer's home. Retouching using the in-store terminal 12 is performed inside the photo studio where the in-store terminal 12 is installed. The retouching location (the device to be used for the retouching) is set at a predetermined time, such as before uploading begins.

[0042] Hereinafter, retouching work using the customer terminal 13 will be referred to as "out-of-store retouching," and retouching work using the in-store terminal 12 will be referred to as "in-store retouching."

[0043] FIG. 6 shows the main differences between the functions of off-site retouching and in-site retouching.

[0044] 6 shows the functions for out-of-store retouching, and the third row shows the functions for in-store retouching. As shown in the second column from the left, the device used for out-of-store retouching is the customer terminal 13, and the device used for in-store retouching is the in-store terminal 12.

[0045] As shown in the third column from the left, there is no time limit for off-site retouching. On the other hand, there is a time limit for in-store retouching. With off-site retouching, customers can complete the retouching work without worrying about time, but with in-store retouching, the retouching work must be completed within the time limit.

[0046] As shown in the fourth column from the left, in off-site retouching, it is possible to interrupt the retouching work by closing the retouching screen display, for example. When retouching work is interrupted, the retouching parameters set up to that point are saved. When retouching work is resumed, the retouching work is resumed from the state at the time of interruption. In-store retouching does not allow interruptions of retouching work in this way.

[0047] As shown in the fifth column from the left, identity verification is performed before starting out-of-store retouching. On the other hand, identity verification is not performed before starting in-store retouching. Identity verification is performed by presenting a portion of the captured image.

[0048] The functions of off-site retouching and in-site retouching will be described in detail later. Note that retouching work may also be performed inside the photo studio using the customer terminal 13. This retouching work is also considered off-site retouching because it is performed using a device different from the device used to upload the photographed image.

[0049] <<Overall processing flow in the information processing system>> Here, we will explain the basic flow of processing that takes place between the devices in the information processing system 1. Details of the processing of each device will be described later with reference to flowcharts and the like.

[0050] <Out-of-store retouching> Fig. 7 shows the sequence when "outside store retouching" is selected as the retouching location. Each process shown in Fig. 7 is performed in parallel with other processes or in a different order than other processes, as appropriate. The same applies to the sequence in Fig. 8.

[0051] 7 is started after a photo shoot at a photo studio, for example. After the photo shoot at the photo studio, customer information including information such as name, telephone number, and email address is registered by, for example, the customer. The customer operates customer terminal 13 to access image processing server 11, and registers the customer information using a registration site, which is a website displayed by the browser of customer terminal 13.

[0052] The image processing server 11 acquires the customer information entered into the registration site and manages it in association with the photo studio. In this way, the information on customers of each photo studio is managed in the image processing server 11. The customer information may be registered before photography begins. Alternatively, the customer information may be registered by photo studio staff using the in-store terminal 12.

[0053] In step S1, the in-store terminal 12 accesses the image processing server 11 and performs retouching settings. For example, when a photo studio staff member operates the in-store terminal 12 to launch a browser and selects a customer who will upload a photographed image after logging in to the website, a setting screen used for retouching settings is displayed on the display of the in-store terminal 12. Each photo studio is provided with an account to be used for logging in.

[0054] In this way, the screen operated by the photo studio staff is basically a screen displayed by the browser of the in-store terminal 12 based on information transmitted from the image processing server 11. Various settings such as retouching locations are made using the setting screen.

[0055] In step S11, the image processing server 11 receives and manages the information entered on the setting screen. Information indicating the retouching settings is managed in association with the uploaded photographed image as image management information for managing the uploaded photographed image. In this way, it is possible to change the retouching location settings for each upload.

[0056] In step S2, the in-store terminal 12 captures the captured images and stores and manages them in a predetermined folder. For example, 100 captured images are captured.

[0057] In step S3, the in-store terminal 12 uploads the captured image.

[0058] In step S12, the image processing server 11 receives the photographed image uploaded from the in-store terminal 12, and stores it in step S13.

[0059] In step S14, the image processing server 11 transmits an access destination notification based on the customer information. If an email address is registered as the customer information, the access destination notification is transmitted by email. In the example of Fig. 7, the customer terminal 13 is the destination of the access destination notification.

[0060] The access destination notification may be sent via a medium other than email, such as an SMS message, a social media message, etc. The access destination notification includes a URL to be accessed when performing the retouching work.

[0061] As will be described later, the access destination notification is sent when the upload of some of the photographed images to be retouched is completed. After the upload of some of the photographed images to be retouched is completed, the upload of photographed images that are not to be retouched continues.

[0062] In step S31, the customer terminal 13 receives the access destination notification sent from the image processing server 11. A URL may be provided to the customer terminal 13 from another device that has received the access destination notification.

[0063] In step S32, the customer terminal 13 accesses the image processing server 11 based on the URL included in the access destination notification in response to an operation by the customer.

[0064] In step S15, the image processing server 11 accepts access from the customer terminal 13, and in step S16, displays a retouch screen.

[0065] In step S33, the customer terminal 13 receives the data transmitted from the image processing server 11 and displays a retouch screen. The screen displayed on the customer terminal 13 is also basically a screen displayed by the browser of the customer terminal 13 based on the information transmitted from the image processing server 11.

[0066] In step S34, the customer terminal 13 transmits to the image processing server 11 information indicating the content of the customer's operation on the retouch screen.

[0067] In step S17, the image processing server 11 receives the information transmitted from the customer terminal 13, and performs retouching processing in step S18. The captured image is retouched by performing image processing in accordance with the customer's operation.

[0068] When the retouching process is completed, in step S19, the image processing server 11 performs a full retouching process. The same retouching as that performed on some of the photographed images selected as targets for the retouching process is applied to photographed images that are not selected as targets for the retouching process.

[0069] When all images have been reflected, in step S20, the image processing server 11 transmits a download available notification to the in-store terminal 12. The download available notification is information notifying that the captured images are now available for download.

[0070] In step S4, the in-store terminal 12 receives the notification that downloading is possible. The fact that the captured image is now available for download is displayed, for example, on a customer list screen.

[0071] In step S5, the in-store terminal 12 requests the image processing server 11 to download the captured images in response to an operation by the photo studio staff member.

[0072] In step S21, the image processing server 11 receives a request from the in-store terminal 12, and in step S22, transmits the retouched photographed image to the in-store terminal 12.

[0073] In step S6, the in-store terminal 12 receives and downloads the retouched photographed images transmitted from the image processing server 11. The downloaded retouched photographed images are stored, for example, in a predetermined folder in the in-store terminal 12. When downloading of all retouched photographed images is complete, the series of processes ends. Through the above processes, the retouched photographed images obtained through off-site retouching are provided to the in-store terminal 12.

[0074] <In-store retouching> Fig. 8 is a diagram showing a sequence when in-store retouching is selected as the setting of the retouching location. Of the processes shown in Fig. 8, descriptions of processes that are the same as the processes shown in Fig. 7 will be omitted as appropriate.

[0075] In step S51, the in-store terminal 12 accesses the image processing server 11 and performs retouching settings.

[0076] In step S71, the image processing server 11 receives the information input on the setting screen and manages it as image management information.

[0077] In step S52, the in-store terminal 12 captures the captured image and stores it in a predetermined folder for management.

[0078] In step S53, the in-store terminal 12 uploads the captured image.

[0079] In step S72, the image processing server 11 receives the photographed image uploaded from the in-store terminal 12, and stores it in step S73.

[0080] In step S54, the in-store terminal 12 transmits a request to start retouching to the image processing server 11. The request to start retouching is transmitted, for example, in response to a photo studio staff member operating a button that is operated to start retouching work. The retouching start button that is operated to start retouching work is displayed on the screen of the in-store terminal 12.

[0081] As will be described later, the retouching start button is displayed when the upload of the photographed image to be retouched is completed. After the upload of the photographed image to be retouched is completed, the upload of the photographed image not to be retouched continues.

[0082] In step S74, the image processing server 11 receives the request from the in-store terminal 12, and in step S75, displays a retouch screen.

[0083] In step S55, the in-store terminal 12 receives the data transmitted from the image processing server 11 and displays a retouching screen. The customer performs retouching work using the retouching screen displayed on the in-store terminal 12. For example, a photo studio staff member operates the in-store terminal 12 until the retouching screen is displayed, and then hands the in-store terminal 12 over to the customer.

[0084] In step S56, the in-store terminal 12 transmits to the image processing server 11 information indicating the content of the customer's operation on the retouch screen.

[0085] In step S76, the image processing server 11 receives the information transmitted from the in-store terminal 12, and performs retouching processing in step S77. The captured image is retouched by applying image processing in accordance with the customer's operation.

[0086] When the retouching process is completed, the image processing server 11 performs a full reflection in step S78.

[0087] When all the updates have been completed, in step S79, the image processing server 11 transmits a notification that download is possible to the in-store terminal 12. The subsequent processing is the same as the processing described with reference to FIG.

[0088] That is, in step S57, the in-store terminal 12 receives the notification that download is possible, and in step S58 requests the image processing server 11 to download the captured image.

[0089] In step S80, the image processing server 11 receives a request from the in-store terminal 12, and in step S81 transmits the retouched photographed image to the in-store terminal 12.

[0090] In step S59, the in-store terminal 12 receives and downloads the retouched photographed images transmitted from the image processing server 11. When downloading of all retouched photographed images is complete, the series of processes ends. Through the above process, the retouched photographed images obtained by in-store retouching are provided to the in-store terminal 12.

[0091] Since off-site retouching and in-site retouching are available as retouching location settings for photographed images, customers can select the equipment to use for retouching their photographed images. Customers can choose whether to have retouching done at a time and place of their choice, or at a photo studio after the photograph is taken. Whether the retouching location is off-site retouching or in-site retouching can be selected, for example, according to the customer's preference.

[0092] <<Details of processing for each device>> <Processing at upload> Processing on in-store terminal 12 The processing of the in-store terminal 12 during uploading will be described with reference to the flowcharts of Figures 9 and 10. Descriptions of the same processing as that described with reference to Figures 7 and 8 will be omitted as appropriate. The processing of each step shown in Figures 9 and 10 is performed in parallel with the processing of other steps, or out of order with the processing of other steps, as appropriate. The same applies to other flowcharts. Here, it is assumed that customer information has been registered in advance.

[0093] In step S101, the in-store terminal 12 displays a customer list screen and accepts the selection of a customer to upload a photographed image. The customer list screen is a screen listing customer information. For example, the customer list screen is displayed when a login operation is performed on the website of the image processing server 11.

[0094] FIG. 11 is a diagram showing an example of a customer list screen.

[0095] As shown in Fig. 11, the information in each row displayed in table format is the customer management information of each customer. In the example of Fig. 11, customer management information #1 to #4 are displayed. A customer list screen is displayed based on the information managed in the image processing server 11.

[0096] The customer management information includes information such as the phase, customer name, upload date and time, and person in charge. The phase indicates the status of the customer's captured images. In the example of FIG. 11, the phases of customer management information #1 to #4 are "Uploadable," "Downloaded," "Downloadable," and "Retouching," respectively.

[0097] The "Uploadable" phase indicates a state in which the captured image has not yet been uploaded. The "Downloaded" phase indicates a state in which the captured image has been downloaded. The "Downloadable" phase indicates a state in which all images have been updated and the retouched captured image can be downloaded. The "Retouching in progress" phase indicates that retouching work is currently being carried out. The "Retouching in progress" phase is set when retouching work has started but has been interrupted.

[0098] The upload button 101 displayed in the right column of customer management information #1 is the button that is operated when starting to upload the photographed images of the customer ("Sato"). The upload button 101 is displayed in an operable state in the column of customer management information where the phase is "upload allowed."

[0099] The download button 102 displayed in the right column of customer management information #3 is the button that is operated to start downloading the photographed images of the customer ("Tanaka"). The download button 102 is displayed in an operable state in the column of customer management information where the phase is "downloadable."

[0100] It should be noted that customer management information #3 and customer management information #4 are information about one customer, "Tanaka." In commemorative wedding photography, the photos may be taken on a day other than the wedding ceremony and reception (so-called "pre-wedding photos" and "post-wedding photos"). In the information processing system 1, images taken by the same customer can be managed together in a predetermined unit, such as by date. Images may also be managed together by location, such as images taken at the wedding hall and images taken at the reception.

[0101] In step S101 of FIG. 9, an operation to start uploading a photographed image is accepted for a customer whose phase is "upload allowed."

[0102] For example, when the upload button 101 is operated to instruct the start of uploading a captured image, in step S102 the in-store terminal 12 displays a setting screen and accepts settings before uploading. The retouching settings described with reference to Figures 7 and 8 are the settings before uploading made using the setting screen.

[0103] FIG. 12 is a diagram showing an example of the setting screen.

[0104] The setting screen shown in Fig. 12 is a screen used to set the retouching location. The screen used to set the retouching location displays an in-store button 121 that is operated when selecting in-store retouching, and an out-of-store button 122 that is operated when selecting out-of-store retouching. The photo studio staff will operate either button to set the retouching location.

[0105] The setting screen shown in Fig. 12 is a screen used to set whether retouching work will be performed using the in-store terminal 12 or the customer terminal 13. The in-store terminal 12 is the first device that uploads the captured image, and the customer terminal 13 is the second device that is accessed based on the URL notified by the image processing server 11 as access destination information after the captured image has been uploaded.

[0106] In addition to setting the retouching location, each screen can be used to set the number of photos to select, the retouching deadline, the mode, whether to turn the age estimation function on or off, whether to turn the upper arm slimming function on or off, and the retouching time limit.

[0107] The number of selected images is the number of selected images that will be the subject of retouching. The number of selected images is set before uploading. Any number, such as 5 or 10, can be set as the number of selected images. It is also possible to select all captured images as selected images.

[0108] The retouching deadline is the date and time by which the retouching work is due, and the customer is required to complete the retouching work by the date and time set as the retouching deadline.

[0109] The mode is the customer's photography mode. For example, there are a single-person mode for taking photos by one person and a two-person mode for taking photos with multiple people. For example, the single-person mode is used for taking commemorative photos for coming-of-age ceremonies and ID photos, and the two-person mode is used for taking commemorative photos for weddings and family photos.

[0110] The on / off setting of the age estimation function indicates whether or not to use the function for estimating the age of a person appearing in a captured image. The estimated age is used by the image processing server 11 to set retouching parameters and display a retouching screen. When the age estimation function is set to on, age estimation processing is performed on each captured image in the image processing server 11 after uploading. The image processing server 11 is provided with various machine learning models, such as a model for age estimation, that have been generated by machine learning.

[0111] The on / off setting of the upper arm slimming function indicates whether or not the upper arm slimming function is to be used. The upper arm slimming function is a function that slims down the upper arms of people appearing in photographed images. When the upper arm slimming function is set to on, arm recognition processing is performed on each photographed image in the image processing server 11 after uploading.

[0112] Since age estimation and arm recognition are performed on each captured image, if the age estimation function or upper arm slimming function is on, it will take longer to start retouching than if those functions are off. For example, if in-store retouching is selected, the age estimation function and upper arm slimming function are set to off.

[0113] The retouching time limit is a time limit for retouching work. For example, a retouching time limit is set when in-store retouching is set as the retouching location.

[0114] 9, various settings before uploading, including the setting of the retouching location, are accepted. Information indicating the settings is received by the image processing server 11 and managed as image management information.

[0115] In step S103, the in-store terminal 12 acquires the photographed images to be uploaded. In response to the selection of a folder storing the photographed images, the photographed images stored in the folder are acquired as images to be uploaded.

[0116] In step S104, the in-store terminal 12 displays an image selection screen and accepts the selection of a select image. The image selection screen is a screen used to select a select image. As described above, a select image is a photographed image that is the target of retouching work. Of all the photographed images of a certain customer, photographed images other than the select image are not the target of retouching work, but are the target of setting retouching parameters by reflecting all of them.

[0117] Hereinafter, the captured images for which retouching parameters are set by full reflection will be referred to as “reflected images.” In other words, selecting selected images is the process of dividing, for example, 100 captured images into 5 selected images and 95 reflected images.

[0118] FIG. 13 is a diagram showing an example of the image selection screen.

[0119] A message is displayed at the top of the image selection screen, instructing you to select five captured images. The image selection screen in Figure 13 is the screen that is displayed when "5" is set as the number of images to select.

[0120] A horizontally long thumbnail area 111 is provided on the right side of the image selection screen. Thumbnails of the captured images are displayed side by side in the thumbnail area 111. By switching pages, the thumbnails displayed in the thumbnail area 111 are switched. One captured image selected using the thumbnails displayed in the thumbnail area 111 is enlarged and displayed in the display area 112 on the left side.

[0121] Selection area 113 provided below the image selection screen is a display area for thumbnails of photographed images that have been selected as selected images. When selection button 112A provided at the bottom right of display area 112 is operated, the photographed image being displayed in display area 112 is selected as the selected image. When "5" is set as the number of selected images, five photographed images are selected using this screen.

[0122] When the selection of the select image is completed, the in-store terminal 12 starts uploading the select image in step S105 of Fig. 9. The upload of the photographed image is performed in such a way that the select image is uploaded first, and then the reflected image is uploaded after the upload of the select image.

[0123] In step S106, the in-store terminal 12 determines whether or not the upload of the selected image has been completed.

[0124] If it is determined in step S106 that the upload of the selected images is not complete, the in-store terminal 12 displays an upload screen in step S107. The upload screen is a screen that includes information indicating the progress of the upload.

[0125] FIG. 14 is a diagram showing an example of the upload screen.

[0126] At the top of the upload screen shown in Figure 14, information is displayed indicating that the second of five selected images is currently being uploaded. Information displayed at the bottom of the upload screen indicates the upload status of the reflected image. In Figure 14, "Selected Photo" represents the selected image, and "Exported Photo" represents the reflected image.

[0127] This upload screen is displayed while the upload is in progress. The process returns to step S106 in Fig. 9 and the upload of the selected images continues.

[0128] On the other hand, if it is determined in step S106 that the upload of the selected image has been completed, the in-store terminal 12 starts uploading the reflected image in step S108.

[0129] In step S109 of FIG. 10, the in-store terminal 12 determines whether or not in-store retouching has been selected as the setting of the retouching location.

[0130] If it is determined in step S109 that in-store retouching has been selected, then in step S110 the in-store terminal 12 displays an upload screen including a retouching start button.

[0131] FIG. 15 is a diagram showing an example of an upload screen including a retouch start button.

[0132] When the upload of the selected images is complete, information indicating this is displayed at the top of the upload screen, as shown in Fig. 15. In addition, a retouch start button 131 is displayed. When the retouch start button 131 is operated, the retouch screen is displayed in place of the upload screen.

[0133] At the bottom of the upload screen in FIG. 15, information indicating that uploading of the second reflected image is in progress is displayed.

[0134] In other words, when the upload of the selected image is completed, the retouching work can be started before the upload of the reflected image is completed. When an instruction to start the retouching work is given, the upload of the reflected image continues behind the scenes while the customer is retouching the image.

[0135] When the retouching start button 131 is operated to instruct the start of retouching work, the in-store terminal 12 starts retouching work in parallel with the process of Figure 10. The retouching work performed by the in-store terminal 12 is a process of displaying a retouching screen and accepting retouching work by the customer. The retouching work will be described later.

[0136] In step S111, the in-store terminal 12 determines whether or not the upload of the reflected image has been completed.

[0137] If it is determined in step S111 that the upload of the reflected image is not complete, the process returns to step S110 and the subsequent processes are repeated. At the in-store terminal 12, the upload of the reflected image and the display of the upload screen continue.

[0138] On the other hand, if it is determined in step S109 that out-of-store retouching has been selected, the in-store terminal 12 displays an upload screen in step S112.

[0139] FIG. 16 is a diagram showing an example of the upload screen displayed in step S112.

[0140] At the top of the upload screen in Figure 16, information indicating that the upload of the selected images has been completed is displayed. Also displayed is information indicating that an access destination notification containing a URL to be accessed when retouching the images will be sent.

[0141] At the bottom of the upload screen in FIG. 16, information indicating that uploading of the second reflected image is in progress is displayed.

[0142] That is, when the upload of the selected image is completed, an access destination notification is sent from the image processing server 11 before the upload of the reflected image is completed, so that the retouching work can be started. If off-site retouching is selected, the retouching screen is not displayed on the in-store terminal 12.

[0143] In step S113 of FIG. 10, the in-store terminal 12 determines whether or not uploading of the reflected image has been completed.

[0144] If it is determined in step S113 that the upload of the reflected image is not complete, the process returns to step S112 and the subsequent processes are repeated. At the in-store terminal 12, the upload of the reflected image and the display of the upload screen continue.

[0145] If it is determined in step S111 or step S113 that the upload of the reflected image is complete, the upload process ends.

[0146] As described above, whether in-store or out-of-store retouching is used, retouching work can begin as soon as the upload of the selected image is complete. If retouching work could not begin until the upload of all captured images, including the reflected image, was complete, customers would have to wait a long time until retouching work could begin, but this can be prevented.

[0147] An upload program, which is a program dedicated to uploading captured images, may be provided in the in-store terminal 12. The upload program is started when an operation to start uploading is performed on a screen displayed by a browser, for example, and executes uploading of the captured images.

[0148] The upload program continues to operate and uploads captured images even if the browser is closed. By providing the upload program, it becomes possible to continue uploading even if, for example, the browser display is closed after uploading has started.

[0149] Image processing server 11 processing The processing of the image processing server 11 at the time of uploading will be described with reference to the flowchart in Fig. 17. Descriptions of processing that is the same as the processing described with reference to Figs. 7 and 8 will be omitted as appropriate. The processing shown in Fig. 17 corresponds to the processing of the in-store terminal 12 described with reference to Figs. 9 and 10. For example, the processing shown in Fig. 17 starts when a login operation is performed on the in-store terminal 12.

[0150] In step S151, the image processing server 11 displays screens including a customer list screen, a setting screen, and an image selection screen in response to an operation on the in-store terminal 12. This processing corresponds to the processing of steps S101, S102, and S104 in Fig. 9. The image processing server 11 receives information indicating the settings made using the setting screen, and manages the information as image management information.

[0151] In step S152, the image processing server 11 receives and saves the selected images uploaded from the in-store terminal 12.

[0152] In step S153, the image processing server 11 determines whether or not the upload of the selected images has been completed.

[0153] If it is determined in step S153 that the upload of the select images is not complete, then in step S154 the image processing server 11 causes an upload screen to be displayed on the in-store terminal 12. This process corresponds to the process of step S107 in Fig. 9. The process returns to step S153, and reception of the uploaded select images continues.

[0154] On the other hand, if it is determined in step S153 that the upload of the select image has been completed, then in step S155 the image processing server 11 receives and saves the reflection image that was uploaded following the select image.

[0155] In step S156, the image processing server 11 determines whether or not in-store retouching has been selected as the setting of the retouching location.

[0156] If it is determined in step S156 that in-store retouching has been selected, in step S157 the image processing server 11 causes an upload screen including a retouching start button to be displayed on the in-store terminal 12. This process corresponds to the process of step S110 in Fig. 10. The in-store terminal 12 can then start the retouching work.

[0157] When a retouching start button is operated on the in-store terminal 12 to request the start of retouching work, the image processing server 11 starts retouching work in parallel with the process of Fig. 17. The retouching work performed by the image processing server 11 involves displaying a retouching screen for in-store retouching on the in-store terminal 12 and retouching each selected image in accordance with the retouching work performed by the customer. The retouching work will be described later.

[0158] In step S158, the image processing server 11 determines whether or not the upload of the reflected image has been completed.

[0159] If it is determined in step S158 that the upload of the reflected image has not been completed, the process returns to step S157, and the subsequent processes are repeated. The image processing server 11 continues to receive the reflected image.

[0160] On the other hand, if it is determined in step S156 that off-site retouching has been selected, in step S159, the image processing server 11 sends an access destination notice to the customer terminal 13 based on pre-registered customer information. If the upload of the reflected image has not been completed, the access destination notice will be sent before the upload of the reflected image is completed. The customer terminal 13 can start the retouching work by accessing the image processing server 11 based on the URL included in the access destination notice.

[0161] When access is made based on the URL and a request to start retouching is made from customer terminal 13, image processing server 11 starts retouching in parallel with the processing of Fig. 17. The retouching process performed by image processing server 11 involves displaying a retouching screen for off-site retouching on customer terminal 13 and retouching each selected image in accordance with the retouching work requested by the customer. The retouching process will be described later.

[0162] In step S160, the image processing server 11 displays an upload screen on the in-store terminal 12. This process corresponds to the process of step S112 in FIG.

[0163] In step S161, the image processing server 11 determines whether the upload of the reflected image has been completed. If it is determined in step S161 that the upload of the reflected image has not been completed, the process returns to step S160, and the subsequent processes are repeated. The image processing server 11 continues to receive the uploaded reflected image.

[0164] If it is determined in step S158 or step S161 that the upload of the reflected image is complete, the upload process ends.

[0165] <Retouching> Processing on in-store terminal 12 (in-store retouching) The retouching process of the in-store terminal 12 will be described with reference to the flowchart in Fig. 18. Since this is retouching process at the in-store terminal 12, the process shown in Fig. 18 is the process for performing in-store retouching. The process shown in Fig. 18 is performed in parallel with the uploading of the reflected image as appropriate.

[0166] In step S201, the in-store terminal 12 displays an in-store retouching screen based on the information transmitted from the image processing server 11. The display of the retouching screen begins before the upload of the reflected image is completed. The in-store retouching screen is a retouching screen used for in-store retouching.

[0167] FIG. 19 is a diagram showing an example of an in-store retouch screen.

[0168] A display area 201 is formed in approximately the center of the in-store retouching screen. The display area 201 is an area in which one photographed image selected as the target for retouching is enlarged and displayed. A thumbnail area 202 formed above the display area 201 displays a thumbnail of the photographed image selected as the selected image. In the example of FIG. 19, thumbnails of photographed images P1 to P5 are displayed. For example, photographed image P1 selected using the thumbnail is enlarged and displayed in the display area 201.

[0169] On the left side of the in-store retouch screen, from the top, a comparison button 203, a person selection button 204, and a batch retouch button 205 are displayed.

[0170] The compare button 203 is a button that is operated when switching the display in the display area 201 to the state before retouching. While the compare button 203 is being operated, the captured image in the state before retouching is displayed in the display area 201. By using the compare button 203, it is possible to compare the state after retouching with the state before retouching and check whether the degree of retouching is too strong, for example.

[0171] The person selection button 204 is a button used to select a person to be retouched. By using the person selection button 204, it is possible to select whether to retouch only the person appearing on the left side of the selected photographed image, or only the person appearing on the right side. The person selection button 204 is displayed, for example, when a two-person shooting mode is selected.

[0172] The batch retouch button 205 is a button used for batch retouching. Batch retouching is a function for selecting a combination of multiple types of retouching all at once. In the example of FIG. 19, the parameter levels for face slimming, skin texture, and eye size are selected. By using the batch retouching function, the retouching details can be selected with a single operation, without having to select each retouching detail individually.

[0173] A vertically long retouching selection area 206 is formed on the right side of the in-store retouching screen. The retouching content is selected using operation images such as buttons and slide bars displayed in the retouching selection area 206. Each time a retouching content is selected using the operation images displayed in the retouching selection area 206, the selected content is reflected in the display in the display area 201. The customer selects the retouching content for the captured image while viewing the display in the display area 201.

[0174] Fig. 20 is an enlarged view of the display of retouching selection area 206. In the example of Fig. 20, the "Face / Body" tab is selected, and operation images used for retouching related to "Face / Body" are displayed in retouching selection area 206. When the "Brightness" tab is selected, the display of retouching selection area 206 switches to a state in which operation images used for retouching related to "Brightness" are displayed.

[0175] The retouch selection area 206 is formed from an area 206-1 which is a display area for an operation image used for adjusting the entire face and body, and an area 206-2 which is a display area for an operation image used for adjusting the eyes and makeup.

[0176] Area 206-1 displays adjustment buttons 231 to 236 used to adjust skin type, small face, face shape, chin length, slim upper arms (thinness of arms), and style.

[0177] Five levels (degree of adjustment) of skin texture are available for adjustment using the adjustment button 231. Skin texture "0" indicates that the original skin texture is left unchanged. Skin texture "1" indicates that the skin texture level is increased by one level based on the parameter corresponding to "1." By selecting one of levels "1" to "4," the degree of smoothness of the skin of the person appearing in the captured image displayed in the display area 201 is adjusted.

[0178] There are five levels of small face feeling that can be adjusted using the adjustment button 232. Small face feeling "0" indicates that the size of the face is not changed and remains at the original size. Small face feeling "1" indicates that the size of the face is reduced by one level based on the parameter corresponding to "1". By selecting one of the levels "1" to "4", the size of the face in the captured image displayed in the display area 201 is adjusted.

[0179] The face shapes that can be adjusted using the adjustment button 233 include "round," "egg," and "slim." By selecting one of these shapes, the shape of the face that appears in the captured image displayed in the display area 201 is adjusted. By selecting "no retouching," it is also possible to leave the shape of the face unchanged.

[0180] Three levels of chin length are available for adjustment using adjustment button 234. A chin length of "0" indicates that the chin length is not changed and remains at its original length. A chin length of "1" indicates that the chin length is shortened by one level based on the parameter corresponding to "1." By selecting one of the levels "0" to "2," the chin length of the face appearing in the photographed image displayed in display area 201 is adjusted.

[0181] Four levels of arm thinness are available that can be adjusted using adjustment button 235. Arm thinness "0" indicates no processing, i.e., the arm thinness is not changed and remains the original thinness. Arm thinness "1" indicates that processing is performed based on the parameter corresponding to "1", making the arm thinness one level thinner. Arm thinness "2" indicates that processing is performed based on the parameter corresponding to "2", making the arm thinness one level thinner than "1". By selecting one of the levels, the thinness of the arm in the captured image displayed in display area 201 is adjusted.

[0182] Three levels of style enhancement are available that can be adjusted using the adjustment button 236. Style enhancement "0" indicates no processing, i.e., the style is left unchanged and remains the original style. Furthermore, style enhancement "weak" indicates that processing, such as making the legs longer, is performed based on the parameters corresponding to "weak," improving the overall balance by one level. By selecting one of the levels, the style of the person appearing in the photographed image displayed in the display area 201 is adjusted.

[0183] Area 206-2 displays adjustment buttons 241 to 245 used to adjust the size and shape of the eyes, the strength of the makeup, the bridge of the nose, and the downy hair around the mouth.

[0184] Five levels of eye size are available for adjustment using the adjustment button 241. Eye size "0" indicates no processing, i.e., the eye size is left unchanged at its original size. Eye size "1" indicates that the eye size is increased by one level based on the parameter corresponding to "1." By selecting one of levels "1" to "4," the size of the eyes in the captured image displayed in the display area 201 is adjusted.

[0185] The eye shapes that can be adjusted using the adjustment button 242 include "round eyes," "droopy eyes," and "cat eyes." By selecting one of these shapes, the eye shape that appears in the photographed image displayed in the display area 201 is adjusted. By selecting "no processing," it is also possible to leave the eye shape unchanged.

[0186] Five levels of makeup intensity are available that can be adjusted using adjustment button 243. A makeup intensity of "0" indicates that the makeup intensity will remain unchanged at its original intensity. A makeup intensity of "1" indicates that the makeup intensity will be increased by one level based on the parameter corresponding to "1." By selecting one of the levels "1" to "4," the intensity of makeup of the person appearing in the photographed image displayed in display area 201 is adjusted.

[0187] It is possible to select ON / OFF as an adjustable nose bridge using the adjustment button 244. By selecting ON or OFF, it is possible to select whether to emphasize the highlight part of the nose bridge of the person appearing in the photographed image displayed in the display area 201 or to leave it in its original state.

[0188] Three levels are provided for the thickness of the downy mouth hair that can be adjusted using the adjustment button 245. A downy hair thickness of "0" indicates that the appearance of the downy hair is not changed and remains the original appearance. A downy hair thickness of "1" indicates that processing is performed based on the parameter corresponding to "1" to make the thickness of the downy mouth hair one level thinner. By selecting one of the levels, the thickness of the downy mouth hair of the person appearing in the captured image displayed in the display area 201 is adjusted.

[0189] Using these various functions, customers can select the retouching content for the captured image. Retouching work includes at least adjusting the appearance of the entire face and body, adjusting the appearance of parts of the face such as the eyes and makeup, and adjusting brightness. When the "Brightness" tab is selected, for example, adjustment buttons used to adjust the brightness of the skin, adjustment buttons used to adjust the brightness of the entire captured image, and adjustment buttons used to adjust the brightness of the background are displayed.

[0190] The captured image to be retouched is switched using the thumbnail, and the retouching work is performed sequentially on each of the captured images P1 to P5. When the retouching work on each of the captured images P1 to P5 is completed and the done button 212 displayed in the upper right corner of the screen is operated, the retouching work using the in-store retouching screen is completed.

[0191] The in-store retouch screen has a time limit area 211 at the top left corner. The remaining time of the time limit set in the retouch setting is displayed in the time limit area 211.

[0192] 18, in-store terminal 12 transmits information indicating the details of the customer's operation on the in-store retouching screen to image processing server 11. In response to the transmission of the information indicating the details of the customer's operation, image processing server 11 performs image processing on the selected captured image and reflects the results on the display of the in-store retouching screen.

[0193] In step S203, the in-store terminal 12 determines whether or not retouching is complete. If it is determined in step S203 that retouching is not complete, the process returns to step S201, and the retouching work continues using the in-store retouching screen.

[0194] When the completion button 212 is operated or when the time limit has elapsed, the retouching ends. When it is determined in step S203 that the retouching has ended, the processing of FIG.

[0195] The above processing is performed at the in-store terminal 12 when in-store retouching is selected as the setting of the retouching location.

[0196] Processing of customer terminal 13 (out-of-store retouching) The retouching process of the customer terminal 13 will be described with reference to the flowchart in Fig. 21. Since this is retouching process in the customer terminal 13, the process shown in Fig. 21 is the process for performing retouching outside the store. The process shown in Fig. 21 is started when access is instructed based on the URL included in the access destination notification.

[0197] In step S211, the customer terminal 13 displays an identity verification screen based on the information transmitted from the image processing server 11.

[0198] 22 is a diagram showing an example of an identity verification screen. The layout of each piece of information changes depending on the shape of the display provided in the customer terminal 13. The same applies to other screens displayed on the display of the customer terminal 13.

[0199] As shown in Figure 22, the identity verification screen displays thumbnails of the photographed images uploaded as selected images. For example, the number of thumbnails displayed on the identity verification screen changes depending on the number of photographed images uploaded as selected images. A confirmation button is provided below the thumbnails to operate when you confirm that the photo is yours.

[0200] Since photo studio staff can check whether the retouching screen using the customer's own photograph is displayed, identity verification using the identity verification screen is not required for in-store retouching. By verifying identity in advance for out-of-store retouching, it is possible to prevent problems such as accidentally presenting a customer with a retouching screen using someone else's photograph. If identity verification is not performed, the retouching screen will not be displayed.

[0201] After the customer has confirmed their identity using the confirmation button, in step S212 of FIG. 21, the customer terminal 13 displays an outside-store retouch screen based on the information transmitted from the image processing server 11.

[0202] Fig. 23 is a diagram showing an example of a retouch screen for use outside a store. Of the components shown in Fig. 23, the same components as those shown in Fig. 19 are given the same reference numerals. Duplicate explanations will be omitted where appropriate.

[0203] The configuration of the outside-store retouching screen differs from the configuration of the inside-store retouching screen in that there is no time limit displayed in the upper left corner of the screen and a save / done button 212A is displayed in the upper right corner of the screen, allowing customers to perform retouching work without worrying about the time limit.

[0204] The save / complete button 212A is operated to interrupt or end the retouching work. When the retouching work is interrupted, the retouching parameters indicating the operations performed by the customer up to that point are saved in the image processing server 11. The customer can perform the retouching work on the same captured image multiple times.

[0205] 21, the customer terminal 13 transmits information indicating the details of the customer's operation on the outside-store retouching screen to the image processing server 11. In response to the transmission of the information indicating the details of the customer's operation, the image processing server 11 performs image processing on the selected captured image, and the results are reflected in the display of the outside-store retouching screen.

[0206] In step S214, the customer terminal 13 determines whether or not the retouching is complete. If it is determined in step S214 that the retouching is not complete, the process returns to step S212, and the retouching work continues using the outside-store retouching screen.

[0207] When the save / complete button 212A is operated to request interruption of the retouching work or termination of the retouching work, the retouching ends. If it is determined in step S214 that the retouching has ended, the processing in FIG. 21 ends.

[0208] The above process is performed on the customer terminal 13 when outside retouching is selected as the setting of the retouching location.

[0209] Image processing server 11 processing The retouching process of the image processing server 11 will be described with reference to the flowchart in Fig. 24. Descriptions of processes that are the same as those described with reference to Fig. 7 and Fig. 8 will be omitted as appropriate. The process shown in Fig. 24 corresponds to the process of the in-store terminal 12 described with reference to Fig. 18 and the process of the customer terminal 13 described with reference to Fig. 21.

[0210] For example, the process shown in Fig. 24 is started when a retouching start button is operated on the in-store terminal 12 and a request to start in-store retouching is made from the in-store terminal 12. The process shown in Fig. 24 is also started when a request to start out-of-store retouching is made from the customer terminal 13 by accessing based on the URL included in the access destination notification.

[0211] In step S231, the image processing server 11 determines whether or not to start in-store retouching.

[0212] If it is determined in step S231 that in-store retouching should be started, in step S232 the image processing server 11 causes an in-store retouching screen to be displayed on the in-store terminal 12. This process corresponds to the process in step S201 in FIG.

[0213] In step S233, the image processing server 11 retouches the selected image in response to the customer's operation on the in-store retouch screen.

[0214] In step S234, the image processing server 11 determines whether or not retouching has ended. If it is determined in step S234 that retouching has not ended, the process returns to step S232, and in-store retouching continues.

[0215] On the other hand, if it is determined in step S231 that out-of-store retouching is to be started, in step S235, the image processing server 11 causes an identity verification screen to be displayed on the customer terminal 13. This process corresponds to the process of step S211 in FIG.

[0216] In step S236, the image processing server 11 displays the outside-store retouch screen on the customer terminal 13. This process corresponds to the process in step S212 of FIG.

[0217] In step S237, the image processing server 11 retouches the selected image in response to the customer's operation on the outside-of-store retouching screen.

[0218] In step S238, the image processing server 11 determines whether or not retouching has ended. If it is determined in step S238 that retouching has not ended, the process returns to step S236, and out-of-store retouching continues.

[0219] If it is determined in step S234 that in-store retouching has ended, or if it is determined in step S238 that out-of-store retouching has ended, the image processing server 11 determines in step S239 whether or not reception of all reflected images has been completed. The process waits until it is determined that reception of all reflected images has been completed.

[0220] If it is determined in step S239 that all of the reflected images have been received, then in step S240 the image processing server 11 performs a full reflection, and applies the same retouching to each of the reflected images as to the selected image. By performing a full reflection, the reflected images are retouched, and the retouching parameters are set for each of the reflected images.

[0221] In step S241, the image processing server 11 transmits a notification that download is possible to the in-store terminal 12, and then ends the process.

[0222] In response to receiving the download availability notification, the customer list screen display on the in-store terminal 12 switches to a state indicating that the photographed images are available for download. When an operation to start downloading is performed on the in-store terminal 12 and a request for downloading is transmitted, downloading of the photographed images from the image processing server 11 to the in-store terminal 12 begins. The retouched photographed images downloaded to the in-store terminal 12 include retouched selected images and retouched reflected images with full retouching.

[0223] <<Configuration of each device>> <Configuration of image processing server 11> 25 is a block diagram showing an example of the hardware configuration of the image processing server 11. The image processing server 11 is configured by a computer as shown in FIG.

[0224] A CPU (Central Processing Unit) 301 , a ROM (Read Only Memory) 302 , and a RAM (Random Access Memory) 303 are interconnected by a bus 304 .

[0225] An input / output interface 305 is further connected to the bus 304. An input unit 306, an output unit 307, a recording unit 308, a communication unit 309, and a drive 310 are connected to the input / output interface 305.

[0226] The input unit 306 is composed of a keyboard, a mouse, and the like.

[0227] The output unit 307 is configured by a display or the like.

[0228] The recording unit 308 is configured by storage such as a hard disk, non-volatile memory, etc. The recording unit 308 records various information such as programs executed by the CPU 301 and captured images uploaded from the in-store terminal 12.

[0229] The communication unit 309 is an interface to a network such as the Internet, a LAN, etc. The communication unit 309 transmits and receives the above-mentioned various data to and from the in-store terminal 12 and the customer terminal 13.

[0230] The drive 310 controls writing of data to and reading of data from the removable medium 311 .

[0231] FIG. 26 is a block diagram showing an example of the functional configuration of the image processing server 11.

[0232] 26, in the image processing server 11, a predetermined program is executed by the CPU 301 to implement an information processing unit 321. The information processing unit 321 is made up of an image management unit 331, a display control unit 332, a setting management unit 333, an image processing unit 334, and a communication control unit 335.

[0233] The image management unit 331 manages the captured images uploaded from the in-store terminal 12 and received by the reception control unit 335-2 by having the recording unit 308 record them.

[0234] FIG. 27A is a diagram showing an example of image management information used to manage captured images.

[0235] 27A, the image management information includes information on the upload date and time, retouching deadline, phase, mode, retouching location, age estimation AI, upper arm slimming, and URL. The retouching deadline, mode, retouching location, age estimation AI, and upper arm slimming indicate the information set on the in-store terminal 12 using the setting screen before uploading the captured image. Duplicate explanations will be omitted as appropriate.

[0236] The upload date and time indicates the date and time when the captured image was uploaded.

[0237] The phase indicates the status of the customer's captured image. One of the following phases is set: "Available for uploading," "Available for downloading," "Downloaded," or "Retouching."

[0238] The URL indicates the location where the uploaded photographed image is saved. If the retouching location is set to off-site retouching, the URL information will be notified to the customer via the access destination notification.

[0239] The image management unit 331 in Fig. 26 generates image management information including such information and manages the photographed images uploaded as customer images. The photographed images managed by the image management unit 331 are read out by other processing units. The processing of step S13 in Fig. 7 and step S73 in Fig. 8 is performed by the image management unit 331.

[0240] The display control unit 332 displays various screens on a device accessed by a program such as a browser. When the retouching location is set to off-store retouching, the display control unit 332 displays screens such as a setting screen and an image selection screen on the in-store terminal 12 and a retouching screen on the customer terminal 13. When the retouching location is set to in-store retouching, the display control unit 332 displays screens such as a setting screen, an image selection screen, and a retouching screen on the in-store terminal 12. The processing of step S16 in FIG. 7 and step S75 in FIG. 8 is performed by the display control unit 332. Similarly, the processing of steps S151, S154, S157, and S160 in FIG. 17 and steps S232, S235, and S236 in FIG. 24 is also performed by the display control unit 332.

[0241] The setting management unit 333 generates and manages store setting information based on information set in the in-store terminal 12. The store setting information is information that indicates the contents set for each photo studio store. The store setting information is set by photo studio staff at a timing such as before a photo shoot for a customer.

[0242] FIG. 27B is a diagram showing an example of store setting information.

[0243] As shown in FIG. 27B, the image management information includes information on the number of selected images, retouching deadline, mode, retouching location, age estimation AI, slim upper arms, and retouching time limit.

[0244] The number of selected images indicates the default number of photographed images to be selected as selected images.

[0245] The retouching deadline indicates the default retouching deadline.

[0246] The mode setting indicates whether the default mode is single-player or two-player. Mode settings can also be selected for each upload.

[0247] The retouching location indicates whether in-store retouching or off-store retouching is used as the default retouching location setting. It is also possible to select the retouching location setting for each upload.

[0248] Age estimation AI indicates whether the default age estimation AI setting is on or off. Age estimation AI settings can also be selected for each upload.

[0249] The "Slim upper arms" option indicates whether the default slim upper arms setting is on or off. The slim upper arms setting can also be selected for each upload.

[0250] The retouching time limit indicates the default retouching time limit when the retouching location setting is in-store retouching.

[0251] This information contained in the store setting information is reflected in the image management information. By being able to set information according to the service status of each store as store setting information, photo studios can easily set up the upload.

[0252] 27B is not necessarily included in the store setting information, but it is possible for at least some of the information shown in Fig. 27B to be included in the store setting information. The setting management unit 333 has a function of managing at least one of the settings of the number of selected images, retouching deadline, mode, retouching location, age estimation AI, slim upper arms, and retouching time limit in accordance with operations on the setting screen displayed on the in-store terminal 12.

[0253] The image processing unit 334 in Figure 26 performs various types of image processing related to retouching in response to customer operations on the retouching screen. The processing of step S18 in Figure 7 and step S77 in Figure 8 is performed by the image processing unit 334. Similarly, the processing of steps S233 and S237 in Figure 24 is also performed by the image processing unit 334.

[0254] For example, when an operation related to adjusting skin texture is performed, the image processing unit 334 performs image processing such as adjusting skin color. When an operation related to adjusting face size (feeling of small face) or face shape is performed, the image processing unit 334 performs image processing such as shifting the position of the face outline. Specifically, when one of the levels is selected using the adjustment button 232 (FIG. 20), the image processing unit 334 performs image processing to shift each position of the facial outline of the person to be retouched toward the center of the face according to the selected level, thereby making the face smaller.

[0255] When an operation for adjusting the size or shape of the eyes is performed, the image processing unit 334 performs image processing such as modifying the outline of the eyes. Specifically, when one of the levels is selected using the adjustment button 241 (FIG. 20), the image processing unit 334 extracts the eye region of the person to be retouched by masking or the like. The image processing unit 334 enlarges the image of the extracted eye region according to the selected level, and performs eye enlargement processing by pasting the enlarged image of the eye region at the position from which it was extracted.

[0256] When an operation related to adjusting the intensity of makeup is performed, the image processing unit 334 performs image processing such as combining an image of blush of a predetermined color on the cheeks or combining an image of lipstick of a predetermined color on the lips. Furthermore, when an operation related to modifying a specific part of a person, such as slimming the upper arms or improving the figure, the image processing unit 334 performs image processing for those functions.

[0257] Information representing the results of image processing by the image processing unit 334 is supplied to the display control unit 332 and used to display the retouch screen. Retouching parameters indicating the details of retouching for each captured image are supplied to the image management unit 331 and managed in association with the captured image.

[0258] The image processing unit 334 includes a recognition unit 334-1 and a total reflection processing unit 334-2.

[0259] The recognition unit 334-1 recognizes various features of the subject of the uploaded photographed image. The recognition results by the recognition unit 334-1 are used for retouching, etc.

[0260] For example, the recognition unit 334-1 recognizes the faces of people appearing in each captured image using an inference model for face recognition. The recognition unit 334-1 is provided with an inference model for face recognition that receives a captured image as input and outputs information indicating the position of a face.

[0261] The recognition unit 334-1 recognizes facial features. The facial features include the positions of each part of the face such as the eyes, nose, and mouth, the positions of contour points, the direction of the face, etc. The recognition unit 334-1 is provided with an inference model for facial feature recognition that receives a facial image as input and outputs information on facial features.

[0262] The recognition unit 334-1 estimates the gender of each person. The recognition unit 334-1 is provided with an inference model for gender estimation that receives an image of a person as input and outputs gender information. The gender information estimated by the recognition unit 334-1 is used to switch available retouching functions, switch retouching parameters, and so on.

[0263] The recognition unit 334-1 identifies each person. The recognition unit 334-1 is provided with an inference model for person identification that takes a person's image as input and outputs a person ID. Information on the person identification result by the recognition unit 334-1 is used, for example, to identify the same person in a full reflection.

[0264] The recognition unit 334-1 estimates the age of each person. The recognition unit 334-1 is provided with an inference model for age estimation that takes a person's image as input and outputs age information. Age estimation is performed when the age estimation function is turned on.

[0265] Various features other than facial features, gender, and age are recognized by the recognition unit 334-1. The features recognized by the recognition unit 334-1 include bone points, the position of the upper arms, the extent of exposed skin, the type of clothing, etc. The recognition unit 334-1 is provided with inference models according to the features to be estimated, such as an inference model for bone structure estimation, an inference model for upper arms estimation, an inference model for skin region estimation, and an inference model for clothing estimation.

[0266] The inference model for bone structure estimation is an inference model that takes an image of a person as input and outputs information on the position of bone points, while the inference model for upper arm estimation is an inference model that takes an image of a person as input and outputs information on the position of the upper arms. The inference model for skin region estimation is an inference model that takes an image of a person as input and outputs information on the exposed area of ​​skin, while the inference model for clothing estimation is an inference model that takes an image of a person as input and outputs information on clothing. For example, an inference model for clothing determination is generated by learning using a large number of images of people in various clothing labeled with information on the type of clothing as learning data. Note that the position of the upper arms is estimated when the upper arm slimming function is turned on.

[0267] The recognition results by the recognition unit 334-1 are supplied to the image management unit 331 and managed in association with the captured image. Feature recognition by the recognition unit 334-1 may be performed by analyzing the image rather than using an inference model. Since the recognition results by the recognition unit 334-1 are used for retouching, for example, retouching of the captured image begins after uploading of the selected images is complete and feature recognition for all the selected images is finished.

[0268] When retouching of the selected image is complete and uploading of the reflection image is complete, the complete reflection processing unit 334-2 performs a complete reflection on the uploaded reflection image. Various features recognized by the recognition unit 334-1, such as face size, face direction, number of people in the subject, and gender, are used for the complete reflection. The reflection image for which retouching parameters have been set by the complete reflection processing unit 334-2 is managed by the image management unit 331. The processing of step S19 in FIG. 7 and step S78 in FIG. 8 is performed by the complete reflection processing unit 334-2. Similarly, the processing of step S240 in FIG. 24 is also performed by the complete reflection processing unit 334-2.

[0269] The communication control unit 335 controls the communication unit 309, and controls communication between the in-store terminal 12 and the customer terminal 13. The communication control unit 335 includes a transmission control unit 335-1 and a reception control unit 335-2.

[0270] The transmission control unit 335-1 controls the transmission of various data to the in-store terminal 12 and the customer terminal 13. The processing of steps S14, S20, and S22 in Fig. 7 and steps S79 and S81 in Fig. 8 is performed by the transmission control unit 335-1. Similarly, the processing of step S159 in Fig. 17 and step S241 in Fig. 24 is also performed by the transmission control unit 335-1.

[0271] The reception control unit 335-2 controls the reception of various data transmitted from the in-store terminal 12 and the customer terminal 13. The reception control unit 335-2 performs steps S11, S12, S15, S17, and S21 in Fig. 7 and steps S71, S72, S74, S76, and S80 in Fig. 8. Similarly, the reception control unit 335-2 also performs steps S152 and S155 in Fig. 17.

[0272] <Configuration of in-store terminal 12> The in-store terminal 12 has the same hardware configuration as that shown in Fig. 25. The configuration shown in Fig. 25 will be cited as the configuration of the in-store terminal 12 in the following description.

[0273] FIG. 28 is a block diagram showing an example of the functional configuration of the in-store terminal 12.

[0274] 28, in the in-store terminal 12, an information processing unit 351 is realized by the CPU 301 executing programs such as a browser and an upload program. The information processing unit 351 is made up of an image acquisition unit 361, a display control unit 362, an operation detection unit 363, an image selection unit 364, and a communication control unit 365.

[0275] The image acquisition unit 361 reads and acquires the captured images from the recording unit 308. When the image capture is completed, a plurality of captured images, such as 100 images, that have been imported into the in-store terminal 12 are recorded in the recording unit 308. The captured images acquired by the image acquisition unit 361 are supplied to, for example, the transmission control unit 365-1 and uploaded to the image processing server 11. The processing of step S2 in FIG. 7 and step S52 in FIG. 8 is performed by the image acquisition unit 361. Similarly, the processing of step S103 in FIG. 9 is also performed by the image acquisition unit 361.

[0276] The display control unit 362 displays various screens on the display constituting the output unit 307 based on information transmitted from the image processing server 11. When the retouching location is set to outside-store retouching, the display control unit 362 displays screens such as a setting screen and an image selection screen. When the retouching location is set to inside-store retouching, the display control unit 362 displays screens such as a setting screen, an image selection screen and a retouching screen. The processing of step S55 in FIG. 8 is performed by the display control unit 362. Similarly, the processing of steps S101, S102, S104, and S107 in FIG. 9, steps S110 and S112 in FIG. 10, and step S201 in FIG. 18 are also performed by the display control unit 362.

[0277] The operation detection unit 363 detects an operation performed by a user using the input unit 306. The photo studio staff and customers are users of the in-store terminal 12. For example, information indicating the content of the operation detected by the operation detection unit 363 is supplied to the transmission control unit 365-1 and transmitted to the image processing server 11.

[0278] The image selection unit 364 selects a predetermined number of select images from among the captured images based on an operation on the image selection screen.

[0279] The communication control unit 365 controls the communication unit 309 and controls communication with the image processing server 11. The communication control unit 365 includes a transmission control unit 365-1 and a reception control unit 365-2.

[0280] The transmission control unit 365-1 controls the transmission of various types of data to the image processing server 11. The transmission control unit 365-1 performs the processes of steps S1, S3, and S5 in Fig. 7 and steps S51, S53, S54, S56, and S58 in Fig. 8. Similarly, the transmission control unit 365-1 also performs the processes of steps S105 and S108 in Fig. 9 and step S202 in Fig. 18.

[0281] The reception control unit 365-2 controls the reception of various data transmitted from the image processing server 11. The processes of steps S4 and S6 in Fig. 7 and steps S57 and S59 in Fig. 8 are performed by the reception control unit 365-2.

[0282] <<Details of some features>> <Retouching work interrupted> When outside retouching is selected as the setting of the retouching location, the retouching work can be interrupted as described above.

[0283] Fig. 29 is a diagram showing an example of a screen used when retouching work is interrupted. When the save / complete button 212A is operated on the outside-store retouching screen (Fig. 23), a screen such as that shown in Fig. 29 is displayed on the display of the customer terminal 13 under the control of the display control unit 332 of the image processing server 11.

[0284] In the example of FIG. 29, a pop-up containing a temporary save button 401 and a complete button 402 is displayed superimposed on the outside-store retouching screen. The temporary save button 401 is operated to interrupt the retouching work and save the retouching parameters that have been set up to that point. The complete button 402 is operated to end the retouching work. When the temporary save button 401 is operated, the retouching parameters set for each selected image are linked to the captured image and managed by the image management unit 331.

[0285] Restarting the retouching work can be done, for example, by accessing the same URL that was used as the access destination when the work was started. When access is made based on the URL, a screen such as that shown in Figure 30 is displayed on the display of the customer terminal 13.

[0286] 30, a resume button 411 is displayed, which is operated to resume retouching work from a suspended state, and a start button 412 is displayed, which is operated to reset the work up to that point and start the retouching work from the beginning. When the resume button 411 is operated, the same retouching screen as when the work was suspended is displayed on the display of the customer terminal 13, based on the retouching parameters managed by the image management unit 331.

[0287] In this way, by selecting out-of-store retouching, the customer can retouch the selected images in multiple batches.

[0288] <Reflect all numbers> FIG. 31 is a diagram showing an example of a screen displayed before a select image is selected.

[0289] Select images are selected after checking the notes displayed on the screen as shown in Fig. 31. As shown in the approximate center of Fig. 31, the images are selected by selecting, for example, multiple types of captured images in which the faces in the images are different in size.

[0290] As described above, in full reflection, the selected image that will be the source of the retouching parameters for a certain reflected image is selected based on the size of the faces in the image, etc. By selecting photographed images with faces of different sizes as selected images and performing retouching, it becomes possible to perform full reflection on reflected images with various compositions.

[0291] Furthermore, the selected image is selected by selecting an image in which the face is facing forward and the eyes are open. If the face is facing diagonally or the eyes are closed, recognition by the image processing server 11 may fail or unnatural image processing may be performed. By selecting an image in which the face is facing forward and the eyes are open as the selected image, it is possible to prevent such problems from occurring.

[0292] Before the start of full reflection, for example, in the recognition unit 334-1 of the image processing server 11, the face ratio, which is the ratio of the size of the face in each uploaded photographed image to the size of the photographed image, is calculated for each uploaded photographed image.

[0293] FIG. 32 is a diagram showing an example of calculation of the face ratio.

[0294] Fig. 32A shows an example of calculating the face ratio for a captured image P101, which is a horizontally long image. Fig. 32B shows an example of calculating the face ratio for a captured image P102, which is a vertically long image. Note that Fig. 32 shows a captured image containing one person, but if multiple people are included in the image, the same process is performed for each person.

[0295] Through face recognition, the area enclosed by the frame F is recognized as the area of ​​the face. Furthermore, based on the width X of the frame F and the length Y of the short side of the captured image, the face ratio Z is calculated, for example, using the following formula: Face ratio Z = width of frame F X ÷ length of short side Y

[0296] For the horizontally elongated captured image P101 shown in Fig. 32A, the face ratio is calculated based on the width of the frame F and the height of the captured image indicated by the dashed-dotted arrow. On the other hand, for the vertically elongated image shown in Fig. 32B, the face ratio is calculated based on the width of the frame F and the width of the image indicated by the dashed-dotted arrow.

[0297] Such calculation of the face ratio is performed for, for example, each of the five selected images and the 95 reflected images.

[0298] Furthermore, the selected image with a similar face ratio is used as the reflection source, and the retouching is reflected on each reflection image.

[0299] For example, assume that the face ratios of the selected images P101 to P105 are calculated as 0.30, 0.40, 0.50, 0.60, and 0.70. In this case, the reflected image with a calculated face ratio of 0.32 is retouched based on the retouching parameters set for the selected image P101 with the closest face ratio. The reflected image with a calculated face ratio of 0.80 is retouched based on the retouching parameters set for the selected image P105 with the closest face ratio. Instead of the selected image with the closest face ratio, retouching based on the retouching parameters set for the selected image with the closest face ratio by more than a threshold may be applied to all images.

[0300] Note that retouching is reflected based not only on the size of the face but also on the results of personal identification, so that retouching set for the same person is reflected. If a captured image containing a person identified as person #1 and a person identified as person #2 is retouched as the selected image, the same retouching as that for person #1 will be reflected on person #1 in the reflected image. Similarly, the same retouching as that for person #2 will be reflected on person #2 in the reflected image.

[0301] It is possible to select which of the retouched selected images will be used as the source image based on characteristics other than the size of the face and the results of personal identification, such as the gender and facial direction of the person appearing as the subject.

[0302] FIG. 33 is a diagram showing an example of full reflection.

[0303] 33 shows an example in which 100 photographed images have been uploaded, and five photographed images #1 to #5 have been selected as selected images. 95 photographed images #6 to #100 become reflected images.

[0304] When retouching is performed on five selected images, photographed images #1 to #5, retouching parameters are set for photographed images #1 to #5, as shown in the middle of Fig. 33. In the example of Fig. 33, retouching parameters #1 to #5 are set for photographed images #1 to #5. In Fig. 33, photographed images shown in color indicate that they have been retouched.

[0305] When the retouched selected image to be used as the source image is selected as described above and all images are reflected, the retouching parameters will be set for captured images #6 to #100, as shown in the lower part of Figure 33.

[0306] 33, parameter #2, which is the same retouching parameter as set for photographed image #2, is set for photographed image #6, and parameter #3, which is the same retouching parameter as set for photographed image #3, is set for photographed image #7. Furthermore, parameter #1, which is the same retouching parameter as set for photographed image #1, is set for photographed image #8, and parameter #3, which is the same retouching parameter as set for photographed image #3, is set for photographed image #9. Similarly, parameters, which are the same as the retouching parameters set for the selected photographed image selected as the source, are set for each of photographed images #10 to #100.

[0307] The photographed images #6 to #100, which are reflection images to which retouching parameters have been set, are downloaded as retouched photographed images together with the photographed images #1 to #5, which are retouched selected images.

[0308] <Transfer of retouching parameters> When images of the same customer are taken multiple times on different days, the status of the groups of images grouped together based on the date of shooting is managed on the customer list screen. For example, when images are imported after the first and second shootings, the status of the group of images obtained by the first shooting and the group of images obtained by the second shooting are managed. For convenience of explanation, a group of images managed together is called a "photographed image group." The group of images obtained by the first shooting is called photographed image group #1, and the group of images obtained by the second shooting is called photographed image group #2.

[0309] When a customer who has taken multiple photos has completed retouching using one of the groups of photographed images, the retouching parameters set during the retouching can be reflected in the group of photographed images that has not been retouched. The retouching parameters are reflected (carried over) in the same way as when all photographed images are reflected, with each photographed image included in the group of photographed images that has not been retouched being treated as a reflected image.

[0310] FIG. 34 is a diagram showing an example of taking over retouching parameters.

[0311] The selected images shown in the upper part of Fig. 34 are selected from the photographed images included in photographed image group #1 and have been retouched. As in the case described with reference to Fig. 33, retouching has been performed on five selected images, photographed images #1 to #5, and retouching parameters #1 to #5 have been set. Although not shown in the figure, retouching parameters have been set by full reflection for photographed images #6 to #100, which are the 95 reflection images included in photographed image group #1.

[0312] For example, if it is selected to inherit the retouching parameters of photographed images #101 to #200 included in photographed image group #2 as the reflected images, the same retouching parameters as those set for photographed images #1 to #5 will be set for each of photographed images #101 to #200, as shown in the lower part of Figure 34.

[0313] 34, parameter #5, which is the same retouching parameter as set for photographed image #5, is set for photographed image #101, and parameter #4, which is the same retouching parameter as set for photographed image #4, is set for photographed image #102. Furthermore, parameter #3, which is the same retouching parameter as set for photographed image #3, is set for photographed image #103, and parameter #2, which is the same retouching parameter as set for photographed image #2, is set for photographed image #104. Similarly, parameters, which are the same as the retouching parameters set for the selected photographed image in photographed image group #1 selected as the reflection source, are set for each of photographed images #105 to #200.

[0314] FIG. 35 is a diagram showing an example of a screen used when taking over retouching parameters.

[0315] The confirmation screen shown in Figure 35 is displayed, for example, when retouching of the photographed images included in photographed image group #1 has been completed and an operation to upload the photographed images included in photographed image group #2 is performed on the customer list screen.

[0316] 35, a takeover button 421 and a new button 422 are displayed. The takeover button 421 is a button that is operated when retouching parameters set using photographed images included in photographed image group #1 are to be taken over to photographed image group #2. The new button 422 is a button that is operated when setting new retouching parameters using photographed images included in photographed image group #2.

[0317] When the takeover button 421 is operated, uploading of the photographed images included in photographed image group #2 begins. When uploading of the photographed images included in photographed image group #2 is completed, the image processing server 11 sets the retouching parameters set using the photographed images included in photographed image group #1 to the photographed images included in photographed image group #2. In this case, a select image cannot be selected from the photographed images included in photographed image group #2, and retouching of the photographed images included in photographed image group #2 is not performed.

[0318] On the other hand, when the New button 422 is operated, a select image is selected from the captured images included in the captured image group #2, just as in normal uploading, and after the select image is selected, uploading of the captured image begins. When the upload of the select image is complete, the select image is retouched by either off-site retouching or in-site retouching, and retouching parameters are set for the reflected images included in the captured image group #2 by reflecting all images.

[0319] In this way, when multiple photos are taken of the same customer and the photos are uploaded, it is possible to reflect all of the photos across the number of times they were taken. By simply retouching a selected image selected from a single group of photos, it is possible to obtain retouched photos for the reflected images included in the same group of photos and all of the photos (reflected images) included in different groups of photos.

[0320] <Example of retouch screen for outside the store> Fig. 36 is a diagram showing another example of the retouch screen for use outside a store. Of the components shown in Fig. 36, the same components as those described above are assigned the same reference numerals. Duplicate explanations will be omitted as appropriate.

[0321] The outside-store retouch screen shown in FIG. 36 is a screen displayed on a customer terminal 13 having a vertically long display, such as a smartphone.

[0322] A display area 201 is formed in the center of the display of the customer terminal 13 so as to occupy most of the screen. A horizontally long, strip-shaped thumbnail area 202 is formed above the display area 201 at the top edge of the screen. Thumbnails of photographed images P1 to P5 are displayed in the thumbnail area 202. For example, photographed image P1 selected using the thumbnails is enlarged and displayed in the display area 201. If there are five or more selected images, the thumbnail images displayed in the thumbnail area 202 can be switched by the customer's operation.

[0323] For example, the display range of the captured image can be changed by performing a swipe operation on the display area 201. The display of the captured image can be enlarged or reduced by performing a pinch-in / out operation.

[0324] At the bottom of the outside-store retouching screen, sheet 451 is displayed, which is a display area for buttons used for various operations. When an operation to expand sheet 451 is performed, sheet 451 expands upward, and various buttons appear in area 451A, as shown in FIG. 37. The display of area 451A is variable, and frequently used buttons such as person selection button 204, comparison button 203, and face frame ON / OFF button 462 are displayed horizontally and fixed above it. Face frame ON / OFF button 462 is a button that is operated to switch ON / OFF the display of the frame showing the person to be retouched.

[0325] In this way, the sheet 451 can be expanded only when necessary. By closing the sheet 451, the range of the display area 201 can be expanded, making it easier to check the retouching status of the captured image. The display of the thumbnail area 202 may be switched on and off. When the display of the thumbnail area 202 is turned off, the display area 201 is enlarged and displayed in the range where the thumbnail area 202 was displayed.

[0326] Tabs 461-1 to 461-4 are displayed horizontally below sheet 451. Tabs 461-1 to 461-4 are for batch retouching, adjusting the face and entire body, adjusting eyes and makeup, and adjusting brightness, respectively. Operating tabs 461-1 to 461-4 switches the display content in area 451A. In the example of FIG. 37, tab 461-1 is selected, and batch retouch button 205, which is used to select the level of batch retouching, is displayed in area 451A.

[0327] FIG. 38 is a diagram showing a display example of the area 451A.

[0328] When tab 461-2 is selected, buttons 471 to 475 used to select functions related to adjustments to the entire face and body are displayed in area 451A. Buttons 471 to 475 are assigned to the functions of face slimming, face shape, slim upper arms, and style, respectively. By swiping horizontally, buttons assigned to other functions not shown in FIG. 38 are displayed.

[0329] When button 471 is selected, adjustment buttons 232 used to select the level of small face feeling are displayed above buttons 471 to 475, as shown in Fig. 38. The upper row of Fig. 38 shows a state in which small face feeling level "1" is selected, and the lower row shows a state in which small face feeling level "2" is selected.

[0330] FIG. 39 is a diagram showing another display example of the area 451A.

[0331] When button 472 is selected while tab 461-2 is selected, adjustment buttons 233 used to select the face shape are displayed above buttons 471 to 475, as shown in FIG.

[0332] In this way, even on customer terminal 13 with a vertically long display, retouching work can be performed using the same functions as in-store retouching. In particular, thumbnail area 202 is formed on display area 201 in the same way as the in-store retouching screen. Customers can perform out-of-store retouching with the same operational feel as when performing in-store retouching.

[0333] If the display of the in-store terminal 12 is a vertically long display, a retouch screen having a configuration similar to that of the screen shown in FIG. 36 etc. may be displayed as the in-store retouch screen.

[0334] <<Others>> Although the selection of the select images is performed by the photo studio staff in the above embodiment, it may be performed by the customer. Furthermore, the selection of the select images may be performed automatically instead of manually. The image selection unit 364 is provided with a function for automatically selecting the select images.

[0335] When automatically selecting a select image, for example, the in-store terminal 12 is provided with a function identical to the function of the recognition unit 334-1 in the image processing server 11. When automatically selecting a select image, the in-store terminal 12 recognizes the features of all captured images and selects a captured image to be used as the select image based on the recognized face size, face orientation, etc. The automatic selection of a select image is also performed by selecting captured images in which faces of different sizes are captured.

[0336] Although the processing is performed on captured images that contain people as subjects, this technology can also be applied to retouching captured images that contain subjects other than people, such as landscapes, animals, objects, etc. Reflecting all captured images that contain subjects other than people is performed, for example, by selecting, based on the feature recognition results, from among the retouched captured images, a captured image that contains the same subject as the subject of the reflection destination captured image in a similar composition (size, position, etc.), as the reflection source captured image.

[0337] In the in-store terminal 12 and the customer terminal 13, various screens are displayed by a browser and the functions of the image processing server 11 are used, but a dedicated application may be provided. By executing the dedicated application, the same processing as that described above is performed in the in-store terminal 12 and the customer terminal 13.

[0338] About the program The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0339] The program to be installed is provided by being recorded on removable media 311 shown in FIG. 25, which may be an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. Alternatively, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be installed in advance in ROM 302 or recording unit 308.

[0340] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0341] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0342] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0343] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0344] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0345] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0346] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. [Explanation of symbols]

[0347] 1 Information processing system, 11 Image processing server, 12 In-store terminal, 13 Customer terminal, 321 Information processing unit, 331 Image management unit, 332 Display control unit, 333 Setting management unit, 334 Image processing unit, 335 Communication control unit, 351 Information processing unit, 361 Image acquisition unit, 362 Display control unit, 363 Operation detection unit, 364 Image selection unit, 365 Communication control unit

Claims

1. In an information processing system including an information processing device and an image processing device connected via a network, The information processing device includes: an image acquisition unit that acquires a plurality of captured images; a display control unit that displays a setting screen used for setting whether to perform retouching work using the information processing device that uploads the captured image or using another device that has been accessed based on access destination information notified from the image processing device after uploading the captured image; and a transmission control unit that transmits a plurality of the captured images to the image processing device to perform uploading; Equipped with when it is set that the retouching work is to be performed using the information processing device, the display control unit displays a retouching screen used for the retouching work after uploading some of the captured images; The image processing device includes: a reception control unit that receives the plurality of captured images uploaded from the information processing device; an image management unit that manages the received photographed images; a transmission control unit that transmits information about the access destination after uploading some of the photographed images, based on customer information that is information about a customer who will receive the retouched photographed images, when the retouching work is set to be performed using the other device; and Equipped with Information processing system.

2. an image acquisition unit that acquires a plurality of captured images; a display control unit that displays a setting screen used for setting whether the retouching work should be performed using a first device that uploads the captured image or a second device that has accessed based on access destination information notified from a server after the captured image has been uploaded; and a transmission control unit that performs uploading by transmitting a plurality of the captured images to the server that is an upload destination; Equipped with When the retouching work is set to be performed using the first device, the display control unit displays a retouching screen used for the retouching work after uploading some of the captured images. The information processing device is the first device.

3. The display control unit does not display the retouch screen when the retouching work is set to be performed using the second device. The information processing device according to claim 2 .

4. an image selection unit that selects a predetermined number of selected images, which are the photographed images to be retouched, from the plurality of photographed images; The transmission control unit uploads the selected image before uploading a reflection image, which is the captured image that is the target of a reflection process that reflects, in the server, the same retouching as that performed on the selected image. The information processing device according to claim 2 .

5. When uploading of the selected image is completed, the display control unit starts displaying the retouch screen before uploading of the reflected image is completed. The information processing device according to claim 4 .

6. an information processing device as a first device, Acquiring a plurality of captured images; displaying a setting screen used for setting whether the retouching work should be performed using the first device that uploads the photographed image or using a second device that has accessed based on the access destination information notified from the server after the photographed image has been uploaded; uploading the captured images by transmitting the captured images to the server that is the upload destination; When the retouching work is set to be performed using the first device, a retouching screen used for the retouching work is displayed after uploading some of the photographed images. An information processing method including:

7. The first device is a computer. Acquiring a plurality of captured images; displaying a setting screen used for setting whether the retouching work should be performed using the first device that uploads the photographed image or using a second device that has accessed based on the access destination information notified from the server after the photographed image has been uploaded; uploading the captured images by transmitting the captured images to the server that is the upload destination; When the retouching work is set to be performed using the first device, a retouching screen used for the retouching work is displayed after uploading some of the photographed images. A program for executing a process including:

8. a reception control unit that receives a plurality of photographed images uploaded from an information processing device that is a first device and sets whether retouching work should be performed using a first device that uploads the photographed images or a second device that accesses the plurality of photographed images based on access destination information notified from a server after uploading the photographed images; and an image management unit that manages the received photographed images; a transmission control unit that transmits information about the access destination after uploading some of the photographed images, based on customer information that is information about a customer who receives the retouched photographed images, when the retouching work is set to be performed using the second device; Equipped with The image processing device is the server.

9. The apparatus further includes a display control unit that, when the second device accesses the apparatus based on the information of the access destination, causes the second device to display a retouch screen used for retouching work. The image processing device according to claim 8 .

10. The uploading of the photographed images is performed by uploading a predetermined number of selected images, which are photographed images to be subjected to a retouching operation selected by the first device from among the plurality of photographed images, before reflection images, which are photographed images to be subjected to a reflection process that reflects the same retouching as the retouching performed on the selected images; When the upload of the selected image is completed, the transmission control unit transmits the information of the access destination while the reflected image is being uploaded. The image processing device according to claim 9 .

11. When the retouching work is set to be performed using the first device, the display control unit starts displaying the retouching screen on the first device before uploading of the reflected image is completed. The image processing device according to claim 10.

12. an image processing unit that performs retouching adjustments to the selected images in response to operations on the retouching screen, and that performs the reflection processing after retouching of the selected images is completed; The image processing device according to claim 11 .

13. After the reflection process is completed, the transmission control unit transmits a notification to the information processing device that the retouched captured image can be downloaded. The image processing device according to claim 12.

14. The image processing device that is the server receiving a plurality of photographed images uploaded from an information processing device that is a first device that acquires the photographed images and sets whether retouching work will be performed using a first device that uploads the photographed images or a second device that has accessed the photographed images based on access destination information notified from the server after uploading the photographed images; Managing the received captured images; If the retouching work is set to be performed using the second device, transmitting the access destination information after uploading some of the photographed images based on customer information that is information about a customer who will receive the retouched photographed images. An information processing method including:

15. On the server computer, receiving a plurality of photographed images uploaded from an information processing device that is a first device that acquires the photographed images and sets whether retouching work will be performed using a first device that uploads the photographed images or a second device that has accessed the photographed images based on access destination information notified from the server after uploading the photographed images; Managing the received captured images; If the retouching work is set to be performed using the second device, transmitting the access destination information after uploading some of the photographed images based on customer information that is information about a customer who will receive the retouched photographed images. A program for executing a process including:

16. receiving, from an image processing device that receives and manages a plurality of photographed images uploaded from an information processing device, access destination information transmitted based on customer information that is information about a customer who will receive the retouched photographed images; performing access based on the information on the access destination, and displaying a retouch screen including thumbnails of some of the plurality of captured images selected using the information processing device before starting upload; In response to an operation being performed on the retouch screen, the captured image is displayed in a state where the content of the user's operation is reflected. A program for causing a computer to execute a process including the above.

17. The retouch screen is a first area that is a display area for the thumbnail; a second area that is a display area for one of the photographed images to be retouched, selected using the thumbnail; a third area that is a display area for information used to select retouching parameters, the third area being displayed so as to overlap a part of the second area when the third area is expanded; This is a screen containing The program according to claim 16.

18. The third area when expanded includes an area in which a plurality of buttons including a button used to select a person to be retouched are fixedly displayed, and an area in which information used to select one type of retouching parameter is displayed and the display content can be switched in response to an operation by the user. The program according to claim 17.

Images