Information processing device, information processing method, and program

Abstract

This information processing device is configured to perform: a control process for providing a user interface that allows operation of designating an extraction condition on a group of images captured in an event concerning marriage; and an extraction process for extracting images from the group of images on the basis of the designated extraction condition.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and in particular to a technology suitable for creating an image gallery for a wedding-related event. [Background technology]

[0002] For example, at wedding-related events such as weddings, wedding receptions, and after-parties, there is a service in which a photographer takes pictures, creates a photo album as a collection of images, and provides the photos to the participants. In this case, the photographer or the like imports the images captured by the imaging device into a personal computer, smartphone, or the like after the event is over, and edits them using editing software or the like to create an image album. It is possible to provide the image album via a social networking service (SNS), cloud service, email, messaging application, or the like. The following Patent Document 1 describes a technology for transferring images taken with a camera to a smartphone or the like for sharing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-81173 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, event photo collections are a source of enjoyment for participants, and they often want to view them as soon as possible after the event has ended. For example, if the captured images could be viewed immediately after the end of the wedding ceremony or reception, while the afterglow of the ceremony is still lingering, the bride and groom and other guests would likely feel even more moved. However, for example, when a large number of photographs are taken at a wedding, simply viewing the photographs in order is troublesome for the viewer, and is not considered appropriate. Therefore, it is considered to create an edited image album using images selected from a large number of images and make it available for viewing by related parties. However, even in this case, it is a difficult task for the photographer or other person creating the image album to decide or select which images to add to the image album from among the large number of images. Therefore, an object of the present disclosure is to provide an information processing function that supports a creator when creating such an image album. [Means for solving the problem]

[0005] An information processing device relating to the present technology includes a user interface control unit that controls the provision of a user interface that enables the specification of extraction conditions for a group of images taken at a wedding-related event, and an extraction processing unit that extracts images from the group of images based on the specified extraction conditions. Marriage-related events include, for example, wedding ceremonies, wedding receptions, after-parties, engagement ceremonies, introductions, matchmaking, and the like, and refer to events in which multiple people, including the bride and groom, participate. The present invention provides an apparatus that allows a user to specify extraction conditions for an image group consisting of a plurality of images taken at such an event, and obtains image extraction results.

[0006] In the information processing device related to the above-mentioned present technology, it is considered that the user interface control unit performs control to display the images extracted by the extraction processing unit and to provide a user interface that enables input operations of rating information for each image. For example, images that satisfy a predetermined condition are displayed in a list, and the user is allowed to arbitrarily set rating information for each image.

[0007] In the information processing device related to the above-mentioned present technology, the extraction processing unit is thought to extract images that meet specified extraction conditions by referring to analysis result data of image content for each image in the image group. For each image, information indicating the image content is obtained by image analysis processing, and for each image, it is determined whether the information resulting from this analysis matches the extraction conditions.

[0008] In the information processing device according to the present technology described above, it is considered that the analysis result data is received from an external device. Each image captured at the event is transmitted to an external device, which then executes image analysis processing, and receives the analysis result data.

[0009] In the information processing device related to the above-mentioned present technology, the user interface control unit may perform control to display an icon presenting extraction conditions for images in the image group, and detect an operation on the icon as an operation for specifying the extraction conditions. Icons indicating the image types, etc. of captured images of wedding-related events are displayed so that the user can select them.

[0010] In the information processing device related to the above-mentioned present technology, it is conceivable that the user interface control unit displays an icon that is the subject of an operation for specifying extraction conditions in a display manner different from that of an icon that is not the subject of the specified operation. For example, the color of the icon or the color of its frame may be changed, or the icon may be highlighted, making the operated icon distinguishable from other icons.

[0011] In the information processing device according to the present technology described above, it is considered that the extraction condition includes a condition for specifying a person. To enable a user to specify people to be photographed at wedding events, such as the bride, groom, relatives (parents, siblings, etc.), friends, guests of honor, etc.

[0012] In the information processing device according to the present technology described above, it is considered that the extraction condition includes a condition capable of designating a scene of an event related to a wedding. A condition is prepared that allows a user to specify a scene in a wedding event, such as a cake cutting or a guest of honor's speech.

[0013] In the information processing device according to the present technology described above, it is considered that the extraction condition includes a condition capable of specifying time information of an event related to a wedding. The time information can be extracted based on conditions such as a specific time, a period from a certain time to another time, or the time from the start of shooting.

[0014] In the information processing device according to the present technology described above, it is considered that the extraction conditions include conditions that enable designation of an object or an animal that is set as a subject. For example, conditions are provided for specifying items that are likely to be photographed at wedding events, such as cakes, candles, rings, pets, and glasses.

[0015] In the information processing device according to the present technology described above, it is considered that the extraction condition includes a condition capable of specifying an emotion or facial expression of a subject. For example, conditions for specifying smiling scenes, touching scenes, crying scenes, scenes with downcast eyes, etc. are prepared.

[0016] In the information processing device according to the present technology described above, it is considered that the user interface control unit presents information indicating the degree of emotion or facial expression of a subject of the image extracted by the extraction processing unit. For example, when images of smiling scenes are extracted, the degree of smiling is indicated for each image.

[0017] In the information processing device according to the present technology described above, it is considered that the extraction condition includes a condition capable of designating an item to be determined based on metadata added at the time of image capture. For example, it is possible to specify a condition that can be determined by referring to metadata associated with an image in the imaging device at the time of imaging, such as information regarding focus.

[0018] In the information processing device according to the present technology described above, it is considered that the user interface control unit presents a focus position for the image extracted by the extraction processing unit. For example, an image showing the in-focus position or area using a focusing frame or the like is superimposed on the extracted image with reference to in-focus area information in the metadata.

[0019] In the information processing device related to the above-mentioned present technology, it is considered that the user interface control unit performs control to display icons of the extraction conditions in a candidate area on the display screen, and to display an icon specified by a predetermined operation in a selection area on the display screen. A candidate area and a selection area are set on a display screen, and the user can operate the icon of the candidate area to be displayed in the selection area.

[0020] In the information processing device relating to the above-mentioned present technology, it is considered that the user interface control unit performs control to terminate display of the icon displayed in the selection area in response to a specified deselection operation. The user can operate the device to return the icon in the selected area to the candidate area.

[0021] In the information processing device according to the present technology described above, it is conceivable that an AND condition, an OR condition, or an exclusion condition is set for each of a plurality of specified extraction conditions to extract images. A variety of extractions are possible using a plurality of extraction conditions prepared for wedding-related events.

[0022] The information processing device according to the present technology described above may include a video editing unit that creates video content using a plurality of video scenes extracted by the extraction processing unit based on the extraction condition. Video scenes are extracted based on extraction conditions and used to create video content.

[0023] In the information processing method of the present technology, an information processing device performs a control process that provides a user interface that enables a user to specify extraction conditions for a group of images taken at a wedding-related event, and an extraction process that extracts images from the group of images based on the specified extraction conditions. These are support processes that make the work of photographers and the like to create an image album more efficient. A program according to the present technology is a program that causes an information processing device to execute processing that corresponds to such an information processing method. [Brief description of the drawings]

[0024] [Figure 1] 1 is an explanatory diagram of a system configuration for providing services according to an embodiment of the present technology; [Diagram 2] FIG. 2 is an explanatory diagram of a work procedure for providing a service according to an embodiment. [Diagram 3] FIG. 2 is a block diagram showing the configuration of a terminal device and a server device according to an embodiment. [Figure 4] FIG. 2 is an explanatory diagram of a functional configuration of a terminal device according to an embodiment. [Diagram 5] FIG. 4 is an explanatory diagram of event data on the terminal device side according to the embodiment; [Figure 6] FIG. 2 is an explanatory diagram of a functional configuration of a server device according to an embodiment; [Figure 7] FIG. 4 is an explanatory diagram of event data on the server device side according to the embodiment; [Figure 8] FIG. 4 is an explanatory diagram of Web gallery information generated by the server device according to the embodiment. [Figure 9] 11 is a flowchart of a presetting process of a terminal device according to an embodiment. [Figure 10]FIG. 13 is an explanatory diagram of a project list screen according to an embodiment. [Figure 11] 13 is a flowchart of a transfer / editing process of the terminal device according to the embodiment. [Figure 12] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 13] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 14] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 15] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 16] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 17] 13 is a flowchart of a selection support process of the terminal device according to the embodiment. [Figure 18] FIG. 4 is an explanatory diagram of an event screen according to an embodiment. [Figure 19] 13 is an explanatory diagram of tab transitions on an event screen according to an embodiment. FIG. [Figure 20] 13A and 13B are diagrams illustrating a pinch operation on the event screen according to the embodiment. [Figure 21] FIG. 2 is an explanatory diagram of a single image display screen according to an embodiment. [Figure 22] 13 is an explanatory diagram of a cover / highlight selection screen according to an embodiment. FIG. [Figure 23] FIG. 13 is an explanatory diagram of a confirmation screen according to the embodiment. [Figure 24] 11 is a flowchart of a process of a server device according to an embodiment. [Diagram 25] 13 is a flowchart of a process performed by the terminal device after receiving a URL according to the embodiment. [Figure 26] FIG. 4 is an explanatory diagram of a preview screen of a web gallery according to an embodiment. [Figure 27] FIG. 13 is an explanatory diagram of a Web gallery delivery screen according to an embodiment. [Figure 28] 13 is a flowchart of a final delivery process of a terminal device according to an embodiment. [Figure 29]11 is a flowchart of an automatic setting process of a terminal device according to an embodiment. [Diagram 30] 11 is a flowchart of an automatic setting process of a terminal device according to an embodiment. [Diagram 31] 11 is a flowchart of an example of a display control process according to the number of flags on according to an embodiment. [Diagram 32] 11A and 11B are diagrams illustrating an example of a display control process according to the number of flags on according to an embodiment. [Diagram 33] 11 is a flowchart of an example of information replacement processing according to an embodiment. [Diagram 34] 11A and 11B are diagrams illustrating examples of display related to information replacement according to an embodiment. [Diagram 35] FIG. 11 is an explanatory diagram of replacement of rating information according to an embodiment. [Diagram 36] FIG. 11 is an explanatory diagram of replacement of selection information according to an embodiment. [Figure 37] FIG. 13 is an explanatory diagram of rating settings. [Figure 38] FIG. 2 is an explanatory diagram of a configuration example for speeding up selection according to an embodiment; [Figure 39] FIG. 2 is an explanatory diagram of an example of a functional configuration according to an embodiment; [Diagram 40] FIG. 2 is an explanatory diagram of image analysis according to an embodiment. [Diagram 41] FIG. 11 is an explanatory diagram of image selection according to the embodiment. [Diagram 42] 11 is a flowchart of a process example of the imaging apparatus according to the embodiment. [Diagram 43] 13 is a flowchart of a process example of a staff terminal according to an embodiment. [Diagram 44] 11 is a flowchart of a process example of a server device according to an embodiment. [Diagram 45] 11 is a flowchart of an image analysis process according to an embodiment. [Figure 46] FIG. 11 is an explanatory diagram of a rating from image extraction according to an embodiment. [Figure 47] 11 is an explanatory diagram of icons indicating extraction conditions according to an embodiment. FIG. [Figure 48]FIG. 13 is an explanatory diagram of one person extraction according to an embodiment. [Figure 49] FIG. 11 is an explanatory diagram of extraction of multiple people according to an embodiment. [Figure 50] FIG. 13 is an explanatory diagram of a one-person close-up image according to an embodiment. [Figure 51] FIG. 2 is an explanatory diagram of a bounding box according to an embodiment. [Figure 52] FIG. 13 is an explanatory diagram of an image of a one-person shooting cut according to an embodiment. [Diagram 53] 1 is an explanatory diagram of an image of a person and an object according to an embodiment; [Figure 54] FIG. 13 is an explanatory diagram of an image of an object according to an embodiment. [Figure 55] FIG. 11 is an explanatory diagram of extraction by person and time according to an embodiment. [Figure 56] FIG. 11 is an explanatory diagram of a display of emotion recognition information according to an embodiment. [Figure 57] FIG. 2 is an explanatory diagram of an example of a user interface according to the embodiment; [Figure 58] FIG. 2 is an explanatory diagram of an example of a user interface according to the embodiment; [Figure 59] FIG. 2 is an explanatory diagram of an example of a user interface according to the embodiment; [Figure 60] FIG. 2 is an explanatory diagram of an example of a user interface according to the embodiment; [Figure 61] FIG. 11 is an explanatory diagram of extraction using emotion recognition information according to an embodiment. [Figure 62] 11A and 11B are explanatory diagrams of narrowing down based on in-focus position information according to an embodiment. [Figure 63] 11A and 11B are explanatory diagrams of narrowing down based on in-focus position information according to an embodiment. [Figure 64] 11 is a flowchart of detection of a pupil-focused image according to an embodiment. [Figure 65] FIG. 13 is an explanatory diagram of extraction of an image focused on an object according to an embodiment. [Figure 66] FIG. 13 is an explanatory diagram of extraction of an image focused on an animal according to an embodiment. [Figure 67] FIG. 10 is an explanatory diagram of an extraction of an image focused on an extract according to an embodiment. [Figure 68] FIG. 11 is an explanatory diagram of extraction based on an OR condition according to the embodiment. [Figure 69] FIG. 11 is an explanatory diagram of extraction using an exclusion condition according to an embodiment. [Figure 70] FIG. 11 is an explanatory diagram of extraction using an exclusion condition according to an embodiment. [Figure 71] FIG. 11 is an explanatory diagram of customization of extraction conditions according to an embodiment. [Figure 72] FIG. 13 is an explanatory diagram of application of the embodiment to a moving image. [Figure 73] FIG. 11 is an explanatory diagram of highlight movie production according to an embodiment. [Figure 74] 13 is a flowchart of a highlight movie production process according to an embodiment. [Figure 75] FIG. 13 is an explanatory diagram of a check screen for images that are candidates for delivery according to an embodiment. [Figure 76] FIG. 11 is an explanatory diagram of face filtering according to an embodiment. [Figure 77] 11 is an explanatory diagram of the presentation of a person not included in the delivery image of the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The embodiments will be described below in the following order. <1. System configuration and operation procedure for image gallery service> <2. Configuration of information processing device> <3. Pre-setting process> <4. Transfer / Editing Process> <5. Selection process> <6. Server processing> <7. Web Gallery Delivery Processing> <8. Delivery Processing> <9. Automatic setting process> <10. Selection Speed-Up Support Processing> <11. Final confirmation support for selecting delivery images> <12. Face-based Presentation> <13. Automatic selection of delivery images> <14. Using facial information to show people not included in the delivery photo> <15. Shooting support using shot lists> <16. Summary and Modifications>

[0026] <1. System configuration and operation procedure for image gallery service> In the embodiment, it is assumed that a photographer takes still images (photographs) at an event such as a wedding ceremony, generates content as an image collection from the captured images, and provides the content to a client with or without payment. It should be noted that the present invention is not limited to still images, and that it is also possible to capture moving images with or without sound and generate an image collection including these moving images, or an image collection made up of multiple moving images.

[0027] For the purpose of explanation, cameramen and their assistants are collectively referred to as camera staff. The term "client" is used to refer not only to the person placing the order, but also to any user who is provided with and can view the image collection. For example, the term "client" refers collectively to the bride and groom, their relatives, friends, and other related parties.

[0028] An image collection broadly refers to content that is a collection of images (still images and videos), such as a photo album, and the form in which it is provided is not important, but in this embodiment, as an example, it is possible to provide an image collection as a web gallery and an image collection of actual delivery.

[0029] The image collection of a web gallery referred to here is an electronic image collection that allows photos of an event, such as a wedding, to be viewed quickly on a website after the event has ended. In the explanation, an example using a website is used; however, this is just one example, and the content file as an image collection may be sent to a client's terminal device using, for example, email, electronic message, SNS, etc.

[0030] The image collection to be delivered is, for example, an image collection of an event that will be officially produced as a photo album by camera staff at a later date and provided to a client. The image data captured at the event may be edited to create an electronic image collection as image album content or slide show content, or an image collection on paper, for example. The electronic image collection may be provided by download or streaming, or may be provided by storing it on a portable storage medium such as a disk-shaped storage medium, a card-shaped storage medium, a tape-shaped storage medium, or a flash memory medium.

[0031] In this embodiment, a web gallery image collection is generated that can be quickly viewed by clients before the final delivery of the image collection. The web gallery image collection is not simply a large number of captured images lined up, but is selected images with a quality that allows viewers to enjoy them. On the other hand, the collection of images delivered here is positioned as something that camera crews can take their time to create by editing in a more creative way.

[0032] FIG. 1 shows terminal devices 1A and 1B, an imaging device 2, a personal computer 3, a server device 4, terminal devices 5A and 5B, and a network 6. The terminal devices 1A and 1B indicate terminal devices used by camera staff, and for ease of explanation, these are collectively referred to as a staff terminal 1. The terminal devices 5A and 5B indicate terminal devices used by users as clients as described above, and for the sake of explanation, these are collectively referred to as client terminal 5.

[0033] As the staff terminal 1 and the client terminal 5, a mobile terminal device such as a smartphone or a tablet device is exemplified here. For example, the terminal device 1A is assumed to be a smartphone used by a cameraman, and the terminal device 1B is assumed to be a smartphone used by an assistant. Terminal device 5A is assumed to be a smartphone used by the bride, and terminal device 5B is assumed to be a tablet device used by a friend at the venue. The staff terminal 1 and the client terminal 5 may be any so-called information processing device, specific examples of which include a personal computer device, a mobile phone device, a game device, an audio device, a video device, a communication device, a television device, etc. In other words, any information processing device capable of performing information processing operations, such as a device with a built-in microcomputer, can be used as the staff terminal 1 and the client terminal 5 of the present disclosure. However, since it is assumed that the device will be used at event venues and the like, it is preferable that the device be a portable terminal such as a smartphone or tablet device.

[0034] Various types of imaging devices such as video cameras and still cameras are assumed to be the imaging device 2. The imaging device 2 shown in the figure is assumed to be a camera used by a photographer at a wedding venue.

[0035] Data communication is possible between the imaging device 2 and the staff terminal 1 (for example, the terminal device 1A). For example, data communication is performed by FTP (File Transfer Protocol) communication. Specifically, it is conceivable that the terminal device 1A serves as an FTP server, and image files (including image data and metadata) captured by the imaging device 2 are uploaded sequentially or collectively via FTP communication. Therefore, it is assumed that the imaging device 2 is configured with an FTP setting for transferring data to the terminal device 1A. The FTP setting information includes the host name of the FTP server, the save path, the user name, the password, the connection type, and the like.

[0036] However, the imaging device 2 and the terminal device 1A may use other protocols instead of FTP. Any communication method may be used between the imaging device 2 and the terminal device 1A as long as images and metadata can be transferred between them. For example, information may be exchanged between them by short-distance wireless communication such as Bluetooth (registered trademark), Wi-Fi (Wireless Fidelity: registered trademark), or NFC (Near Field Communication: registered trademark), or by infrared communication. Furthermore, the imaging device 2 and the terminal device 1A may be capable of communicating with each other via a wired connection such as a wired LAN.

[0037] The personal computer 3 is exemplified as an information processing device used by, for example, camera staff to create the image album content for delivery. This personal computer 3 is used to obtain data relating to images of the event from the server device 4 and to create the image album content for delivery using an image editing application or the like.

[0038] The server device 4 provides and manages various information for the camera staff to create a web gallery or a collection of images for actual delivery. For example, the staff terminal 1 has an application program for an image album providing service installed, and performs the processes described below in accordance with the application program. In response to this, the server device 4 manages data related to the event, creates a web gallery, and so on.

[0039] The staff terminal 1 and the server device 4 are in a constant connection state, for example, while the above application program is running, and event data is synchronized. Examples of the event data will be described later, and include image data captured at the event and data used to generate a web gallery.

[0040] In this disclosure, "synchronization" refers to maintaining the event data stored in the staff terminal 1 and the event data stored in the server device 4 so that at least a portion of the data content is the same. In particular, data to be synchronized includes image data, rating information, selection information, editing information, etc. The contents of each will be described later.

[0041] The network 6 may be, for example, the Internet, a home network, a LAN (Local Area Network), a satellite communication network, or any other type of network. The staff terminal 1 can upload image data and the like to the server device 4 via the network 6 and can transmit information for synchronization. The client terminal 5 can access the web page provided by the server device 4 via the network 6 and view the web gallery. The personal computer 3 can obtain image files for creating the image album content of the actual delivery from the server device 4 via the network 6 .

[0042] The imaging device 2 has a display device such as a display panel provided on the back of the device, a viewfinder, etc., and allows input of operations using various controls, a touch panel, etc. In other words, it has sufficient functionality as a user interface. The imaging device 2 also has a microcomputer mounted thereon, allowing various information processing and communication. For this reason, it is also assumed that the imaging device 2 functions as an information processing device as referred to in this disclosure, communicating with the server device 4 without going through the terminal devices 1A and 1B, and executing processing performed by the terminal devices 1A and 1B as described below. In the embodiment, the processing of the staff terminal 1 and the server device 4 will be mainly described, but an example in which the imaging device 2 performs all or part of the processing of the staff terminal 1 described below is also conceivable.

[0043] An outline of the procedure for providing an image album realized by a system such as that shown in Fig. 1 will be explained with reference to Fig. 2. This is a procedure for operations at the staff terminal 1 based on the operations of the camera staff as the service provider.

[0044] In step S1, advance settings are made. For example, the camera staff performs communication settings between the staff terminal 1 and the imaging device 2, event settings, image retouching settings, etc. This advance setting is performed before the start of the wedding ceremony.

[0045] In step S2, images are captured during the wedding ceremony, transferred, and automatically edited. Image files captured by the cameraman using the imaging device 2 are sequentially transferred to the staff terminal 1 (for example, terminal device 1A). The staff terminal 1 imports image files, transfers the image files to the server device 4, and automatically edits the images according to pre-setup.

[0046] Step S3 shows the work that can be done on the day after the wedding ceremony. The camera staff performs selection operations using the staff terminal 1. Selection includes the selection of images to be posted in the web gallery, the selection of cover images and highlight images, and further image editing. The staff terminal 1 performs update processing of the event data in response to the operations of the camera staff. Processing for synchronizing the event data with the server device 4 is also performed sequentially.

[0047] In step S4, the web gallery is delivered. The server device 4 creates a web gallery based on the event data. The camera staff checks the contents of the web gallery on the staff terminal 1 and then performs an operation to make it available for viewing by the client. This allows the client to view the web gallery, which is a collection of images from the wedding, for example, on the day after the wedding ceremony.

[0048] Step S5 shows the creation of an image collection of the actual deliveries and the processing of the actual deliveries. The camera staff will later create the image album content for the actual delivery using, for example, a personal computer 3 and provide it to the client. This allows the client to receive the image gallery content as the final delivery.

[0049] <2. Configuration of information processing device> 3 shows a configuration example of an information processing device 10 that can be configured as a staff terminal 1, a client terminal 5, or a server device 4 or a personal computer 3. The staff terminal 1, the personal computer 3, or the server device 4 can execute the processes described below by having the following configuration of the information processing device 10. The imaging device 2 can also execute the same processes as the staff terminal 1 described below by having the following configuration of the information processing device 10.

[0050] A CPU (Central Processing Unit) 71 of the information processing device 10 executes various processes according to a program stored in a ROM (Read Only Memory) 72 or a program loaded from a storage unit 79 to a RAM (Random Access Memory) 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes as appropriate. The CPU 71, the ROM 72, and the RAM 73 are interconnected via a bus 74. To this bus 74, an input / output interface 75 is also connected.

[0051] The input / output interface 75 is connected to an input unit 76 including an operator and an operating device. For example, the input unit 76 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. An operation by the user is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

[0052] Further, the input / output interface 75 is connected, either integrally or separately, to a display unit 77 formed of a liquid crystal panel (LCD: Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or the like, and an audio output unit 78 formed of a speaker, or the like.

[0053] The display unit 77 is a display unit that performs various displays, and is configured, for example, by a display device provided in the housing of the information processing device 10, a separate display device connected to the information processing device 10, or the like. The display unit 77 displays images for various types of image processing, moving images to be processed, etc., on the display screen based on instructions from the CPU 71. Furthermore, the display unit 77 displays various operation menus, icons, messages, etc., that is, as a GUI (Graphical User Interface), based on instructions from the CPU 71.

[0054] The input / output interface 75 may be connected to a storage unit 79 configured with a hard disk or solid-state memory, or a communication unit 80 that performs communication using various communication methods.

[0055] The communication unit 80 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like. When this information processing device 10 is considered as the staff terminal 1, the communication unit 80 has a function of FTP communication with the imaging device 2 and a function of communicating with the server device 4 via the network 6. Alternatively, the communication unit 80 may have a function of communicating by short-range wireless communication such as the above-mentioned Bluetooth, Wi-Fi, and NFC, infrared communication, wired connection communication, etc. The communication unit 80 may also have a function of communicating by a communication line for mobile phones such as LTE (Long Term Evolution). When the information processing device 10 is considered as a server device 4 or a personal computer 3, the communication unit 80 has at least a function of performing communication via the network 6.

[0056] A drive 82 is also connected to the input / output interface 75 as required, and a removable recording medium 81 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately mounted thereon. The drive 82 allows data files such as image files and various computer programs to be read from the removable recording medium 81. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the sound output unit 78. Furthermore, the computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as necessary.

[0057] In the information processing device 10, for example, software for the processing of the present disclosure can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0058] Assuming that the staff terminal 1 is the information processing device 10, a functional configuration as shown in FIG. 4 is constructed in the CPU 71 of the information processing device 10 by software (application program). FIG. 4 shows an event data management unit 30, a UI (user interface) control unit 31, a communication control unit 32, and an automatic setting unit 33 as functions provided in the information processing device 10 serving as the staff terminal 1.

[0059] The event data management unit 30 is a processing function that stores image data received through communication with the imaging device 2 and additional information including rating information corresponding to the image data as event data related to a specific event. The event data management unit 30 also performs update processing of the event data in response to operational input by the camera staff.

[0060] FIG. 5 shows an example of the contents of the event data. Event data is a data group including images and various details of an event that is registered by a camera staff member operating the staff terminal 1. Figure 5 shows event data 60 for an event with the event name "event2". In this example, it is assumed that the event data 60 includes preset information 61 , an imaging / editing information table 62 , and trend information 63 .

[0061] The pre-setting information 61 is information that the camera staff set in advance in step S1 of Fig. 2. For example, the pre-setting information 61 includes editing information such as retouching parameters.

[0062] The imaging / editing information table 62 is a table to which information is successively added and updated in the processes of steps S2 and S3 in FIG. In this imaging / editing information table 62, various information is written in correspondence with the image file transferred from the imaging device 2. Image data and metadata (MT1, MT2, . . . ) are included in the image file transferred from the imaging device 2. This image data is managed as original image data, and the added metadata is also managed. The image data may be in any format, such as raw data or data compressed using the JPEG format, but here we use JPEG compressed data, such as "DSC0001.jpg", as an example. The metadata (MT1, MT2, . . . ) includes the image capture date and time of the image data, camera information, data format, data size, field of view information, focus point information, and other information added by the imaging device 2.

[0063] The imaging / editing information table 62 stores rating information corresponding to image data. The rating information is information indicating the rating value given by the photographer or the like to the image, and is, for example, six-level rating information, such as "no stars," "one star," "two stars," "three stars," "four stars," and "five stars," indicated by the number of star marks. For example, the higher the number of stars, the higher the rating. Of course, such rating information is only one example. Such rating information is useful for camera staff in selecting images, so for example, when setting the selection flag (setting the image information to be used), which will be described later, it can be said that images with high ratings indicated by the rating information are images that are likely to be used in a web gallery or for actual delivery. This rating information can be assigned to an image by a cameraman after checking the captured image when the cameraman captures the image. The imaging device 2 is provided with a user interface for inputting the rating information. This rating information may be included in metadata corresponding to the image data, for example, and may be transferred to the staff terminal 1 together with the image data. Alternatively, the rating information may be written in the XMP area of ​​Exif in the image file. The staff terminal 1 extracts and manages rating information from the metadata (or Exif data). This rating information can be added or updated in association with image data by operating the staff terminal 1 side.

[0064] The imaging / editing information table 62 stores selection information corresponding to image data. One of the selection information is the "image used" information that indicates whether or not the image will be included in the Web Gallery. This is the "selection flag" (also simply "flag" as described below), and when the flag is off, the image will not be used in the Web Gallery, and when the flag is on, the image has been selected for use in the Web Gallery. One piece of selection information is cover image information indicating that the image is selected as a cover image to be placed as a cover at the top of the Web gallery. Also, one type of selection information is highlight image information that indicates that the image is selected to be highlighted in the web gallery. Highlight display refers to displaying the image larger than other images. Alternatively, normal images may be monochrome images, while highlight images may be displayed in color. Conversely, in terms of emphasis, normal images may be color images, while highlight images may be displayed in monochrome. Furthermore, normal images may be displayed without a border, while highlight images may be displayed with a border. There are various possible ways of highlighting.

[0065] The editing information includes various image processing parameters such as retouching parameters, tilt correction parameters, etc. The retouching parameters include image quality adjustment parameters such as brightness adjustment values, color adjustment values, contrast adjustment values, and sharpness adjustment values. Retouching parameters can be set in advance, but can also be set for each individual image. Tilt correction parameters are set individually. For these reasons, editing information corresponding to image data is stored so that parameters set for each individual image can be stored.

[0066] As shown in the figure, the imaging / editing information table 62 may store editing information corresponding to image data. Edited image data is image data that has been edited based on preset editing information or individually set editing information for the original image data. In addition to image data that has been retouched based on editing information, edited image data may also refer to image data that has had its quality changed. Examples of quality changes include changes in image size, resolution, cropping, and frame rate for videos. In the figure, edited image data is distinguished from original image data by adding an "E" to the file name, such as "DSC0001E.jpg." This edited image data is used for display on the user interface of the staff terminal 1, etc. Such edited image data does not necessarily have to be created and stored, and for example, when an image is displayed, the editing information may be applied and displayed each time. Therefore, it is sufficient that the editing information is updated and managed in the imaging / editing information table 62, and it is not necessary to store the edited image data. However, it is of course effective in terms of reducing the processing load during display processing by generating and storing edited image data as needed, as shown in the figure.

[0067] In addition to the above-mentioned preset information 61 and shooting / editing information table 62, trend information 63 may be stored in the event data 60. Trend information is, for example, characteristic information such as the selection tendency of camera staff and the tendency of clients (for example, brides) to like images. An example of using this trend information 63 will be described later.

[0068] The CPU 71 of the information processing device 10 serving as the staff terminal 1 performs additional storage and updating of information on the event data 60 as described above using the function of the event data management unit 30 in FIG. The format of the event data 60 and the format of the imaging / editing information table 62 are merely examples, and any format may be used as long as the above contents can be managed for each event. For example, the original image data, metadata, rating information, selection information, editing information, and edited image data included in the imaging / editing information table 62 are not limited to being stored together as a table, and may be stored in any format or manner as long as the metadata, rating information, selection information, editing information, and edited image data are associated with at least one piece of original image data.

[0069] 4 is a function that controls the presentation of images and rating information related to event data 60 of a specific event, and detects an operation input, that is, a function that performs user interface processing. User interface processing includes presenting various types of information, providing an environment in which operational input is possible, detecting operations by the user, and detecting / estimating the user's intentions. Specifically, the UI control unit 31 performs control processing to provide the user with an operation input environment and present information by, for example, causing the display unit 77 and the audio output unit 78 to execute display output, audio output, and other outputs to the user. Alternatively, the UI control unit 31 performs a process of detecting an operation by a user, for example. Alternatively, the UI control unit 31 performs, for example, both a process of providing an operation input environment to the user and a process of detecting an operation by the user. Of course, the UI control unit 31 may perform other user interface processes.

[0070] The communication control unit 32 has a function of controlling communication for synchronizing the event data 60 with the external server device 4 in response to storage or updating of the event data 60. For example, when an image transfer is received from the imaging device 2, the communication control unit 32 controls the process of transferring the image data from the communication unit 80 to the server device 4, and also controls notifying the server device 4 of the update contents in response to updates to the contents of the event data 60.

[0071] The automatic setting unit 33 is a function for automatically setting rating information, selection information, or editing information using the above-mentioned trend information 63. The processing of this automatic setting unit 33 will be described later together with the trend information 63.

[0072] The above functions in Fig. 4 are merely examples. By providing the information processing device 10 with at least the event data management unit 30, the UI control unit 31, and the communication control unit 32, the information processing device 10 can perform processing as the staff terminal 1 of the present embodiment. Each of these functions is realized, for example, by the information processing device 10 installing an application program for creating image album contents.

[0073] Next, assuming that the server device 4 is the information processing device 10 as shown in FIG. 3, a functional configuration as shown in FIG. 6 is constructed in the CPU 71 of the information processing device 10 by software (application program). FIG. 6 shows an event data management section 40, an image album generation section 41, an event data provision control section 42, a data output section 43, and a trend information generation section 44 as functions provided in the information processing device 10 serving as the server device 4.

[0074] The event data management unit 40 is a processing function that performs storage and update management of event data in the server device 4. The contents of the event data 90 on the server device 4 side are shown in Fig. 7. As in Fig. 5, the event data 90 is for an event with the event name "event2". In this example, it is assumed that the event data 90 includes preset information 91 and an imaging / editing information table 92 .

[0075] The pre-setting information 91 is the same as the pre-setting information 61 in the staff terminal 1. For example, the contents set in the pre-setting in step S1 in FIG.

[0076] The imaging / editing information table 92 has almost the same information as the imaging / editing information table 62. That is, the original image data, metadata, rating information, selection information, and editing information can be managed for each image. In addition, the imaging / editing information table 92 does not associate edited image data, but this is because edited images are created when the web gallery is generated. Of course, the server device 4 may also generate edited image data one by one, and associate the edited image data with the original image data in the imaging / editing information table 92. The CPU 71 of the information processing device 10 serving as the server device 4 performs additional storage and updating of information regarding the event data 90 as described above using the function of the event data management unit 40 in FIG.

[0077] The image album generating unit 41 in Fig. 6 has a function for generating image album content as a web gallery of a specific event. In particular, the image album generating unit 41 references the selection information and editing information in the event data 90 to generate image album data, i.e., web page data to be used as a web gallery. An example of the generated web gallery data is shown in FIG. The server device 4 generates page information in HTML (HyperText Markup Language) format or XML (Extensible Markup Language) format as web gallery data, for example as web page setting information 96, and also generates an image folder 97 that collects images called from the web page setting information 96 and displayed on the page. Image folder 97 contains edited image data for images that have been adopted as web galleries.

[0078] The event data provision control unit 42 in FIG. 6 is a function that controls transmission of the event data 90 to the staff terminal 1 for providing the event data 90. For example, the staff terminal 1 logs in to the server device 4, and in response to specifying an event, all or part of the event data 90 for that event is transmitted to the staff terminal 1. The staff terminal 1 manages the received content as event data 60. This allows an assistant to operate the terminal device 1B using the event data 60, for example. Of course, the event data provision control unit 42 may also control the provision of event data to the terminal device 1A and the personal computer 3.

[0079] The data output unit 43 controls the transmission of some or all of the data contained in the event data, such as metadata, rating information, selection information, editing information, or image data (e.g., original image data), to a terminal device (e.g., a personal computer 3) as a file in a specified format. In other words, it is a processing function that provides the camera staff with the data necessary to create the image album content for this delivery.

[0080] The tendency information generating unit 44 is a function that learns the tendency of camera staff to select images and the tendency of users who have become clients to evaluate images, and generates characteristic information thereof. The processing of the tendency information generating unit 44 will be described later together with the processing of the automatic setting unit 33 of the staff terminal 1.

[0081] The above functions in Fig. 6 are merely examples. By providing the information processing device 10 with the event data management unit 40, event data can be synchronized between the staff terminal 1 and the server device 4. Furthermore, by providing the server device 4 with the image album generation unit 41, web galleries can be generated quickly. Each function in FIG. 6 is realized, for example, by the information processing device 10 installing an application program for creating image album contents.

[0082] <3. Pre-setting process> A specific processing example will be described below. First, an example of the presetting process at the staff terminal 1 shown as step S1 in FIG. 2 will be described with reference to FIGS. Note that the process of the staff terminal 1 referred to below is the process performed by the CPU 71 of the information processing device 10 having the functions of FIG. Moreover, each screen example described below with reference to the flowchart assumes that the information processing device 10 serving as the staff terminal 1 is a smartphone, and is an example of the display content on a display formed on the housing of the smartphone.

[0083] FIG. 9 shows the process of the staff terminal 1. For example, a camera staff member starts an application program using the staff terminal 1 and performs an operation for preparation processing. In response to the operation, the CPU 71 of the staff terminal 1 executes the processing from step S10 onward.

[0084] In step S10, a project creation process is performed on the staff terminal 1. This is a process for setting an event for which a photo album is to be created. For example, on the display unit 77 of the staff terminal 1 on which the application program has been started, a project list screen 110 is displayed as shown in FIG. 10A. A list area 111 is provided on the project list screen 110, in which event names that are the subject of each project are displayed in a list. Also displayed are an add icon 112, a camera icon 134, a menu icon 127, and the like.

[0085] The camera staff creates a project for which an image album is to be created on this project list screen 110. For example, the camera staff performs an operation to create one project by inputting an event name, etc. For example, by operating the add icon 112, a project setting screen (not shown) is displayed, and the project title (for example, event name), the date and time of the event, the location, whether or not to apply auto-retouch settings, and the like can be set. The camera staff inputs the appropriate items and performs an operation to set the project contents. In response to this, one project is created on the staff terminal 1 and is displayed in the list area 111.

[0086] Fig. 10B shows an example in which two projects named "event1" and "event2" are displayed. For each project, information such as the event name, date and time, and location is displayed. In step S10 in FIG. 9, the staff terminal 1 performs control to provide such a user interface on the screen by using the function of the UI control section 31, and performs a project creation process in response to the operation of the camera staff member.

[0087] This project creation process is performed, for example, when the staff terminal 1 and the server device 4 are connected online. In this case, the staff terminal 1 uses the function of the communication control unit 32 to transmit information for synchronization processing to the server device 4 in response to the user's operation. This allows the server device 4 to have information about one project entered synchronously.

[0088] 9, the staff terminal 1 performs transfer settings. The transfer settings are settings for FTP transfer between the staff terminal 1 and the imaging device 2, such as settings for the file format to be transferred and the FTP connection server. The staff terminal 1 uses the function of the UI control unit 31 to display a screen for transfer settings to the camera staff, and performs necessary transfer settings in response to operations by the camera staff.

[0089] In step S12, the staff terminal 1 performs auto-retouching settings. The auto-retouching settings are a process for setting parameters for image editing processing that is commonly applied to each image. The staff terminal 1 uses the functions of the UI control unit 31 to display a screen for auto-retouching settings to the camera staff, and performs the necessary auto-retouching settings in response to operations by the camera staff. The parameters of this auto-retouching setting are described as pre-setting information 61 in the event data 60 of the created project. Furthermore, this auto-retouching setting is also performed, for example, while the staff terminal 1 and the server device 4 are connected online, and the staff terminal 1 transmits parameter information for which auto-retouching has been set to the server device 4 for synchronization processing. This allows the server device 4 to also store pre-setting information 91 having the same contents as the pre-setting information 61 of the event data 60 as the event data 90 of the project.

[0090] In step S13, the staff terminal 1 sets a watermark. For example, the name of the photographer can be inserted as a watermark (digital watermark) in the images in the web gallery. The watermark setting is a process for setting the presence or absence of this watermark and its contents. The staff terminal 1 displays a screen for setting a watermark to the camera staff using the function of the UI control unit 31, and sets the necessary watermark in response to the camera staff's operation. This watermark setting information is also described as pre-setting information 61 in the event data 60 of the created project. This watermark setting is also performed while the staff terminal 1 and the server device 4 are in an online connection state, and the staff terminal 1 transmits information on the watermark setting to the server device 4 for synchronization processing. This allows the server device 4 to store pre-setting information 91 synchronized with the pre-setting information 61 in the event data 90 of the project.

[0091] The order of steps S11, S12, and S13 may be different. Furthermore, the processing of each step does not have to be performed as a series of processing, and may be performed each time in response to a user operation during startup, for example. Also, each process may be skipped if not necessary. For example, the transfer setting in step S11 may be performed each time a project is created in step S10, but if the FTP transfer setting has been performed once and there is no particular need to change the setting, step S11 may be skipped after the project is created in step S10 and the process may proceed to step S12. Of course, the same applies to the processes in steps S12 and S13, and if there is no particular need to change the setting, these may be skipped. In addition, communication with the server device 4 for synchronization may be performed at the timing of each process, but the pre-setting information 61, 91 may also be synchronized by sending the information collectively when several settings are made. Furthermore, if there is content that has been set in an environment in which online connection with the server device 4 is not possible, it is preferable that the set information be transmitted and the pre-configured information 61, 91 be synchronized when an online connection is established.

[0092] <4. Transfer / Editing Process> Next, a specific example of the processing of the staff terminal 1 in step S2 in Fig. 2 will be described with reference to Fig. 11. That is, this is processing executed in the staff terminal 1 mainly by the functions of the event data management unit 30 and the communication control unit 32 in Fig. 4 while the wedding ceremony is taking place.

[0093] The staff terminal 1 checks whether or not image file transfer from the imaging device 2 has started in step S101 of Fig. 11. When no image transfer is in progress, the process of Fig. 11 is not particularly performed. For example, during a wedding ceremony, a photographer uses the imaging device 2 to capture images of the ceremony and reception. After capturing still images, for example, the imaging device 2 transfers image files to the staff terminal 1 automatically or in response to the cameraman's operation. Note that the transfer timing is an example. Transfer may be performed each time an image is captured, or multiple images may be transferred together. In the case of video capture, the video may be transferred during the period when recording is not being performed between start and stop, or during the recording period after recording has started. All recorded video may be transferred, or only the initial specified period may be transferred.

[0094] Furthermore, the cameraman can add rating information to the captured image by operating the imaging device 2. For example, he / she checks the image immediately after capturing it and inputs the rating information. This rating information is written in the metadata. The transfer of an image file including image data and metadata may be performed, for example, in response to a cameraman checking a captured image and inputting rating information.

[0095] When the staff terminal 1 detects that such transfer of the image file has started, the process proceeds from step S101 to step S102, and a process of receiving the image data and metadata is performed. In this case, the received image data is managed as original image data in the imaging / editing information table 62 of the event data 60. The received metadata is also managed accordingly.

[0096] The staff terminal 1 performs image compression processing in step S103. For example, the compression processing is performed so as to increase the compression rate of the received image data. This is processing for generating image data to be transmitted to the server device 4.

[0097] In step S104, the staff terminal 1 performs rating information management. This is a process in which the metadata of the received image file is checked, the rating information added by the imaging device 2 is extracted, and the information is managed in an updatable manner in the imaging / editing information table 62 of the event data 60.

[0098] In step S105, the staff terminal 1 transmits information to the server device 4 to synchronize the event data 60 and 90. In this case, the staff terminal 1 transmits the image data, metadata, and rating information compressed in step S103 to the server device 4. In response to receiving this, the server device 4 updates the event data 90 and maintains the synchronized state with the event data 60.

[0099] In step S106, the staff terminal 1 performs an automatic retouch application process. That is, image processing is automatically performed on the original image data using retouch parameters that have been set in advance to generate edited image data, which is then managed in the imaging / editing information table 62. This edited image data is generated for use in UI display on the staff terminal 1. In this example, auto-retouch application processing is performed when an image file is received, but it may be performed at other times as needed, for example, when a request is made to display a single image. It is also possible to perform image processing on the compressed image data generated in step S103. In that case, for example, the order of the processes in steps S105 and S106 may be reversed, and the compressed image data may be subjected to image processing for applying automatic retouching before being transmitted to the server.

[0100] 11, the staff terminal 1 captures images captured by the photographer, and information corresponding to the images is managed by the event data 60. For example, when a wedding ceremony is over, all captured images are managed by the event data 60. Furthermore, by carrying out transmission for successive synchronization, the event data 90 on the server device 4 side is synchronized with the event data 60 on the staff terminal 1 side during and at the end of the wedding ceremony.

[0101] Also, during the processing of Figure 11, a list of images transferred from the imaging device 2 and the communication status are displayed on the event screen 120 described below, so that the camera staff can check the status of image import to the staff terminal 1, etc.

[0102] <5. Selection process> Next, a detailed example of the selection process in step S3 in FIG. 2 will be described with reference to the flowcharts in FIGS. 12 to 17 and the screen examples in FIGS. Note that Figures 12 to 17 show a series of flowcharts in separate parts, with the connections indicated by "c1", "c2", "c3", "c4", "c5", "c6", and "c7". The processing of these flowcharts is performed by the functions of the event data management unit 30, UI control unit 31, and communication control unit 32 in FIG.

[0103] Step S201 in FIG. 12 shows a process of displaying a list of projects (event names, etc.) on the project list screen 110 as shown in FIG. 10B. In step S202, the staff terminal 1 monitors whether or not a project selection operation has been performed on the project list screen 110, and in step S203, monitors whether or not an application program has been terminated. When an operation to terminate the application program is performed, the application program is closed and processing is completed.

[0104] For example, when making a selection immediately after a wedding ceremony, the camera staff performs an operation to select a target project on the project list screen 110. For example, the camera staff performs an operation such as tapping one of the projects displayed in the list area 111. When such an operation is detected, the staff terminal 1 advances the process from step S202 to step S204, and acquires the latest event data for the selected project. For example, the staff terminal 1 requests the event data 90 for the selected project from the server device 4, and imports the transmitted event data 90. It then compares it with the event data 60 it holds, and updates it to the latest information. For example, it compares some information that can determine whether the information is new or old, such as update date and time information or version management information for the event data 60 and the event data 90, and if they differ, it updates the event data 60 to have the latest information.

[0105] For example, assuming that the terminal device 1A is carried by a photographer, during the wedding ceremony an assistant may be entering rating information and selection information on the terminal device 1B side, and this information may not be reflected in the event data 60 on the terminal device 1A side. Furthermore, when rating information or selection information is to be input on the side of terminal device 1B possessed by the assistant, it is appropriate to input the latest information on the side of terminal device 1A as the standard. Therefore, the staff terminal 1 receives the event data 90 of the server device 4, which has the latest information sent from both the terminal devices 1A and 1B, and updates the information of its own event data 60 if it is not the latest.

[0106] In addition, there may be a case where there is only one staff terminal 1 and the information in the staff terminal 1 is always the latest. In such a case, the process of step S204 may be unnecessary.

[0107] Next, the staff terminal 1 proceeds to step S211 in Fig. 13 to display a list of event images. For example, an event screen 120 as shown in Fig. 18 is displayed. The event screen 120 displays an event title 121, an all flags icon 123, a next icon 132, a camera icon 134, a menu icon 127, and the like. The menu icon 127 is an operator for displaying various menu items. The camera icon 134 is an icon for displaying the FTP communication status with the imaging device 2 and the like. The all flag icon 123 is an operator for setting on / off the flag (selection flag) which is the image information to be used for all images. The next icon 132 is an operator for transitioning to the next screen.

[0108] The event screen 120 also has an image area 128, in which images transferred from the imaging device 2 are displayed as thumbnails in a list form. For example, the images are displayed in chronological order according to the imaging time. In addition, an all tab 125 and a candidate tab 126 are provided for displaying the image area 128 . When all tabs 125 is selected, images transferred from imaging device 2 are displayed in image area 128 regardless of whether the flag state as used image information is on or off. For example, assume that Fig. 18 shows a list display when all tabs 125 is selected. On the other hand, when the candidate tab 126 is selected, only images whose flags are set to ON are displayed in a list in the image area 128 out of all images transferred from the imaging device 2. For example, Fig. 19 shows an example of the list display when the candidate tab 126 is selected. The flag is set on / off by the camera staff for each image depending on whether or not the image is to be included in the web gallery.

[0109] Furthermore, a filter designation area 129 is provided on the event screen 120. The filter designation area 129 allows images to be narrowed down by, for example, a six-level rating. For example, an operator is provided as the number of stars, filtering is performed according to the designated operator, and the extracted images are displayed in a list in the image area 128. For example, when a three-star control is operated, images with rating information of "three stars" or higher (i.e., images that fall into any of the categories of "three stars," "four stars," and "five stars") are extracted and displayed in a list. Alternatively, in this case, only images with rating information of "three stars" may be extracted and displayed in a list. In addition, in the filter designation area 129, together with the controls expressed as the number of stars, the number of images to which that rating has been set is also shown.

[0110] In the state where the staff terminal 1 has displayed such an event screen 120 in step S211, the staff terminal 1 performs a monitoring loop process from step S212 to step S219 in FIG.

[0111] 13, the staff terminal 1 monitors the operation of the next icon 132. The process when the operation of the next icon 132 is detected will be described later.

[0112] In step S213, the staff terminal 1 monitors a tab switching operation. A tab switching operation is an operation of the candidate tab 126 displayed in a list in the all tab 125, or an operation of the all tab 125 displayed in a list in the candidate tab 126. When the tab switching operation is detected, the staff terminal 1 performs a process of switching the list display contents in the image area 128 in accordance with the selected tab in step S220.

[0113] In step S214, the staff terminal 1 monitors a pinch operation performed by the camera staff on the image area 128. The pinch operation here refers to a pinch out operation for enlarging an image, and a pinch in operation for reducing an image. When the pinch operation is detected, the staff terminal 1 enlarges or reduces the display state in the image area 128 as a display corresponding to the operation in step S221.

[0114] 20A, 20B, and 20C show examples of changes in the display of image region 128 in response to pinching in / out. Fig. 20A shows the normal display state, the same as Fig. 18. When a pinch-out operation is performed, the images displayed as a list are enlarged as shown in Fig. 20B. When a further pinch-out operation is performed, the images are enlarged as shown in Fig. 20C. Furthermore, when a pinch-in operation is performed, the screen is reduced from the state of FIG. 20C to the state of FIG. 20B, and further pinch-in operation returns the screen to the state of FIG. 20A.

[0115] Also, in this example, although they are not displayed in the normal display state of Fig. 20A, when the image is enlarged by a pinch-out operation, rating icon 130 and flag setting icon 131 are displayed in association with the image as shown in Fig. 20B and Fig. 20C. In other words, when the image is enlarged to a size that is easy to see or operate, rating icon 130 and flag setting icon 131 are displayed in association with the image. Of course, even in the state of FIG. 20A, the rating icon 130 and the flag setting icon 131 may be displayed corresponding to the image. In step S221 of FIG. 13, the staff terminal 1 performs processing to change the display state of the image area 128 in response to a pinch operation as in the above example.

[0116] The rating icon 130 not only displays rating information set by the number of stars, but also serves as an operator for changing the rating. For example, the number of stars can be changed by tapping or sliding the rating icon 130, which is a rating change operation. The flag setting icon 131 expresses the on / off state of the flag by, for example, changing the color or brightness of the icon. The flag setting icon 131 is also used as an operator for switching the flag on / off. For example, the flag can be switched on / off by a tap operation.

[0117] In step S215, the staff terminal 1 monitors whether or not an operation has been performed to select one of the images displayed in the image area 128. For example, this is an operation such as tapping one of the thumbnail images. The process performed when this selection operation is detected will be described later.

[0118] In step S216, the staff terminal 1 monitors the filter operation. That is, the operation of the operator indicating the rating of the filter designation area 129. When the operation of any of the controls in the filter designation area 129 is detected, the staff terminal 1 proceeds to step S223, performs filtering processing according to the control, and extracts corresponding images. Then, in step S224, the extracted images are displayed in a list in the image area 128.

[0119] In step S217 of Fig. 14, the staff terminal 1 monitors rating operations, for example operations on the rating icons 130 as shown in Figs. 20B and 20C. If an operation on the rating icon 130 is detected, the staff terminal 1 proceeds to step S230, and updates the event data 60 so that the rating information of the image corresponding to that rating icon 130 has a value according to the operation. Also, in step S231, the staff terminal 1 transmits the updated rating information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, a synchronized state is maintained.

[0120] In step S218, the staff terminal 1 monitors a flag on / off operation, for example, an operation on the flag setting icon 131 as shown in Figs. 20B and 20C. If an operation on the flag setting icon 131 is detected, the staff terminal 1 proceeds to step S232 and updates the event data 60 so that the flag in the selection information of the image corresponding to the flag setting icon 131 is switched between on and off. Furthermore, in step S233, the staff terminal 1 transmits the updated flag information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, the synchronized state is maintained.

[0121] In step S219, the staff terminal 1 monitors the operation of the all flag icon 123. If an operation on the all flag icon 123 is detected, the staff terminal 1 proceeds to step S234, where the process branches depending on whether the flags of all images are currently on or not. If some of the images have their flags on or all have their flags off, and not all of the images have their flags on, the staff terminal 1 proceeds to step S236, where it updates the event data 60 so that the flags of all images are on. On the other hand, if the all flag icon 123 is operated when all the images are flagged on, the staff terminal 1 proceeds to step S235 and updates the event data 60 so that the flags of all the images are turned off. Furthermore, in step S237, the staff terminal 1 transmits the updated flag information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, the synchronized state is maintained.

[0122] Next, a description will be given of the process when an image selection operation is detected in step S215 of Fig. 13. In this case, the process of the staff terminal 1 proceeds to step S250 of Fig. 15, and a single image display screen 140 as shown in Fig. 21 is displayed.

[0123] A single image display area 142 is provided on the single image display screen 140, in which a selected image is displayed in a large size. The image displayed in this single image display area 142 is, for example, an image based on edited image data managed in event data 60. That is, the image is edited image data generated by the auto retouching process in step S106 in Fig. 11, and is a reproduced image of the image data reflecting the retouching parameters in the pre-setting information 61. Also, a focus point mark 149 may be displayed on the image to indicate the in-focus position within the image.

[0124] Also displayed on the single image display screen 140 are a back icon 133 which is an operator for returning the display to the event screen 120, a rating icon 130 for the displayed image, and a flag setting icon 131. Additionally, on the single image display screen 140, an edit icon 145, a focus point operation icon 146, an export icon 147, and an original icon 148 are displayed for operations relating to the displayed image.

[0125] With such a single-image display screen 140 being displayed, the staff terminal 1 performs a monitoring loop process from step S251 to step S257 in FIG. 15 and FIG.

[0126] In step S251, the staff terminal 1 monitors the operation of the edit icon 145. When the operation of the control of the edit icon 145 is detected, the staff terminal 1 proceeds to step S260 and displays an edit screen (not shown). On the edit screen, a user interface is provided that enables, for example, retouching and tilt correction, and the camera staff can perform image editing operations through the user interface. In the retouching operation, for example, a brightness adjustment value, a color adjustment value, a contrast adjustment value, a sharpness adjustment value, and the like can be set as retouching parameters. As an inclination correction operation, for example, an operator for adjusting the angle of an image is provided. The edit screen may be a screen separate from the single image display screen 140, or may be a screen superimposed on the single image display screen 140 to display various controls.

[0127] When the staff terminal 1 detects a retouching operation, the process proceeds from step S261 to step S262, and a retouching process is performed. This is a process in which the edited image data is further edited using retouching parameters input by the camera staff's operation. The retouching parameters input in this case are stored in the event data 60 as editing information for each image data.

[0128] If the staff terminal 1 detects a tilt correction operation, the process proceeds from step S263 to step S264, and performs a tilt correction process for the image. This is a process of editing the edited image data using the tilt correction parameters input by the operation of the camera staff. In addition, the tilt correction parameters input in this case are stored in the event data 60 as editing information for each image data.

[0129] When the end of the editing operation is detected in step S265, the staff terminal 1 transmits the editing information of the event data 60 updated in accordance with the editing process to the server device 4 in step S266, so that the event data 90 is updated on the server device 4 side. In other words, the synchronized state is maintained. In step S267, the staff terminal 1 ends the editing screen and returns to the normal single image display screen 140. The image displayed at this time reflects the editing results. In this way, by operating the edit icon 145, the camera staff can retouch or correct the tilt of any image.

[0130] In the single image display screen 140 display state, the staff terminal 1 monitors the operation of the focus point operation icon 146 in step S252. If an operation of the focus point operation icon 146 is detected, the staff terminal 1 proceeds to step S270, and branches the process depending on whether or not the focus point mark 149 is currently displayed on the image in the single image display area 142. If the focus point mark 149 is not currently being displayed, the staff terminal 1 proceeds to step S271 to acquire focus point information for the displayed image. The focus point is information that indicates the position in the image that the imaging device 2 focused on when capturing the image, and is described in the metadata. For example, the focus point is described by the coordinate information of a point in the image. The staff terminal 1 acquires focus point information from the metadata of the displayed image, and in accordance with that information, displays a focus point mark 149 on the displayed image, as shown in Fig. 21. For example, the focus point mark 149 is displayed as an area with a predetermined spread centered on the coordinate information of a point in the image stored as the focus point. This allows the user to check the subject that was in focus when the image was captured. The "spread" of the area indicated by focus point mark 149 may be variable, and the amount of spread may be changed depending on, for example, the position within the image or the subject as a result of subject recognition. For example, if the focus is on the person's eyes in Fig. 21, focus point mark 149 may be displayed in a small area about the size of the eyes, and if the focus is on a bouquet of flowers, focus point mark 149 may be displayed broadly to include the entire bouquet. Furthermore, the display form of the focus point mark 149 is a mark that displays the four corners (a square expressed in parentheses) in FIG. 21, but other display forms such as a square frame or a circular frame may also be used. 21 shows a case where there is only one focus point, but there may be cases where there are multiple focus points, such as when multiple points on a subject (e.g., a bride) in an image are in focus, or when there are multiple subjects (e.g., a bride and groom) and each subject is in focus on a part of the subject, etc. Therefore, multiple coordinate positions may be described as metadata for a focus point, in which case multiple corresponding focus point marks 149 will be displayed. Also, while the focus point mark 149 is displayed on the single image display screen as in FIG. 21, the focus point mark 149 may be displayed on the images displayed in a list in the image area of ​​the event screen in FIG. 18, for example.

[0131] If an operation on the focus point operation icon 146 is detected in step S252 while the focus point operation icon 146 is being displayed, the staff terminal 1 proceeds from step S270 to step S273, and ends the display of the focus point operation icon 146. Therefore, the camera staff can use the focus point operation icon 146 to turn on / off the display of the focus point mark 149 .

[0132] The staff terminal 1 monitors the operation of the export icon 147 in step S253 of FIG. When the operation of the export icon 147 is detected, the staff terminal 1 proceeds to step S280 to perform the export process, that is, the staff terminal 1 performs the export process on the information related to the displayed image data.

[0133] In step S254, the staff terminal 1 monitors the image switching operation. In this case, the image switching operation is the operation of the original icon 148. When the operation of the original icon 148 is detected, the staff terminal 1 proceeds to step S281 and switches between an image based on the original image data and an image based on the edited image data. That is, if a playback image based on the edited image data was displayed at that time, the playback image based on the original image data is displayed. Also, if a playback image based on the original image data was displayed at that time, the playback image based on the edited image data is displayed. Therefore, the camera staff can compare the edited image with the original image at will by operating the original icon 148.

[0134] On the single image display screen 140, a rating icon 130 for the image being displayed is displayed. In step S255, the staff terminal 1 monitors the rating operation. If an operation on the rating icon 130 is detected, the staff terminal 1 proceeds to step S282, and updates the event data 60 so that the rating information of the displayed image is set to a value according to the operation. Furthermore, in step S283, the staff terminal 1 transmits the updated rating information to the server device 4, so that the event data 90 is updated on the server device 4 side. In other words, a synchronized state is maintained.

[0135] On the single image display screen 140, a flag setting icon 131 for the image being displayed is displayed. In step S256, the staff terminal 1 monitors the flag on / off operation, that is, the operation of the flag setting icon 131. If an operation on the flag setting icon 131 is detected, the staff terminal 1 proceeds to step S284 and updates the event data 60 so that the flag in the selection information of the displayed image is switched between on and off. Furthermore, in step S285, the staff terminal 1 transmits the updated flag information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, the synchronized state is maintained.

[0136] In step S257, the staff terminal 1 monitors whether the return icon 133 is operated. When the operation of the back icon 133 is detected, the staff terminal 1 proceeds to step S211 in FIG. 13, closes the single image display screen 140, and returns to the state in which the event screen 120 is displayed.

[0137] Next, a process to be performed when an operation on the next icon 132 is detected in step S212 of FIG. 13 will be described. In this case, the process of the staff terminal 1 proceeds to step S300 in FIG. 17, and the cover / highlight selection screen 150 as shown in FIG. 22 is displayed.

[0138] The cover / highlight selection screen 150 is a screen for the camera staff to carry out an operation to select an image to be used as a cover (front page) or an image to be highlighted (highlighted) in the web gallery. The cover / highlight selection screen 150 displays a back icon 133 and a next icon 132 .

[0139] The cover / highlight selection screen 150 also has a list designation area 153, a candidate tab 151 and a selection tab 152. When the candidate tab 151 is selected, a list of images that are flagged on (that is, images selected to be used in the web gallery) is displayed in a list designation area 153. For each image, a highlight button 155 and a cover button 156 are displayed. The highlight button 155 is an operator for designating an image as highlighted, and the cover button 156 is an operator for designating an image as covered. The highlight button 155 may indicate whether or not a highlight is currently set depending on its display mode, and the cover button 156 may indicate the current cover setting status depending on its display mode. When the selection tab 152 is selected, a list of images designated as covers or highlighted is displayed in a list designation area 153 .

[0140] The camera staff operates the cover / highlight selection screen 150 to select an image to be used as a cover or an image to be highlighted.

[0141] With the cover / highlight selection screen 150 displayed in step S300 of FIG. 17, the staff terminal 1 performs a monitoring loop process from step S301 to step S305.

[0142] In step S301, the staff terminal 1 monitors the operation of the cover button 156. If an operation on the cover button 156 is detected, the staff terminal 1 proceeds to step S310, sets the cover image information in the selection information so that the corresponding image is set as the cover image, and updates the event data 60. Also, in step S311, the staff terminal 1 transmits the updated selection information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, a synchronized state is maintained.

[0143] In step S302, the staff terminal 1 monitors the operation of the highlight button 155. If an operation on the highlight button 155 is detected, the staff terminal 1 proceeds to step S320, sets the highlighted display image information in the selection information so that the corresponding image is set as the highlighted image, and updates the event data 60. Also, in step S321, the staff terminal 1 transmits the updated selection information to the server device 4, so that the event data 90 is updated on the server device 4 side. That is, a synchronized state is maintained.

[0144] In step S303, the staff terminal 1 monitors a tab switching operation. The tab switching operation in this case is an operation of the selection tab 152 in the list display of the candidate tab 151, or an operation of the candidate tab 151 in the list display of the selection tab 152. When the tab switching operation is detected, the staff terminal 1 performs a process of switching the list display contents in the list designation area 153 in accordance with the selected tab in step S330.

[0145] In step S305, the staff terminal 1 monitors the operation of the back icon 133. If the operation of the back icon 133 is detected, the staff terminal 1 proceeds to step S211 in FIG. This allows camera crew to abort cover / highlight selection or restart their selection by setting the flags.

[0146] When the camera staff member has completed the operations on the cover / highlight selection screen 150 , he or she operates the next icon 132 . 17, the staff terminal 1 monitors the operation of the next icon 132. If the operation of the next icon 132 is detected, the staff terminal 1 proceeds to step S340 and displays a confirmation screen.

[0147] 23 shows an example of a confirmation screen 160. The confirmation screen 160 has a back icon 133, a next icon 132, and a details area 161 in which the final confirmation content is displayed. For example, detailed information such as the number of images flagged for inclusion in a web gallery, the number of images to be highlighted, the title, and the date are displayed.

[0148] The staff terminal 1 monitors the operation of the next icon 132 in step S341 of FIG. 17, and monitors the operation of the back icon 133 in step S342. When the operation of the back icon 133 is detected, the staff terminal 1 returns to step S300 and returns to the display state of the cover / highlight selection screen 150.

[0149] If an operation of the next icon 132 is detected, the staff terminal 1 proceeds to step S 343 and transmits a web gallery generation request to the server device 4 . 12, and returns to the display state of the project list screen 110. In this case, it is also possible to proceed to step S211 in FIG.

[0150] In the above-mentioned selection process, the camera staff had to repeatedly decide whether to select an image to be used in the web gallery (turning the flag on / off). Therefore, even if similar images (images similar in type of subject, pose of subject, angle of view, etc.) were taken multiple times in succession (also called the case of taking pictures of the same scene), the camera staff had to view the similar images one by one and decide whether to turn the flag on / off when viewing them.

[0151] For example, when a sequence of images of the same scene is displayed, after it has been determined that a predetermined number of the flags are turned on for the images to be used in the web gallery, it is efficient to process the display so that the remaining images of the same scene are not displayed for viewing, but instead the image of the next scene is displayed.

[0152] A specific process flow will be described below, but it is preferable to recognize the shooting scene of each captured image based on information such as the type of subject, the pose of the subject, the angle of view, and the shooting time of each captured image. For example, if the imaging device 2 has a function for recognizing scene information, the captured images can be grouped by scene by including scene information of each image in the metadata of the image file transferred to the staff terminal 1. Alternatively, if the staff terminal 1 has a recognition function, the images can be grouped by scene at the staff terminal 1. In either case, the scene information of each image can be stored in the event data 60 in FIG. 5.

[0153] Other processing examples as the selection in step S3 in FIG. 2 will be described below with reference to the flow chart in FIG. 31 and the screen example in FIG. The processing in the flowchart of Fig. 31 is processing by the function of the UI control unit 31 in Fig. 4. Note that this processing example can also be said to be an example of the case where the processing in steps S218, S232 and / or S233 in Fig. 14 is repeatedly performed on all images.

[0154] Step S381 in Fig. 31 is a process in which the staff terminal 1 displays images such as those shown in Fig. 20A, Fig. 20B, and Fig. 20C. In step S382, the staff terminal 1 monitors flag on / off operations for the displayed image. For example, this is an operation for the flag setting icon 131 as shown in Fig. 20B and Fig. 20C. When an operation on the flag setting icon 131 is detected, the staff terminal 1 updates the event data 60 so that the flag in the selection information of the image corresponding to the flag setting icon 131 is switched between on and off.

[0155] In step S383, the staff terminal 1 receives the result of the flag operation performed in step S382 and counts the flag-on number, which is the number of images in the same grouped scene whose flags have been determined to be on. In step S384, the staff terminal 1 compares a predetermined flag-on number determined in advance for each scene with the flag-on number counted in step S383, and determines whether the predetermined flag-on number has been reached. The predetermined flag-on number may be determined in advance for each scene by the camera staff, or may be determined according to the number of images in the group that are considered to be the same scene.

[0156] If it is determined that the predetermined number of flags has not been reached, the staff terminal 1 proceeds to step S385, where it monitors whether an operation has been performed to display the next image, and if so, proceeds to step S386, where it continues to control the display of the next image in the same scene. On the other hand, if it is determined in step S384 that the predetermined number of flags has been reached, the staff terminal 1 proceeds to step S387, where it monitors whether an operation has been performed to display the next image, and if so, proceeds to step S388, where it controls the display of an image of the next group of scenes different from the scenes in the previous group. This process can be repeated until the processing of all groups of scenes is completed.

[0157] Figures 32A and 32B are example screens showing the order in which images are displayed. Figure 32A is an example in which all images (images A to E) are displayed in order, regardless of the scene of the image. Figure 32B is an example in which some images are not displayed (display skipped) depending on the scene and the number of flags on. For example, if the predetermined flag-on number for scene SN1 is set to "1" and image A is flag-off processed, the flag-on number is "0 (zero)" and the predetermined flag-on number has not been reached, so as shown in FIG. 32B, the display of the next image B in scene SN1, which is the same scene, is controlled. Next, if image B is flag-on processed, the flag-on number becomes "1" and the predetermined flag-on number has been reached, so as shown in the figure, the next image C in the same scene is not displayed, and image D, which is the image of the next scene SN2, is controlled to be displayed. Furthermore, if the predetermined flag-on number for scene SN2 is set to "1" and image D is flag-on processed, the flag-on number becomes "1" and the predetermined flag-on number has been reached, so the next image E in the same scene is not displayed, and the image of the next scene is displayed.

[0158] Next, as another example of the selection process, a process of exchanging rating information and selection information between images will be described. There may be cases where the camera staff wants to review and replace the rating information and selection information that were once set in the selection of step S3 in Fig. 2. In such cases, the information may be changed by again operating the rating icon 130 and the flag setting icon 131, but the selection process can be carried out more efficiently if the rating information and selection information assigned to the two images can be exchanged.

[0159] Other processing examples as the selection of step S3 in Fig. 2 will be described below with reference to the flowchart in Fig. 33 and the screen examples in Fig. 34, Fig. 35, and Fig. 36. The processing in the flowchart in Fig. 33 is processing by the function of the UI control unit 31 in Fig. 4.

[0160] Step S420 in FIG. 33 is a step in which the staff terminal 1 displays a single image display screen 140 as shown in FIG. 34A. In FIG. 34A, the same numbers are given to the same items as in FIG. 21, but a replacement icon 170 for instructing the replacement of information is newly added, and the displayed image is image W. In step S421, the staff terminal 1 monitors the operation of the replacement icon 170. If it is determined that the replacement icon 170 has been operated, in step S422, the staff terminal 1 displays an image list including images near the displayed image. Here, the term "nearby" may refer to images taken close in order, images with close numbers as file names of image data, or images in the same scene. FIG. 34B shows a state in which images X, Y, and Z are displayed as a list below image W as images near image W.

[0161] In step S423, the staff terminal 1 monitors the operation of selecting an image from the image list. When an image is selected from the list, in step S424, the information added to the original image is replaced with the information of the selected image, and in the following step S425, the selected image is displayed. In FIG. 34C, among the images in the vicinity of FIG. 34B, Drawing In response to the selection of image Y, the rating information and selection information of the original image W and the selected image Y are swapped, and image Y is displayed.

[0162] Note that steps S424 and S425 may be reversed, that is, step S425 may be performed first. The information replacement process as in the above example may be a process of replacing both the rating information and the selection information, or a process of replacing only one of the rating information and the selection information. In that case, an operation step may be added to indicate which information is to be replaced. Fig. 35 is a conceptual diagram showing that rating information has been swapped. As shown in Fig. 35A, the rating information of image W was "three stars" and the rating information of image Y was "no stars", but as shown in Fig. 35B, by the process of Fig. 33 in response to the user's swapping operation, image W has been swapped to "no stars" and image Y to "three stars". Fig. 36 is a conceptual diagram showing that flags as selection information have been swapped. As shown in Fig. 36A, the flag of image W was off and the flag of image Y was on, but as shown in Fig. 36B, the flag of image W has been swapped to on and the flag of image Y to off in the process of Fig. 33 in response to the user's swapping operation.

[0163] <6. Server processing> Up to this point, the processing of the staff terminal 1 up to the selection stage in step S3 in FIG. 2 has been explained, but now the processing on the server device 4 side will be explained. Fig. 24 shows the processing of the server device 4 in response to communication from the staff terminal 1. Note that the processing of the server device 4 referred to below refers to processing performed by the CPU 71 of the information processing device 10 having the functions of Fig. 6. In Fig. 24, the processing is mainly executed by the functions of the event data management unit 40 and the image album generation unit 41.

[0164] At step S1 in FIG. 2, pre-setting information is transmitted from the staff terminal 1 to the server device 4. When the server device 4 receives the information regarding the project setting from the staff terminal 1, the process proceeds from step S500 to step S510, and the server device 4 performs a project setting process.

[0165] The communication of project setting from the staff terminal 1 is the communication that the staff terminal 1 makes to the server in the process from step S10 to step S13 in Fig. 9. For example, information such as the event name, date and time, and location of the created project is transmitted. The server device 4 sets up a new project based on this information, and sets the event data 90. Furthermore, on the staff terminal 1 side, not only the settings of the project itself but also auto-retouch settings, watermark settings, etc. are performed, and the setting information is successively transmitted to the server device 4. The server device 4 describes this information as pre-setting information 91 of the event data 90, thereby synchronizing the event data 60 with the event data 60 in the pre-setting state.

[0166] 2, the image transferred from the imaging device 2, metadata including rating information, and the like are transmitted to the server device 4 by the staff terminal 1 through the process of step S105 in FIG. When the server device 4 receives the image and metadata from the staff terminal 1, the process proceeds from step S501 to step S520 in Fig. 24, and the image and metadata are imported as information in the event data 90 of the corresponding project. The rating information in the metadata is extracted from the metadata and managed in the event data 90 in a state in which it can be updated later.

[0167] 2, various pieces of information for maintaining synchronization with the imaging device 2 are transmitted from the staff terminal 1 to the server device 4 during the process of the staff terminal 1 in FIGS.

[0168] When the server device 4 receives the rating information from the staff terminal 1, the process proceeds from step S502 to step S530 in FIG. 24, and updates the rating information for the corresponding image in the event data 90 of the corresponding project.

[0169] When the server device 4 receives the selection information from the staff terminal 1, the process proceeds from step S503 to step S540, and updates the selection information for the corresponding image in the event data 90 of the corresponding project. That is, the server device 4 is also configured to successively update the image information used by the flag setting, the cover image information as the cover, and the highlighted image information by the highlight setting.

[0170] When the server device 4 receives the editing information from the staff terminal 1, the process proceeds from step S504 to step S550, and updates the editing information for the corresponding image in the event data 90 of the corresponding project.

[0171] When the server device 4 receives the web gallery editing information from the staff terminal 1, the process proceeds from step S505 to step S560. This is the information sent by the staff terminal 1 in step S343 of FIG. 24, the server device 4 first performs image editing processing using parameters as editing information (including editing information based on the preset information 61) for each image flagged as use image information, and writes the images out as images for the web gallery. That is, it generates each edited image to be stored in the image folder 97 in FIG. 8.

[0172] In step S561 in Fig. 24, the server device 4 performs a web gallery generation process. That is, for example, the web page setting information 96 in Fig. 8 is generated as HTML data, and entered as a web page. In this way, a web gallery is formed. 24, the server device 4 transmits the URL (uniform resource locator) of the web gallery to the staff terminal 1. This enables the staff terminal 1 to view the web gallery. At this point, the web gallery is still set to private and can only be viewed on the staff terminal 1 to which the URL was sent.

[0173] The above is the process of the server device 4 corresponding to the process of the staff terminal 1 up to step S3 in FIG. Although not shown in FIG. 24, a process of providing event data 90 to the staff terminal 1 that has logged in to the server device 4 is performed as necessary (see the explanation of step S201 in FIG. 12).

[0174] After the web gallery is generated, in step S4 of FIG. 2, a share request is transmitted from the staff terminal 1 to the server device 4 (described later with reference to FIG. 25). In this case, the staff terminal 1 proceeds from step S506 to step S570 in Fig. 24 and sets the corresponding web gallery to be public, so that it can be viewed from the client terminal 5, for example.

[0175] <7. Web Gallery Delivery Processing> The process of the staff terminal 1 at the stage of the web gallery delivery process in step S4 in FIG. 2 will be described with reference to FIG. When a web gallery generation request is sent to the server device 4 as described above, the server device 4 generates a web gallery based on the event data 90, and transmits the URL of the web page. Figure 25 shows the process of the staff terminal 1 after acquiring the URL.

[0176] The staff terminal 1 accesses the web page using the URL acquired in step S350, which allows the staff terminal 1 to preview the web gallery. This is a preview for the camera staff to check, and the web gallery for this preview looks like a preview screen 180 in FIG. 26, for example. For example, the created web gallery is displayed in a preview area 183, and an edit button 181 and a share button 182 are provided.

[0177] The web gallery contents displayed in the preview area 183, i.e., the web gallery created by the server device 4, are configured by arranging images with flags set to on, for example, in chronological order. An image with cover image information set is displayed at the top as a cover. Images that are highlighted using highlight image information are displayed in a larger size than images that are not highlighted. Even images that are not highlighted may be automatically displayed in a larger size. This is done to give balance and sharpness to the entire gallery.

[0178] The staff terminal 1 monitors the browsing end operation in step S351 of FIG. 25, monitors the operation of the edit button 181 in step S352, and monitors the operation of the share button 182 in step S353. When an operation to end browsing is detected, web page viewing is terminated. When the operation of the edit button 181 is detected, the staff terminal 1 proceeds to step S360 and performs an editing process.

[0179] The camera staff can check the web gallery contents in the preview area 183 and, if necessary, operate the edit button 181 to perform further editing. In the editing process of step S360, the staff terminal 1 displays an interface image for editing, accepts operations by the camera staff, and edits the preview image accordingly. For example, the background color, title font, title color, title position, slide show music, etc. can be edited. Then, in response to the editing completion operation, the editing information is transmitted to the server device 4 so that the editing operation is reflected in the web gallery.

[0180] The camera staff performs editing operations as necessary, and then operates the share button 182. In this case, the staff terminal 1 proceeds to step S370 and transmits a share request to the server device 4. As described above, in response to the share request, the server device 4 sets the web gallery to be public in step S570 of FIG.

[0181] In step S371 of FIG. 25, the staff terminal 1 displays a screen indicating completion of delivery of the web gallery and performs processing corresponding to operations on the screen. For example, the staff terminal 1 displays a web gallery delivery screen 190 in Fig. 27. This web gallery delivery screen 190 displays a notification that the creation of the web gallery has been completed, a validity period, and a URL 193 of the web gallery. Also displayed are a copy button 191 and a share button 192. The camera staff can operate the copy button 191 to copy the URL, attach it to an e-mail or the like, and send it to the client terminal 5. Also, by operating the share button 192, the camera staff can provide the URL to an SNS or the like, so that the client can view the web gallery via the SNS. This allows clients to view the web gallery as early as possible, such as after the wedding ceremony has ended or on the day itself. Note that distributing a URL to a client to allow the client to view the image collection as a web gallery is just one example. For example, image collection content equivalent to the above-mentioned web gallery may be created and distributed to the client as a content file.

[0182] <8. Delivery Processing> The actual delivery process is carried out at step S5 in FIG. The process at the final delivery stage is shown in Fig. 28. This process is carried out, for example, on the personal computer 3 or the staff terminal 1. The following description will be given assuming that the process is carried out on the personal computer 3.

[0183] The camera staff operates the personal computer 3 to access the server device 4 in step S400 and makes a request to read information for the actual delivery. Then, in step S401, the event data 90 is obtained from the server device 4. In this case, for example, the server device 4 may allow the rating information and selection information or editing information to be downloaded as an XMP file so that it can be applied to the RAW image file in the personal computer 3 . That is, the rating information, used image information indicating that a flag has been set, cover image information indicating that the image has been used as a cover image, and highlighted image information indicating that a highlight has been set are transferred to the personal computer 3. All of the event data 90 may be provided to the personal computer 3 .

[0184] The camera staff imports RAW image data captured by the imaging device 2 into the personal computer 3, and uses this RAW image data to create an image collection for delivery. In this case, by acquiring an XMP file based on the event data 90 from the server device 4, the creation work can be made more efficient.

[0185] In step S402, the personal computer 3 performs necessary editing processing to complete the image collection for delivery. Then, in step S403, processing for actual delivery is performed, such as generating content data as image album content, transmitting it electronically, recording it on a recording medium, or making it into print data. The created content data, a recording medium on which the content data is recorded, or a collection of images printed on paper is then delivered to the client.

[0186] <9. Automatic setting process> The above describes a series of processes from pre-setting to actual delivery. Here, as a modified example, an automatic setting process that makes the selection in FIG. 2 more efficient will be described.

[0187] Figure 29 shows an example of a process in which, using the function of the automatic setting unit 33 in Figure 4, selection information is automatically set based on tendency information generated based on past selection information of the camera staff member who is the user of the staff terminal 1.

[0188] For example, the server device 4 manages event data 90 for a plurality of projects of a certain camera crew member. Then, the server device 4 can perform a learning process on the event data 90 related to the camera crew member by using the function of the tendency information generating unit 44 in Fig. 6, and generate tendency information indicating the selection tendency of the camera crew member. For example, as a machine learning technique, images that are flagged on, images that are highlighted, images that are covered, etc. are each used as learning data, and trend information is generated that includes characteristic information of images that are flagged on, characteristic information of images that are highlighted, characteristic information of images that are covered, etc.

[0189] On the staff terminal 1 side, the process of FIG. 29 is carried out, for example, at the timing when the event screen 120 is displayed in step S211 of FIG. 13 or at a timing prior to that. In step S600, the staff terminal 1 acquires the tendency information of the camera staff member, which is generated by the server device 4. In step S601, the staff terminal 1 judges each image transferred from the imaging device 2 based on the trend information, and determines whether the image has a feature that is to be flagged on, a feature that is to be highlighted, a feature that is to be covered, etc.

[0190] In step S602, the staff terminal 1 automatically sets flags, highlights, and covers for each image according to the above-mentioned determination results. With such automatic settings in place, the camera staff can carry out the selection operations described with reference to Figures 13 to 17. This increases the likelihood that the work of the camera staff can be made more efficient.

[0191] The above process can also be applied to ratings, and automatic rating information setting is also possible. That is, the server device 4 uses the event data 90 of the camera staff's past projects through the function of the trend information generating section 44 to analyze images of each rating level, and generates feature information of "no stars" images, feature information of "one star" images, feature information of "two stars" images, feature information of "three stars" images, feature information of "four stars" images, and feature information of "five stars" images. Of course, it is also conceivable to generate only feature information of "five star" images, rather than all of these. The staff terminal 1 then acquires trend information including such characteristic information, and determines the match with the characteristic information of each image, thereby automatically setting rating information for all or some of the images. This will also make operations more efficient when it comes to camera staff selection.

[0192] Next, FIG. 30 shows an example in which the staff terminal 1 performs processing for automatically setting selection information based on tendency information generated based on the client user's inclinations, using the function of the automatic setting section 33.

[0193] For example, the server device 4 determines the taste of the client (the bride who is the client who ordered the image collection) from her past actions and sets trend information. For example, the server device 4 determines the trend of images that the client likes from her SNS behavior (such as giving "likes" to images) and images posted on her SNS, and sets the trend information.

[0194] On the staff terminal 1 side, the process of FIG. 30 is carried out, for example, at the timing when the event screen 120 is displayed in step S211 of FIG. 13 or at a timing prior to that. In step S610, the staff terminal 1 acquires the tendency information of the client (for example, the bride) of the project, which is generated by the server device 4. In step S611, the staff terminal 1 judges each image transferred from the imaging device 2 based on the trend information, and judges whether the image has characteristics that match the client's preferences.

[0195] In step S612, the staff terminal 1 automatically sets a flag for each image according to the above judgment results. In other words, images that match the client's taste are adopted for the web gallery. Of course, highlight settings and cover settings may also be performed. With these automatic settings in place, the camera staff can carry out the selection operations described in Figures 13 to 17. This not only makes the camera staff's work more efficient, but also increases the likelihood of creating an image collection that increases client satisfaction.

[0196] The above process can also be applied to ratings, and automatic rating information setting is also possible. That is, the server device 4 uses the function of the trend information generating unit 44 to generate trend information including feature information of images that clients like. The staff terminal 1 acquires trend information including such characteristic information, and automatically sets rating information by determining the match with the characteristic information of each image. This not only makes it easier for camera staff to make selections, but also improves client satisfaction.

[0197] <10. Selection Speed-Up Support Processing> Various further processing examples of the embodiment will be described below.

[0198] As described above, for example, if a photographer takes pictures at a wedding ceremony, wedding reception, etc. and creates an image collection, the photographer may take several thousand to more than 10,000 pictures during one day of the wedding event. An effective method for choosing which photographs to include in the aforementioned web gallery or in the final image collection from such a huge number of images is to select them based on ratings.

[0199] As mentioned above, rating information is linked to images and managed. As an example, the rating information may be a six-level rating ranging from "no stars" to "five stars," and is set by camera staff after evaluating each image. FIG. 37A shows that rating information "3 stars" has been added to a certain image.

[0200] By having the camera staff view and evaluate all the images taken and set the rating information, it is possible, for example, to extract highly rated images and select from them the images to be used in the image collection. Therefore, the rating information is useful from the viewpoint of selecting images to be used in the image collection from among a large number of images. For example, Fig. 37B shows an example of extracting images with a rating of "three stars or more". This allows only highly rated images with three stars or more to be extracted, allowing the camera staff to select images to be included in the image album. This is extremely efficient compared to selecting images to be included in the image album from all images taken.

[0201] However, with this method, rating information must be set for every image taken, which can be a time-consuming and tedious task. In addition, the evaluation criteria for ratings can fluctuate over time, and camera staff can be lenient or strict in their evaluations, making it difficult to evaluate all images using uniform criteria.

[0202] In the processing example described below, when processing a group of images taken at a wedding event, each image is evaluated as efficiently as possible according to a uniform standard, and the setting of rating information is also made efficient. In some cases, it is also possible to assume that it is not necessary to set rating information for all images.

[0203] For this purpose, the results of scene analysis for each image and metadata added by the image capture device 2 are used to enable image extraction. For example, icons corresponding to the results of scene analysis are provided, and images that meet the conditions can be extracted by using the icons corresponding to the extraction conditions, and rating information can be set for them. As a result, this will provide support that will speed up image selection by camera staff.

[0204] Note that allowing ratings to be set for the extracted images is just one example, and a process of selecting images to be included in the image album from among the extracted images may also be performed. That is, the extraction process using icons based on scene analysis or the like described below will be described as image extraction for rating setting, but is not necessarily limited to processing aimed at rating setting. In addition, wedding-related events refer to events in which multiple people including the bride and groom participate, such as wedding ceremonies, wedding receptions, after-parties, engagement ceremonies, meetings, matchmaking, etc. As a selection speed-up support specialized for such events, it is possible to set appropriate extraction conditions.

[0205] An example of the processing subject will be described with reference to Figs. 38A, 38B, and 38C. The process for accelerating selection based on scene analysis, etc. basically involves analyzing captured images, setting extraction conditions, and extracting images by referring to the scene analysis results, etc. according to the extraction conditions. Such processing can be executed by any one of the imaging device 2, the staff terminal 1, and the server device 4 shown in FIG.

[0206] FIG. 38A shows a case where the imaging device 2 executes this process. The imaging device 2 can display images, icons, etc. on the display unit 2a, for example. Also, various types of controls are provided. In this imaging device 2, analysis processing is performed on each image of a group of images captured and recorded at a certain event, and analysis result data is obtained. Then, in response to an extraction operation, an extraction condition applicability determination is performed for each image using the analysis result data, and the images extracted as a result are displayed on the display unit 2a. The camera staff can set ratings for the extracted images, etc.

[0207] It is also possible for the imaging device 2 to transmit each captured image to the server device 4, and for the server device 4 to analyze each image in the image group and transmit the analysis result data to the imaging device 2. In this case as well, the imaging device 2 can perform extraction using the analysis result data in response to an extraction operation by the camera staff, and display the extracted image as the extraction result on the display unit 2a.

[0208] FIG. 38B shows a case where the staff terminal 1 executes the selection speed-up support process. Images captured by the imaging device 2 are transferred to the staff terminal 1. The staff terminal 1 transmits, for example, each image of the image group received from the imaging device 2 as a reduced image to the server device 4 in the cloud 7. The server device 4 analyzes each image of the image group and transmits the analysis result data to the staff terminal 1. The staff terminal 1 performs extraction using the analysis result data in response to an extraction operation by the camera staff member, and can display the extracted image on the display unit 1a.

[0209] Image analysis may be performed within the staff terminal 1. That is, the staff terminal 1 analyzes each image in the image group received from the imaging device 2 and obtains analysis result data. Then, extraction is performed using the analysis result data in response to an operation of specifying extraction conditions by the camera staff, and the extracted images are displayed on the display unit 1a.

[0210] FIG. 38C shows a case where the server device 4 in the cloud 7 executes the process. The images captured by the imaging device 2 are transmitted to the server device 4. The server device 4 analyzes each image in the image group received from the imaging device 2 and obtains analysis result data. The server device 4 then provides an interface to the staff terminal 1 via, for example, a browser or an application program. The camera staff can perform extraction operations on the web page screen viewed in the browser on the staff terminal 1. In response to this extraction operation, the server device 4 performs extraction using the analysis result data, and provides the extracted image as the extraction result to the display unit 1a of the staff terminal 1 via the browser or the like.

[0211] The images captured by the imaging device 2 may be first transferred to the staff terminal 1, and then the staff terminal 1 may transmit the images to the server device 4, after which the server device 4 may analyze the images and provide an interface via a browser or the like. Furthermore, if the imaging device 2 has a web browsing function or the like, camera staff can use the imaging device 2 to specify extraction conditions and view extracted images.

[0212] In any of the above cases, the camera crew can obtain the image extraction results for speeding up the selection process. This type of selection speed-up support process can support the camera crew in setting ratings for a large number of images and in selecting images to be included in the image collection.

[0213] An example of the configuration of the information processing device 10 that serves as the staff terminal 1 or the server device 4 has been described with reference to Fig. 3. The imaging device 2 also has the configuration of the information processing device 10 in Fig. 3 since it has a built-in microcomputer and the like. The process for speeding up the selection is realized by providing the CPU 71 in FIG. 3 with the function of FIG. 39A, for example.

[0214] Figure 39A shows a UI control unit 31, an extraction processing unit 34, a rating information management unit 35, an analysis result data receiving unit 36, an analysis unit 37, and a video editing unit 38 as functions provided in a CPU 71 of an information processing device 10 serving as an imaging device 2, a staff terminal 1, and a server device 4. For the selection speed-up support process, at least the UI control unit 31 and the extraction processing unit 34 are provided, and either the analysis result data receiving unit 36 ​​or the analysis unit 37 may be provided.

[0215] As described with reference to FIG. 4, the UI control unit 31 has a function of controlling the presentation of images, rating information, and the like, and also of detecting operational inputs. Specifically, with regard to the process for speeding up the selection, the UI control unit 31 performs UI processing that enables the user to specify extraction conditions that can classify an image group that is a plurality of images taken at a wedding-related event. Specifically, the UI control unit 31 also performs processes such as displaying icons (described later) and detecting operations corresponding to the icons.

[0216] The extraction processing unit 34 is a function for extracting one or more images from a group of images of a certain event based on designated extraction conditions. For example, the extraction processing unit 34 performs image extraction by referring to the image analysis result data based on the extraction conditions specified by the icons. Various specific examples will be described later.

[0217] The rating information management unit 35 is a function that performs processing to update the rating information of an image when a rating is set for the extracted image. In the case of the staff terminal 1, it can be considered that this is included in the function of the event data management unit 30 described in FIG.

[0218] The analysis result data receiving unit 36 ​​has a function of receiving analysis result data. The analysis unit 37 has a function of performing image analysis on each image and generating analysis result data. It is sufficient that either the analysis result data receiving unit 36 ​​or the analysis unit 37 is provided.

[0219] For example, when the imaging device 2 alone performs the selection speed-up process as shown in FIG. 38A, the CPU 71 of the imaging device 2 is provided with an analysis unit 37 to obtain analysis result data, and therefore the function of the analysis result data receiving unit 36 ​​is not required.

[0220] In addition, in the case where the imaging device 2, for example, causes the server device 4 to perform an analysis and receives analysis result data from the server device 4, the CPU 71 of the imaging device 2 does not need to have the functions of the analysis unit 37, and only needs to have the functions of the analysis result data receiving unit 36. In this case, the CPU 71 of the server device 4 is provided with an analysis unit 37 as shown in FIG. 39B.

[0221] Also, when the staff terminal 1 receives the analysis result data from the server device 4 as shown in Fig. 38B, the CPU 71 of the staff terminal 1 does not need the function of the analysis unit 37, and only needs to have the function of the analysis result data receiving unit 36. In that case, the CPU 71 of the server device 4 is provided with the analysis unit 37 as shown in Fig. 39B. Furthermore, when image analysis is performed within the staff terminal 1, the CPU 71 of the staff terminal 1 has the function of the analysis unit 37. The analysis result data receiving unit 36 ​​is not necessary.

[0222] When image analysis and UI processing and extraction processing via a browser or the like are performed in the server device 4 as in FIG. 38C, the analysis result data receiving unit 36 ​​becomes unnecessary.

[0223] The video editing unit 38 indicates a function for performing editing processing on the extracted video in particular, which is provided when generating a highlight video, which will be described later.

[0224] The scene analysis by the analysis unit 37 will now be described. The analysis unit 37 analyzes images taken by camera staff or the like, and performs scene recognition and person recognition. A representative example of this technique is machine learning. For example, images can be analyzed using machine learning to obtain information about people and objects (scenes). ·Person information Gender discrimination, face information, blinking information, face size on screen, emotion, smile, type of facial expression, etc. Scene information Cake, groom, bride, bouquet, church, etc.

[0225] 40A shows an example in which an image is analyzed and it is determined that the image contains an object that is considered to be a "bouquet." The recognition rate is a value of the reliability of the analysis result. FIG. 40B shows an example in which an image is analyzed and the analysis results show that the image contains people as a "bride" and a "groom" and an object as a "cake."

[0226] As a result of such image analysis, analysis result data indicating that the image contains a "bouquet" or analysis result data indicating that the image contains a "bride," "groom," and "cake" are associated with each image.

[0227] Such analysis result data is then referenced to extract the image. For example, when the user specifies the extraction condition "bouquet," images associated with "bouquet" are extracted as analysis result data. In other words, by extracting AND and OR conditions using the information that is the analysis result data, the user can selectively view images of desired scenes. This allows the user to set ratings and select images to be used in the image collection from images of desired scenes without having to look at all the images that were taken, making the selection work of the camera staff more efficient.

[0228] A specific example of this technique is shown in Figures 41A and 41B. FIG. 41A shows a display example on the display unit 1a of the staff terminal 1 or the display unit 2a of the imaging device 2, for example. On the display unit 1a (or 2a), an icon area 200 is set together with an image area 128. In the image area 128, before extraction, for example, a list of images of the image group to be processed, that is, a series of images taken at a wedding event, is displayed. An icon IC for designating extraction conditions for extracting a desired scene or person from such a list image is displayed in the icon area 200. Although details will be described later, icon IC allows the user to specify extraction conditions using analysis result data for each image in the image group.

[0229] Users such as camera staff can specify extraction conditions by selecting any icon IC. For example, if the icon of "bride" is selected, "bride" becomes the extraction condition, and images including brides are extracted. Furthermore, as shown in FIG. 41B, the user can arbitrarily specify a plurality of icons IC to set extraction conditions such as an extraction formula including so-called AND conditions, OR conditions, and the like. For example, when an image of a cake cutting scene is to be extracted, the icon ICs of "Bride", "Groom" and "Cake" are designated by AND conditions. Also, if you want to extract scenes with an impressive close-up of the bride, you can easily narrow down the desired images by selecting the icon IC for "Bride," "Alone," "Close-up (close-up)," or "Bouquet," or the icon IC for "Specify time," which specifies a specific period of time.

[0230] An example of a selection speed-up support process that extracts images using this method will be described below. The following example assumes that after image analysis, image extraction is performed in response to an operation based on an icon IC, and the user sets a rating for the extracted image.

[0231] Therefore, in this case, the selection speed-up support can realize efficient rating setting, but as described above, the selection speed-up support process is not limited to being used for efficient rating setting. For example, the selection speed-up support process may be executed when selecting images to be posted in the aforementioned web gallery, when selecting cover images, highlight images, and when selecting images to be edited, etc. Therefore, the rating setting process in each of the following processes can be considered as being replaced with these image selection processes.

[0232] FIG. 42 shows an example of performing the selection speed-up support process in the imaging device 2 as described with reference to FIG. 38A. The CPU 71 in the imaging device 2 stores image data obtained by shooting in step S700. This collectively indicates a process of sequentially storing the shot image data in the internal storage unit 79. A large number of images are stored in the storage unit 79 as the cameraman takes a series of shots.

[0233] In step S701, the CPU 71 performs image analysis on each of the stored images. As described above, person information and scene information are obtained from each image through analysis, and analysis result data is generated. The analysis result data is recorded as information associated with the image, that is, as metadata. This image analysis process may be performed each time an image is captured, or may be performed periodically on new captured images, or may be performed on all stored images at once when the cameraman performs a specific operation, such as an operation for extracting an image.

[0234] In step S702, the CPU 71 displays an icon IC in the icon area 200 on the display unit 2a. The types of icons IC to be displayed may be preset icons for wedding-related events, such as "bride," "groom," "cake," and "bouquet." Moreover, such icon ICs should not be displayed in a fixed manner, but should be selected and displayed based on the analysis result data. That is, icon ICs corresponding to the people and scene information obtained by analyzing each image in step S701 are selected and displayed. Alternatively, corresponding icon ICs may be generated and displayed based on people and objects obtained as the analysis result data. In this way, it is possible to display only icon ICs from which images can be extracted. For example, if there are no images containing "cake" among all images in the image group to be processed, then the "cake" icon IC is unnecessary. This is because no image will be extracted even if the user specifies the "cake" icon IC. By presenting the user with only icon ICs corresponding to people or objects obtained as analysis result data, it is possible to prevent meaningless icon ICs (from which images will not be extracted) from being specified.

[0235] In step S703, the CPU 71 checks the extraction conditions, that is, detects an icon IC designation operation by the user through UI processing.

[0236] In step S704, the CPU 71 performs image extraction processing based on the specified extraction conditions and controls the display of the extracted images. That is, the analysis result data associated with each image in the image group to be processed is referenced, and images that match the extraction conditions are extracted. The extracted images are then displayed in the image area 128 on the display unit 2a.

[0237] Thereafter, if the user performs a rating operation on the image displayed in image area 128, CPU 71 proceeds from step S705 to step S706 and stores the rating information. In the case of CPU 71 of imaging device 2, the rating information is stored as metadata associated with the image.

[0238] Next, Fig. 43 shows an example of performing the selection speed-up support process in the staff terminal 1 as described in Fig. 38B. However, this is an example of a case where image analysis is also performed in the staff terminal 1. In step S720, the CPU 71 in the staff terminal 1 receives images from the imaging device 2. This is a process in which images stored in the imaging device 2 by a series of shots taken by a cameraman are imported into the storage unit 79 in the staff terminal 1 by short-distance wireless communication, wired communication, or delivery by a recording medium.

[0239] In step S721, the CPU 71 of the staff terminal 1 performs image analysis on each image received from the imaging device 2. Person information and scene information are obtained from the analysis of each image, and analysis result data is generated. The analysis result data is stored as information associated with the image. In this case, the analysis result data may be stored as information associated with each image in event data 60 as shown in Fig. 5, for example. Specifically, the analysis result data corresponding to each image is saved in imaging / editing information table 62 of event data 60. For example, the analysis result data may be saved in addition to metadata MT1, MT2, etc. in event data 60 of Fig. 5, or the analysis result data may be stored in association with each image in imaging / editing information table 62 separately from these metadata MT1, MT2, etc. Whatever the storage form, it is sufficient that the analysis result data is saved in a state associated with the image.

[0240] It is assumed that this image analysis process is performed on each image transferred, for example, when the images are transferred from the imaging device 2. In addition, when the camera staff performs a predetermined operation, for example, an operation for executing image extraction, the process may be performed on all the stored images at once.

[0241] In step S722, the CPU 71 displays the icon IC in the icon area 200 on the display unit 1a. As in the case of the imaging device 2 described above, the type of the icon IC to be displayed may be an icon previously set for a wedding-related event, but it is particularly preferable to present to the user only the icon IC corresponding to a person or item obtained as the analysis result data.

[0242] In step S723, the CPU 71 checks the extraction conditions, that is, detects an icon IC designation operation by the user through UI processing.

[0243] In step S724, the CPU 71 performs image extraction processing based on the specified extraction conditions and controls the display of the extracted images. That is, the analysis result data associated with each image in the image group to be processed is referenced, and images that match the extraction conditions are extracted. The extracted images are then displayed in the image area 128 on the display unit 1a.

[0244] Thereafter, if the user performs a rating operation on the image displayed in the image area 128, the CPU 71 proceeds from step S725 to step S726 and stores the rating information. In the case of the CPU 71 of the staff terminal 1, the set rating information is stored and updated as rating information in the imaging / editing information table 62 of the event data 60 as shown in FIG. Note that after step S724, the camera staff may operate the staff terminal 1 to select, for example, images to be posted in a web gallery, a cover image, a highlight image, or an image to be edited. In such cases, the event data 60 will be updated in response to these operations.

[0245] Next, FIG. 44 shows an example of the case where the selection speed-up support process is performed by the server device 4 as described with reference to FIG. 38C. In step S740, the CPU 71 in the server device 4 receives images from the staff terminal 1 or the imaging device 2. For example, images captured by a cameraman in a series of shots and stored in the imaging device 2, or images transferred to the staff terminal 1, are uploaded to the server device 4 in the cloud 7.

[0246] In step S741, the CPU 71 of the server device 4 performs image analysis on each of the received images. Person information and scene information are obtained from each image by the analysis, and analysis result data is generated. The analysis result data is stored as information associated with the image. In this case, the analysis result data may be stored as information associated with each image in the event data 90 as shown in FIG. 7, for example.

[0247] Steps S742 to S746 are processes in which the server device 4 provides a UI image to the staff terminal 1 via a browser, an application program, etc. For example, the processes from step S742 onwards are performed in response to the staff terminal 1 requesting display from the server device 4 via a browser, etc.

[0248] In step S742, the CPU 71 displays an icon IC on the display unit 1a of the staff terminal 1, for example, as a display of the icon area 200 on the web page. In this case, the type of the icon IC to be displayed may be an icon previously set for an event related to a wedding, but it is particularly preferable to present to the user only the icon IC corresponding to a person or item obtained as the analysis result data.

[0249] In step S743, the CPU 71 receives extraction conditions based on operations performed on the staff terminal 1 or the like.

[0250] In step S744, the CPU 71 performs image extraction processing based on the specified extraction conditions and controls the display of the extracted images. That is, the analysis result data associated with each image of the image group to be processed is referenced, and images that match the extraction conditions are extracted. The extracted images are then displayed on the display unit 1a or the like as image area 128 on a web page or the like.

[0251] After that, when the user performs a rating operation on the image displayed in the image area 128, the CPU 71 S745 Step from S746 In the case of the CPU 71 of the server device 4, the set rating information is saved and updated as rating information in the imaging / editing information table 92 of the event data 90 as shown in FIG. Note that after step S744, the camera staff may operate the staff terminal 1 to select, for example, an image to be posted in a web gallery, a cover image, a highlight image, or an image to be edited. In this case, the CPU 71 of the server device 4 updates the event data 90 in response to these operations. When the event data 90 is updated on the server device 4 side, the event data 60 on the staff terminal 1 side is also synchronized as described above.

[0252] Since the imaging device 2 has a web browsing function or the like, the processing in FIG. 44 can be realized not only by the staff terminal 1 but also as an interface processing using the imaging device 2.

[0253] 38A and 38B, an example has been described in which image analysis is performed in the server device 4 when the selection speed-up support process is performed in the imaging device 2 or the staff terminal 1. In that case, the image analysis process in step S701 in FIG. 42 and step S721 in FIG. 43 may be performed as shown in FIG. 45. For example, steps S800 and S803 in FIG. 45 are processed by the function of the analysis result data receiving unit 36 ​​(FIG. 39A) in the CPU 71 of the staff terminal 1 or the imaging device 2, and steps S801 and S803 are processed by the function of the analysis unit 37 (FIG. 39B) in the CPU 71 of the server device 4.

[0254] In step S800, the imaging device 2 or the staff terminal 1 uploads the captured image to the server device 4. In this case, taking into consideration, for example, the amount of communication and the communication speed, the captured image may be transmitted as a reduced image with a lower resolution. Since the original image is large in size and takes time to transfer and analyze, by reducing the image to a size sufficient for analysis, the transfer time can be reduced and the analysis processing load on the server device 4 can also be reduced. It is also possible to upload an image without reducing it, in which case the server device 4 may create a reduced image for analysis.

[0255] In step S801, the server device 4 performs an analysis process on each image and generates analysis result data. In step S802, the server device returns analysis result data for each image to the imaging device 2 or the staff terminal 1.

[0256] In step S803, the imaging device 2 or the staff terminal 1 receives and stores the analysis result data from the server device 4. For example, the imaging device 2 stores the data as metadata for the image, or stores the data in a separate database. The staff terminal 1 may store the data in the event data 60.

[0257] By performing the above process in step S701 in FIG. 42 or step S721 in FIG. 43, it is possible to eliminate the need for the imaging device 2 or the staff terminal 1 to perform analysis processing that involves a relatively high processing load.

[0258] A specific example of the support for speeding up selection achieved by the above-described processing example will now be described. Although not mentioned each time, the display examples of each figure described below are display examples on the display unit 1a of the staff terminal 1 or the display unit 2a of the imaging device 2. The display examples on the display units 1a and 2a also include the display of images provided by the server device 4 using a browser or an application program.

[0259] FIG. 46 shows an example of a user interface for supporting high speed selection. 46A, a list of captured images is displayed in the image area 128, and various icons IC are displayed in the icon area 200. These icons IC are displayed according to, for example, the analysis results of each image.

[0260] FIG. 46B shows an example in which icons IC1 and IC4 are designated by the user to determine extraction conditions, and an image extracted by extraction processing corresponding to the conditions is displayed in image area 128. The designated icons IC1 and IC4 are displayed in a manner different from other icons IC in the icon area 200. In this example, the icons are displayed with thick borders, allowing the user to recognize which icon IC is designated.

[0261] There are various possible different display modes. The designated icon IC may have a different color, brightness, or size for the icon itself or its frame compared to other icon ICs, or an icon that is normally displayed in two dimensions may be displayed in three dimensions. In particular, when it is assumed that the user will specify a plurality of icons, it is desirable for the user to clearly present the specified icons IC by changing the display mode.

[0262] Each image in image region 128 is shown with a rating icon 130 . The user can set rating information for each image by operating the rating icon 130. After checking the extraction results as shown in Fig. 46B, the user sets a rating for each image. Fig. 46C shows the state in which the rating information has been updated by the operation.

[0263] For example, in this way, the user can use the selection speed-up support process to view images that meet the desired extraction criteria, and then set ratings while comparing the extracted images. Of course, in addition to rating settings, it may also be possible to select images to be published in a web gallery, select a cover image, select a highlight image, select an image to edit, and so on.

[0264] Examples of icons IC displayed in the icon area 200 are shown in FIG.

[0265] Icon IC1: Image showing a bride Icon IC2: Image showing the groom Icon IC3: Image with cake Icon IC4: An image with only one person Icon IC5: Image containing multiple people Icon IC6: A close-up image of a large face (for example, an image in which the face takes up more than 20% of the width of the screen) Icon IC7: A "pull-out" image with a small face (for example, an image where the face size is less than 5% of the screen width) Icon IC8: Image showing glasses Icon IC9: Image showing a bouquet Icon IC10: Icon for specifying time (for example, an icon for specifying a time, time range, etc. to separate time segments or scenes)

[0266] The above are just examples, and various other icons are conceivable, but it is desirable that the icons are suitable for image extraction in wedding-related events. As described above, suitable icons may be learned based on the results of image analysis, and added as icon ICs for setting extraction conditions. For example, if it is determined through learning that a certain object appears in a relatively large number of images, an icon corresponding to that object may be added. In this way, items that are likely to be the subject of a wedding ceremony or the like can be automatically iconized.

[0267] Various examples of extraction of examination images will be explained. In addition, unless otherwise specified in the following explanation, when multiple icons IC are specified, the extraction conditions represented by those icons IC will be AND conditions.

[0268] (Example 1: Example of using person information as extraction condition) In Fig. 48, icon IC1 representing a bride and icon IC4 representing a single person are specified, and images in which only the bride appears are extracted. This is useful for rating or selecting images of the bride, who is the main character at a wedding ceremony. Similarly, by specifying icon IC2 representing the groom and icon IC4 representing a single person, an image showing only the groom is extracted.

[0269] (Example 2: Example of extracting based on person and number of people) In Fig. 49, icon IC1 representing a bride and icon IC5 representing multiple people are specified, and images that include the bride and multiple people are extracted. This is useful for rating or selecting images that include the bride and related people. Similarly, by specifying icon IC2 representing the groom and icon IC5 representing multiple people, an image containing multiple people including the groom is extracted.

[0270] (Example 3: Extraction conditions are people, number of people, and face size) FIG. 50 shows a case where icon IC1 representing a bride, icon IC4 representing a single person, and icon IC6 representing a close-up image are designated, and as a result, an image in which only the bride is shown in a close-up is extracted. A close-up or long shot of the face can be determined from the size of the rectangle (bounding box BB) that contains the face. 51A and 51B show an example of a bounding box BB. FIG. 51A is an example in which a bounding box BB is defined by the upper left coordinates and the lower right coordinates. FIG. 51B is an example in which a bounding box BB is defined by the top left coordinate, width, and height.

[0271] (Example 4: Extraction conditions are people, number of people, and face size) FIG. 52 shows a case where icon IC1 representing a bride, icon IC4 representing a single person, and icon IC7 representing a "long shot" image are specified, resulting in the extraction of an image in which only the bride is shown "long shot."

[0272] (Example 5: Example of extracting people and objects) In Figure 53, icon IC1 representing a bride and icon IC3 representing a cake are specified, and images showing a bride and cake are extracted. For example, this can be used when you want to check an image of a cake-cutting scene. By using objects as the extraction condition in this way, it becomes easier to extract the desired scene.

[0273] (Example 6: Example of using objects as extraction conditions) 54 shows the case where icon IC9 representing a bouquet is specified, and images in which a bouquet is detected as a subject are extracted. This is useful when extracting images that show items that symbolize a wedding ceremony, etc.

[0274] (Example 7: Example of extracting person and time information) Fig. 55 shows a case where an icon IC1 representing the bride, an icon IC2 representing the groom, and an icon IC10 for designating a time are designated. When the icon IC10 for designating a time is operated, a time designation frame 202 for inputting a time range appears, and the user inputs the desired time. In the example shown, images that show the bride and groom are extracted from images taken between 1pm and 2pm. Specifying time information can be considered as a condition for extracting a scene at a specific time.

[0275] (Example 8: Example of displaying the recognition level using a person as an extraction condition) 56 shows the case where an icon IC1 representing a bride and an icon IC2 representing a groom are designated, and images in which the bride and groom are photographed are extracted. In this case, the emotion / facial expression recognition level 201 is displayed for each image. The emotion / facial expression recognition level is a value indicating the degree of emotion or facial expression when recognizing emotion or facial expression, such as a smile, a surprised face, or a crying face. For example, when performing processing to recognize a "smile" during image recognition, the degree of the smile is quantified as a recognition level, and this is included in the analysis result data. In this case, as shown in the figure, the emotion / facial expression recognition level 201 can be displayed. The user can set or select a rating by referring to this emotion / facial expression recognition level 201.

[0276] So far, we have described an example of specifying extraction conditions using icons IC corresponding to analysis result data, and we will now explain an example of the UI in this case. For example, the display unit 2a of the imaging device 2, or the display unit 1a in the case of a smartphone or the like as the staff terminal 1, may have a smaller display area than, for example, a desktop PC, and may have poor visibility and operability. A suitable UI example for the use of such relatively small display units 2a and 1a will be described.

[0277] 57 and 58 show the display unit 2a of the imaging device 2. Also shown are a cross key 2c and a wheel 2d provided on the imaging device 2 as examples of controls. The cross key 2c is an operator that can be operated up, down, left, right, and the center key by pressing the key. The wheel 2d is an operator that can be rotated and the center key by pressing the key.

[0278] In the display unit 2a, for example, an image area 128 is set in the upper part of the screen, and an icon area 200 is set in the lower part of the screen. As this icon area 200, a candidate area AR1 and a selection area AR2 are set. Candidate area AR1 is an area at the bottom of the screen where icon ICs are lined up in a horizontal row, and it is shown diagrammatically that by operating the left or right key of the directional pad 2c or by rotating the wheel 2d, the displayed icon ICs are scrolled horizontally while one icon IC located in the center is selected with the cursor K. In the figure, the row of icons IC is depicted so as to extend beyond the screen of the display unit 2a, but this is meant to show diagrammatically that each icon IC is displayed within the screen by scrolling. The selected area AR2 is an area adjacent to and above the candidate area AR1.

[0279] The user performs a scroll operation on the candidate area AR1 to select a desired icon IC with the cursor K. Then, the user performs an enter operation by pressing the center key of the cross key 2c or the center key of the wheel 2d. In this case, the icon IC relating to the enter operation is displayed in the selection area AR2 as shown in FIG.

[0280] In this way, the icon IC designated by the user is displayed in the selection area AR2, so that the user can easily check the designated icon type. Although it is useful to indicate the designated state by changing the display mode as described above, the difference in the display mode may be difficult to see, especially on a relatively small screen such as the display unit 2a of the imaging device 2. Also, when it is not possible to display everything at once and it is handled by scrolling, even if the display mode is changed, the icon itself may not be displayed, making it difficult to check. If a selection area AR2 is prepared separately from the candidate area AR1 as shown in FIG. 58 and displaying it there indicates the designated state, the designated icon IC can be easily checked even on a small screen.

[0281] When one or more icons IC are displayed in selection area AR2, by performing a predetermined operation, extraction is performed under extraction conditions corresponding to those icons IC, and the extracted image is displayed in image area 128. In the example of Fig. 58, icon IC1 representing a bride and icon IC4 representing a single person are displayed in selection area AR2. In this case, these are set as extraction conditions, and an image showing a single bride is displayed in extracted image area 128.

[0282] In addition, an icon IC that has been displayed in the selection area AR2 can be erased from the selection area AR2 by the user performing a deselection operation. For example, the scroll target is moved to the selection area AR2 by operating the up key of the cross key 2c. Then, the cursor K is moved to an arbitrary icon IC by a scroll operation, and the enter operation is performed. This cancels the selection state of the icon IC, and it is removed from the selection area AR2.

[0283] In the examples of Figs. 57 and 58, the icon IC4 is designated, but as shown in Fig. 58, the icon IC4 is displayed in both the candidate region AR1 and the selection region AR2. Alternatively, the designated icon IC may be moved from the candidate area AR1 to the selection area AR2. For example, in the example of Fig. 58, the icons IC1 and IC4 are displayed in the selection area AR2, but at this time, the icons IC1 and IC4 are erased from the candidate area AR1 (they do not appear even when scrolling). Then, as the number of icons IC in the selection area AR2 increases, the number of selectable candidate icons in the candidate area AR1 decreases, making it easier for the user to make a selection and also making it possible to avoid duplicate selection operations. Furthermore, when moving icons IC in this manner, the icon IC that has been deselected in the selection area AR2 is restored to display in the candidate area AR1.

[0284] 59 and 60 show the display unit 1a of the staff terminal 1, particularly the display unit 1a of a small portable device such as a smartphone. The display unit 1a is provided with a touch panel mechanism, and the user can perform operations such as tapping, double tapping, flicking, dragging, and pinching in / out.

[0285] 57, on the display unit 1a, for example, an image area 128 is set in the upper part of the screen, and an icon area 200 is set in the lower part of the screen. As this icon area 200, a candidate area AR1 and a selection area AR2 are set. Candidate area AR1 is an area at the bottom of the screen where icon ICs are lined up in a horizontal row, and is diagrammatically shown as follows: by flicking left or right, the displayed icon ICs are scrolled horizontally, while one icon IC located in the center is indicated by cursor K.

[0286] The user scrolls through the candidate area AR1 to select a desired icon IC with the cursor K. Then, the user performs an operation such as dragging the icon IC toward the selection area AR2, so that the icon IC is displayed in the selection area AR2 as shown in FIG.

[0287] When a predetermined operation is performed while one or more icons IC are displayed in selection area AR2, extraction is performed according to extraction conditions corresponding to those icons IC, and the extracted image is displayed in image area 128.

[0288] In addition, an icon IC that has been displayed in the selection area AR2 can be erased from the selection area AR2 by the user performing a deselection operation. For example, the user performs a tap operation on an icon IC in the selection area AR2, whereby the selected state of the icon IC is cancelled and the icon IC is removed from the selection area AR2.

[0289] As described above, in the case of the display unit 1a of the staff terminal 1, the icon IC designated by the user is displayed in the selection area AR2, so that the user can easily confirm the designated icon type. Of course, in this case too, the designated icon IC may be moved from the candidate area AR1 to the selection area AR2, and the deselected icon IC may be moved from the selection area AR2 to the candidate area AR1.

[0290] Next, Fig. 61 shows further examples of icon ICs that are useful when targeting images of weddings, etc. This is a further variation of the example of icon IC shown in Fig. 47.

[0291] FIG. 61A shows an icon IC20 having an image of a scene of a bride with her eyes downcast as an extraction condition. FIG. 61B shows an icon IC21 with an extraction condition of an image of a crying scene. FIG. 61C shows an icon IC22 having an image of an emotional or surprised expression as an extraction condition.

[0292] These are useful icon ICs for extraction conditions to extract images that show the emotions and expressions of the subjects. In particular, it is desirable to use scenes rich in emotions and expressions in an image collection of a wedding event. For this reason, extraction conditions that include the elements of emotions and expressions can be set, and images can be easily extracted using the corresponding icon IC.

[0293] Images of scenes with downcast eyes can be identified using eyes closed, facial angle, and machine learning. Images of crying scenes can be identified by detecting tears and the gesture of wiping tears with a handkerchief or hand. Images of moving or surprising scenes can be identified by analyzing facial expressions, particularly the eyes, mouth, and gestures. Therefore, by determining these scenes through image analysis and including them in the analysis result data, it becomes possible to extract images based on extraction conditions for icons IC20, IC21, IC22, etc.

[0294] 61D shows icons IC20 and IC31 of the faces of the individuals obtained by individual face recognition. By extracting the faces of individuals who attended a wedding ceremony or the like and making them into icons IC, it becomes possible to extract an image of a specific person. By performing personal identification when analyzing each image, an icon IC can be generated using the facial image of each participant, and such individual face icons can be created and used as icons to assist in speeding up selection.

[0295] Next, in addition to the analysis result data, the link with the metadata added by the image capture device 2 will be considered. For example, in the imaging device 2, in-focus position information is associated with an image as metadata at the time of shooting. By combining this in-focus position information with analysis result data, it is also possible to narrow down the search by the focused position.

[0296] (Example 9: Example of extraction conditions being people and eye focus) FIG. 62 shows a case where an icon IC1 representing a bride is specified and a condition that the image is focused on the eyes is added. For each image, information on the pupil frame EP, which indicates the pupil position at the time of capture, and information on the focus frame FW, which indicates the focus position, are included in the metadata. This makes it possible to extract images that include the bride based on the analysis result data, and further to extract images in which the bride's eyes are in focus. For example, in this case, an image in which the focus position and pupil position are almost in the same position can be determined to be an image in which the bride's eyes are in focus.

[0297] (Example 10: Example of extraction conditions being people and focus) 63 shows a case where icon IC1 representing a bride is designated and focusing is added as a condition, which means that an image in which the bride is in focus is extracted. The figure shows a bounding box BB that indicates the bride's body area, and an image in which the focusing frame FW is located within this bounding box BB is an image in which the bride is in focus. Therefore, it is possible to extract images that include the bride based on the analysis result data, and then further extract images from those in which the bride is in focus.

[0298] As described above, a UI for performing extraction using focus information in metadata may be provided that allows the user to specify icons such as "pupil focus" and "focus". An example of extraction processing using focus information in metadata is shown in Fig. 64. Fig. 64 shows an example of extraction of an image focused on the bride's eyes, as in Example 9.

[0299] The CPU 71 performs the process of FIG. 64 for all target images in order to set a flag for each image indicating that the bride's eyes are in focus. First, in step S900, the CPU 71 invalidates (initializes) a flag indicating that the bride's eyes are in focus for one target image. In step S901, the CPU 71 refers to the analysis result data for the image and judges whether or not the image includes a bride. If the image does not include a bride, the process for that image ends.

[0300] If it is determined in step S901 that the image includes the bride, the CPU 71 proceeds to step S902 and determines whether or not the distance between the in-focus position and the bride's eye position is equal to or less than a threshold value, i.e., whether or not the image is in approximately the same position and focused on the eye. If the distance exceeds the threshold, the image is deemed not to be pupil-focused, and processing of that image is terminated.

[0301] On the other hand, if it is determined that the focus position and the bride's eye position are almost the same and the eye is in focus, the CPU 71 proceeds to step S903, where it activates a flag indicating that the bride's eye is in focus, and ends processing for that image.

[0302] By carrying out such processing on all images in the target image group, images with an enabled flag are extracted as images focused on the bride's eyes. Note that while Figures 62 and 63 show a focus frame FW and pupil frame EP, by displaying such a focus frame FW and pupil frame EP when displaying the extracted image in the image area 128 of the display units 1a and 2a, the user can easily understand the focus position and focus state, and can also use this as a reference for setting ratings and selecting images.

[0303] (Example 11: Example of object and focus as extraction conditions) Further, an example in which the conditions are "pupil focus" and "focus" will be given. 65 shows a case where icon IC3 representing a cake is specified and focusing is added as a condition, which means that an image in which the cake is in focus is extracted. As shown in the figure, the position of the focusing frame FW is an image that is located in a position that is recognized as a cake during image analysis, making it possible to extract an image that meets these conditions.

[0304] (Example 12: Extraction conditions are animals and focus) 66A shows a case where an icon IC40 representing an animal is specified and pupil focusing is added as a condition. In this case, an image containing an animal and in which a pupil frame EP indicating pupil focusing exists within a bounding box BB of the animal is extracted. 66B shows a case where an icon IC40 representing an animal is specified and focusing is added as a condition, in which images containing an animal and in which the focusing frame FW is within the bounding box BB of the animal are extracted.

[0305] (Example 13: Example of object and focus as extraction conditions) 67 shows a case where an icon IC41 representing a car is specified and focusing is added as a condition, and images in which the in-focus frame FW exists within a bounding box BB of a car recognized from the image are extracted.

[0306] (Example 14: Example of extraction using OR conditions) So far, we have explained the AND condition of extraction conditions using multiple icon ICs, and the AND condition of an icon IC and metadata, but it is also possible to expand the range of images to be extracted by extraction using OR conditions.

[0307] FIG. 68A shows a case where an icon IC1 representing a bride and an icon IC2 representing a groom are specified by an OR condition, whereby images showing either the bride or the groom are extracted.

[0308] You can also narrow down your search by combining OR and AND conditions. FIG. 68B shows a case where icon IC1 representing a bride and icon IC2 representing a groom are specified under an OR condition, and icon IC3 representing a cake and icon IC7 representing an image of a "pull" are further specified under an AND condition. This allows for extraction of images that include either the bride or groom, a cake, and are taken from a distance.

[0309] (Example 15-1, Example 15-2: Examples of extraction using exclusion conditions) Exclusion conditions (NOT conditions) can also be used. FIG. 69A shows a group of images to be processed, including an image of a person blinking. FIG. 69B shows a case where icon IC1 representing a bride is specified as an extraction condition, and icon IC50 representing a blinking eye is specified as an exclusion condition. This results in an image of the bride, as shown in FIG. 69C, being extracted, in which the bride is not blinking.

[0310] FIG. 70A shows the images in the image group to be processed, which includes an image of a cake. FIG. 70B shows a case where an AND condition is specified for an icon IC1 representing a bride and an icon IC2 representing a groom as an extraction condition, and an icon IC3 representing a cake is specified as an exclusion condition. This results in an image being extracted that includes the bride and groom but does not include the cake, as shown in Figure 70C.

[0311] By setting such exclusion conditions, it becomes possible for the camera staff to extract images that are not to be subject to rating settings or image selection, thereby making the work more efficient.

[0312] (Example 16: Registering extraction conditions) Incidentally, it is also possible to set the above extraction conditions in advance, or to set various extraction conditions in chronological order. For example, extraction conditions for extracting the shots envisioned by a photographer or the like can be registered in advance, enabling the quick extraction of the shots required for delivery to a web gallery, etc. Furthermore, by setting extraction conditions according to the standard flow of a wedding, etc., it becomes convenient to extract images of each scene.

[0313] An example is shown in Figure 71. As condition c1, in order to extract an image of a scene with a church in the background, the condition "bride" & "groom" & "church" is set. For example, an icon IC1 representing a bride, an icon IC2 representing a groom, and an icon IC51 representing a church are registered by a designation operation. As condition c2, in order to extract an image of a scene with a pastor, the condition "bride" & "groom" & "pastor" is set. For example, the icon IC1 representing the bride, the icon IC2 representing the groom, and the icon IC52 representing the pastor are registered by a designation operation. As condition c3, the condition "bride" & "ring" is set in order to extract an image of a scene in which a ring is put on. For example, an icon IC1 representing a bride and an icon IC53 representing a ring are registered by a designation operation. As condition c4, in order to extract an image of a scene of a bride and groom in a rice shower, the condition "bride" & "groom" & "rice shower" is set. For example, the icon IC1 representing the bride, the icon IC2 representing the groom, and the icon IC54 representing a church rice shower are registered by specifying them. As condition c5, in order to extract an image of a scene in which a child delivers flowers to the bride and groom, the condition "bride", "groom", and "child" is set. For example, the icon IC1 representing the bride, the icon IC2 representing the groom, and the icon IC55 representing the child are registered by a designation operation.

[0314] By registering such conditions in chronological order, images that meet the conditions can be extracted in chronological order with extremely simple operations, making the work of camera staff more efficient, for example by allowing them to quickly select images needed for delivery. In order to extract images of scenes such as "church" and "rice shower," it is possible to use machine learning to train the system in advance on images that correspond to those scenes.

[0315] The registration of extraction conditions may be performed by the camera staff specifying an icon IC for each event, but extraction conditions specifying typical scenes at weddings, etc., such as the above conditions c1 to c5, may be registered and these may be selected for each event. For example, you can set the extraction conditions "Bride", "Groom" and "Church" as "Church Scene". You can prepare multiple settings like this and use the registered settings for each event. This allows you to register extraction conditions. itself This also increases efficiency for camera staff.

[0316] (Example 17: Application to video) So far, the process for speeding up selection has been described from the perspective of still image selection, but this process can also be applied to video. An example is shown in Figure 72. For example, by specifying icon IC1 representing a bride, icon IC2 representing a groom, and icon IC9 representing a bouquet, it is possible to extract a video section (in point-out point) corresponding to a scene including the bride and groom holding a bouquet.

[0317] Also, by specifying icon IC1 representing the bride, icon IC2 representing the groom, and icon IC3 representing the cake, the video section (in point-out point) corresponding to the cake-cutting scene can be extracted. In addition, by specifying icon IC1 representing the bride, icon IC2 representing the groom, and icon IC5 representing multiple people, it is possible to extract the video section (in point-out point) corresponding to the scene in which the bride and groom are with friends, etc.

[0318] It is also possible to pre-register the conditions for extracting these videos as shown above (Example 16).

[0319] (Example 18: Automatic production of highlight videos) By registering video extraction conditions such as those described above, it is possible to automatically extract scenes and create a highlight video. An example is shown in Figure 73. Figure 73 illustrates scene extraction conditions similar to those in Figure 72, and when these are registered, the CPU 71 extracts video segments of the corresponding scenes according to the extraction conditions and connects them together to automatically create a highlight video.

[0320] FIG. 74 shows a highlight movie production process performed by the CPU 71 using the functions of the movie editing unit 38 in FIG. In step S1000, the CPU 71 registers one or more extraction conditions in response to a user operation.

[0321] In step S1001, the CPU 71 extracts a scene according to each of the registered extraction conditions, that is, sets an in point and an out point as a scene according to each extraction condition. There are various ways to extract a specific video section. For example, images that meet the extraction conditions are searched for frame by frame, but in the case of video, the extraction conditions are not necessarily met in consecutive frames. For example, even in a scene that includes a bride, the bride may momentarily go out of frame. For this reason, it is a good idea to perform processing such as setting an out point when the conditions are not met for a certain number of consecutive frames.

[0322] In step S1002, the CPU 71 performs scene editing, that is, connects images from the in point to the out point of each scene set in step S1001 to generate a series of moving images. In step S1003, the series of videos is saved as a highlight video.

[0323] Through such processing, a highlight video is automatically generated, which can be viewed by camera staff as video content or provided to clients. Video editing, which is generally a time-consuming task, can now be completed simply by registering extraction conditions, making highlight video production extremely easy for camera staff.

[0324] <11. Final confirmation support for selecting delivery images> Regarding the delivery images, the camera staff simply looks at a list of the delivery candidate images and checks the balance of people included in the delivery images, etc. For example, FIG. 75 shows a state in which a list of delivery candidate images is displayed.

[0325] In the embodiment, the "delivery image" may be considered as a delivery image for a web gallery (an image selected as a photograph to be used in a web gallery), or as an image selected for actual delivery.

[0326] And while checking the delivery image like this -Is the balance of guests for the bride and groom appropriate? -Is the number of group photos appropriate? -Is the ratio of close-up / long-distance shots appropriate? Check the following:

[0327] In order to make such work more efficient and make the final check easier, it is conceivable that image analysis, such as facial recognition and object recognition, could be used to filter images based on various factors.

[0328] For example, it will recognize the faces of people in photos and display information about how many photos each person appears in. FIG. 76 shows a schematic example of narrowing down the search results by face. For example, on the screen of the "Face Filter" tab, a face selection area 210 is provided so that a face image can be selected by scrolling. When the camera staff arbitrarily selects a face image, an image in which the person with that face appears is extracted and displayed in the image area 128. This makes it easy to check whether the images of each participant are included in a balanced manner.

[0329] Furthermore, if you set the attributes of each person in advance, such as their name or their position at the event (for a wedding, the bride and groom, friends, family, etc.), you will be able to narrow down your search to more useful information.

[0330] It is also possible to narrow down images by using the number of people appearing in the image. For example, is it a photo of just one person, or a group photo? When checking delivered images, this is useful for checking whether there are any photos in which important people appear alone, or for checking people who appear only in group photos.

[0331] It is also possible to narrow down photos by the objects they contain. As for the delivered images, you can check whether or not images of important items such as cakes and rings are included in the image collection for weddings, etc.

[0332] <12. Face-based Presentation> If the facial information of attendees is registered in advance, it is possible to show which people have not been photographed. This allows the delivery status to be displayed during shooting.

[0333] It can detect faces contained in photos taken / delivered photos and count how many people appear in them. It is also possible to check whether there are fewer parents of the bride than parents of the groom, allowing for balance.

[0334] <13. Automatic selection of delivery images> By analyzing images, tags are automatically assigned to the images. -Person information (Groom / Bride / Bridemaid / etc) -Object information (ring / cake / welcome board / etc) -Scene information (cake cutting / bride and groom entrance / etc.) These tags can be used to assist with image selection at the time of delivery.

[0335] Also, if you define a template in advance, it is possible to automatically select the photos to be delivered. An example of a template is as follows: - Deliver one photo of the bride and groom's entrance -Deliver 5 photos of the groom - Deliver 8 photos of the bride - Deliver 3 cake cuttings

[0336] <14. Using facial information to show people not included in the delivery photo> When selecting photos to be delivered from the photos taken, the system detects "people who appear in the photos taken but not in any of the photos to be delivered" and presents information about them. For example, as shown in FIG. 77, the camera staff is notified that no photographs of person C are included in the photographs to be delivered.

[0337] <15. Shooting support using shot lists> A shot list is intended to be what might be called planning data for creating an image album, such as setting the number of images to be included in the image album for each of the bride, groom, family, relatives, friends, etc., and setting the number of images for each scene. Interview the bride in advance and create and input a list of photos she would like taken (shot list). For example, a shot list may include the following: Scene: I want a cake cutting scene. · Person: I want a photo of my aunt and the bride Items: Cake, ring

[0338] Based on such a shot list, selections can be made automatically during the wedding ceremony to ensure that photos are picked for delivery. By checking this shot list while shooting, the photographer can check for any shots that he or she has not yet captured.

[0339] <16. Summary and variations> Regarding the selection speed-up support process described with reference to FIG. 37 and subsequent figures, the following effects can be obtained in the embodiment.

[0340] The information processing device of the embodiment, i.e., the staff terminal 1, the server device 4, or the imaging device 2, is equipped with a UI control unit 31 that enables the user to specify extraction conditions for an image group, which is a plurality of images taken at a wedding-related event, and an extraction processing unit 34 that extracts one or more images from the image group based on the specified extraction conditions. For example, when a large number of images are taken at a wedding event, and a selection is made from the images to create a web gallery or a collection of images for final delivery, it is necessary to first evaluate each image, which takes a lot of time and effort. Therefore, we have made it possible to specify conditions specific to weddings, etc., and to select images based on those conditions. This will improve the work efficiency of camera staff. In particular, by providing an interface that allows users to classify the content of images of wedding events, it becomes possible to adequately support the task of creating an image collection of wedding events.

[0341] In the embodiment, an example has been described in which the UI control unit 31 performs control to display the images extracted by the extraction processing unit 34 and provide a user interface that enables an operation to input rating information for each image. This makes it possible to extract images corresponding to people, scenes, etc. that may be included in an image collection, and to set rating information while viewing the images. This makes the work extremely efficient compared to inputting rating information while viewing all captured images one by one. Moreover, since ratings can be set while comparing similar images, it is easy to standardize the evaluation criteria, which is desirable for setting appropriate ratings. In particular, being able to rate similar images while comparing them collectively makes it easier to make appropriate evaluations. This also makes it easier to select desirable images when selecting images to add to an image collection based on the rating information, which increases the value of the rating information for selection purposes.

[0342] In the embodiment, the extraction processing unit 34 refers to the analysis result data of the image contents of each image in a group of images of a certain event, and extracts images that match designated extraction conditions. By associating each image with analysis result data indicating the content of the image through image analysis, it is possible to easily determine whether or not the image satisfies the extraction conditions by referring to the analysis result data.

[0343] In the embodiment, an example in which the analysis result data is received from an external device, for example, the server device 4, has been given. This makes it possible to obtain analysis result data indicating the content of each image without performing image analysis, which is a relatively heavy load, within the staff terminal 1 or the imaging device 2, for example.

[0344] In the embodiment, the UI control unit 31 performs control to display an icon IC presenting extraction conditions for images in an image group, and detects an operation on the icon IC as an operation for specifying extraction conditions. Icon IC clearly presents the extraction conditions for images of wedding-related events, and also makes it easy to specify the extraction conditions. It was mentioned earlier that icon IC should be set according to the results of image analysis. For example, if the image recognition results for a certain image are "bride" and "bouquet," icon IC1 for the bride and icon IC9 for the bouquet should be prepared. By preparing icons based on the analysis results for all images to be analyzed, it becomes possible to prepare the icons necessary for image extraction for the target image group. In other words, icons that are unnecessary for the target image group do not have to be displayed. For example, if there is not even one image containing "cake" in the image group of a certain event, the cake icon IC3 will not be displayed, and unnecessary extraction can be prevented.

[0345] In the embodiment, an example has been described in which the UI control unit 31 displays an icon IC that has been the target of an operation for specifying an extraction condition in a display mode different from that of an icon IC that has not been the target of the operation. For example, the icon IC that has been specified can be distinguished from other icon ICs by changing the color of the icon or the color of the frame, or by highlighting the icon IC. By displaying the icon designated as the extraction condition in a identifiable manner, the extraction condition itself is clearly indicated to the user, making it easy to understand the type of image to be extracted. In addition, when combining extraction conditions, the icons (conditions) that have already been specified will be easier to understand, and the combined state of the conditions will be clear at a glance. In addition, icons are displayed together with the list of extracted images to indicate the specified conditions, so that it is easy to understand under what conditions or combination of conditions the extracted images were obtained.

[0346] In the embodiment, the extraction conditions include conditions for specifying people, such as the bride, groom, relatives (parents, siblings, etc.), friends, guests of honor, etc. The most important thing to consider in an image gallery of a wedding event is to properly feature the people who are attending, especially the bride and groom. Therefore, in terms of supporting the creation of an image gallery, it is appropriate to provide extraction conditions that allow images to be extracted by people.

[0347] In the embodiment, the extraction conditions include a condition that allows a scene of a wedding-related event to be specified. For example, there are extraction conditions that can specify the scene itself, such as "cake cutting," "ring exchange," and "bouquet toss," extraction conditions that assume a certain scene, such as "cake" and "ring," and extraction conditions that can specify a scene by specifying the time corresponding to the scene, etc. This makes it easy to extract an image of a specific scene, which is convenient when rating a large number of images in that scene.

[0348] In the embodiment, the extraction conditions include a condition that can specify time information of a wedding-related event. For example, images of specific scenes can be extracted by specifying the time corresponding to scenes such as "cake cutting," "ring exchange," "bouquet toss," etc. This is useful when rating images of specific scenes.

[0349] In the embodiment, the extraction conditions include conditions that allow a subject or animal to be specified. For example, conditions are provided for specifying items that are likely to be photographed at wedding events, such as cakes, candles, rings, pets, and glasses. In the case of images of wedding events, it is often appropriate to extract similar images by specifying an object. By providing extraction conditions for specifying an object, it becomes possible to easily extract similar images, which is convenient for rating and selecting images.

[0350] In the embodiment, the extraction conditions include conditions that allow the emotion or facial expression of the subject to be specified. For example, conditions for specifying smiling scenes, touching scenes, crying scenes, scenes with downcast eyes, etc. are prepared. In terms of creating an image collection for a wedding event, it is appropriate to select more emotional images and give them a higher rating in order to express the emotions of the participants. The availability of extraction conditions that allow images to be extracted based on emotion and facial expression recognition results, such as smiling scenes, makes it convenient to rate and select images.

[0351] In the embodiment, an example has been given in which information indicating the degree of emotion or facial expression of the subject of an image extracted by the extraction processing unit 34 is presented. As explained in Fig. 56, displaying numerical values ​​obtained as the result of emotion and facial expression recognition for images under certain extraction conditions can be useful as a reference for rating and selection, and is useful as a support for creating an image gallery. While Fig. 56 shows an example of the degree of smiling face, it is envisioned to display the degree of emotion and facial expression analysis results for images, such as the degree of crying face, the degree of surprised face, the degree of downcast eyes, the degree of joy, etc.

[0352] In the embodiment, the extraction conditions include conditions that can specify items to be determined based on metadata added at the time of image capture. For example, it is possible to specify a condition that can be determined by referring to metadata associated with an image in the imaging device at the time of imaging, such as information regarding focus (see Figs. 62 to 67). By using metadata added at the time of shooting, such as information on the focus position achieved by face focus, eye focus, etc., as one of the extraction conditions, it becomes possible to extract images in which the bride's eyes are in focus, for example. By taking the shooting conditions into account, it becomes possible to extract not only the image content but also only images that are in focus, which is appropriate in terms of extracting images that are candidates for inclusion in an image collection from a large number of images. As an example of using metadata, the embodiment uses information on "pupil focus" and "focus", but of course other metadata can be used. For example, when information such as F-number, ISO value, shutter speed, and zoom magnification is included in the metadata, these can be added to the extraction conditions to narrow down the results.

[0353] In the embodiment, an example has been given in which the extraction processing unit 34 presents the focus position for the extracted image. As explained in Figures 62 and 63, by superimposing images showing the in-focus position or area and related information such as the focus frame FW, pupil frame EP, bounding box BB, etc., camera staff can use this as a reference when selecting and rating appropriate images.

[0354] In the embodiment, an example has been described in which icons IC of extraction conditions are displayed in candidate area AR1 on the display screen, and an icon IC designated by a predetermined operation is displayed in selection area AR2 on the display screen. The icon ICs that can be specified can be viewed in the candidate area AR1, and the icon IC that has been selected is displayed in the selection area AR2, so the user can always recognize the icon IC that is being selected. This is easy to understand and easy to operate when extracting images by combining multiple icon ICs (extraction conditions). In particular, in the case of the relatively small display units (1a, 2a) of small devices such as the staff terminal 1 and the imaging device 2, it is possible for camera staff to easily determine whether an icon IC is selected or not by the area in which the icon IC is displayed, rather than by changing the display mode of the icon IC, thereby improving visibility and understanding of the selection state. As in the examples described in Figures 57, 58, 59, and 60, the candidate area AR1 and selection area AR2 are arranged adjacent to each other vertically (or horizontally), and an icon is displayed in the selection area AR2 in response to an operation, thereby clearly presenting the selection state. Also, the selected state can be clearly indicated by moving the icon from the candidate area AR1 to the selection area AR2 in response to an operation. Furthermore, by scrolling the candidate area AR1, a large number of icons can be displayed even on a small display screen. When moving from candidate area AR1 to selection area AR2, it is also useful in preventing the same icon from being selected in candidate area AR1 and in making it easier to select with cursor K by narrowing down the number of icons to be scrolled.

[0355] In the embodiment, an example has been described in which control is performed to terminate the display of an icon in selection area AR2 in response to a predetermined deselection operation. In other words, the user can return the icon in the selected area to the candidate area by operation. This allows the selection of the icon IC (extraction condition) that was once selected as an extraction condition to be cancelled. For camera staff, this allows flexible icon operation for setting appropriate conditions for image extraction. When an icon IC designated in the candidate area AR1 is moved to the selection area AR2, the selected state can be made easier to understand by returning the icon IC to the candidate area AR1 in response to a predetermined deselection operation.

[0356] In the embodiment, an example has been described in which the extraction processing unit 34 extracts images by setting an AND condition, an OR condition, or an exclusion condition for each of a plurality of specified extraction conditions. This allows the camera staff to set conditional expressions by combining AND conditions, OR conditions, and exclusion conditions for multiple extraction conditions set for wedding events, and set them as extraction conditions, thereby making it possible to more appropriately extract desired images.

[0357] In the embodiment, an example has been given in which the video editing unit 38 creates video content using a plurality of video scenes extracted by the extraction processing unit 34 based on the extraction conditions. As described in Fig. 72 to Fig. 74, the extraction process based on the extraction conditions can also be applied to scene extraction as a video. In that case, the extracted scenes can be linked together to generate video content as a highlight video. For camera staff, this allows easy video production by selecting extraction conditions, making it ideal for rapid highlight video production.

[0358] Furthermore, the processing of the embodiment described with reference to FIGS. 1 to 36 provides the following effects. The staff terminal 1 of the embodiment has an event data management unit 30 that stores image data received through communication with the imaging device 2 and metadata (additional information) including at least rating information corresponding to the image data as event data 60 related to a specific event, and updates the event data 60 in response to an operation input. The staff terminal 1 also has a UI control unit 31 that controls the presentation of images and rating information related to the event data 60 of a specific event as a project target, and detects an operation input. The staff terminal 1 also has a communication control unit 32 that controls communication with an external server device 4 to synchronize the event data 60 with the event data 90 in response to the storage or update of the event data 60. As a result, the information processing device 10 as the staff terminal 1 performs an event data management process including a process of storing image data received through communication with the imaging device 2 and additional information including at least rating information corresponding to the image data as event data related to a specific event and a process of updating the event data in response to operational input, a UI control process that controls the presentation of images and rating information related to the event data of a specific event and detects operational input, and a communication control process that controls communication to synchronize the event data with an external server device in response to the storage or update of the event data.

[0359] Such a staff terminal 1 can manage images and metadata successively transferred from the imaging device 2 as event data for a specific event and present this to the camera crew, for example, in a list. Therefore, the camera crew can use the staff terminal 1 to check images that will be the basis of an image collection, such as a web gallery. At this time, the rating information given to each image by the cameraman using the image capture device 2 is also displayed as a rating icon 130. For This will enable camera staff, including the cameraman and his assistants, to assist in selecting images to be included in the photo album. Furthermore, communication is performed so that the event data 60 on the staff terminal 1 and the event data 90 on the server device 4 are kept synchronized. As a result, even if, for example, a photographer performs an operation such as selection on terminal device 1A and then an assistant operates terminal device 1B, terminal device 1B logged in to server device 4 can perform operations in a state that reflects the latest synchronized event data content of terminal device 1A. This makes it possible to efficiently create an image album. Although the imaging device 2 and the staff terminal 1 are separate devices in the above embodiment, it is also possible to integrate the imaging device 2 and the terminal device 1A of the staff terminal 1 into one device.

[0360] In the embodiment, in response to the selection of a specific event, the event data management unit 30 acquires the event data of the specific event from the server device 4 (see step S204 in FIG. 12). Event data is synchronized in the server device 4 through communication from the staff terminal 1, so the event data in the server device 4 is the latest event data. When an application program is started up on the staff terminal 1, a list of events is displayed, and when the user selects an event, the latest event data can be obtained by first loading the event data on the server device 4 side. Of course, the rating information is also the latest. For example, even if an assistant uses the terminal device 1B, he or she can check the latest rating when the cameraman previously set or changed the rating on the imaging device 2 or staff terminal 1.

[0361] In the embodiment, the event data 60 includes preset information 61, and the UI control unit 31 displays an image based on edited image data that is edited according to the editing information in the preset information for the image data included in the event data 60 (see step S250 in FIG. 15). By setting editing information such as retouching parameters as the pre-setting information 61, each captured and imported image can be automatically edited and displayed. This is effective for creating a uniform atmosphere for each image in a web gallery, etc. Furthermore, automatic editing eliminates the need to perform the same editing operations on each individual image, making the camera crew's work much easier.

[0362] In the embodiment, the UI control unit 31 performs control to provide a user interface that enables input of editing information for image data included in the event data 60, and the event data management unit 30 performs processing to update the event data in response to the editing information input operation (see step S251, and steps S260 to S267 in FIG. 15). By making it possible to input editing information such as retouching parameters, for example, a camera staff member can arbitrarily edit each image. In particular, when combined with pre-set retouching parameters, it becomes possible to give each image a common tone and then fine-tune or change the atmosphere of each individual image.

[0363] In the embodiment, the UI control unit 31 performs control to provide a user interface that enables input operations of rating information for image data included in the event data 60, and the event data management unit 30 performs processing to update the event data in response to the input operations of the rating information (see steps S217, S230, S231 in FIG. 14 and steps S255, S282, S283 in FIG. 16, etc.). By making it possible to input rating information, the camera staff can later change the ratings they set for each image at the time of shooting. For example, by reviewing the images after the wedding ceremony and re-rating them, it may be possible to make a more appropriate selection.

[0364] In the embodiment, the UI control unit 31 performs control to provide a user interface that enables the setting operation of selection information for creating an image album by selectively using image data included in the event data 60, and the event data management unit 30 performs processing to update the event data in accordance with the setting operation of the selection information (see steps S218, S219, S232 to S237 in FIG. 14, steps S256, S284, S285 in FIG. 16, and steps S301, S302, S310, S311, S320, S321 in FIG. 17, etc.). By setting the selection information, the cameraman's intentions regarding the photo album can be included in the event data 60. Then, when the server device 4 automatically creates a web gallery based on the synchronized event data 90, an image album that reflects the cameraman's intentions can be created. reflection This allows you to provide customized web galleries to your clients extremely quickly.

[0365] In the embodiment, the selection information includes used image information indicating that an image has been selected to be included in the image album. In other words, it is now possible to set flags as image information to indicate which images will be used in the web gallery. By turning the flags on and off, camera staff can easily select which images to use.

[0366] In the embodiment, the selection information includes cover image information indicating that the image has been selected as the cover image of the image album. In other words, the cover image can be set as selection information indicating which image will be used as the cover of the web gallery. By setting the cover image, the camera staff can easily select the image to be used as the cover.

[0367] In this embodiment, the selection information includes highlighted image information indicating that an image has been selected as an image to be highlighted in the image album. In other words, the highlight setting allows the photographer to select which images to highlight in the web gallery, for example by making them larger than other images. The highlight setting allows the photographer to select which images to highlight. The highlight setting also makes the web gallery a more diverse collection of images, even if they are automatically generated. The server device 4 may create a web gallery in which images that are not highlighted are displayed in a highlighted manner. Moreover, the highlight setting may be such that, instead of increasing the size, the image is highlighted by brightness, color, frame display, or the like.

[0368] In the embodiment, an example has been described in which the staff terminal 1 includes the automatic setting unit 33 that performs processing for automatically setting selection information based on tendency information generated based on past selection information of camera staff members. Past selection settings are learned, and trend information is set regarding what images are adopted, used as cover images, and used as highlight images. That is, the characteristic points of images that are likely to be flagged on, images that are likely to be set as highlights, images that are likely to be used as covers, etc. are analyzed and used as trend information. By using this trend information to automatically set selections, the work of the camera staff can be made easier to a certain extent. Of course, the camera crew can manually set the options after the automatic selection, allowing them to create an image collection that reflects their own intentions. The selection information may be automatically set on the server device 4 side, and event data reflecting this may be provided to the staff terminal 1. Furthermore, although the trend information is generated on the server device 4 side, the staff terminal 1 may be provided with a trend information generating unit and generate the trend information.

[0369] In the embodiment, an example has been described in which the staff terminal 1 includes the automatic setting unit 33 that performs a process of automatically setting selection information based on tendency information generated based on the inclinations of client users. By learning the client's preferred image tendencies and analyzing the features of the preferred images, the system can create trend information. By using this trend information to automatically set the selection, it becomes possible to easily create an image collection that is in line with the client's preferences to a certain extent. The selection information may be automatically set on the server device 4 side, and event data reflecting this may be provided to the staff terminal 1. Furthermore, although the trend information is generated on the server device 4 side, the staff terminal 1 may be provided with a trend information generating unit and generate the trend information.

[0370] The server device 4 of the embodiment is equipped with an event data management unit 40 which manages the storage and updating of event data 90 including image data and rating information corresponding to the image data in response to receiving information regarding a specific event from the staff terminal 1. The server device 4 adds and updates the event data 90 for a specific event so that the image data and rating information transmitted from the staff terminal 1 are reflected in the event data 90. This allows the event data 90 to be stored in which the image and rating information are synchronized with the event data 60 on the staff terminal 1 side. In particular, the rating information can be changed arbitrarily on the staff terminal 1 side, but the server device 4 can hold the latest event data 90 by reflecting the changes. Although the server device 4 is configured to receive image data and rating information corresponding to the image data from the staff terminal 1, the server device 4 may also be configured to receive metadata (additional information) including image data and rating information corresponding to the image data directly from the imaging device 2.

[0371] The server device 4 of the embodiment includes an event data provision control unit 42 that causes the staff terminal 1 to execute transmission for providing event data. Event data is synchronized in the server device 4 through communication from the staff terminal 1, so the event data in the server device 4 is the latest event data. By providing event data to the staff terminal 1, when an application program is started on the staff terminal 1 and an event is selected, display based on the latest event data can be executed. Therefore, the cameraman and assistants can perform various tasks in the latest state using the staff terminal 1.

[0372] In the embodiment, the event data management unit 40 of the server device 4 manages the storage and updating of event data 90 including selection information or editing information corresponding to image data in response to receiving information regarding a specific event from the staff terminal 1. In response to receiving editing information such as retouching parameters and selection information regarding flags, covers, highlights, etc., the server device 4 also updates the event data. This makes it possible to maintain a synchronized state with the event data in the staff terminal 1.

[0373] The server device 4 of the embodiment includes an image album generating section 41 that generates a web gallery, which is image album data for a specific event, using the selection information and editing information. Since the event data 60, 90 is synchronized between the server device 4 and the staff terminal 1, by referring to the editing information and selection information in the event data 90 on the server device 4 side, it is possible to generate an image collection (web gallery) according to the intentions of the camera staff using the staff terminal 1. Furthermore, by generating the image album on the server device 4 side, the processing load on the staff terminal 1, which is a terminal device with limited resources, can be reduced.

[0374] In the embodiment, the selection information in the event data 90 on the server device 4 side includes used image information indicating that an image has been selected to be included in the image album. This makes it possible to generate an image collection (web gallery) using images selected by the camera staff using the staff terminal 1 by referring to the selection information.

[0375] In the embodiment, the selection information in the event data 90 on the server device 4 side includes cover image information indicating that the image has been selected as the cover image of the image album. By this means, by referring to the selection information, it is possible to generate an image collection (web gallery) using as its cover the cover image selected by the camera staff using the staff terminal 1.

[0376] In the embodiment, the selection information in the event data 90 on the server device 4 side includes highlighted image information indicating that an image has been selected as an image to be highlighted in the image album. In this way, by referring to the selection information, it is possible to generate an image collection (web gallery) in which highlight images selected by the camera staff using the staff terminal 1 are emphasized.

[0377] The server device 4 of the embodiment includes a data output unit 43 that transmits all or a part of the event data 90 to a terminal device (personal computer 3 or staff terminal 1) as a file in a predetermined format. This allows, for example, camera staff to use the personal computer 3 or staff terminal 1 to efficiently obtain materials for the image album to be delivered, perform necessary editing, etc., and easily generate the image album for delivery.

[0378] In the embodiment, the image file from the imaging device 2 is transferred to the staff terminal 1, and the staff terminal 1 uploads it to the server device 4. However, the imaging device 2 may upload it directly to the server device 4.

[0379] In the embodiment, the generation of an image collection of still images has been described, but the image collection may include moving images as part of the content, or an image collection may be generated using multiple moving images. In either case, the process described in the embodiment can be applied in the same manner. In this case, the moving image may be a moving image with sound or a moving image without sound that is only image information.

[0380] The program of the embodiment is a program that causes, for example, a CPU, a DSP, or a device including these to execute the processes shown in FIGS. 42 to 45 and 74. In other words, the program of the embodiment is a program that causes an information processing device such as a staff terminal 1, a server device 4, or an imaging device 2 to execute a control process that provides a user interface that enables the user to specify extraction conditions for a group of images taken at a wedding-related event, and an extraction process that extracts images from the group of images based on the specified extraction conditions. By using such a program, the staff terminal 1 and server device 4 that perform the above-mentioned selection speed-up support process can be realized in, for example, a mobile terminal device, a personal computer, or other devices capable of performing information processing. Alternatively, the selection speed-up support process can be performed by the imaging device 2.

[0381] A program for realizing such a staff terminal 1, server device 4, or imaging device 2 can be pre-recorded on a recording medium such as an HDD built into a computer device or the like, or on a ROM in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0382] Furthermore, such a program is suitable for a wide range of provision of the staff terminal 1, server device 4, or imaging device 2 of the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a still camera, a video camera, a game device, a video device, a PDA (Personal Digital Assistant), or the like, the smartphone or the like can function as the staff terminal 1, the server device 4, or the imaging device 2 of the present disclosure.

[0383] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0384] The present technology can also be configured as follows. (1) a user interface control unit that controls the provision of a user interface that allows a user to specify an extraction condition for a group of images taken at a wedding event; an extraction processing unit that extracts images from the image group based on specified extraction conditions; Information processing device. (2) The user interface control unit controls displaying the images extracted by the extraction processing unit and providing a user interface that enables an input operation of rating information for each image. The information processing device according to (1) above. (3) The extraction processing unit refers to analysis result data of image contents for each image in the image group, and extracts images that meet designated extraction conditions. The information processing device according to (1) or (2) above. (4) The analysis result data is received from an external device. The information processing device according to (3) above. (5) The user interface control unit includes: Control is performed to display an icon presenting extraction conditions for the images in the image group, and an operation on the icon is detected as an operation for specifying the extraction conditions. An information processing device according to any one of (1) to (4) above. (6) The user interface control unit includes: Icons that are the target of the specified extraction conditions are displayed in a different manner from icons that are not the target of the specified operations. The information processing device according to (5) above. (7) The extraction conditions include a condition for specifying a person. An information processing device according to any one of (1) to (6) above. (8) The extraction conditions include a condition that can specify a scene of a wedding event. An information processing device according to any one of (1) to (7) above. (9) The extraction conditions include a condition that can specify time information of an event related to a wedding. An information processing device according to any one of (1) to (8) above. (10) The extraction conditions include conditions that allow designation of an object or animal to be a subject. An information processing device according to any one of (1) to (9) above. (11) The extraction conditions include conditions that can specify the emotion or facial expression of the subject. An information processing device according to any one of (1) to (10) above. (12) The user interface control unit presents information indicating a degree of emotion or facial expression of a subject of the image extracted by the extraction processing unit. An information processing device according to any one of (1) to (11) above. (13) The extraction conditions include conditions that can specify items to be determined based on metadata added at the time of image capture. An information processing device according to any one of (1) to (12) above. (14) The user interface control unit presents a focus position for the image extracted by the extraction processing unit. An information processing device according to any one of (1) to (13) above. (15) The user interface control unit includes: An icon of the extraction condition is displayed in a candidate area on a display screen, Control is performed so that an icon designated by a predetermined operation is displayed in a selection area on the display screen. An information processing device according to any one of (1) to (14) above. (16) The user interface control unit includes: The display of the icon displayed in the selection area is terminated in response to a predetermined deselection operation. The information processing device according to (15) above. (17) For each of the multiple extraction conditions specified, images are extracted by setting AND conditions, OR conditions, or exclusion conditions. An information processing device according to any one of (1) to (16) above. (18) A video editing unit that creates video content using the plurality of video scenes extracted by the extraction processing unit based on the extraction conditions. An information processing device according to any one of (1) to (17) above. (19) A control process for providing a user interface that enables a user to specify extraction conditions for a group of images taken at a wedding event; an extraction process for extracting images from the image group based on specified extraction conditions; The information processing method is performed by an information processing device. (20) A control process for providing a user interface that enables a user to specify extraction conditions for a group of images taken at a wedding event; an extraction process for extracting images from the image group based on specified extraction conditions; A program for causing an information processing device to execute the above. [Explanation of symbols]

[0385] 1 Staff terminal 1a Display section 2. Imaging device 2a Display section 2c D-pad 2d wheels 3. Personal Computers 4. Server equipment 5. Client terminal 6 Network 10. Information processing device 30 Event Data Management Department 31 UI control section 32 Communication control section 33 Automatic setting section 34 Extraction processing section 35 Rating Information Management Department 36 Analysis result data receiving unit 37 Analysis Department 38 Video Editing Department 71 CPU 200 Icon Area 201 Emotion and facial expression recognition level 202 Time slot 210 Face Selection Area AR1 candidate area AR2 Selected Area

Claims

1. a user interface control unit that controls the provision of a user interface that allows a user to specify an extraction condition for a group of images taken at a wedding event; an extraction processing unit that extracts images from the image group based on specified extraction conditions; The user interface control unit includes: An icon capable of extracting an image is set according to an analysis result of the image contents of all images in the image group, and the icon is displayed as a display of an extraction condition for the images in the image group. An operation on the icon is detected as an operation for specifying the extraction condition. Information processing device.

2. The user interface control unit controls displaying the images extracted by the extraction processing unit and providing a user interface that enables an input operation of rating information for each image. The information processing device according to claim 1 .

3. The extraction processing unit refers to analysis result data of image contents for each image in the image group, and extracts images that meet designated extraction conditions. The information processing device according to claim 1 .

4. The analysis result data is received from an external device. The information processing device according to claim 3 .

5. The user interface control unit includes: Icons that are the target of the specified extraction conditions are displayed in a different manner from icons that are not the target of the specified operations. The information processing device according to claim 1 .

6. The extraction conditions include a condition for specifying a person. The information processing device according to claim 1 .

7. The extraction conditions include a condition that can specify a scene of a wedding event. The information processing device according to claim 1 .

8. The extraction conditions include a condition that can specify time information of an event related to a wedding. The information processing device according to claim 1 .

9. The extraction conditions include conditions that allow designation of an object or animal to be a subject. The information processing device according to claim 1 .

10. The extraction conditions include conditions that can specify the emotion or facial expression of the subject. The information processing device according to claim 1 .

11. The user interface control unit presents information indicating a degree of emotion or facial expression of a subject of the image extracted by the extraction processing unit. The information processing device according to claim 1 .

12. The extraction conditions include conditions that can specify items to be determined based on metadata added at the time of image capture. The information processing device according to claim 1 .

13. The user interface control unit presents a focus position for the image extracted by the extraction processing unit. The information processing device according to claim 1 .

14. The user interface control unit includes: An icon of the extraction condition is displayed in a candidate area on a display screen, Control is performed so that an icon designated by a predetermined operation is displayed in a selection area on the display screen. The information processing device according to claim 1 .

15. The user interface control unit includes: The display of the icon displayed in the selection area is terminated in response to a predetermined deselection operation. The information processing device according to claim 14.

16. For each of the multiple extraction conditions specified, images are extracted by setting AND conditions, OR conditions, or exclusion conditions. The information processing device according to claim 1 .

17. A video editing unit that creates video content using the plurality of video scenes extracted by the extraction processing unit based on the extraction conditions. The information processing device according to claim 1 .

18. A control process for providing a user interface that enables a user to specify extraction conditions for a group of images taken at a wedding event; an extraction process for extracting images from the image group based on specified extraction conditions; The information processing device performs the above. In the control process, an icon capable of extracting an image is set according to an analysis result of image contents of all images in the image group, and the icon is displayed as a display icon presenting extraction conditions for the images in the image group. An operation on the icon is detected as an operation for specifying the extraction conditions. An information processing method performed by an information processing device.

19. A control process for providing a user interface that enables a user to specify extraction conditions for a group of images taken at a wedding event; an extraction process for extracting images from the image group based on specified extraction conditions; and causing an information processing device to execute the above. In the control process, an icon capable of extracting an image is set according to an analysis result of image contents of all images in the image group, and the icon is displayed as a display icon presenting extraction conditions for the images in the image group. An operation on the icon is detected as an operation for specifying the extraction conditions. A program to be executed by an information processing device.

Images