Electronic album generation device, method for operating electronic album generation device, and program for operating electronic album generation device

Abstract

This electronic album generation device is for preferentially selecting images having higher evaluation values as candidate images for generating an electronic album, and comprises a processor. The processor derives a degree of similarity between the candidate images and remaining images, and sets an evaluation value of a similar image of which the degree of similarity is within a first setting range and an evaluation value of a dissimilar image of which the degree of similarity is within a second setting range to be lower than those of the images of which the degrees of similarity are outside the first and second setting ranges.

Description

Electronic album creating device, operating method for electronic album creating device, and operating program for electronic album creating device

[0001] The technology of the present disclosure relates to an electronic album creating device, an operating method for an electronic album creating device, and an operating program for an electronic album creating device.

[0002] The image evaluation device described in Japanese Patent No. 4,934,380 includes an evaluation value acquisition means, an evaluation value determination means, an evaluation value change means, and a control means. The evaluation value acquisition means acquires a provisional evaluation value representing a provisional evaluation result for each of a plurality of images classified into a plurality of groups. The evaluation value determination means performs an evaluation value determination process to determine the provisional evaluation value of at least one image, the provisional evaluation value being a predetermined evaluation criterion, as the evaluation value of the image. The evaluation value change means performs an evaluation value change process to reduce the provisional evaluation values ​​of all images in the group to which the image with the determined evaluation value is classified, in accordance with the predetermined evaluation criterion. The control means controls the evaluation value determination means and the evaluation value change means to repeatedly perform the evaluation value determination process and the evaluation value change process using the reduced provisional evaluation value until evaluation values ​​are determined for the predetermined number of images.

[0003] The image processing device described in Japanese Patent Laid-Open No. 2013-222304 includes an evaluation unit, a presentation unit, and an update unit. The evaluation unit evaluates each of a plurality of images based on at least one of the image content and attribute information. The presentation unit presents some or all of the plurality of images as selection candidates that can be selected in response to a user instruction based on the evaluation. The update unit updates the evaluation of images other than a selected image among the plurality of images based on at least one of the image content and attribute information of the selected image.

[0004] The image processing device disclosed in Japanese Patent Laid-Open No. 2013-033453 edits the layout of images on each page of photo content consisting of multiple pages. The image processing device includes an image division unit, an image analysis unit, an image extraction unit, an image arrangement unit, an image display unit, and an image editing unit. The image division unit divides multiple images into a predetermined number of groups based on accompanying information of the images. The image analysis unit analyzes each image and generates image analysis information. For each group, the image extraction unit extracts a predetermined number of images from the images included in the group based on the image analysis information. For each group, the image arrangement unit arranges the images extracted from the group by the image extraction unit on the page corresponding to the group. The image display unit displays the page on which the images are arranged by the image arrangement unit. The image editing unit edits the layout of the images on the page on which the images are arranged by the image arrangement unit based on a user instruction. The image display unit displays, on the display screen, an image editing area in which an image arranged on a first page to be edited designated by a user is displayed, and a candidate image display area in which candidate images included in a first group corresponding to the first page and used in editing the image layout by the image editing unit are displayed. The image editing unit edits the layout of the image on the first page displayed in the image editing area based on an instruction from the user, using the candidate images displayed in the candidate image display area.

[0005] The image processing device described in JP 2016-058923 A includes an image acquisition means for acquiring a plurality of images, a selection means for selecting a predetermined number of images from the plurality of images based on accompanying information regarding the use of the plurality of images acquired by the image acquisition means, and a generation means for generating new images from the predetermined number of images selected by the selection means.

[0006] One embodiment of the technology of the present disclosure provides an electronic album creation device, an operating method for the electronic album creation device, and an operating program for the electronic album creation device that are capable of creating an electronic album that is not monotonous and that also takes into consideration a sense of unity.

[0007] The electronic album creation device disclosed herein is an electronic album creation device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, and is equipped with a processor that derives the similarity between the candidate images and the remaining images, and sets the evaluation values ​​of similar images whose similarity is within a first set range and dissimilar images whose similarity is within a second set range lower than the evaluation values ​​of images whose similarity is outside the first set range and the second set range.

[0008] It is preferable that an evaluation value be derived according to the image quality of the image.

[0009] It is preferable that an evaluation value be derived according to the composition of the image.

[0010] The processor preferably sets an evaluation value for an image whose composition differs from that of the candidate image higher than an evaluation value for an image whose composition matches that of the candidate image.

[0011] The processor preferably divides the images into a plurality of groups and selects candidate images for creating an electronic album for each group.

[0012] Preferably, the processor performs grouping based on the date and time of capture.

[0013] Preferably, the processor groups the images based on their similarity to one another.

[0014] Preferably, the processor excludes images having evaluation values ​​outside a third set range from the candidate images.

[0015] The processor preferably extracts, from the plurality of images, a group of images to be arranged in a specified order in a unit area of ​​the electronic album, and sets the evaluation value of the images belonging to the group of images higher than the evaluation values ​​of the other images.

[0016] The processor derives the distance in the space of feature vectors between the feature vector representing the features of the candidate image and the feature vector representing the features of the remaining images as similarity, and treats the remaining images whose similarity is less than a first threshold as similar images, and treats the remaining images whose similarity is equal to or greater than a second threshold as dissimilar images.

[0017] The operating method of the electronic album creation device disclosed herein is a method for operating an electronic album creation device in which images with higher evaluation values ​​are preferentially selected as candidate images for creating an electronic album, and includes deriving the similarity between the candidate images and the remaining images, and setting the evaluation values ​​of similar images whose similarity is within a first set range and dissimilar images whose similarity is within a second set range to be lower than the evaluation values ​​of images whose similarity is outside the first set range and the second set range.

[0018] The operating program of the electronic album creation device disclosed herein is an operating program of an electronic album creation device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, and causes a computer to execute processes including deriving the similarity between the candidate images and the remaining images, and setting the evaluation values ​​of similar images whose similarity is within a first set range and dissimilar images whose similarity is within a second set range lower than the evaluation values ​​of images whose similarity is outside the first set range and the second set range.

[0019] 1 is a diagram illustrating a user terminal and an image management server. FIG. 1 is a block diagram illustrating computers constituting a user terminal and an image management server. FIG. 2 is a block diagram illustrating a processing unit of a CPU of a user terminal. FIG. 3 is a block diagram illustrating a processing unit of a CPU of an image management server. FIG. 4 is a diagram illustrating data stored in an image information DB. FIG. 5 is a diagram illustrating image information. FIG. 6 is a diagram illustrating the processing of an evaluation value derivation unit. FIG. 7 is a diagram illustrating the processing in the learning phase of an evaluation value derivation model. FIG. 8 is a block diagram illustrating the detailed configuration of an album creation unit. FIG. 9 is a block diagram illustrating the detailed configuration of an album creation unit. FIG. 10 is a diagram illustrating the processing of a selection unit. FIG. 11 is a diagram illustrating the process of deriving the Euclidean distance of a feature vector as similarity. FIG. 12 is a diagram illustrating the similarity derivation result. FIG. 13 is a diagram illustrating the processing of an evaluation value setting unit, where (A) shows a case where the similarity is less than 10, (B) shows a case where the similarity is 200 or more, and (C) shows a case where the similarity is 10 or more and less than 200. FIG. 14 is a diagram illustrating the evaluation value setting result. FIG. 15 is a flowchart illustrating the processing procedure of an image management server. FIG. 16 is a flowchart illustrating the processing procedure of an image management server. FIG. 17 is a diagram illustrating an example of grouping based on the similarity between images. FIG. 18 is a diagram illustrating the processing unit of a second embodiment. FIG. 19 is a diagram illustrating the processing of an evaluation value setting unit when the composition matches that of a candidate image. FIG. 10 is a diagram showing the processing of an evaluation value setting unit when a composition differs from that of a candidate image. FIG. 11 is a diagram showing the processing unit of a third embodiment. FIG. 12 is a diagram showing the processing of a group image extraction unit. FIG. 13 is a diagram showing the processing of an evaluation value setting unit for images belonging to a group of images. FIG. 14 is a diagram showing a state in which images belonging to a group of images corresponding to an introduction, development, twist, and conclusion are arranged on a double-page spread.

[0020] [First Embodiment] As an example, as shown in FIG. 1 , a user U owns a user terminal 10. The user terminal 10 is a device having a camera function, an image playback / display function, an image editing function, an image transmission / reception function, and the like. The camera function has an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and obtains an image 56 of a subject (see FIG. 5 ) by focusing subject light captured through a lens on the imaging element. Specifically, the user terminal 10 is a digital camera, a smartphone, a tablet terminal, a notebook personal computer, or the like. The user U uses the camera function to capture an image 56 and uses the image editing function to edit the image 56 to his or her liking.

[0021] The user terminal 10 is connected to the image management server 12 via a network 11 so as to be able to communicate with each other. The network 11 is, for example, a wide area network (WAN) such as the Internet or a public communication network. The user terminal 10 transmits (uploads) images 56 to the image management server 12. The user terminal 10 also receives (downloads) images 56 from the image management server 12.

[0022] The image management server 12 is, for example, a server computer, a workstation, or the like, and is an example of an “electronic album creation device” according to the technology of the present disclosure. A plurality of user terminals 10 of a plurality of users U are connected to the image management server 12 via a network 11.

[0023] 2, the computers that make up the user terminal 10 and the image management server 12 basically have the same configuration, and include a storage 20, a memory 21, a CPU (Central Processing Unit) 22, a communication unit 23, a display 24, and an input device 25. These are interconnected via a bus line 26.

[0024] The storage 20 is a hard disk drive built into the computer that constitutes the user terminal 10 and the image management server 12, or connected via a cable or network. Alternatively, the storage 20 is a disk array consisting of multiple hard disk drives. The storage 20 stores control programs such as an operating system, various application programs (hereinafter abbreviated as APs (Application Programs)), and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0025] The memory 21 is a work memory for the CPU 22 to execute processing. The CPU 22 loads a program stored in the storage 20 into the memory 21 and executes processing in accordance with the program. In this way, the CPU 22 comprehensively controls each part of the computer. The CPU 22 is an example of a "processor" according to the technology of the present disclosure. The memory 21 may be built into the CPU 22.

[0026] The communication unit 23 is a network interface that controls the transmission of various information via the network 11, etc. The display 24 displays various screens. The various screens are equipped with operation functions using a GUI (Graphical User Interface). The computers that make up the user terminal 10 and the image management server 12 accept input of operation instructions from an input device 25 via the various screens. The input device 25 is a keyboard, mouse, touch panel, microphone for voice input, etc.

[0027] In the following explanation, the parts of the computer that make up the user terminal 10 (storage 20, CPU 22, display 24, and input device 25) are distinguished by adding the suffix "A" to their symbols, and the parts of the computer that make up the image management server 12 (storage 20 and CPU 22) are distinguished by adding the suffix "B" to their symbols.

[0028] As an example, as shown in Figure 3, an image AP 30 is stored in the storage 20A of the user terminal 10. The image AP 30 is installed in the user terminal 10 by the user U. The image AP 30 is an AP for playing back, displaying, and editing an image 56 on the user terminal 10. When the image AP 30 is started, the CPU 22A of the user terminal 10 functions as a browser control unit 32 in cooperation with the memory 21 and the like. The browser control unit 32 controls the operation of a web browser dedicated to the image AP 30.

[0029] The browser control unit 32 receives screen data for various screens from the image management server 12. The browser control unit 32 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 24A. The browser control unit 32 also accepts various operation instructions input by the user U from the input device 25A via the various screens. The browser control unit 32 transmits various requests corresponding to the operation instructions to the image management server 12.

[0030] 4, an operating program 35 is stored in storage 20B of image management server 12. Operating program 35 is an AP that causes a computer constituting image management server 12 to function as an "electronic album creation device" according to the technology of the present disclosure. In other words, operating program 35 is an example of an "electronic album creation device operating program" according to the technology of the present disclosure.

[0031] The storage 20B also stores an image information database (hereinafter referred to as DB (Data Base)) 36, an evaluation value derivation model 37, and creation auxiliary information 38. Although not shown in the figure, the storage 20B also stores, as account information of the user U, a user ID (Identification Data) for uniquely identifying the user U, a password set by the user U, and a terminal ID for uniquely identifying the user terminal 10.

[0032] When the operating program 35 is started, the CPU 22B of the image management server 12 cooperates with the memory 21 and the like to function as a request receiving unit 45, an evaluation value derivation unit 46, a read / write (hereinafter referred to as RW (Read Write)) control unit 47, an album creation unit 48, and a distribution control unit 49.

[0033] The request receiving unit 45 receives various requests from the user terminal 10. The request receiving unit 45 outputs various requests to the evaluation value derivation unit 46 and / or the RW control unit 47 and the distribution control unit 49. The various requests include a storage request for storing the image 56 in the image information DB 36 and a distribution request for distributing the image 56 stored in the image information DB 36. The various requests also include a request for creating an electronic album 60 (see FIG. 5 ). The various requests include the terminal ID of the user terminal 10 that sent the various requests.

[0034] The evaluation value derivation unit 46 derives an evaluation value 58 (see FIG. 5 ) from the image 56 for which a storage request has been made. The evaluation value 58 is a numerical value that serves as an index for determining whether or not to select the image 56 for creating the electronic album 60. The evaluation value derivation unit 46 outputs the derived evaluation value 58 to the RW control unit 47 together with the image 56.

[0035] The RW control unit 47 controls the storage of various data in the storage 20B and the reading of various data from the storage 20B. In particular, the RW control unit 47 controls the storage of images 56 in the image information DB 36 and the reading of images 56 from the image information DB 36. The RW control unit 47 also controls the storage of evaluation values ​​58 in the image information DB 36 and the reading of evaluation values ​​58 from the image information DB 36.

[0036] The RW control unit 47 reads the evaluation value derivation model 37 from the storage 20B and outputs the read evaluation value derivation model 37 to the evaluation value derivation unit 46. The RW control unit 47 also reads the creation auxiliary information 38 from the storage 20B and outputs the read creation auxiliary information 38 to the album creation unit 48.

[0037] The album creating section 48 creates an electronic album 60 in response to the creation request based on the evaluation value 58 while referring to the creation auxiliary information 38. The album creating section 48 outputs the created electronic album 60 to the distribution control section 49.

[0038] The request to create the electronic album 60 includes a user ID and specification information for the electronic album 60. The specification information is composed of designation information for the images 56 for which the electronic album 60 is to be created, a layout frame ID for the layout frame to be used in the electronic album 60, etc. The designation information is information that designates at least one of the period of the date and time of the image 56, the location where the image was taken, and the subject such as a person.

[0039] The distribution control unit 49 controls the distribution of various data including screen data to the user terminal 10. The distribution control unit 49 distributes the screen data to the user terminal 10 in the form of screen data for web distribution created using a markup language such as XML (Extensible Markup Language). Note that instead of XML, other data description languages ​​such as JSON (Javascript (registered trademark) Object Notation) may be used.

[0040] 5, the image information DB 36 has a storage area 55 for each user U, such as user U1 and user U2. User IDs are registered in the storage area 55. The storage area 55 stores a plurality of pairs of images 56, associated information 57, and evaluation values ​​58. Hereinafter, the pairs of images 56, associated information 57, and evaluation values ​​58 will be referred to as image information 59.

[0041] The images 56 stored in the memory area 55 are related to the user U whose user ID is registered in the memory area 55. The images 56 include images taken by the user U using the camera function of the user terminal 10, images scanned by the user U, images downloaded by the user U from an internet site, or images given to the user U by an acquaintance. The images 56 "related" to the user U are typically images 56 stored by the user U himself on the user terminal 10 or a cloud server such as the image management server 12. Images 56 stored by someone other than the user U include images 56 "related" to the user U if the user U has been given access rights to the images 56 and the user U can freely edit them.

[0042] An electronic album 60 is also stored in the storage area 55. The electronic album 60 is also called a photo album, photo book, or the like, and is created by appropriately laying out all or part of a plurality of images 56 designated by the user U in a designated layout frame. Note that, of course, the electronic album 60 is not stored in the storage area 55 of a user U who has not created an electronic album 60.

[0043] As an example, as shown in FIG. 6 , an image 56, additional information 57, and evaluation value 58 that constitute image information 59 are associated with each other by an image ID. The additional information 57 includes multiple items such as the date and time of shooting, the location of shooting, and a tag. The date and time when the image 56 was captured using the camera function of the user terminal 10 is registered as the date and time of shooting. The location of shooting is registered with an address and / or a landmark name determined from longitude and latitude information obtained using the GPS (Global Positioning System) function of the user terminal 10. The tag is a word that succinctly describes the subject appearing in the image 56. Tags include those manually entered by the user U and those derived using a trained model for subject discrimination. Although not shown, the additional information 57 also includes items such as exposure value, ISO (International Organization for Standardization) sensitivity, shutter speed, focal length, and whether or not a flash is used.

[0044] 7 , the evaluation value derivation unit 46 inputs an image 56 for which a storage request has been made to the evaluation value derivation model 37. Then, the evaluation value derivation model 37 outputs an evaluation value 58. The evaluation value derivation model 37 is a trained model constructed using, for example, a convolutional neural network.

[0045] The evaluation value derivation model 37 derives an evaluation value 58 based on the results of evaluating multiple evaluation items related to the image quality of the image 56. The multiple evaluation items include, for example, whether the exposure value, shutter speed, and aperture value of the image 56 are appropriate, whether blurring or out-of-focus occurs, and whether the image is sharp. The multiple evaluation items also include whether the composition is likely to be selected for the electronic album 60, whether a person's face is included, whether a person's face is smiling if a person's face is included, the size of the person's face, the number of people, and so on. In other words, the evaluation value 58 varies depending on the image quality of the image 56 and takes a value between 0 and 100. 0 indicates that the image 56 has the lowest image quality. 100 indicates that the image 56 has the highest image quality. The evaluation value 58 can also be described as the probability that the image 56 will be included in the electronic album 60.

[0046] As an example, as shown in FIG. 8 , in the learning phase of the evaluation value derivation model 37, learning data (also called teacher data or training data) 70 is prepared. The learning data 70 is composed of a pair of a learning image 56L and a correct answer evaluation value 58CA. A plurality of pieces of learning data 70 are prepared. The correct answer evaluation value 58CA is a numerical value corresponding to the result of an annotator's actual evaluation of the learning image 56L for the above-mentioned plurality of evaluation items, and is, so to speak, data for checking the answer.

[0047] In the learning phase, a training image 56L is input to the evaluation value derivation model 37, which then outputs a training evaluation value 58L. This training evaluation value 58L is compared with the correct evaluation value 58CA. Then, based on the comparison result, a loss calculation is performed for the evaluation value derivation model 37 using a loss function. Then, depending on the result of the loss calculation, various coefficients of the evaluation value derivation model 37 (such as the coefficients of the filters in the convolution layer) are updated, and the evaluation value derivation model 37 is updated according to the update setting.

[0048] In the learning phase of the evaluation value derivation model 37, the above-described series of processes, including input of the training image 56L to the evaluation value derivation model 37, output of the training evaluation value 58L from the evaluation value derivation model 37, loss calculation, update setting, and update of the evaluation value derivation model 37, are repeatedly performed while the training data 70 is exchanged. The repetition of the above-described series of processes is terminated when the derivation accuracy of the training evaluation value 58L reaches a predetermined set level. The evaluation value derivation model 37 whose derivation accuracy has thus reached the set level is stored in the storage 20B of the image management server 12. Note that learning may be terminated when the above-described series of processes has been repeated a set number of times, regardless of the derivation accuracy of the training evaluation value 58L.

[0049] 9 and 10 , the album creation unit 48 includes an acquisition unit 75, a grouping unit 76, a selection unit 77, a similarity derivation unit 78, an evaluation value setting unit 79, and a layout unit 80. The creation auxiliary information 38 includes setting ranges 85A, 85B, and 85C.

[0050] The RW control unit 47 reads out a plurality of pieces of image information 59 including the image 56 specified by the specification information from the storage area 55 corresponding to the user ID of the request to create the electronic album 60. Then, the RW control unit 47 outputs an image information group 90, which is a collection of the plurality of pieces of image information 59 that have been read out, to the acquisition unit 75. The acquisition unit 75 acquires the image information group 90 and outputs it to the grouping unit 76.

[0051] The grouping unit 76 divides the image information 59 (images 56) of the image information group 90 into a plurality of groups, such as a first group G1 and a second group G2. The grouping unit 76 performs grouping based on the date and time the images 56 were captured, such as by year, by half a year, by month, by day, by hour, or by morning and afternoon. The grouping unit 76 outputs the grouping results 91 to the selection unit 77, the similarity derivation unit 78, and the evaluation value setting unit 79. Note that, hereinafter, a plurality of groups, such as the first group G1 and the second group G2, will be collectively referred to as a group GK (K=1 to N (N is a natural number equal to or greater than 2)).

[0052] A setting range 85A is input to the selection unit 77. As shown in FIG. 11 as an example, the selection unit 77 first excludes from each group GK image information 59 (images 56) whose evaluation value 58 is outside the setting range 85A. The setting range 85A is, for example, content in which the evaluation value 58 is 50 or greater. In this case, the selection unit 77 excludes from each group GK image information 59 including images 56 whose evaluation value 58 is less than 50. The setting range 85A is an example of a "third setting range" according to the technology of the present disclosure.

[0053] Next, the selection unit 77 selects the image 56 with the highest evaluation value 58 from among the images 56 of the remaining image information 59 as the image 56 for creating the electronic album 60. Hereinafter, the image 56 selected by the selection unit 77 will be referred to as the candidate image 56C. Note that Fig. 11 illustrates an example in which, after three image information 59 are excluded from the ten image information 59 in the first group G1, the image 56 with the highest evaluation value 58 is selected and set as the candidate image 56C.

[0054] The selection unit 77 outputs the selection result 92 to the similarity derivation unit 78 and the layout unit 80. The selection result 92 is a set of candidate images 56C for each group GK.

[0055] The similarity derivation unit 78 first derives feature quantities representing the characteristics of the candidate image 56C and the remaining images 56 of each group GK. The feature quantities are, for example, a collection of representative pixel values ​​(average, mode, maximum, minimum, etc.) of the candidate image 56C and the remaining images 56, the sharpness, degree of blur, and degree of defocus of the candidate image 56C and the remaining images 56, and feature quantities obtained by inputting the candidate image 56C and the remaining images 56 into a trained model such as an autoencoder. In other words, the feature quantities can be considered multidimensional feature vectors. In addition to the above, feature quantities may also include a numerical value representing the type of main subject (person, animal, still life, landscape, etc.), the area occupied by the main subject, a numerical value corresponding to the position of the main subject, and the color of the main subject. Here, the trained model may be generated by supervised learning such as the exemplary autoencoder, or by unsupervised learning. The trained model may output one or more feature quantities. The features output from a trained model cannot be interpreted semantically by humans, but if a unique numerical value is output when an image is input to the trained model, it can be used as a feature representing the characteristics of the image.

[0056] The similarity derivation unit 78 derives a similarity 102 (see FIG. 12 ) between the candidate image 56C and the remaining images 56 for each group GK. More specifically, as shown in the graph 100 of FIG. 12 as an example, the similarity derivation unit 78 derives the Euclidean distance D in a feature vector space (hereinafter referred to as the feature space) 101 between the feature vector of the candidate image 56C and the feature vector of the remaining images 56 as the similarity 102. The Euclidean distance D can be used as an index representing the similarity between the candidate image 56C and the remaining images 56. In other words, the closer the Euclidean distance D between the candidate image 56C and the remaining images 56 in the feature space 101, the more similar the candidate image 56C and the remaining images 56 are. Conversely, the farther the Euclidean distance D between the candidate image 56C and the remaining images 56 in the feature space 101, the less similar the candidate image 56C and the remaining images 56 are. 12, for convenience of explanation, the feature space 101 is represented in two dimensions along the D1 and D2 axes, but the actual dimension of the feature space 101 is several hundred to several thousand. The same applies to the graph 105 in the following FIG. 18. Furthermore, instead of the Euclidean distance D, the Mahalanobis distance between the feature vector of the candidate image 56C and the feature vector of the remaining images 56 may be derived as the similarity 102. Alternatively, the cosine similarity between the feature vector of the candidate image 56C and the feature vector of the remaining images 56 may be derived as the similarity 102.

[0057] The similarity derivation unit 78 outputs the similarity derivation result 93 to the evaluation value setting unit 79. As an example, as shown in Fig. 13 , the similarity derivation result 93 is a registration of the similarity 102 of the remaining images 56 with respect to the candidate image 56C and the evaluation value 58 for each group GK.

[0058] Setting ranges 85B and 85C are input to evaluation value setting unit 79. As shown in FIG. 14 as an example, setting range 85B specifies that similarity 102 is less than 10. Setting range 85C specifies that similarity 102 is 200 or greater. Setting range 85B is an example of a "first setting range" according to the technology of the present disclosure. Setting range 85C is an example of a "second setting range" according to the technology of the present disclosure.

[0059] 14A , the evaluation value setting unit 79 resets the evaluation value 58 of the remaining images 56 for which the similarity 102 to the candidate image 56C is within the set range 85B, i.e., the similarity 102 is less than 10, to 0. A similarity 102 of less than 10 means that the Euclidean distance D in the feature space 101 is relatively close, and the image 56 can be said to be relatively similar to the candidate image 56C. Hereinafter, an image 56 whose similarity 102 to the candidate image 56C is within the set range 85B will be referred to as a similar image 56S. Note that the value of 10 in the set range 85B is an example of a "first threshold" according to the technology of the present disclosure.

[0060] 14B , the evaluation value setting unit 79 also resets the evaluation value 58 of the remaining images 56 whose similarity 102 to the candidate image 56C is within the set range 85C, i.e., whose similarity 102 is 200 or greater, to 0. A similarity 102 of 200 or greater means that the Euclidean distance D in the feature space 101 is relatively far, and the image 56 is relatively dissimilar to the candidate image 56C. Hereinafter, an image 56 whose similarity 102 to the candidate image 56C is within the set range 85C will be referred to as a dissimilar image 56DS. Note that 200 in the set range 85C is an example of a "second threshold" according to the technology of the present disclosure.

[0061] 14C , evaluation value setting unit 79 leaves the original evaluation value 58 of any remaining image 56 whose similarity 102 to candidate image 56C is outside set range 85B and set range 85C, i.e., whose similarity 102 is equal to or greater than 10 and less than 200. A similarity 102 of equal to or greater than 10 and less than 200 means that the Euclidean distance D in feature space 101 is neither extremely close nor extremely far, and while that image 56 is not completely dissimilar to candidate image 56C, it cannot be said that it is very similar either.

[0062] In this way, the evaluation value setting unit 79 resets the evaluation value 58 of the similar image 56S whose similarity 102 is within the setting range 85B and the evaluation value 58 of the dissimilar image 56DS whose similarity 102 is within the setting range 85C to 0. Furthermore, the evaluation value setting unit 79 leaves the evaluation value 58 of the image 56 whose similarity 102 is outside the setting range 85B and outside the setting range 85C at its original value. In this way, the evaluation value setting unit 79 sets the evaluation value 58 of the similar image 56S whose similarity 102 is within the setting range 85B and the evaluation value 58 of the dissimilar image 56DS whose similarity 102 is within the setting range 85C to be lower than the evaluation value 58 of the image 56 whose similarity 102 is outside the setting range 85B and outside the setting range 85C.

[0063] The evaluation value setting unit 79 outputs the evaluation value setting result 94 to the selection unit 77. As an example, as shown in Fig. 15 , the evaluation value setting result 94 is a registration of the evaluation values ​​58 of the remaining images 56 for each group GK, including those that have been reset to 0.

[0064] The selection unit 77 again selects candidate images 56C for creating the electronic album 60 shown in Fig. 11 based on the evaluation value setting result 94. That is, the selection unit 77 excludes from each group GK the image information 59 (images 56) whose evaluation value 58 is outside the set range 85A. At this time, the similar image 56S and the dissimilar image 56DS are excluded as they are outside the set range 85A because their evaluation values ​​58 are set to 0.

[0065] Thereafter, the selection unit 77 selects the image 56 with the highest evaluation value 58 as a candidate image 56C for creating the electronic album 60. The similarity derivation unit 78 derives the similarity 102 between the candidate image 56C selected by the selection unit 77 and the remaining images 56 based on the evaluation value setting result 94. The evaluation value setting unit 79 sets the evaluation value 58 for the remaining images 56 as shown in Fig. 14. The selection unit 77, the similarity derivation unit 78, and the evaluation value setting unit 79 repeat these processes until the set number of candidate images 56C for each group GK are stored in the layout unit 80.

[0066] The set number may be the same for all groups GK or may be different for each group GK. The set number may be set by the user U and included in the request to create the electronic album 60, or may be automatically set by the album creating unit 48 based on the number of candidate images 56C for each group GK.

[0067] The layout unit 80 creates the electronic album 60 by appropriately laying out the candidate images 56C in the layout frame of the layout frame ID specified in the request to create the electronic album 60. At this time, the layout unit 80 lays out the candidate images 56C in units of group GK, such as by collectively laying out the candidate images 56C of one group GK on a double-page spread 120 (see FIG. 25 ). The layout unit 80 outputs the created electronic album 60 to the distribution control unit 49.

[0068] If the highest evaluation value 58 falls outside the set range 85A, the selection unit 77 stops selecting candidate images 56C for creating the electronic album 60. The layout unit 80 creates the electronic album 60 using the candidate images 56C that have been stored up to that point. Alternatively, if the highest evaluation value 58 falls outside the set range 85A, the set range 85A may be loosened again.

[0069] The distribution control unit 49 generates screen data for the display screen of the electronic album 60 and distributes this to the user terminal 10 that sent the creation request. The browser control unit 32 reproduces the display screen based on the screen data for the display screen of the electronic album 60 and displays this on the display 24A. The user U views the electronic album 60 displayed on the display 24A.

[0070] The user U issues an instruction to store the electronic album 60 as necessary. When an instruction to store the electronic album 60 is issued, the browser control unit 32 transmits a request to store the electronic album 60 to the image management server 12. The image management server 12 receives the request to store the electronic album 60 at the request receiving unit 45, and stores the electronic album 60 in the image information DB 36 by the RW control unit 47.

[0071] Next, the operation of the above configuration will be described with reference to the flowcharts shown in Figures 16 and 17 as an example. As shown in Figure 3, the CPU 22A of the user terminal 10 functions as a browser control unit 32 when the image AP 30 is activated. As shown in Figure 4, the CPU 22B of the image management server 12 functions as a request receiving unit 45, an evaluation value derivation unit 46, an RW control unit 47, an album creation unit 48, and a distribution control unit 49 when the operating program 35 is activated. As shown in Figures 9 and 10, the album creation unit 48 includes an acquisition unit 75, a grouping unit 76, a selection unit 77, a similarity derivation unit 78, an evaluation value setting unit 79, and a layout unit 80.

[0072] The user U takes an image 56 using the camera function of the user terminal 10. After taking the image 56, a request to store the image 56 is sent to the image management server 12 under the control of the browser control unit 32.

[0073] In the image management server 12, the request receiving unit 45 receives a request to store the image 56 (YES in step ST100 of FIG. 16). The request receiving unit 45 outputs the request to the evaluation value derivation unit 46 and the like.

[0074] 7 , in the evaluation value derivation unit 46, the image 56 is input to the evaluation value derivation model 37, which then outputs an evaluation value 58 (step ST110). The evaluation value 58 is output together with the image 56 from the evaluation value derivation unit 46 to the RW control unit 47. Then, under the control of the RW control unit 47, the image 56, the incidental information 57, and the evaluation value 58 are associated with each other by the image ID and stored as image information 59 in the image information DB 36 of the storage 20B (step ST120).

[0075] When an instruction to create an electronic album 60 for user U is issued via the input device 25A of the user terminal 10, a request to create the electronic album 60 is transmitted from the browser control unit 32 to the image management server 12. In the image management server 12, the request to create the electronic album 60 is received by the request receiving unit 45 (YES in step ST200 of FIG. 17 ). The request to create the electronic album 60 is output from the request receiving unit 45 to the RW control unit 47. Then, as shown in FIG. 9 , under the control of the RW control unit 47, multiple pieces of image information 59, including the image 56 specified by the designation information, are read from the storage area 55 corresponding to the user ID of the creation request. An image information group 90, which is a collection of multiple pieces of image information 59, is output from the RW control unit 47 to the acquisition unit 75 and acquired by the acquisition unit 75 (step ST210). The image information group 90 is output from the acquisition unit 75 to the grouping unit 76.

[0076] The grouping unit 76 divides the image information 59 (images 56) in the image information group 90 into a plurality of groups GK based on the shooting date and time (step ST220). The grouping results 91 are output from the grouping unit 76 to the selection unit 77, the similarity derivation unit 78, and the evaluation value setting unit 79.

[0077] 11, the selection unit 77 excludes from each group GK the image information 59 (images 56) whose evaluation value 58 is outside the set range 85A (step ST230). Next, from among the images 56 of the remaining image information 59, the image 56 with the highest evaluation value 58 is selected as a candidate image 56C for creating the electronic album 60 (step ST240). The selection result 92 is output from the selection unit 77 to the similarity derivation unit 78 and the layout unit 80.

[0078] 12, the similarity deriving unit 78 derives the similarity 102 between the candidate image 56C and the remaining images 56 for each group GK (step ST260). The similarity derivation result 93 is output from the similarity deriving unit 78 to the evaluation value setting unit 79.

[0079] As shown in FIG. 14A, the evaluation value setting unit 79 resets the evaluation value 58 of the similar image 56S, among the remaining images 56, whose similarity 102 to the candidate image 56C is within the set range 85B, to 0. Also, as shown in FIG. 14B, the evaluation value 58 of the dissimilar image 56DS, among the remaining images 56, whose similarity 102 to the candidate image 56C is within the set range 85C, is also reset to 0 (step ST270). Meanwhile, as shown in FIG. 14C, the evaluation value 58 of the images 56, among the remaining images 56, whose similarity 102 to the candidate image 56C is outside the set range 85B and the set range 85C, is left unchanged. The evaluation value setting result 94 is output from the evaluation value setting unit 79 to the selection unit 77. Thereafter, the selection unit 77 performs the processes of steps ST230 and ST240 again.

[0080] The processes of steps ST230, ST240, ST260, and ST270 are repeated until the set number of candidate images 56C are stored in the layout section 80 for each group GK (NO in step ST250).

[0081] When the set number of candidate images 56C for each group GK have been stored in the layout unit 80 (YES in step ST250), the layout unit 80 lays out the candidate images 56C in the layout frames and creates the electronic album 60 (step ST280). The electronic album 60 is output from the layout unit 80 to the distribution control unit 49. The electronic album 60 is distributed in the form of screen data for the display screen to the user terminal 10 that sent the creation request under the control of the distribution control unit 49 (step ST290).

[0082] As described above, the similarity derivation unit 78 derives the similarity 102 between the candidate image 56C and the remaining images 56. The evaluation value setting unit 79 sets the evaluation value 58 of the similar image 56S whose similarity 102 is within the set range 85B and the evaluation value 58 of the dissimilar image 56DS whose similarity 102 is within the set range 85C lower than the evaluation values ​​58 of the images 56 whose similarity 102 is outside the set ranges 85B and 85C. This makes it less likely that the similar image 56S and the dissimilar image 56DS will be selected as candidate images 56C for creating the electronic album 60. This prevents the creation of a monotonous electronic album 60 containing duplicate similar images 56, or, conversely, the creation of an electronic album 60 containing unexpected images 56 that are completely dissimilar to the other images 56. This makes it possible to create an electronic album 60 that is not monotonous and that also takes into consideration a sense of unity. As a result, it is possible to lead to an order for printing of the electronic album 60 or to give the user U an awareness of know-how for creating the electronic album 60 .

[0083] 7, an evaluation value 58 is derived according to the image quality of the image 56. Therefore, images 56 with relatively good image quality can be preferentially adopted in the electronic album 60, and images 56 with relatively poor image quality can be excluded from adoption in the electronic album 60.

[0084] As shown in Fig. 9, the grouping unit 76 divides the images 56 into a plurality of groups GK. As shown in Fig. 11, the selection unit 77 selects candidate images 56C for creating the electronic album 60 for each group GK. Therefore, compared to the case where candidate images 56C for creating the electronic album 60 are selected regardless of the group GK, it is possible to create an electronic album 60 with a more unified look.

[0085] 9, the grouping unit 76 performs grouping based on the shooting date and time, so that it is possible to create an electronic album 60 that has a sense of temporal unity, such as an electronic album 60 in which the images 56 are arranged in chronological order.

[0086] 11 , the selection unit 77 excludes from the candidate images 56C any images 56 whose evaluation values ​​58 are outside the set range 85A. This prevents images 56 with relatively poor image quality from being used in the electronic album 60. Note that excluding images 56 whose evaluation values ​​58 are outside the set range 85A from the candidate images 56C includes both a case where, as in this example, the images 56 whose evaluation values ​​58 are outside the set range 85A are excluded before the candidate images 56C are selected so that they are not used as candidate images 56C, and a case where the candidate images 56C whose evaluation values ​​58 are outside the set range 85A are excluded from the selected candidate images 56C after the candidate images 56C have been selected.

[0087] Although the example has been given in which the grouping unit 76 performs grouping based on the shooting date and time, this is not limiting. Grouping may also be performed based on the similarity 102 between the images 56. More specifically, as shown in the graph 105 in FIG. 18 as an example, the grouping unit 76 derives a feature vector for each image 56. Then, clustering is performed on the derived feature vectors to divide the feature vectors into a plurality of clusters 106. The clusters 106 are collections of images 56 that have a high similarity 102 to each other (i.e., the Euclidean distance D between the feature vectors is short). The grouping unit 76 treats images 56 that belong to the same cluster 106 as images 56 of a single group GK.

[0088] For records of a child's daily life, travel records, and the like, it is preferable to arrange the images 56 in chronological order in the electronic album 60. In contrast, for photo books of female models and photo books of landscapes, the images may have different shooting dates, so it is better to prioritize appearance over chronological order when arranging the images 56 in the electronic album 60. In such cases where appearance is a priority, it is preferable to group the images 56 based on the similarity 102 between the images 56, as in this example. For example, in the case of a photo book of a female model, grouping based on the similarity 102 between the images 56 would result in images 56 taken outdoors and indoors, images 56 taken by the pool and images 56 taken around the bed, images 56 taken in red clothing and images 56 taken in light blue clothing, and so on, being divided into different groups GK. This allows for the creation of an electronic album 60 with a consistent appearance.

[0089] The configuration may be such that the user U can select whether to group the images 56 based on the shooting date and time or the similarity 102 between the images 56. Alternatively, the grouping unit 76 may analyze the images 56 and, depending on the analysis results, suggest to the user U whether to group the images 56 based on the shooting date and time or the similarity 102 between the images 56, and leave it up to the user U to choose. In these cases, the result of the user U's selection may be included in a request to create the electronic album 60. Alternatively, the grouping unit 76 may select, depending on the analysis results of the images 56, whether to group the images 56 based on the shooting date and time or the similarity 102 between the images 56.

[0090] The page layout of the electronic album 60 may be changed according to the similarity of the groups GK, for example, by making pages of groups GK whose centers of gravity are close to each other in Euclidean distance D into a contiguous group.

[0091] Grouping may be performed based on both the shooting date and time and the similarity 102. Also, grouping may be performed based on criteria other than the shooting date and time and the similarity 102, such as the shooting location or the people in the photograph.

[0092] [Second Embodiment] As shown in FIG. 19 as an example, the album creation unit of the second embodiment functions as a composition analysis unit 110 in addition to the processing units 75 to 80 of the first embodiment (only the evaluation value setting unit 79 is shown in FIG. 19 ). The composition analysis unit 110 is arranged in parallel with the similarity derivation unit 78 between the selection unit 77 and the evaluation value setting unit 79. The composition analysis unit 110 receives the candidate image 56C and the remaining images 56. The composition analysis unit 110 analyzes the composition of the candidate image 56C and the remaining images 56 by making full use of well-known techniques and trained models. The composition includes multiple items such as shooting styles (portrait / snapshot / landscape, close-up / long shot, and vertical / horizontal shot), how the person is viewed (facing forward / sideways), face / upper body / full body, and standing / lying / sitting position. The composition analysis unit 110 outputs a composition analysis result 111 to the evaluation value setting unit 79 .

[0093] As an example, as shown in Figures 20 and 21, the evaluation value setting unit 79 compares the composition analysis result 111C of the candidate image 56C with the composition analysis result 111 of the remaining image 56. If the composition analysis result 111C and the composition analysis result 111 match as shown in Figure 20, the evaluation value setting unit 79 resets the evaluation value 58 of the image 56 to 0. The composition analysis result 111C and the composition analysis result 111 match when all of the multiple items match. Alternatively, the composition analysis result 111C and the composition analysis result 111 match when, for example, 80% or more of the multiple items match. In other words, if the number of items is 20, 16 or more items match.

[0094] On the other hand, as shown in FIG. 21 , if the composition analysis result 111C and the composition analysis result 111 differ, the evaluation value setting unit 79 adds an appropriate value, in this case, 10, to the evaluation value 58 of the image 56. The composition analysis result 111C and the composition analysis result 111 differ when all of the multiple items differ. Alternatively, this means that, for example, 80% or more of the multiple items differ. In other words, if the number of items is 20, this means that 16 or more items differ. Although not shown, if the composition analysis result 111C and the composition analysis result 111 do not meet the definitions of either a match or a difference, the evaluation value setting unit 79 leaves the evaluation value 58 of the image 56 unchanged. The composition analysis unit 110 and the evaluation value setting unit 79 repeat these processes until the set number of candidate images 56C for each group GK are stored in the layout unit 80.

[0095] In this way, in the second embodiment, the evaluation value 58 is derived according to the composition of the image 56. Then, the evaluation value setting unit 79 sets the evaluation value 58 of the image 56 whose composition differs from that of the candidate image 56C higher than the evaluation value 58 of the image 56 whose composition matches that of the candidate image 56C. This makes it easier for the image 56 whose composition differs from that of the candidate image 56C to be selected as the candidate image 56C for creating the electronic album 60. Therefore, it is possible to avoid creating a monotonous electronic album 60 in which images 56 with roughly the same composition are lined up. In other words, it is possible to create an electronic album 60 with a wide variety of compositions.

[0096] A user U with relatively high photography skills, such as a professional photographer, takes composition into consideration more when taking images 56 than an average user U. For this reason, the second embodiment, which can create an electronic album 60 that clearly reflects the user's preference for composition, can be said to be significantly effective for a user U with relatively high photography skills.

[0097] Note that if the composition analysis result 111C and the composition analysis result 111 match, instead of resetting the evaluation value 58 of the image 56 to 0, an appropriate value is subtracted from the evaluation value 58 of the image 56. If the composition analysis result 111C and the composition analysis result 111 differ, an appropriate value is added to the evaluation value 58 of the image 56, or the evaluation value 58 of the image 56 is left at its original value. In this way, the evaluation value 58 of the image 56 whose composition differs from that of the candidate image 56C may be set higher than the evaluation value 58 of the image 56 whose composition matches that of the candidate image 56C. Furthermore, if the composition analysis result 111C and the composition analysis result 111 differ, the evaluation value 58 of the image 56 whose composition differs from that of the candidate image 56C may be set higher than the evaluation value 58 of the image 56 whose composition matches that of the candidate image 56C by resetting the evaluation value 58 of the image 56 to 100.

[0098] [Third Embodiment] As shown in FIG. 22 as an example, the album creation unit of the third embodiment functions as a grouped image extraction unit 115 in addition to the processing units 75 to 80 of the first embodiment (only the evaluation value setting unit 79 is shown in FIG. 22 ). The grouped image extraction unit 115 is disposed after the grouping unit 76. The grouped image extraction unit 115 receives the grouping result 91 from the grouping unit 76. The grouped image extraction unit 115 also receives a grouped image extraction model 116. The grouped image extraction model 116 is stored in the storage 20B. Like the evaluation value derivation model 37, the grouped image extraction model 116 is a trained model constructed using a convolutional neural network or the like. The grouped image extraction model 116 is trained using training data created by an expert such as a professional art director.

[0099] The group image extraction unit 115 extracts a group image 56G from the multiple images 56 of each group GK using the group image extraction model 116. More specifically, as shown in FIG. 23 , the group image extraction unit 115 inputs all of the images 56 belonging to one group GK to the group image extraction model 116, and causes the group image extraction model 116 to output a group image extraction result 117. The group image extraction result 117 includes the extracted group image 56G. The group image extraction unit 115 outputs the group image extraction result 117 to the evaluation value setting unit 79.

[0100] A group of images 56G conveys a message to the viewer through a combination of multiple images 56. FIG. 23 illustrates an example in which four images 56 corresponding to an introduction, development, twist, and conclusion (introduction, development, twist, and conclusion), i.e., an introduction image 56I, a development image 56D, a twist image 56T, and a conclusion image 56CC, are extracted as the group of images 56G. The introduction image 56I, the development image 56D, the twist image 56T, and the conclusion image 56CC are examples of "images belonging to a group of images" according to the technology of the present disclosure. Note that, instead of an introduction, development, twist, and conclusion, three images 56 corresponding to an introduction, development, and climax may be extracted as the group of images 56G.

[0101] As an example, as shown in FIG. 24 , the evaluation value setting unit 79 resets the evaluation value 58 of the image 56 belonging to the group of images 56G (the original image 56I is exemplified in FIG. 24 ) to 100. By doing so, the evaluation value setting unit 79 sets the evaluation value 58 of the image 56 belonging to the group of images 56G higher than the evaluation values ​​58 of the other images 56. Even if the image 56 belonging to the group of images 56G is determined to be a similar image 56S or a dissimilar image 56DS, the evaluation value setting unit 79 maintains the evaluation value 58 of only the image 56 belonging to the group of images 56G at 100. This makes it easier for the image 56 belonging to the group of images 56G to be selected as a candidate image 56C for creating the electronic album 60. Therefore, it is possible to create an electronic album 60 with a storyline.

[0102] In this case, as shown in FIG. 25 as an example, the layout unit 80 arranges the initiation image 56I on the top of the left page of the spread page 120 of the electronic album 60 and the completion image 56D below. The layout unit 80 also arranges the transfer image 56T on the top of the right page of the spread page 120 and the final image 56CC below. The spread page 120 is an example of a "unit area of ​​an electronic album" according to the technology of the present disclosure. In this way, the group image 56G is an image 56 to be arranged in a specified order in the unit area of ​​the electronic album 60. Note that FIG. 25 is an example when the electronic album 60 is bound on the left side. When the electronic album 60 is bound on the right side, the initiation image 56I and the completion image 56D are arranged on the right side of the spread page 120, and the transfer image 56T and the final image 56CC are arranged on the left side of the spread page 120, in the opposite manner to this example.

[0103] Instead of resetting the evaluation value 58 of the image 56 belonging to the group of images 56G to 100, an appropriate value may be added to the evaluation value 58 of the image 56 belonging to the group of images 56G. Alternatively, an appropriate value may be subtracted from the evaluation value 58 of the image 56 other than the image 56 belonging to the group of images 56G.

[0104] A group image 56G may be extracted from a plurality of selected candidate images 56C, instead of a plurality of images 56. Also, instead of the images 56, a plurality of pages may be arranged in a sequence corresponding to an introduction, development, twist, conclusion, or the like.

[0105] In addition to or instead of the exemplary embodiment in which the electronic album 60 is created in response to a creation instruction from the user U, the album creating unit 48 may periodically and automatically create the electronic album 60 without waiting for a creation instruction from the user U. Also, the setting ranges 85A to 85C may be configured so that the user U can change the settings.

[0106] The electronic album 60 is not limited to an arrangement of a plurality of images 56 in an orderly fashion without overlapping. It may be a so-called collage-style electronic album 60 in which overlapping is permitted and a plurality of images 56 are arranged at random angles. It may also be an electronic album 60 that makes full use of three-dimensional computer graphics.

[0107] The hardware configuration of the computer that makes up the image management server 12 can be modified in various ways. For example, the image management server 12 can be configured with multiple computers separated as hardware to improve processing power and reliability. For example, the functions of the request reception unit 45 and evaluation value derivation unit 46 and the functions of the RW control unit 47, album creation unit 48, and distribution control unit 49 can be distributed and performed by two computers. In this case, the image management server 12 is configured with two computers. In addition, all or part of the functions of the image management server 12 may be performed by the user terminal 10.

[0108] In this way, the hardware configuration of the computers of the user terminal 10 and the image management server 12 can be changed as appropriate depending on the required performance, such as processing power, safety, reliability, etc. Furthermore, not only the hardware, but also APs such as the image AP 30 and the operating program 35 can be duplicated or stored in multiple storage devices in order to ensure safety and reliability.

[0109] In each of the above embodiments, the hardware structure of the processing units that execute various processes, such as the browser control unit 32, the request receiving unit 45, the evaluation value derivation unit 46, the RW control unit 47, the album creation unit 48, the delivery control unit 49, the acquisition unit 75, the grouping unit 76, the selection unit 77, the similarity derivation unit 78, the evaluation value setting unit 79, the layout unit 80, the composition analysis unit 110, and the group image extraction unit 115, can be any of the various processors shown below. The various processors include CPUs 22A and 22B, which are general-purpose processors that execute software (image AP 30 and operating program 35) and function as various processing units, as well as programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and / or dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed specifically for executing specific processing.

[0110] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0111] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0112] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0113] From the above description, the technology described in the following supplementary paragraphs can be understood.

[0114] [Supplementary Item 1] An electronic album creation device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, comprising a processor, wherein the processor derives similarities between the candidate images and the remaining images, and sets the evaluation values ​​of similar images whose similarities are within a first set range and the evaluation values ​​of dissimilar images whose similarities are within a second set range lower than the evaluation values ​​of the images whose similarities are outside the first set range and outside the second set range. [Supplementary Item 2] The electronic album creation device according to Supplementary Item 1, in which the evaluation values ​​are derived according to the image quality of the images. [Supplementary Item 3] The electronic album creation device according to Supplementary Item 1 or Supplementary Item 2, in which the evaluation values ​​are derived according to the composition of the images. [Supplementary Item 4] The electronic album creation device according to Supplementary Item 3, in which the processor sets the evaluation values ​​of the images whose composition differs from that of the candidate images higher than the evaluation values ​​of the images whose composition matches that of the candidate images. [Supplementary Item 5] The electronic album creation device according to any one of Supplementary Items 1 to 4, wherein the processor divides the images into a plurality of groups, and selects the candidate images for creating the electronic album for each group. [Supplementary Item 6] The electronic album creation device according to Supplementary Item 5, wherein the processor divides the images into groups based on a date and time of shooting. [Supplementary Item 7] The electronic album creation device according to Supplementary Item 5 or 6, wherein the processor divides the images into groups based on a similarity between the images. [Supplementary Item 8] The electronic album creation device according to any one of Supplementary Items 1 to 7, wherein the processor excludes the images whose evaluation value is outside a third set range from the candidate images. [Supplementary Item 9] The electronic album creation device according to any one of Supplementary Items 1 to 8, wherein the processor extracts, from the plurality of images, a group of images to be arranged in a specified order in a unit area of ​​the electronic album, and sets the evaluation value of an image belonging to the group of images higher than the evaluation values ​​of other images.[Supplementary Item 10] The electronic album creation device described in any one of Supplementary Items 1 to 9, wherein the processor derives the distance in feature vector space between a feature vector representing the features of the candidate image and a feature vector representing the features of the remaining images as the similarity, treats the remaining images whose similarity is less than a first threshold as the similar images, and treats the remaining images whose similarity is equal to or greater than a second threshold and is greater than the first threshold as the dissimilar images.

[0115] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs, but also to storage media that non-temporarily store programs, and computer program products that include programs.

[0116] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0117] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0118] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An electronic album creation device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, the electronic album creation device comprising a processor that derives the similarity between the candidate images and the remaining images, and sets the evaluation value of similar images whose similarity is within a first set range and the evaluation value of dissimilar images whose similarity is within a second set range lower than the evaluation values ​​of images whose similarity is outside the first set range and outside the second set range.

2. The electronic album creating device according to claim 1, wherein the evaluation value is derived according to the image quality of the image.

3. The electronic album creating device according to claim 1, wherein the evaluation value is derived according to the composition of the image.

4. An electronic album creation device according to claim 3, wherein the processor sets the evaluation value of the image whose composition differs from that of the candidate image higher than the evaluation value of the image whose composition matches that of the candidate image.

5. The electronic album creating device according to claim 1, wherein said processor divides said images into a plurality of groups and selects said candidate images for creating said electronic album for each group.

6. The electronic album creating device according to claim 5, wherein the processor performs grouping based on the date and time of photography.

7. The electronic album creating device according to claim 5, wherein said processor performs grouping based on the similarity between said images.

8. The electronic album creating device according to claim 1, wherein said processor excludes from said candidate images said images whose evaluation values ​​are outside a third set range.

9. The electronic album creation device according to claim 1, wherein the processor extracts a group of images to be arranged in a specified order in a unit area of ​​the electronic album from the plurality of images, and sets the evaluation value of the images belonging to the group of images higher than the evaluation values ​​of other images.

10. The electronic album creation device according to claim 1, wherein the processor derives the similarity as the distance in the space of feature vectors between a feature vector representing the characteristics of the candidate image and a feature vector representing the characteristics of the remaining images, treats the remaining images whose similarity is less than a first threshold as the similar images, and treats the remaining images whose similarity is equal to or greater than a second threshold and is greater than the first threshold as the dissimilar images.

11. A method for operating an electronic album creating device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, the method comprising: deriving the similarity between the candidate images and the remaining images; and setting the evaluation value of similar images whose similarity is within a first set range and the evaluation value of dissimilar images whose similarity is within a second set range lower than the evaluation values ​​of images whose similarity is outside the first set range and outside the second set range.

12. An operating program for an electronic album creation device that preferentially selects images with higher evaluation values ​​as candidate images for creating an electronic album, the operating program causing a computer to execute processes including: deriving the similarity between the candidate images and the remaining images; and setting the evaluation value of similar images whose similarity is within a first set range and the evaluation value of dissimilar images whose similarity is within a second set range lower than the similarity of images whose similarity is outside the first set range and outside the second set range.

Images