Program, image processing device, and image processing method

Abstract

To reflect the details of a result of editing of common spreads performed in a certain user's album selectively according to the user.SOLUTION: An embodiment of the present invention is a program for causing a computer to function as: editing means that edits the layouts of a plurality of albums having an attribute of common spreads common to the albums and an attribute of individual spreads in each album as attributes of spreads; first determination means that determines, in a predetermined album on which the editing is performed, if the edited spread is the common spread; and second determination means that, when a result of determination made by the first determination means is true, for each album of the plurality of albums excluding the predetermined album, determines if the details of a result of editing on the predetermined album is reflected on the album.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present invention relates to a technique for creating an album. [Background technology]

[0002] The number of photos taken by users is rapidly increasing due to the widespread use of digital cameras, the widespread use of smart devices, and the improvement of the camera performance of smart devices. Patent Document 1 proposes a method for multiple users to jointly create an album.

[0003] It is also known that when creating multiple albums, album data is automatically generated that includes a spread created according to a general standard common to multiple albums (referred to as a common spread) and a spread created according to individual standards specific to each album (referred to as an individual spread). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-66328 Summary of the Invention [Problem to be solved by the invention]

[0005] After creating albums for multiple users, if a common spread is edited in one user's album, the edited content is usually reflected in the albums of the other users. However, it is necessary to consider the case where some users do not want the edited content reflected in their own albums. In this case, it is not appropriate to uniformly reflect the edited content of the common spread in one user's album in the albums of all users except that one user.

[0006] In view of the above problem, one embodiment of the present invention aims to selectively reflect the contents of the editing results depending on the user when a common spread editing is performed in a user's album. [Means for solving the problem]

[0007] In one embodiment of the present invention, a computer is configured to read a common spread that is common to each album. It is said that Attributes and each album Not common in Individual spreads It is said that and editing means for editing the layout of a plurality of albums having the attribute as a spread attribute, and for a predetermined album in which editing has been performed, editing means for editing the layout of the edited spread as a common spread. It is said that a first determination means for determining whether the album is an attribute of the predetermined album; and if the determination result of the first determination means is true, reflecting the content of the editing result for the predetermined album in each of the albums excluding the predetermined album from among the plurality of albums. or not A second determination means for determining a reflecting means for reflecting the content of the editing result of the predetermined album to an album determined to reflect the content of the editing result of the predetermined album among the albums excluding the predetermined album from the plurality of albums; Function as Control is performed so that the editing results in the predetermined album are reflected in the first album, based on the fact that by reflecting the content of the editing results in the predetermined album in a first album among the plurality of albums that is not the predetermined album, the score based on the main subject corresponding to the first album becomes higher, and control is performed so that the editing results in the predetermined album are not reflected in the second album, based on the fact that by reflecting the content of the editing results in the predetermined album in a second album among the plurality of albums that is not the predetermined album, the score based on the main subject corresponding to the second album becomes lower. This is a program characterized by: [Effects of the Invention]

[0008] According to one embodiment of the present invention, when a common spread is edited in a user's album, the contents of the edited result can be selectively reflected depending on the user. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of hardware capable of executing an application according to the present invention. [Figure 2] FIG. 2 is a software block diagram of an album creation application. [Figure 3] FIG. 2 is a diagram illustrating an image layout. [Figure 4] FIG. 10 is a diagram of a display screen displayed by an application. [Figure 5] FIG. 10 is a process flow diagram of automatic layout. [Figure 6] FIG. 10 is a diagram illustrating image group division. [Figure 7] FIG. 1 is a diagram illustrating scene classification. [Figure 8] FIG. 10 is a diagram illustrating image selection. [Figure 9] FIG. 1 is a block diagram of template selection. [Figure 10] FIG. 10 is a process flow diagram of template selection. [Figure 11] FIG. 2 is a diagram illustrating an image layout. [Figure 12] FIG. 1 is a block diagram of double-page binding. [Figure 13] FIG. 10 is a process flow diagram for combining spreads. [Figure 14] FIG. 10 is a software block diagram of an album editing application. [Figure 15A] FIG. 10 is a diagram illustrating an example of an album editing screen. [Figure 15B] FIG. 10 is a diagram illustrating an example of an album editing screen. [Figure 16] FIG. 10 is a diagram showing an example of adding a double-page spread. [Figure 17] FIG. 2 is a block diagram showing details of an album editing unit. [Figure 18] FIG. 10 is a diagram showing a spread addition UI. [Figure 19] FIG. 10 is a software block diagram of an album editing application. [Figure 20] FIG. 10 is a flow diagram of a process for adding a double-page spread. [Figure 21] FIG. 10 is a flowchart showing a process for reflecting the results of editing a predetermined album in other albums. [Figure 22] FIG. 10 is a software block diagram of an album editing application. [Figure 23] FIG. 10 is a flowchart showing a process for reflecting the results of editing a predetermined album in other albums. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] (Embodiment 1) In this embodiment, an application for creating an album (hereinafter referred to as "application" or simply "app") is run on an image processing device, and album data in which multiple photographic images are automatically laid out is generated on the application. This album data can be printed to create multiple albums as printed materials. In this embodiment, a spread common to at least two albums and individual spreads for each album are created. Each album has individual spreads with different configurations for each subject (referred to as "main subject") that is to be particularly emphasized in the layout, for example. The total number of spreads constituting each album (referred to as the total number of spreads) is the same for all albums, as specified by the user operating the app. However, the respective numbers of common spreads and individual spreads for each album vary depending on the image group and settings. The spreads and the images arranged within them are arranged in chronological order of their capture.

[0012] By using individual spreads, the multiple albums created will be albums that are partially specialized for each main subject. In other words, it is assumed that the multiple albums created in this embodiment will not be completely identical albums in which the same images are arranged in the same layout on all spreads. As a result, some of the multiple albums created may end up being completely identical albums.

[0013] Furthermore, in this embodiment, the user can edit the album data of an album created to include a common spread and an individual spread. In the following explanation, the process is broadly divided into a process for automatically generating album data and a process for a user to edit the generated album data. The process for a user to edit the album data is also performed automatically. First, the configuration of the process for automatically generating album data will be explained. Then, the process for a user to edit the generated album data will be explained.

[0014] <<Album data generation process>> <About the hardware configuration of the image processing device> The hardware configuration of the image processing device in this embodiment will be described below with reference to Fig. 1. Note that, although the following describes a case where the image processing device is an information processing device (PC), other devices such as a smartphone may also be used as the image processing device.

[0015] 1, the image processing device 100 includes a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display 105, a keyboard 106, a mouse 107, and a data communication device 108. These components are connected by a data bus 109, and can transmit and receive data between them.

[0016] The CPU 101 controls the entire image processing device 100. The CPU 101 also executes the image processing method described in this embodiment according to a program. Although the image processing device in FIG. 1 has one CPU, the image processing device may have multiple CPUs.

[0017] The ROM 102 stores programs executed by the CPU 101. The RAM 103 provides memory for temporarily storing various information when the CPU 101 executes the programs. The HDD 104 stores databases and the like that hold image files or processing results such as image analysis, and in this embodiment, the HDD 104 stores an application program for creating an album. This application program is sometimes called an album creation application, and will be described later using FIG. 2 and the like.

[0018] The display 105 is a device for displaying the user interface (hereinafter referred to as UI) of this embodiment or the image layout results and presenting them to the user. The display 105 may have a touch sensor function. The keyboard 106 is one of the input devices included in the image processing device 100, and is used, for example, to input predetermined information on the GUI displayed on the display 105. In this embodiment, the user inputs the number of double-page spreads in the album via the keyboard 106. The mouse 107 is one of the input devices included in the image processing device 100, and is used, for example, to click and press buttons on the GUI displayed on the display 105.

[0019] The data communication device 108 is a device for communicating with an external device such as a printer or a server. For example, album data resulting from automatic layout is transmitted via the data communication device 108 to a printer or a server connected to the image processing device 100. The data bus 109 connects the above-mentioned components with the CPU 101. The above is the content of the hardware configuration of the image processing device in this embodiment.

[0020] <Software configuration of image processing device> The software configuration of the image processing device in this embodiment, in other words, the functional configuration realized by an album creation application installed in the image processing device, will be described below with reference to Fig. 2. The album creation application is launched when the user double-clicks the application icon displayed on the display 105 using the mouse 107. The album creation application has various functions, but the following description will mainly focus on the automatic layout function provided by the automatic layout processing unit 202.

[0021] As shown in Fig. 2, this application has an album creation condition setting unit 201, an automatic layout processing unit 202, an album display unit 214, and an album information storage unit 215. The "automatic layout function" is a function that classifies and selects images of photographs taken based on their content and attributes, and then lays them out to generate album data. The generated album data is displayed on the display 105.

[0022] The album creation condition setting unit 201 sets album creation conditions in the automatic layout processing unit 202 according to the user's mouse operation. In this embodiment, the album creation conditions include the image group to be used in the album, the total number of spreads per album, the commercial materials for the album, the number of albums, and the main subject for each album. The image group may be set using, for example, accompanying information or attribute information of individual image data, such as the date and time of shooting, or may be set based on the structure of the file system in which the image data is stored, such as a device or directory designation. This image group may be a storage destination for storing image data from each user who may be each main subject, or a storage destination for storing image data collected by a specific user. Note that a "spread" corresponds, for example, to a single display window in a display, or to a pair of adjacent pages (i.e., two pages) that a user can view at once when a book is opened. Note that the two pages in a spread may be printed on different sheets of paper and bound adjacently to form a spread, or may be printed on a single sheet of paper and folded in the middle to form a spread. Here, the explanation will be given using the concept of a spread, but pages can also be used. The main subject is a subject that is particularly emphasized in the layout of the album. Examples include a specific person, a category of objects such as a dog, a cat, a flower, or food, or the color of an object or the entire image. For the sake of explanation, a person will be used as the main subject here.

[0023] Here, the album created in this embodiment will be described with reference to Fig. 3. In this embodiment, a plurality of different variations of albums are created based on the number of albums and the main subject of each album set by the album creation condition setting unit 201. In the example of Fig. 3, album 1, album 2, and album 3 are created as a plurality of different variations of albums.

[0024] Each album is made up of a "common spread" in which the same images are laid out in the same arrangement across multiple different variations of albums, and "individual spreads" in which images or arrangements are different from those in other albums. In the example of Figure 3, albums 1 to 3 each include a common spread 302. Meanwhile, album 1 includes an individual spread 303, album 2 includes an individual spread 304, and album 3 includes an individual spread 305.

[0025] Image 301 is an image arranged on a spread, and text in image 301 represents the main subject appearing in the image. As shown in FIG. 3, common spread 302 includes images showing multiple main subjects. Meanwhile, individual spread 303 includes an image showing only main subject A, and so on, with individual spreads 303-305 each including an image showing only a specific main subject. In this way, when multiple albums are created in this embodiment, the multiple albums each include spreads created using different standards. In other words, because the albums are created using different standards, the resulting spreads are also different.

[0026] The automatic layout processing unit 202 has an image acquisition unit 203, an image analysis unit 204, a main subject setting unit 205, a scene division unit 206, a scene classification unit 207, an image scoring unit 208, a spread creation parameter generation unit 209, and a spread creation unit 210. It also has a spread combination unit 211, a layout information output unit 212, and an album information output unit 213.

[0027] The image acquisition unit 203 acquires a group of images that satisfy the album creation conditions set by the album creation condition setting unit 201 from the images stored in the HDD 104. The image group here refers to a group of images that will be used as layout candidates when creating an album. For example, if January 1, XX year to December 31, XX year are specified as the shooting date and time, all images taken between January 1, XX year and December 31, XX year correspond to the group of layout candidate images. Examples of images stored in the HDD 104 include still images and clipped images clipped from videos. The still images or clipped images are acquired from an imaging device such as a digital camera or a smart device. The imaging device may be included in the image processing device 100 or may be included in an external device of the image processing device 100. If the imaging device is an external device, the image acquisition unit 203 acquires the images via the data communication device 108. The still images or clipped images may also be images acquired from an external network or server via the data communication device 108. Examples of images acquired from a network or server include social networking service images (hereinafter referred to as SNS images). By executing an OS program, CPU 101 analyzes data accompanying image data for each image and determines the storage source from which the image was acquired. However, by acquiring images from an SNS via an application, the acquisition destination of the images may be managed within the application. The images acquired by image acquisition unit 203 are not limited to those described above, and may be other types of images.

[0028] The image analysis unit 204 analyzes the image data acquired by the image acquisition unit 203. In this embodiment, the image analysis unit 204 derives image features and performs object determination within the image, face detection, facial expression recognition of the detected face, and personal recognition of the detected face. Furthermore, the image analysis unit 204 references data (e.g., Exif information) attached to the image data acquired from the HDD 104 to acquire information on the shooting date and time. Note that the information obtained as a result of the image analysis unit 204 analyzing the image data is referred to as "analysis information."

[0029] Main subject setting section 205 sets a main subject to be emphasized for each album. In this embodiment, main subject setting section 205 acquires information on a main subject to be emphasized for each album sent from album creation condition setting section 201.

[0030] The image scoring unit 208 assigns a score to each image based on the album creation conditions set by the album creation condition setting unit 201 and the analysis information derived by the image analysis unit 204. In this embodiment, each image is assigned a score so that images suitable for the layout receive a high score. When assigning scores so that images suitable for the layout receive a high score, image analysis information, image classification results, image information, album creation conditions, etc. are used. However, other information may be used additionally or alternatively. Examples of images suitable for the layout include images with high aesthetic appeal, such as high contrast or sharp edges, or images that prominently feature a subject (so-called main subject), such as a person, animal, or building. Furthermore, the image scoring unit 208 assigns a score for each main subject set by the main subject setting unit 205. For example, an image that does not include the set main subject may receive a low score even if it is very aesthetically pleasing, while an image that includes a large main subject may receive a high score even if it is taken in poor conditions, such as backlighting.

[0031] The scene division unit 206 divides the group of images acquired by the image acquisition unit 203 into scenes using the analysis information derived by the image analysis unit 204. Here, a "scene" refers to a collection of images (sub-image group) grouped according to the analysis information. For example, with respect to two images taken at different times, the scene division unit 206 determines that images whose difference in shooting time is less than a predetermined threshold belong to the same scene, or determines that images that feature the same people or background belong to the same scene.

[0032] The scene classification unit 207 determines a category for each scene obtained by the scene division unit 206. Examples of scene categories include situations such as indoors or outdoors, or events such as travel or weddings when the images were taken.

[0033] The spread creation parameter generation unit 209 generates multiple parameters for creating a spread based on the main subject set by the main subject setting unit 205 and the analysis information derived by the image analysis unit 204. The multiple parameters include, for example, parameters for a common spread and parameters for individual spreads. The parameters here refer to setting values ​​that determine the creation criteria for a spread. Examples of such parameters include weights or thresholds that determine the priority of scores obtained by the image scoring unit 208 and are used when selecting images to be used in a spread. These setting values ​​select whether to prioritize images that show only a specific main subject or images that show two or more main subjects among images that show the main subject set by the main subject setting unit 205. By using multiple parameters, the creation criteria change depending on the spread for each album. The spread creation parameters also include a parameter that determines a score that takes into account the relationship between images within the spread when the image is placed on the spread.

[0034] Here, for convenience, the parameters used to calculate the score for a single image when selecting an image to be used in a spread are referred to as first parameters, and the parameters used to calculate a score taking into account the relationship when the image is placed in the spread are referred to as second parameters. In this case, the spread creation unit 210, which will be described later, first selects candidate images to be used in the spread according to the first parameters. Then, the spread creation unit 210 determines images and templates suitable for the spread from among the candidate images according to the second parameters. Here, for convenience, the parameters are divided into first parameters and second parameters, but parameters that take these into consideration comprehensively are collectively referred to as spread creation parameters.

[0035] The spread creation unit 210 creates spread data by selecting and arranging images to be used in the spread according to the parameters generated by the spread creation parameter generation unit 209. For example, the spread creation unit 210 selects candidate images according to the first parameter. As described above, the first parameter is a parameter that changes the weight for each set main subject, and is also a parameter that assigns weights according to a predetermined priority. As an example of the priority, a priority based on whether or not cropping is performed may be taken into consideration. For example, since the main subjects in a common spread are often multiple people, priority may be given to images of a size that does not require cropping so that the main subject is not omitted. Since the main subject in an individual spread is one person, it is conceivable that cropping will be performed to show a close-up of the main subject, so there is no need to lower the priority of images to be cropped.

[0036] Such priorities may be determined in advance or may be set by the user as appropriate. By preparing weights or thresholds according to priorities, the user can change the parameters by changing the priorities of various pieces of information. Furthermore, such priorities may be set as priorities that take into consideration various pieces of information in addition to the information described above. Examples of the various pieces of information include information about the scene of the image, information about the number of objects other than the main subject, the number of people other than the main subject, resolution, and background.

[0037] The spread creation unit 210 then applies a second parameter to the scores of the candidate images thus determined to create a spread that maximizes the total score of the images to be placed within the spread. The total score of the images to be placed within the spread is called the spread score. The second parameter is a parameter that takes into account the relationship within the spread, and can, for example, prioritize the use of images that are spaced a certain time apart within the same spread, or assign weights according to the slot size.

[0038] As described above, the spread creation parameters (first and second parameters) are different for common spreads and individual spreads, which allows image selection to be performed based on different spread criteria, resulting in different spreads being created for common spreads and individual spreads.

[0039] The spread combining unit 211 combines the spread data created by the spread creating unit 210. Specifically, the spread combining unit 211 determines the order of the spreads created by the spread creating unit 210, thereby creating the layout of the entire album.

[0040] The layout information output unit 212 outputs album data in accordance with the layout of the entire album created by the spread combining unit 211. The album data is, for example, image data in a bitmap format in which each image is arranged in accordance with a predetermined layout.

[0041] The album information output unit 213 outputs album information used to generate the album. The album information includes, for example, album configuration information and album generation information. The album configuration information is configuration information that configures an album, such as used image information, image layout information, template information, slot shape and size information, background color information, trimming information, text or stamp editing information, and common / individual page information. The album generation information is generation information used when generating album data, such as album creation conditions, image analysis information, score information, or scene information. Note that when an album is generated using this album generation information, the same album can be generated. The album information may include the album data itself. This information is output on an album-by-album basis.

[0042] The album display unit 214 performs display control to display the bitmap data output by the layout information output unit 212 on the display 105 .

[0043] The album information holding unit 215 receives and holds the album information output by the album information output unit 213. When the album is updated due to album editing or the like, the album information holding unit receives and holds the album information adjusted to the update. The album information here includes, in addition to the album configuration information and album generation information described above, album change information. Examples of the album change information may be the user operation history of album editing or the difference information changed by album editing. These pieces of information are held in units of albums.

[0044] When the program of the album creation application in the present embodiment is installed in the image processing apparatus 100, a startup icon of this application is displayed on the top screen (desktop) of the OS (operating system) operating on the image processing apparatus 100. When the user double-clicks the startup icon on the desktop displayed on the display 105 with the mouse 107, the program of the album creation application stored in the HDD 104 is loaded into the RAM 103. Then, the program loaded into the RAM 103 is executed by the CPU 101, and the album creation application is started. The above is the content regarding the software configuration of the image processing apparatus in the present embodiment. Note that the application may be in another form, for example, a web application that displays a screen or the like within a browser operating on the image processing apparatus 100.

[0045] <Regarding the GUI screen> Hereinafter, the GUI screen of the album creation application in the present embodiment will be described with reference to FIG. 4. FIG. 4 is a diagram showing a GUI screen 401 provided by the started album creation application and displayed on the display 105. The user can set the creation conditions of the album via the GUI screen 401.

[0046] The GUI screen 401 has a path box 402 and a folder selection button 403 as setting sections for the photographic images to be included in the album. The path box 402 is a box for indicating the storage location (path) in the HDD 104 of the group of images to be the subject of album creation. The folder selection button 403 is a button for selecting a folder containing the group of images to be the subject of album creation. When the user clicks the folder selection button 403 with the mouse 107, a tree made up of multiple folders is displayed. Then, when the user selects a folder containing the group of images to be the subject of album creation, the folder path of the selected folder is displayed in the path box 402.

[0047] The number of spreads box 404 is a box for specifying the total number of spreads in one album. The user can input a number directly into the number of spreads box 404 using the keyboard 106, or can input a number into the number of spreads box 404 from a list using the mouse 107. The number of albums box 410 is a box for specifying the number of albums to be created.

[0048] Main subject designation section 405 is an element for designating a main subject to be emphasized for each of the number of albums input in album number box 410. For example, the user can designate a main subject by using mouse 107 to drag and drop an image containing the main subject into main subject designation section 405. For example, person A, person B, etc., as described above in FIG. 3, are designated.

[0049] The spread photo number setting section 406 is a slider bar for setting the number of images to be placed on each spread of the album to be created. When the slider is moved to the "more" side, the number of images to be placed on each spread increases. On the other hand, when the slider is moved to the "fewer" side, the number of images to be placed on each spread decreases.

[0050] The product material specification section 407 is an element for setting the product material of the album to be created. Specific items related to the product material to be set include the size of the album or the type of paper used for the album. The cover type or the type of binding section may also be set.

[0051] The OK button 408 is a button for confirming the selected conditions as the album creation conditions. When the user clicks the OK button 408 with the mouse 107, the album creation conditions are confirmed and communicated to the automatic layout processing unit 202 via the album creation condition setting unit 201. More specifically, path information entered in the path box 402 is communicated to the image acquisition unit 203. Furthermore, the value of the total number of spreads entered in the spread number box 404 is communicated to the spread creation unit 210. The value of the number of albums entered in the album number box 410 and information about the main subject specified in the main subject specification unit 405 are communicated to the spread creation parameter generation unit 209.

[0052] A reset button 409 is a button for resetting each setting on the display screen. The above is the content of the GUI screen of the album creation application in this embodiment.

[0053] <About automatic layout processing> The automatic layout process in this embodiment will be described below with reference to Fig. 5(a). Fig. 5(a) is a flowchart of the process of executing the automatic layout of the album creation application according to this embodiment. The flowcharts shown in Figs. 5(a) to 5(c) are realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or RAM 103 and executing it.

[0054] In step S501, the album creation condition setting unit 201 sets album creation conditions. Note that hereinafter, "step S~" will be simply abbreviated as "S~". In this embodiment, the album creation conditions set include a group of images to be used in the album, the number of spreads, the number of albums, the main subject of each album, the number of images to be arranged on each spread, and the commercial materials for creating the album (see FIG. 4).

[0055] In S502, the image acquisition unit 203 reads images that satisfy the conditions for the image group to be used in the album set in S501 from the HDD 104. Then, the image acquisition unit 203 develops the read images in the RAM 103.

[0056] In S503, the image analysis unit 204 analyzes the image developed in S502 in the RAM 103. Here, the image analysis in this step will be described with reference to FIG.

[0057] In S50301, the image analysis unit 204 acquires information on the shooting date and time corresponding to the image data acquired by the image acquisition unit 203. In this embodiment, the information on the shooting date and time is acquired based on the Exif information attached to each image data.

[0058] In S50302, the image analysis unit 204 derives image quality features for the image data acquired by the image acquisition unit 203. Examples of image quality features include the amount of focus. An edge detection method can be used to determine the amount of focus, and a known Sobel filter can be used as the edge detection method. The Sobel filter detects the edges of the image, and the brightness gradient, i.e., the slope of the edge, is calculated by dividing the brightness difference between the start and end points of the edge by the distance between the start and end points. The average slope of the edges in the image is calculated, and an image with a larger average slope can be considered to be more in focus than an image with a smaller average slope. In this embodiment, multiple thresholds are set to measure the magnitude of the calculated average slope of the edges, and the amount of focus is determined by determining whether the calculated edge slope is equal to or greater than a threshold. Specifically, two different slope thresholds, a first slope threshold and a second slope threshold (where the first slope threshold is greater than the second slope threshold), are set, and the amount of focus is determined using three levels: true, false, or false. If the average gradient of the edges in the image is equal to or greater than the first threshold, the image is determined to have a good level of focus (denoted by a circle). If the average gradient of the edges in the image is less than the first threshold and equal to or greater than the second threshold, the image is determined to have an acceptable level of focus (denoted by a triangle). If the average gradient of the edges in the image is less than the second threshold, the image is determined to have an unacceptable level of focus (denoted by an x).

[0059] In S50303, the image analysis unit 204 performs object detection and classification of the detected object in the image data acquired by the image acquisition unit 203. In this embodiment, a face is detected as the object. Any known method can be used as the face detection method, such as AdaBoost, which creates a strong classifier from multiple weak classifiers. In this embodiment, a human face is detected using a strong classifier created by AdaBoost. In S50303, a face is detected and the top left and bottom right coordinates of the detected face area in the image are acquired. By obtaining these two coordinates, the position and size of the face can be determined. Furthermore, by simultaneously executing AdaBoost, which detects animals such as dogs or cats and food, as well as human faces, it is possible to detect human, animal, and food objects and simultaneously classify the objects in the image. Note that the objects to be detected are not limited to the above and may also be flowers, buildings, ornaments, etc. Furthermore, although the case where AdaBoost is used to detect (classify) objects has been described here, object detection (classification) may also be performed using a trained model such as a neural network.

[0060] In S50304, the image analysis unit 204 performs personal recognition on the face detected by the image analysis unit 204. First, the similarity between the extracted face image and the representative face image stored for each personal ID in the face dictionary database is derived. Then, the personal ID with the derived similarity equal to or greater than a predetermined threshold and having the highest similarity is determined as the ID corresponding to the extracted face image. In other words, the person corresponding to the personal ID with the highest similarity equal to or greater than a predetermined threshold is identified as the person in the extracted face image. The image analysis unit 204 obtains the similarity between each of the representative face images and the face image in the image to be processed. Note that if all of the similarities derived for each personal ID are less than the predetermined threshold, the person in the extracted face image is considered a new person, and a new personal ID is assigned and registered in the face dictionary database. The face dictionary database is stored, for example, in the HDD 104.

[0061] Returning to the explanation of Figure 5(a), in S504, the image analysis unit 204 determines whether the processes from S502 to S503 have been completed for all images in the image group that satisfy the conditions set in S501. If the determination result in this step is true, proceed to S505. On the other hand, if the determination result is false, return to S502.

[0062] In S505, the scene division unit 206 performs scene division on the group of images acquired by the image acquisition unit 203. Scene division refers to dividing the acquired group of images into multiple sub-groups of images using image information. Specifically, the image group is divided into multiple groups based on the time difference between the shooting dates and times of the images, using the shooting date and time information acquired in S50301 as image information. An example of actual division is as follows:

[0063] The image with the oldest (or newest) shooting time among the image group is first focused on, and the time difference between it and the next oldest (or newest) image is calculated, and it is determined whether the calculated time difference is equal to or greater than a predetermined threshold. This process is performed on all images, sequentially replacing the image of interest with one with a newer (or older) shooting time. Note that "division" in this embodiment means dividing the image group into two images with newer and older shooting times, with the difference between the two images as a boundary. In this embodiment, if the difference between the shooting time of the image of interest and the shooting time of the next oldest (or newest) image is 16 hours or more, the image group is divided so that these images belong to different sub-image groups. Note that the time difference threshold used for division is not limited to this. Figure 6 shows the results of dividing the image group using the above-mentioned scene division method.

[0064] In this embodiment, scenes are divided using information about the date and time of shooting. However, this information is not limited to this. For example, images may be divided using information about the shooting location so that images taken at nearby locations belong to the same scene. As another example, personal recognition results may be used. For example, in a yearbook, images of students belonging to a certain group (such as a class or club) may be registered in advance in an application, and a group of images containing the students in that group may be grouped together as a single scene (sub-image group). As another example, information other than image information may be used. Here, a method of grouping similar events into a single scene will be described. The name of the folder in which the images are saved or tag information added via a social networking service or the like is acquired as meta information associated with the images. For example, by using search terms such as "athletic meet" or "school trip," a group of images with meta information containing the search terms may be grouped into a single scene. Regarding search terms, the user may select from search terms pre-installed in the application via the album creation condition setting unit 201, or may enter the search terms in a text box.

[0065] In S506, the scene classification unit 207 performs scene classification for each of the sub-image groups obtained in S505. Note that, in the following, an example will be described in which each sub-image group (each scene) is classified into one of three categories: "travel," "daily life," and "ceremony," but the classification items are not limited to these.

[0066] First, multiple sub-image groups that have been previously determined to be classified as travel, everyday, or ceremonial are acquired, and shooting features are acquired for each of the acquired multiple sub-image groups. The shooting features acquired here include, for example, the shooting period, the number of images taken, and the number of people photographed. The shooting period is the time difference between the shooting time of the oldest image and the shooting time of the newest image among the images included in the sub-image group. The number of images taken is the number of images included in the sub-image group (i.e., the number of photos). The number of people photographed is the number of faces in images that contain faces, i.e., the number of faces included in one image. Then, for the multiple sub-image groups, the average and standard deviation of the shooting period, the average and standard deviation of the number of images taken, and the average and standard deviation of the number of people per image are calculated. In this embodiment, the number of faces per image is taken as the number of people per image.

[0067] FIG. 7 shows an example of the average and standard deviation of the shooting period (hours), the average and standard deviation of the number of images taken, and the average and standard deviation of the number of people per image, calculated for multiple sub-image groups. These calculated values ​​are pre-installed in the album creation application program. That is, at the design stage of the application, parameters are generated by learning using pre-collected images, and the generated parameters are pre-installed in the program. After the album creation application is launched, the average values ​​of the shooting period, the number of images taken, and the number of people per image are calculated for each sub-image group obtained as a result of dividing the image group specified by the user via the pass box 402 in S505. Then, for each scene, a score is calculated according to the following formula using the parameters pre-installed in the application (specifically, the average values ​​and standard deviations of the feature quantities of the shooting period, number of images taken, and number of people taken for each sub-image group). Score for each scene and feature = 50 - |10 × (average value for each scene and feature - feature for each sub-image group) / standard deviation for each scene and feature |···Equation (1) Average score per scene = (score of shooting period per scene + score of number of shots per scene + score of number of people shooting per scene) / number of feature items ··· Equation (2)

[0068] Using equation (1), a score is calculated for each scene and each feature of the target sub-image group. For example, for a travel scene, scores are calculated for the shooting period, number of photos taken, and number of people photographed. These scores are averaged using equation (2) to determine the score for the travel scene. Similarly, scores are obtained for other scenes such as everyday life and ceremonies. Using the method described above, the average score for travel scenes, the average score for everyday life scenes, and the average score for ceremony scenes are calculated for each sub-image group. Note that the number of feature items in equation (2) is three.

[0069] The scene with the highest score among the scores calculated for each scene in each sub-image group is classified as the scene of that sub-image group. If two or more scenes have the same score, the scenes are classified according to a predetermined priority. For example, in this embodiment, the order of priority is predetermined as follows: daily life > ceremony > travel, with daily life having the highest priority. Note that the order of priority is not limited to this, and the user may also be able to change the order of priority.

[0070] Sub-image groups (1) to (8) in FIG. 6 represent sub-image groups obtained by scene segmentation of an image group. For sub-image group (5) among these sub-image groups, assume that the shooting period was 36 hours, the number of images taken was 300, and the number of people photographed was 1.7. The average score for travel, the average score for daily life, and the average score for ceremony were 45.32, 18.38, and -29.92, respectively. In this case, the scenes in sub-image group (5) are classified as travel. The classified sub-image groups are managed by associating them with scene IDs to enable scene identification. While this embodiment describes a case in which the shooting period, the number of images taken, and the number of people photographed are used as feature quantities for photography, these are not the only possible features. For example, to distinguish between indoor and outdoor scenes, a flag indicating whether the sky is visible or not may be used as a feature quantity using image segmentation using machine learning. Alternatively, ceremonies may be further classified into more detailed categories. For example, when classifying a scene as either a wedding or a wedding reception, the results of personal recognition can be used to identify the two people who appear most frequently in all images as the bride and groom. Scenes with many images showing only the bride and groom can be classified as weddings, while scenes with many images showing people other than the bride and groom can be classified as wedding receptions. Furthermore, feature values ​​can be derived using machine learning. For example, scene classification can be performed by preparing a scene to be classified and a group of images representing that scene, and using a Convolution Neural Network (CNN) to learn images as input and output the scene classification results.

[0071] In S507, main subject setting section 205 sets a main subject for each album. In this embodiment, main subject setting section 205 acquires information on the main subject for each album sent from album creation condition setting section 201. In this embodiment, an example in which three types of albums are generally created is described, but for the sake of simplicity, an example in which main subject A is set in album 1 and main subject B is set in album 2 for two types of albums will be described here.

[0072] In S508, the image scoring unit 208 assigns a score to each image. This process is called image scoring. The "score" derived in this step is a parameter for evaluating suitability for use in an album. A score is assigned for each image data item based on the criteria described below, and is referenced during image selection, as described below. In this embodiment, in addition to the quality of image quality, a score is assigned for each main subject set in S507. First, a score is assigned based on the image quality feature derived in S50302. Here, a score of 1 is assigned for a focus level of ○ (good), 0.5 points for a focus level of △ (acceptable), and 0 points for a focus level of × (unacceptable). This allows for a higher score for an image that is in focus. Note that, although focus level is used as the image quality feature in this embodiment, other image quality features may be used. For example, image size, shooting information such as lens information used during shooting, or the compression format of the image input to the application may be used.

[0073] After assigning scores according to the image quality features, the image scoring unit 208 assigns a score for each set main subject. In other words, in this example, three types of scores are assigned to each image, specifically, scores based on three evaluation criteria: whether the image quality is good, whether main subject A is included, and whether main subject B is included. The face size derived in S50303 and the individual recognition results in S50304 can be used to assign scores for each main subject. If no main subject is included in the image, a score of 0 is assigned. If a main subject is included, the proportion of the face of the individual set as the main subject relative to the image size is used as the score for each main subject. While this embodiment assigns scores for each main subject using face size, factors other than face size may also be used. For example, the facial expression of the main subject may be determined, and points may be added if the person is smiling. Furthermore, if a non-human subject is set as the main subject, the results of object detection and classification in S50303 are used to similarly assign scores according to object size. As a result of the above processing, for example, a score for image quality, a score for main subject A, and a score for main subject B are calculated for the first image. For example, an image that shows main subject A but not main subject B will be given a score of 1 for image quality, 1 for main subject A, and 0 for main subject B.

[0074] In S509, the image scoring unit 208 determines whether the scoring in S508 has been completed for all images in the image group acquired by the image acquisition unit 203. If the determination result in this step is true, the process proceeds to S510, whereas if the determination result is false, the process returns to S508.

[0075] In S510, the spread creation parameter generation unit 209 receives album creation conditions from the album creation condition setting unit 201 and information on the main subject for each album from the main subject setting unit 205, and generates spread creation parameters. As described above, the spread creation parameters are setting values ​​used when selecting and laying out images that compose a spread. For example, when selecting images to be used in a spread based on the scores assigned to the images in S508, the spread creation parameter generation unit 209 generates weights that control which score is prioritized between the score according to image quality and the score for each main subject, or thresholds that serve as a criterion for image selection. The spread creation parameter generation unit 209 generates, as spread creation parameters, "common parameters" that are common to all albums and "individual parameters" that differ for each album. That is, the spread creation parameter generation unit 209 generates (the number of albums + 1) spread creation parameters. This makes it possible to create "common spreads" in which spreads with the same images and layout that compose the spread appear in multiple albums, and "individual spreads" in which the images and layout that compose the spread appear only in a specific album. Then, using these spread creation parameters (first parameters), the spread creation unit 210 calculates a score representing the image from the scores (three types of scores in this example) assigned to the image in S509. That is, scores equal to the number of spread creation parameters are calculated for each image. Images to be placed on the spread are then selected using the scores calculated in this way. For example, when creating a spread, the score obtained for a certain image X using the common parameters will differ from the score obtained using the individual parameters. Below, the spread creation parameters and an example of deriving the scores of images when creating a spread using the spread creation parameters will be described.

[0076] In this embodiment, the common parameter is set to a weight that allows each score assigned to an image to be evaluated equally. i The score that represents an image I when using I score S is expressed by the following equation (3).

[0077]

number

[0078] where: I score i is the score assigned to image I, and w i represents the weight for each score. In this embodiment, w i By setting the ratio to 1 / 3, the image quality score, the score of main subject A, and the score of main subject B are evaluated equally. This gives priority to images with good image quality and a high number of people in them, which gives them a higher score. Here, the image quality score reflects the score according to priority, such as whether or not the image has been cropped, as described above.

[0079] On the other hand, for the individual parameters, weights are set to prioritize images that include the main subject for each album. o The score that represents an image I when using I score I is expressed by the following equation (4).

[0080]

number

[0081] where: I score q is the quality score assigned to image I, I score o is the score of the main subject O assigned to image I, and w q is the weight for the image quality score, w o represents the weight for the main subject O. In this embodiment, w q =w o By setting ≠1, only images that include the main subject are evaluated, and the image to be prioritized is determined based on the image quality and the size of the main subject. In this embodiment, weights for scores are used as parameters for creating spreads, but spread parameters are not limited to this. I score S andI score I In both cases, a higher score is calculated when the main subject is captured, but this is not limited to this. For images that do not capture a main subject at all, such as landscapes, the score may be calculated based on a separate image quality score or image information. For example, for an image that is captured at a time that is close to a high-scoring image, the image quality score may be used as the representative score. This allows images that are likely to be related to images that capture a main subject to be given a high score. Furthermore, these relationships may be specified as weights to calculate the score.

[0082] In S511, the spread creation unit 210 creates spread data for the album using the parameters generated in S510. The spread creation unit 210 creates spread data so that the spread score is high. As described above, the spread score is the sum of the scores of each image in the spread. That is, it is the sum of the scores obtained by multiplying the scores of each image obtained according to the first parameter by the second parameter. As described above, the spread score is the sum of the results of multiplying the scores of each image by a weight corresponding to the size of the slot in which the image will be placed. In this example, candidate images are first determined, and then a layout including the slot and placement is performed. As a result, a spread with a high score as a spread is created. The spread creation process in this step will now be described with reference to FIGS. 2(b) and 5(c). FIG. 2(b) is a detailed block diagram of the spread creation unit 210. FIG. 5(c) is a detailed flowchart of the spread creation process in S511.

[0083] In S51101, the spread creation parameter setting unit 21001 reads one of the parameters generated in S510. In this embodiment, the individual parameters of album 1, the individual parameters of album 2, and the common parameters are read in this order.

[0084] In S51102, the spread candidate image acquisition unit 21002 acquires candidate images to be used for the spread from the entire group of images acquired by the image acquisition unit 203, based on the scores assigned in S508 and the spread creation parameters set in S51101. In this embodiment, based on the scores calculated in S508 and the spread creation parameters set in S51101, scores are assigned to each image again according to Equation (3) or Equation (4) as described above (image re-scoring). Images whose scores obtained as a result of the re-scoring are equal to or greater than a predetermined threshold are acquired as candidate images. That is, in the first S51102 in this example, each image is re-scoring when the individual parameters of Album 1 are applied. Images whose scores are equal to or greater than a predetermined threshold when the individual parameters of Album 1 are applied are acquired as candidate images. If there are no candidate images, the image with the highest score is designated as the candidate image. Note that, although candidate images are acquired from the entire group of images here, this is not a limitation. Images already selected in the image selection process described below may be excluded.

[0085] In S51103, the spread number setting unit 21003 determines whether the spread creation parameters set in the most recent S51101 are individual parameters. If the determination result in this step is true, the process proceeds to S51104, whereas if the determination result is false, the process proceeds to S51105.

[0086] If the answer is YES in S51103 (that is, if individual parameters were set in the most recent S51101), proceed to S51104. In S51104, the spread number setting unit 21003 sets the number of spreads (that is, the number of individual spreads corresponding to the individual parameters set in S51101) based on the scene division results performed in S505 and the candidate images acquired in S51102. In this embodiment, the number of spreads is set according to the following equations (5) and (6). Number of spreads per scene = ceil (number of candidate images in a scene ÷ maximum number of images per spread) Equation (5) Number of individual spreads = Σ Number of spreads per scene ··· Equation (6)

[0087] The number of spreads created based on the individual parameters is calculated using equations (5) and (6). The maximum number of images per spread is set based on user input via the spread photo number setting unit 406. For example, when scenes are divided as shown in FIG. 6, the number of spreads is determined based on the number of candidate images in each scene. Each scene always includes images acquired by the image acquisition unit 203, but does not necessarily include candidate images (candidate images acquired as a result of rescoring based on the individual parameters). Naturally, for scenes that do not include candidate images, the number of spreads is set to 0 in this process (processing according to the individual parameters). On the other hand, a scene that includes many images of the main subject corresponding to the individual parameters will be assigned an appropriate number of spreads. Note that if the number of individual spreads is equal to or greater than the total number of spreads set in S501, the following equation (7) is followed. Number of individual spreads = total number of spreads - 1 Equation (7)

[0088] Equation (7) allows spreads created with different parameters, namely individual parameters and common parameters, to be mixed in one album. This allows the number of individual spreads to be set according to the number of images that capture the main subject in high quality for each scene. Note that upper and lower limits may also be set for the number of individual spreads.

[0089] On the other hand, if NO in S51103 (that is, if the common parameters were set in the immediately preceding S51101), in S51105 the spread number setting unit 21003 sets the number of spreads according to the following formula (8). Number of common spreads = total number of spreads - MIN (total number of individual spreads for each album) Equation (8)

[0090] Formula (8) allows the common number of spreads to be set to match both the number of individual spreads and the total number of spreads. Formula (8) determines the common number of spreads to match the number of individual spreads in the album with the fewest number of individual spreads. Therefore, for other albums, adding the common number of spreads in formula (8) to the number of individual spreads in that album will result in the total number of spreads being exceeded. In this case, adjustments will be made during the process of combining spreads, which will be described later.

[0091] Here, the case where the number of spreads is set according to the number of candidate images has been described, but the method for setting the number of spreads is not limited to this. For example, the spread creation parameter generation unit 209 may set the number of spreads in the parameters, and the spread number setting unit 21003 may read the number of spreads set in the parameters. Furthermore, for example, the spread creation parameter generation unit 209 may set the number of spreads pre-installed in the program of the album creation application, or may calculate the number of spreads based on the ratio between individual spreads and common spreads specified by the user. The number of individual spreads and the number of common spreads have both been described above. Below, we will continue to explain the processing for each parameter set in S51101.

[0092] In S51106, the spread allocation unit 21004 divides the candidate images acquired in S51102 (i.e., candidate images corresponding to the spread generation parameters to be processed) into sub-image groups equal to the number of spreads set in S51104 or S51105 and allocates them. In this embodiment, the candidate images are arranged in the order of their shooting times acquired in S50301, and are divided at locations where the time difference between shooting times of adjacent images is large. This process is repeated until the image is divided into the number of spreads set in S51104 or S51105. In other words, division is performed (number of spreads - 1) times. This makes it possible to create an album in which images are arranged in order of their shooting time. In other words, division is performed as described in FIG. 6, but differs from the example described in FIG. 6 in that the image groups to be divided are candidate images corresponding to the spread generation parameters to be processed acquired in S51102. Note that the process of S51106 may be performed on a scene-by-scene basis. For example, the spread allocation unit 21004 arranges and divides the images to which a scene "travel" is assigned among the candidate images in candidate time order. Next, the spread allocation unit 21004 arranges and divides the images to which a scene "daily life" is assigned in candidate time order. As described above, the spread allocation unit 21004 may perform the division process while switching scenes.

[0093] In S51107, the spread image selection unit 21005 selects images to be used in the layout from the group of candidate images assigned to the spread to be processed in S51106. At this point, the number of candidate images assigned to the spread to be processed may be greater than the maximum number of images per spread. Of course, there may be cases where the number is the same or less. In other words, the images to be used in the layout have not yet been determined, so in S51107, processing is performed to select images to be used in the layout. Below, using Figure 8, an example will be described in which four images are selected from the group of candidate images assigned to a certain spread.

[0094] Figure 8(a) shows the time difference between the shooting dates and times of the first and last images in the candidate image group assigned to a spread, in other words, the shooting period of the candidate image group. Figure 8(b) will be used to explain how to select the first image when selecting four images. Of all the images captured during the shooting period of the candidate image group shown in Figure 8(b), the image with the highest rescoring score in S51102 is selected as the first image. Then, for the second and subsequent selections, the shooting period of the candidate image group is subdivided and images are selected so that the selected images are not concentrated in a single part of the shooting period of the candidate image group. First, as shown in Figure 8(c), the shooting period of the candidate image group is divided into two shooting periods (periods). In other words, the shooting period is divided into two equal shooting periods (groups). Next, as shown in Figure 8(d), the image with the highest score from among the images belonging to the shooting period (the section indicated by the solid line) in which the first image was not selected is selected as the second image. Next, as shown in Fig. 8(e), each shooting section in Fig. 8(d) is divided into two equal parts. In Fig. 8(f), the image with the highest score among the images shot in the shooting sections indicated by the solid lines in Fig. 8(e), i.e., the two shooting sections where the first and second images are not selected (images corresponding to the two shooting sections), is selected as the third image.

[0095] Next, a case where no image exists in the shooting section within the range for image selection and an image cannot be selected will be described using the selection of the fourth image as an example. In this embodiment, the shooting period is divided according to time regardless of the number of images. Therefore, there are cases where no image exists in the shooting section obtained by dividing. For example, as shown in FIG. 8(g), suppose that a fourth image is to be selected from a shooting section in which no image has yet been selected (shooting section indicated by diagonal lines), but no image exists in this shooting section. In such a case, as shown in FIG. 8(h), each shooting section in which images have already been selected is divided into two. Next, as shown in FIG. 8(i), the image with the highest score is selected as the fourth image from among the images shot in the shooting section in which images 1 to 3 indicated by solid lines in the figure have not yet been selected. Note that images are selected in the same manner as described above even when the number of images to be selected increases or decreases from four.

[0096] Returning to the explanation of Fig. 5(c), in S51108, the template setting unit 21006 selectively sets a plurality of templates to be used for layout from among the templates stored in the HDD 104. In this step, a template with a design that satisfies the album creation conditions set in S501 is set.

[0097] In S51109, the spread layout unit 21007 determines the image layout for the spread to be processed. Specifically, the spread layout unit 21007 determines a template suitable for laying out the images selected in S51107 from among the multiple templates set in S51108. Here, the method for determining a template in this step will be described with reference to Fig. 9. Fig. 9 is a functional block diagram related to template determination.

[0098] The selected image information acquisition unit 901 acquires the number of images selected by the spread image selection unit 21005 and image information related to the images. The acquired image information includes the image width and height, shooting date and time information, and scores calculated by the image scoring unit 208. The template selection unit 902 selects a template having a number of slots matching the number of selected images from among the templates set by the template setting unit 21006. The selected image sorting unit 903 sorts the selected images in order of shooting time. The template selection unit 904 selects, from the templates selected by the template selection unit 902, a template whose main slot for arranging the main images matches the order of the main images arranged in order of shooting time. Here, the "main image" is the image with the highest score obtained as a result of re-scoring in S51102 among the multiple images selected by the spread image selection unit 21005. On the other hand, images among the multiple images that are not main images are referred to as "sub-images." When selecting a template, in addition to the order, the selection condition is that the aspect ratio of the main image and the aspect ratio of the slot in which the main image is to be placed also match. In this embodiment, the image with the older capture time is laid out in the upper left of the template, and the image with the newer capture time is laid out in the lower right. From the templates selected by the template selection unit 904, the template selection unit 905 selects a template in which the slot in which the sub-images are to be placed matches the order of the sub-images arranged in chronological order, and the aspect ratios of the image and slot match. Figure 10 is a flowchart of the template determination process.

[0099] In S1001, the selected image information acquisition unit 901 acquires selected image information. The selected image information acquired in this step includes the number of selected images. For the sake of explanation, it is assumed here that the number of selected images is 3.

[0100] In S1002, the template selection unit 902 selects a template whose number of selected images matches the number of slots. Here, it is assumed that the number of selected images acquired in S1001 is 3, and a template with 3 slots is selected. Below, an example will be described in which templates (1-1) to (4-4) shown in Fig. 11 are selected in this step.

[0101] In S1003, the selected image sorting unit 903 arranges the selected images in order of their shooting time. Here, it is assumed that the relationship shown in Fig. 11A is obtained as a result of arranging the selected images in order of their shooting time. Furthermore, it is assumed that image 1105 is the main image, and images 1106 and 1107 are sub-images.

[0102] In S1004, the template selection unit 904 selects a template in which the position of the main image when arranged in shooting order matches the position of the slot for the main image (referred to as the main slot). Furthermore, a template in which the aspect ratio of the main image matches that of the main slot is selected. Here, image 1105 is the most recently shot image for the main slot, has a portrait aspect ratio, and so templates (3-1) to (3-4) in Fig. 11 are candidates.

[0103] In S1005, the template selection unit 905 selects a template whose aspect ratio matches that of the sub-image and the slot for the sub-image (referred to as a sub-slot). In the example of FIG. 11, the older image 1106 for the sub-slot, i.e., the image to be placed in the upper left, is a portrait image, and the newer image 1107 for the sub-slot is a landscape image. Therefore, the template (3-2) is determined as the template most suitable for the selected three images. In this way, in S51109, the template to be used for layout for each spread and information that can identify which image should be laid out in which slot of the template are determined. The information determined here is called spread data.

[0104] Returning to the explanation of Figure 5(c), in S51110, the spread layout unit 21007 determines whether the processes of S51107 to S51109 have been completed for all spreads to be processed. If the determination result of this step is true, the process proceeds to S51111. On the other hand, if the determination result is false, the process returns to S51107.

[0105] In S51111, the spread creation parameter setting unit 21001 determines whether the processes of S51101 to S51110 have been completed for all parameters generated in S510. If the determination result of this step is true, the process proceeds to S512. On the other hand, if the determination result is false, the process returns to S51101.

[0106] Returning to the explanation of Figure 5(a), in S512, the spread combining unit 211 determines the spread number in the album for each spread created by the spread creating unit 210, and creates the layout of the entire album. In this embodiment, the spreads are arranged based on the shooting times of the images arranged on the spreads so that the images are in chronological order of their shooting times.

[0107] The spread combining in S512 will be described in detail below with reference to Fig. 12 and Fig. 13. Fig. 12 is a block diagram showing the detailed configuration of the spread combining unit 211 in this embodiment. Fig. 13 is a flowchart of the processing executed by the spread combining unit 211.

[0108] In S1301, the spread acquisition unit 1201 acquires spread data for one album from the spreads created by the spread creation unit 210. In this embodiment, for example, the spread data for album 1 acquired in this step includes spread data created with common parameters and spread data created with individual parameters for album 1.

[0109] In S1302, the sorting reference value calculation unit 1202 calculates a sorting reference value for each spread data acquired by the spread acquisition unit 1201. In this embodiment, the sorting reference value is the average shooting time of the images arranged in the spread data. Note that the sorting reference value is not limited to the shooting time. For example, it may be the average shooting position (latitude and longitude) of the images arranged in the spread, the number of images, or image analysis information such as a color histogram.

[0110] In S1303, the sorting unit 1203 sorts the spread data acquired by the spread acquisition unit 1201 based on the sorting reference value calculated by the sorting reference value calculation unit 1202. In this embodiment, the spreads are sorted in ascending order starting from the spread with the earliest average shooting time.

[0111] In S1304, the spread number adjustment unit 1204 determines whether the number of spread data in the album to be processed is the same as the total number of spreads set in S501. If the determination result in this step is true, the process proceeds to S1306. On the other hand, if the determination result in this step is false, that is, if the number of spreads in the album to be processed was set to be greater than or equal to the total number of spreads in S51104, and as a result the number of spreads in the album exceeds the total number of spreads, the process proceeds to S1305.

[0112] In S1305, the spread number adjustment unit 1204 deletes unnecessary spread data from the spread data acquired by the spread acquisition unit 1201, thereby adjusting the number of spreads in the album to approach the total number of spreads set in S501. In this embodiment, the number of spreads is adjusted by deleting common spread data. Specifically, of the common spread data in the album, common spread data whose shooting time calculated by the sorting standard value calculation unit 1202 is closest to the shooting time of the individual spread data is deleted. After deletion, the process proceeds to S1304. Note that in this embodiment, common spread data is deleted, but this is not limited to this. The total number of spreads may also be adjusted by deleting individual spread data.

[0113] In S1306, the spread acquisition unit 1201 determines whether the processes of S1301 to S1305 have been completed for all albums. If the determination result of this step is true, the process proceeds to S513, whereas if the determination result is false, the process returns to S1301. This makes it possible to arrange images throughout the album in chronological order of their shooting time. The above is the content of the automatic layout process in this embodiment. In this embodiment, the number and positions of common spreads and individual spreads can be automatically set for each album. It is possible to automatically layout multiple albums with different main subjects, each with the same common spread across multiple albums and individual spreads that differ for each album.

[0114] <<Editing the album data>> <Editing process overview> So far, we have described a process for automatically generating album data having common spreads and individual spreads as spread attributes. Below, we will describe the functions and processes of an album editing application that allows a user to edit the generated album data. In this embodiment, we will assume that the editing user is the same user as the user who operates the album creation application, but this does not have to be the same user. Below, we will first provide a general overview of the album data editing process, and then describe the detailed process of this embodiment.

[0115] 14 is a software block diagram of an album editing application 1400 in this embodiment. In this embodiment, the album editing application 1400 is described as being part of the functions of an album creation application stored in the HDD 104, but it may also be provided as an application separate from the album creation application. Furthermore, the image processing device on which the album editing application is started may be the same device as the image processing device 100 that generated the album data, or may be a different image processing device.

[0116] 14, the album editing application 1400 includes an editing mode instruction unit 1401, an album information receiving unit 1402, an album editing unit 1403, an album display unit 1404, and an album information holding unit 215. It should be noted that the album information holding unit 215 does not necessarily have to be included in the album editing application 1400.

[0117] The edit mode instruction unit 1401 transitions the album editing application 1400 to edit mode when it detects that the user has clicked on the edit mode panel 1508 (see FIG. 15A) by UI operation using the mouse 107, for example. Edit mode is a mode in which operations to change the contents of the album, such as adding a spread, trimming, replacing images, changing the template, changing the size, changing the layout, inserting text or stamps, or deleting a spread, can be performed. Here, an example has been described in which the edit mode is transitioned to when a click on the edit mode panel is detected, but the edit mode may always be transitioned to after album data is generated. Alternatively, the edit mode may be transitioned to when any album is selected.

[0118] When the edit mode is entered by the edit mode instructing unit 1401, the album information receiving unit 1402 receives album information from the album information holding unit 215. As described above, the album information is information including album configuration information, album creation information, and album modification information. The album information receiving unit 1402 receives album information on an album-by-album basis.

[0119] The album editing unit 1403 uses the album information received by the album information receiving unit 1402 from the album information storage unit 215 to edit the album in response to UI operations by the user using the mouse 107 or keyboard. As described above, the album information includes album configuration information and album generation information, and by using the album information, album data similar to album data created by an album creation application can be obtained. When the album editing unit 1403 updates the album, the album editing unit 1403 outputs the album information to the album information storage unit 215 to update the album information. The album editing unit 1403 can also create recommendations to be displayed to the user when editing the album. Examples of recommendations include recommended images to replace in a spread that the user is editing, or recommended spreads including templates or layouts. When the edit mode instructing unit 1401 transitions to edit mode, the album editing unit 1403 can create various recommendations and output the created recommendations to the album display unit 1404.

[0120] The album display unit 1404 displays the bitmap data output by the album editing unit 1403 on the display 105. The album display unit 1404 also displays an editing screen currently being edited by the album editing unit 1403 on the display 105. The album display unit 1404 also displays recommendations created by the album editing unit 1403.

[0121] The album data obtained as a result of editing by the album editing application 1400 may be uploaded to a server via the Internet based on a user instruction. Based on the uploaded album data, printing and bookbinding processes are executed, an album (photo book) is created as a booklet, and delivered to a designated user.

[0122] 15A and 15B are diagrams illustrating an example of a UI display of an album editing screen in this embodiment. FIG. 15A is an example of a screen displayed on the display 105 after an album is created. As described above, the album data in this embodiment is assumed to be data for three albums, each with an album target (main subject) of Person A (Album 1), Person B (Album 2), and Person C (Album 3). Image 1501 is an example of an image arranged on a double-page spread, and the text in image 1501 indicates the main subject appearing in the image. Common double-page spread 1502 is an example of a layout centered around an image featuring multiple main subjects. Meanwhile, individual double-page spreads 1503, 1504, and 1505 are respectively centered around images featuring main subject A, main subject B, and main subject C. As described above, in this embodiment, multiple albums are created, each consisting of double-page spreads created according to different standards. Edit mode panel 1508 is a panel that transitions to edit mode when clicked with mouse 107. Characters 1506 and 1507 are characters that respectively indicate that the spread is a common spread and an individual spread on the display 105. Although characters are used here, the background color of the spread may be different between a common spread and an individual spread.

[0123] FIG. 15B(1) shows a screen displayed on the display 105 by the album editing application 1400 after clicking the edit mode panel 1508 with the mouse 107. That is, FIG. 15B(1) shows an example of an editing screen. FIG. 15B(1) shows an editing screen for each album. That is, each spread for each album is displayed. Note that if an album or spread does not fit within the screen, the entire screen may be viewed by scrolling, or the size of the displayed spread may be changed to display all spreads on the screen. In this embodiment, different processing is performed depending on whether the spread to be edited is a common spread or an individual spread. Details will be described later. Note that in the example of the editing screen in FIG. 15B(1), the "spread to be edited" refers to each spread for each album. That is, in the example of FIG. 15B(1), a total of 12 spreads are "spreads to be edited."

[0124] The image insertion panel 1509 enters a state in which an image can be inserted when clicked with the mouse 107. For example, to determine the image to be inserted, all images acquired in the image acquisition unit when the album was created may be displayed, allowing the user to select one of the images. Images not used in the album may be selected instead of all images. The text insertion panel 1510 enters a state in which text can be inserted when clicked with the mouse 107. For example, the mouse cursor may be changed to a text insertion cursor in response to a click of the mouse 107 on the text insertion panel 1510, allowing text to be entered by clicking the mouse 107 within the album. The text size, font, or text color may be set at this time. The stamp insertion panel 1511 enters a state in which a stamp can be inserted when clicked with the mouse 107. For example, stamps may be displayed in response to a click of the mouse 107 on the stamp insertion panel, allowing the user to select any stamp from the displayed stamps and add it to the album. The two-page spread addition panel 1512 is a panel that displays a two-page spread addition UI when clicked with the mouse 107. The plus mark 1513 is an icon that, when clicked with the mouse 107, fixes the position at which a spread is added for insertion to the plus mark and adds a spread.

[0125] FIG. 15B(2) shows an edit mode in which the focus is on any spread in any album. FIG. 15B(2) shows an example in which the focus is on spread 1 in album 1 in FIG. 15B(1). FIG. 15B(2) shows an edit screen for editing in spread units. In the edit screen of FIG. 15B(2), the "spread to be edited" is the focused spread. That is, in the example of FIG. 15B(2), the "spread to be edited" is spread 1 in album 1. To enter the spread-based edit mode shown in FIG. 15B(2), for example, clicking any spread in FIG. 15B(2) with the mouse 107 may transition to spread-based edit mode with the focus on that spread. Note that clicking spread 1 in album 2, which is displayed small in FIG. 15B(2), displays an edit screen focused on spread 1 in the clicked album 2.

[0126] In the examples up to this point, the explanations have been given assuming that various panels are left-clicked with the mouse 107, but it is also possible to display a menu by right-clicking the mouse 107, and then left-click that menu with the mouse 107 to enter the desired editing state.

[0127] The edit mode instruction unit 1401 also sets the timing for reflecting the edit results. Reflecting the edit results is a process of applying edits made to one album to other albums based on change information due to editing. The timing for reflecting the edit results may be when the user performs some kind of editing operation, when the user saves the album, when the spread to be edited is changed, or when the album to be edited is changed. The timing may also be specified by user settings, or the timing may be determined in advance by software and used. When these various timing conditions are met, the album editing unit 1403 performs processing for reflecting the edit results. In this embodiment, the timing for reflecting the edit results is when the user performs some kind of editing operation. This concludes the general description of the album editing process.

[0128] Fig. 16 is a diagram showing an example of adding a spread. Fig. 16 shows an example in which a spread is added to Album 1 in the album configuration shown in Fig. 15A, and an edited spread reflected in response to this edit operation of adding a spread is added to Album 2, as displayed on display 105. However, in this example, the edited spread is not added to Album 3.

[0129] Fig. 16 shows an example in which a common spread has been added. Specifically, a common spread has been added to Album 1. The added common spread is added at the same position as Spread 2, and the subsequent spread numbers are moved down.

[0130] The same common spread is also added to Album 2 as Album Spread 2, just as in Album 1. This makes the number of spreads in Album 1 and the number of spreads in Album 2 the same, eliminating any sense of unfairness. In this example, since the additional spread 1601 is a common spread, a spread with the same layout is also added to Album 2 as Edit-reflected spread 1602. Note that the additional spread 1601 and the edit-reflected spread 1602 are either identical spreads with no difference at all, or similar spreads.

[0131] Also, since the editing content of Album 1 has not been reflected in Album 3, there is one less spread compared to Albums 1 and 2.

[0132] 17 is a block diagram showing details of the album editing unit 1403. The album editing unit 1403 can include various functions, but in this embodiment, the description will focus on blocks related to the function of adding a spread. The album editing unit 1403 has a spread addition condition setting unit 1701, a spread addition position determination unit 1702, a spread reference determination unit 1703, an image acquisition unit 1704, a used image determination unit 1705, an image analysis unit 1706, a scene division unit 1707, and a scene classification unit 1708. It also has a main subject setting unit 1709, an image scoring unit 1710, a spread creation parameter generation unit 1711, a spread creation unit 1712, a spread addition unit 1713, and an album information output unit 1714.

[0133] The spread addition condition setting unit 1701 sets the conditions for starting the spread addition process. This spread addition process is a process of adding a spread automatically by the album editing application 1400. In this embodiment, this spread addition process includes a process of reflecting the editing results of the added spread in other albums, that is, a process of adding spreads to other albums. The spread addition conditions are set when the user specifies the spread addition conditions through UI operations using the mouse 107 or keyboard and instructs their execution. The spread addition conditions include the group of images to be used in the album, the number of spreads to be added, and the album to which the spreads are to be added.

[0134] The number of spreads to be added may be specified by the user, or a fixed number of spreads may always be added. The number of spreads to be added may also be determined based on the number of images not actually used among the images used in the album. The image scores of images not used in the album may also be used. Specifically, an image score threshold may be set, and the number of images with image scores equal to or greater than the image score threshold may be calculated to determine the number of spreads to be added to the album. The number of images equal to or greater than the image score threshold may be divided by a fixed value, and the quotient obtained as a result may be used as the number of spreads to be added.

[0135] As another example, a threshold value for the spread score may be determined and used. The spread score is the sum of scores obtained by multiplying the images included in the spread by a predetermined weight. The weight here may be, for example, a weight based on the time information within the spread, the size of the slot in which the image is included within the spread, and placement information. Alternatively, a weight based on the size of the slot in which each image is placed may be used. The larger the slot in which an image is placed, the more important the image within that spread is, so this is intended to emphasize the elements of such an image. In this way, the sum of the images within the spread multiplied by the predetermined weight becomes the score for the spread. For example, a spread may be created using images whose image scores are equal to or greater than a threshold, and as many spreads as are necessary to create a spread whose spread score is equal to or greater than the spread score threshold may be added. As another example, the maximum number of spreads that does not increase the album price may be added.

[0136] The album to which spreads are to be added may be specified by the user, or spreads may be added to the album currently being edited. The album to be added set in the spread addition conditions is, so to speak, the primary addition album. When spreads are added to this primary addition album, the same number of spreads as those added to the primary addition album will also be added to other albums (which may also be called secondary addition albums).

[0137] 18 shows a display screen 1801 provided by the album creation application for adding a spread, which is displayed on the display 105. The user can specify the conditions for creating the additional spread via the display screen 1801. The display screen 1801 is a spread addition UI that is displayed when, for example, the spread addition panel 1512 in FIG. 15B(1) is clicked.

[0138] A path box 1802 is a box for indicating the storage location (path) of the image group for which an additional spread is to be created in the HDD 104. The storage location may be the same as the storage location when the album was created, or may be different.

[0139] The folder selection button 1803 is a button for selecting a folder containing a group of images for which an additional spread is to be created. When the user clicks the folder selection button 1803 with the mouse 107, a tree consisting of multiple folders is displayed. Then, when the user selects a folder containing a group of images for which an album is to be created, the folder path of the selected folder is displayed in the path box 1802.

[0140] The number of double-page spreads to add box 1804 is a box for specifying the number of double-page spreads to add to the album. The user can use the keyboard 106 to directly input a number into the number of double-page spreads box 1804, or can use the mouse 107 to input a number into the number of double-page spreads box 1804 from a list. In this embodiment, an example will be described in which one double-page spread is added to one album. Note that in this spread addition UI, an example has been described in which the user specifies the number of double-page spreads to add, but as mentioned above, the number of double-page spreads to add may also be determined automatically. Furthermore, if no number is input into the number of double-page spreads box 1804, the number of double-page spreads to add may also be determined automatically.

[0141] The additional spread position box 1805 is a box for specifying a spread number as the position of a spread to be added when adding a spread to an album. The user can use the keyboard 106 to directly input a number into the additional spread position box 1805, or can use the mouse 107 to input a number into the additional spread position box 1805 from a list. Note that if the user clicks the plus mark 1513 in Figure 15B(1) and then clicks the additional spread panel 1512, the position of the plus mark 1513 may be used as the position for adding the spread.

[0142] The additional spread attribute check box 1806 is a box for specifying attributes when creating an additional spread. The user can specify the check box using the keyboard 106 or mouse 107. A spread with the selected attributes is automatically generated. If the check box is not checked, the attributes may be automatically determined, or an automatic determination check box may be added.

[0143] The spread image number setting section 1807 is a slider bar for setting the number of images to be placed on the additional spread to be created. When the slider is moved to the "more" side, the number of images to be placed on each spread increases. On the other hand, when the slider is moved to the "fewer" side, the number of images to be placed on each spread decreases.

[0144] The album selection button 1808 is a button for selecting an album into which an additional spread will be inserted. When the user clicks the album selection button 1808 with the mouse 107, a list of currently created albums is displayed. When the user then specifies one or more albums from the list, the specified albums are selected as the albums into which an additional spread will be inserted. As described above, the album currently being edited may be automatically determined as the album into which an additional spread will be inserted, or the addition UI may be configured so that the album currently being edited is selected by default. In this embodiment, spreads will also be added to albums other than the album selected with the album selection button 1808. However, spreads will not be added to all albums.

[0145] The OK button 1809 is a button for confirming the selected conditions as the additional spread creation conditions. When the user clicks the OK button 1809 with the mouse 107, the additional spread creation conditions are confirmed, and the additional spread conditions are communicated to each section of the album editing section 1403 via the additional spread condition setting section 1701. The reset button 1810 is a button for resetting each setting information on the display screen.

[0146] Returning to the description of FIG. 17 , the spread addition position determination unit 1702 determines the position in the album where the additional spread will be inserted. If an additional spread position is specified by the spread addition condition setting unit 1701, the spread addition position determination unit 1702 can set that position as the spread addition position. Alternatively, the spread addition position determination unit 1702 may automatically determine the spread addition position. An example of automatically determining the spread addition position is adding the spread to the spread next to the spread currently being edited by the user. As another example, the spread may always be added to the end of the spread. If the album is divided into two groups of common spreads and individual spreads, the additional spread may be inserted as the last spread of either the common spread group or the individual spread group according to the spread criteria specified in the spread addition conditions. Furthermore, after creating the layout of the additional spread, the additional spread may be positioned to match the time axis of the layout, or may be positioned to maximize the overall album score. The score for the entire album may be determined, for example, by the arrangement of the spreads in accordance with the combining rules (such as the time axis) at the spread combining section 211.

[0147] As an example of automatically determining the position for adding a spread after creating a spread, the position for adding a spread may be determined at a position where the time axis matches the reference in the album, using time axis information for the created spread and information about the spread standard. In the case of an album in which the positions of the common spread group and the individual spread group are divided into two, the spread standard is given priority to determine which group it is, and then the position for adding a spread may be determined at a position where the time axis matches. In the case of an album in which the positions of the common spread group and the individual spread group are not divided into two, the position for adding a spread may be determined at a position where the time axis matches. Note that a recommended position for adding a spread may be determined and displayed to the user.

[0148] The spread standard determination unit 1703 determines the spread creation standard for the added spread. Specifically, it determines which standard to use: common spread, individual spread, or other spread. Other spreads are spread creation standards that are neither common nor individual and are not used during automatic album creation. For example, the standard may use only the image score without determining the main subject, or the standard may be a weighted sum of the common spread standard and the individual spread standard.

[0149] If an additional spread standard is set in the spread addition condition setting unit 1701, the spread standard determination unit 1703 determines the spread standard according to the setting. If an additional spread standard is not set, or in other cases, the spread standard determination unit 1703 may automatically determine the spread standard. An example of automatically determining the spread standard may be determining the spread standard based on the position of the spread to be added. For example, the spread standard used for the spreads on both sides of the spread to be added may be used. If the spread standards used for the spreads on both sides are different, the number of images suitable for a common spread and the number of images suitable for an individual spread among images not used in the album may be compared, and the standard with the largest number of images may be used. Note that an image suitable for a common spread or an image suitable for an individual spread is an image with a high score compared based on each spread standard. When comparing the scores, a threshold may be set, and an image suitable for a high-score standard may be determined when the difference between the score for the common spread and the score for the individual spread is equal to or greater than the threshold. If an image is newly added during editing and the image score has not been calculated, the image score may be calculated here.

[0150] As another example of automatically determining the spread standard, the spread standard for the additional spreads may be determined based on the balance between the number of common spreads and the number of individual spreads in the album. An example of determining the standard using the balance between the number of common spreads and the number of individual spreads in the album is determining the standard so that the ratio between the number of common spreads and the number of individual spreads approaches 5:5. Furthermore, when a user selects an image to be used in a spread and instructs the addition of a spread, a spread standard suitable for the selected image may be determined. Furthermore, spreads may be automatically created using multiple creation standards, and the standard with the highest spread score may be adopted as the spread standard. One of the examples described above may be used, or multiple examples may be combined to determine the spread standard. Spreads automatically created using multiple creation standards may be displayed on the display 105, and the spread standard selected by the user may be determined as the spread standard.

[0151] The image acquisition unit 1704 acquires the image group set by the spread addition condition setting unit 1701. The image acquisition unit 1704 is a processing unit similar to the image acquisition unit 203 of the album creation application. Note that if the user does not specify an image group, or if the user's specification is the same as when the album was created and the data acquired by the image acquisition unit 203 when the album was created is retained, the acquired images may be used as is without image acquisition. If the storage location of the image group set by the spread addition condition setting unit 1701 is different from the folder acquired by the image acquisition unit 203 when creating the album, or if the HDD 104 is updated and the number of images has changed, the set image group is acquired from the HDD 104.

[0152] The used image determination unit 1705 determines whether the image acquired by the image acquisition unit 1704 is an image used in the album based on the image usage information included in the album information received by the album information receiving unit 1402. The determination of whether an image is used may differ depending on the determination made by the spread standard determination unit 1703. For example, if the spread standard determination unit 1703 determines the spread standard to be individual, the used image determination unit 1705 may determine whether an image is used based on the image usage information for the album to be edited. On the other hand, if the spread standard determination unit 1703 determines the spread standard to be common, the used image determination unit 1705 may determine whether an image is used based on the image usage information for all albums. If an image determined to be a used image is placed in an album, duplication of images will occur within one of the albums. For this reason, used images are not used to create additional spreads. As an example of a method for not using the image data to create additional spreads, the acquired image data may be discarded. Furthermore, a used image flag or a used label may be used to mark the image as used, so that the processing described below is not performed, or the image is not selected as an image for creating a double-page spread. In this embodiment, a used image flag is attached to a used image, and processing is performed to prevent the image from being selected as an image for creating an additional double-page spread.

[0153] The image analysis unit 1706 performs analysis processing on the image data acquired by the image acquisition unit 1704. The image analysis unit 1706 is a processing unit similar to the image analysis unit 204. For images acquired by the image acquisition unit 1704 that are the same as those at the time of album creation, the analysis results at the time of album creation may be used.

[0154] The scene dividing unit 1707 divides the group of images acquired by the image acquiring unit 1704 into scenes based on the analysis information of the image analyzing unit 1706. The scene dividing unit 1707 is a processing unit similar to the scene dividing unit 206. If the images acquired by the image acquiring unit 203 include the same images as those at the time of album creation, the division results at the time of album creation may be used.

[0155] The scene classification unit 1708 determines the category of the group of images in the scenes obtained by the scene division unit 1707. The scene classification unit 1708 is a processing unit similar to the scene classification unit 207. When using the same images as when the album was created, the classification results when the album was created may be used.

[0156] Main subject setting unit 1709 sets a main subject to be emphasized for each album. Main subject setting unit 1709 is a processing unit similar to main subject setting unit 205. Main subject setting unit 1709 sets a main subject corresponding to the double-page spread standard determined by double-page spread standard determination unit 1703. Note that it may be set that there is no main subject.

[0157] Image scoring unit 1710 assigns a score to each image for each main subject set by main subject setting unit 1709, based on the analysis information obtained by image analysis unit 1706. Image scoring unit 1710 is a processing unit similar to image scoring unit 208. In the case of the same image and the same subject as when the album was created, the score at the time the album was created may be used without assigning a new score.

[0158] The spread creation parameter generation unit 1711 generates parameters for creating a spread based on the main subject set by the main subject setting unit 1709 and the analysis information obtained by the image analysis unit 1706. The spread creation parameter generation unit 1711 is a processing unit similar to the spread creation parameter generation unit 209. The spread creation parameter generation unit 1711 sets parameters for creating a spread corresponding to the spread standard determined by the spread standard determination unit 1703. An example of this spread creation parameter is a weight assigned to the score obtained by the image scoring unit 1710. Examples of information used for calculating the weight include the various types of information described in connection with the spread creation parameter generation unit 209. Furthermore, the weight may be determined using the spread creation parameters such as image information used, time information within the spread containing the image, and the size and placement information of the slot within the spread containing the image. For example, as explained in the spread creation parameter generation unit 209, when placing an image in a slot, the weight may be changed depending on whether the image has been trimmed or not, and whether the spread in which the image is used is a common spread or an individual spread. Furthermore, the score of a user-used image that has been additionally inserted by the user may be weighted, thereby adjusting the score to be higher. Furthermore, images that have already been used in an album may be weighted less to avoid duplication, so that they are not selected.

[0159] The spread creation parameters also include a parameter (second parameter) that derives the score for the spread. In the example of FIG. 15B(2), the score is the sum of the scores obtained by multiplying the scores of each image in spread 1 of album 1 by a weight based on the size of the slot in which each image is placed. The larger the slot in which an image is placed, the more important the image is in that spread, so this is to strengthen the elements of such an image. Note that, although an example has been described in which the scores of each image weighted by slot size are summed up, weighting is not necessary, and the score for the spread can also be the average of each image.

[0160] The spread creation unit 1712 creates a spread using the results of the image scoring unit 1710, the used image information from the used image determination unit 1705, and the spread creation parameters generated by the spread creation parameter generation unit 1711. The spread creation unit 1712 selects images with high final image scores as images to be used in the spread, and creates a spread by arranging them. The spread creation unit 1712 is a processing unit similar to the spread creation unit 210. The spread creation unit 1712 creates the number of spreads set by the spread addition condition setting unit 1701. Furthermore, in this embodiment, the spread creation unit 1712 performs processing to create additional spreads in other albums in response to creating an additional spread in the album to which spreads are to be added as set in the spread addition conditions.

[0161] A spread adding unit 1713 adds the additional spreads created by the spread creating unit 1712 to the album according to the result of the spread adding position determining unit 1702, determines the spread numbers of the spreads in the album, and creates the layout of the entire album.

[0162] When the album is updated, the album information output unit 1714 outputs the album information to the album information holding unit 215 based on the editing results.

[0163] In this embodiment, the album editing process has been described mainly with an example of automatically adding a double-page spread, but the album editing process may also be performed in response to user operations. For example, as described above, the album editing unit 1403 can edit the album based on user input even in cases of trimming, image replacement, template change, size change, layout change, insertion of text stamps, or deletion of a double-page spread.

[0164] FIG. 19 is a software block diagram showing a group of modules relating to the reflection of editing results, which are included in the album editing application 1400 according to this embodiment.

[0165] The edit result reflection timing determination unit 1901 determines whether or not the conditions for the edit result reflection timing specified by the edit mode instruction unit 1401 are met, that is, whether or not it is time to reflect the results of editing performed on one album in another album.

[0166] The album information receiving unit 1902 is a processing unit similar to the album information receiving unit 1402, and receives album information on an album-by-album basis from the album information holding unit 215. Specifically, this album information is album configuration information, album creation information, and album change information. The album change information includes information on changes due to editing. The information on changes due to editing may be difference information changed by editing the album, or may be the operation history of the user who edited the album.

[0167] The edited spread determination unit 1903 determines whether the edited spread is a common spread, using album change information included in the album information received by the album information receiving unit 1902. In this example, it is determined whether the edited spread is a common spread based on whether the standard used when the spread was automatically generated is a common standard. However, the method for determining whether the spread is a common spread is not limited to this. For example, when the same spread is used between albums, the same spread may be determined as a common spread, or common spread management may be performed using a flag, or a spread with a certain fixed spread number may be determined as a common spread.

[0168] The editing result reflection condition setting unit 1904 sets conditions for reflecting the editing results of a spread determined to be a common spread by the edit spread determination unit 1903 to other albums. Specifically, the conditions may be layout information such as background color or template, or editing content such as text insertion or stamps, or the position such as the spread number or a certain area of ​​the album. As another example, the editing results may always be reflected without setting the reflection condition settings. In this embodiment, the editing results are reflected for edits other than text insertion and stamp insertion.

[0169] The editing result reflection album setting unit 1905 sets which albums the editing results should be reflected in. In this embodiment, a setting is made so that the editing results are not reflected in at least one album. Examples of methods for determining which albums the editing results should be reflected in for a specific album include grouping the albums by user in advance and reflecting the results on a group basis, or reflecting the results in albums that will result in higher spread scores when the editing results are reflected. Alternatively, for image editing, albums to which the editing results should be reflected may be set based on image scores. As an example of setting albums to which the editing results should be reflected based on image scores, the editing results are evaluated based on the image scores using the main subjects of individual spreads, and the editing results are reflected in albums with higher scores. In this way, in this embodiment, the editing results are reflected in albums with higher scores, but are not reflected in albums that will result in lower spread scores when the editing results are reflected. Furthermore, if the scores of the spreads of all albums increase as a result of reflecting the editing results, the editing results are not reflected in the album with the smallest increase in score. This allows appropriate editing content to be reflected for each target user linked to an album.

[0170] An editing results reflection album determination unit 1906 determines whether or not to reflect the editing results for each album, based on the setting of whether or not the album reflects the editing results set by the editing results reflection album setting unit 1905. For albums determined by the editing results reflection album determination unit not to reflect the editing results, the following processing units do not perform processing.

[0171] The editing result reflection area determination unit 1907 determines whether the target area satisfies the conditions set by the editing result reflection condition setting unit 1904. In this embodiment, areas that satisfy the editing result reflection conditions and areas that do not are managed by using the determination result of the editing result reflection area determination unit 1907. As an example of a method for managing the determination results, the result reflection may be managed using a flag indicating whether or not the editing results are reflected in each spread area, or only the user operation history of editing that satisfies the editing result reflection conditions may be passed to the editing result reflection unit 1908.

[0172] The editing result reflecting unit 1908 reflects the edits made to the spread determined by the edit spread determining unit 1903 as a common spread in other albums using the determination result of the editing result reflecting area determining unit 1907 and the editing change information received by the album information receiving unit 1902. In this example, the editing reflecting process involves applying the edits made to one album to other albums as they are, specifically, adding the same spread as added to one album to the other albums. One possible method for creating the same spread is to create an album automatically generated using album generation information and then perform the same operations as described in the album change information. This method allows for the creation of identical spreads with a completely identical layout and arrangement. Alternatively, the album configuration information received by the album information receiving unit may be used to create identical spreads with a completely identical layout and arrangement. The editing result reflecting unit 1908's processing allows the common spread layout to be the same between the edited album and some of the albums excluding the edited album.

[0173] When the album is updated, the album information output unit 1909 outputs album information based on the editing results to the album information holding unit 215. Specifically, this album information includes album configuration information, album creation information, and album change information.

[0174] In this embodiment, with the above-described configuration, editing performed in a specific album, specifically, a change in the common spread, is reflected in other albums of the specific album, but is not reflected in at least some of the albums, thereby allowing the layout of each individual album to be adjusted.

[0175] Fig. 20 is a flowchart showing the process of adding a double-page spread in the album editing application 1400 according to this embodiment. The flowchart shown in Fig. 20 is realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or RAM 103 and executing it.

[0176] In S2001, the spread addition condition setting unit 1701 sets the spread addition conditions. In this embodiment, the spread addition conditions are set by setting the group of images to be used in the album, the number of spreads to be added, and the album to which the spreads will be added. Note that here, the user sets the target album to which the spreads will be added, but in the processing of this embodiment, spreads will also be added to albums for which the setting for album addition has not been made here.

[0177] In S2002, the spread addition position determination unit 1702 determines the spread addition position. In S2003, the spread reference determination unit 1703 determines which reference to use for the added spread. Specifically, as described above, it determines which spread reference to use: common spread, individual spread, or other spread.

[0178] In S2004, the image acquisition unit 1704 reads the images included in the image group set by the double-page spread addition condition setting unit 1701 from the HDD 104 and loads them into the RAM 103. If the images have already been loaded into the RAM 103, no processing is performed.

[0179] In S2005, the used image determination unit 1705 determines whether the acquired image is a used image based on the album information (used image information) acquired by the album information receiving unit 1402 and the spread criteria determined by the spread criteria determination unit 1703. If the additional spreads are based on a common criteria, the used image information of all albums is used. If the additional spreads are based on an individual criteria or other criteria, the used image information of the album currently being edited is used. The flowchart will be explained below using an example in which the additional spreads are based on a common criteria.

[0180] In S2006, the image analysis unit 1706 analyzes the image data expanded in the RAM 103. The image analysis process has the same processing flow as the process described in FIG.

[0181] In S2007, the image analysis unit 1706 determines whether the processes of S2005 and S2006 have been completed for all images in the image group on the HDD 104 specified by the double-page spread addition condition setting unit 1701. If not, the process returns to S2004. If completed, the process proceeds to S2008.

[0182] In S2008, the scene division unit 1707 performs scene division on the group of images acquired by the image acquisition unit 1704. S2008 is the same processing as S505. In S2009, the scene classification unit 1708 performs scene classification on the group of sub-images divided by the scene division unit 1707. S2009 is the same processing as S506.

[0183] In S2010, main subject setting unit 1709 sets a main subject for each album. Main subject setting unit 1709 sets a main subject for the album to be added based on the album information received by album information receiving unit 1402, the album to which a spread is to be added that was set in S2001, and the spread standard determined in S2003. In this example, it is assumed that the additional spread processing is to be performed on album 1. Also, in this example, the common spread standard is used. Therefore, main subjects ABC are set in album 1.

[0184] In S2011, the image scoring unit 1710 scores each image. S2011 is the same process as S508. In S2012, the image scoring unit 1710 determines whether the image scoring in S2011 has been completed for all images. If not, the process returns to S2011. If completed, the process proceeds to S2013.

[0185] In S2013, the spread creation parameter generation unit 1711 generates spread creation parameters based on the spread addition conditions set in S2001 and the main subject of the additional spread set in S2010. S2013 is the same process as S510.

[0186] In S2014, the spread creation unit 1712 creates an additional spread for the album to be added using the spread creation parameters generated in S2013. The spread creation process is the same as that in S511.

[0187] In S2015, the album editing unit 1403 determines whether the number of spreads created in S2014 has reached the number of additional spreads set in the spread addition condition setting unit 1701. If the number of additional spreads has not been reached, the process returns to S2002. If the number of additional spreads has been reached, the process proceeds to S2016.

[0188] In S2016, the spread adding unit 1713 inserts the spread created by the spread creating unit 1712 into the determined position of the additional spread, combines the albums, and creates a layout for the entire album.

[0189] In S2017, the album information output unit 1714 outputs the album information.

[0190] Next, a process of reflecting the editing results, which is included in the automatic layout process in the album creation application according to this embodiment, will be described with reference to Fig. 21. Fig. 21 is a flowchart showing a process of reflecting the editing results by the album editing application 1400 according to this embodiment. The flowchart shown in Fig. 21 is realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or RAM 103 and executing it.

[0191] In S2101, the edit result reflection timing determination unit 1901 determines whether or not the conditions for the edit result reflection timing specified by the edit mode instruction unit 1401 are met. If it is the edit result reflection timing, the process proceeds to S2102. If it is not the edit result reflection timing, the series of processes ends.

[0192] In 2102, the album information receiving unit 1902 receives album information including information on the editing content performed on a specific album from the album information holding unit 215. The album information received in this step is expanded in the RAM 103 and held therein.

[0193] In S2103, the edited spread determination unit 1903 determines whether the album to be determined (first target album) is an edited album. If the determination result in this step is true, the process proceeds to S2104, whereas if the determination result is false, the process proceeds to S2111.

[0194] In S2104, the edited spread determination unit 1903 determines whether the edited spread in the edited album (referred to as the edited spread) is a common spread. If the determination result in this step is true, the process proceeds to S2105, whereas if the determination result is false, the process proceeds to S2111.

[0195] In S2105, the edit result reflection condition setting unit 1904 sets the conditions for reflecting the edit result (edit result reflection conditions).

[0196] In S2106, the edited result reflecting album setting unit 1905 sets the target album in which the edited result is reflected.

[0197] In S2107, the editing result reflection album determination unit 1906 determines whether the album to be determined (second target album) is a target album in which the editing results are reflected, based on the setting in S2106. If the determination result in this step is true, the process proceeds to S2108, whereas if the determination result is false, the process proceeds to 2110.

[0198] In S2108, the edit result reflection area determination unit 1907 determines whether the target area satisfies the conditions set by the edit result reflection condition setting unit 1904. If the determination result in this step is true, the process proceeds to S2109, whereas if the determination result is false, the process proceeds to S2110.

[0199] In S2109, the editing result reflecting unit 1908 reflects the content of the editing results made in the predetermined album to the target area of ​​the album for which both the determinations in S2107 and S2108 are true.

[0200] In S2110, the editing result reflecting unit 1908 determines whether the processes from S2107 to S2109 have been completed for all albums except for the specified album for which editing has been performed. If the determination result in this step is true, the process proceeds to S2111, whereas if the determination result is false, the second focused album is changed to an undetermined one, and the process returns to S2107.

[0201] In S2111, the editing result reflecting unit 1908 determines whether the processes from S2103 to S2110 have been completed for all albums. If the determination result in this step is true, the series of processes ends. On the other hand, if the determination result in this step is false, the first focused album is changed to an undetermined one, and the process returns to S2103.

[0202] <<Effects of this embodiment>> The above is an explanation of the automatic generation of additional spreads in this embodiment and the reflection of the editing results of the common spread in a given album in other albums. In this embodiment, the automatic generation of additional spreads can be performed in response to an instruction from the user for the additional spread. Furthermore, the editing results of the common spread can be applied to each album, allowing editing that is appropriate for each album.

[0203] (Embodiment 2) In the first embodiment, when the editing results of a specified album are reflected in other albums of the specified album, the editing change information corresponding to the specified album is used. In contrast, in the present embodiment, a spread similar to the edited spread in the specified album is created based on the edited spread in the specified album, and the created spread is added to the other album. Note that in the present embodiment, the same components as those in the first embodiment are designated by the same reference numerals and names, and the description thereof will be omitted as appropriate.

[0204] The functional configuration of this embodiment will be described below with reference to Fig. 22. Fig. 22(a) is a software block diagram showing a group of modules relating to the reflection of editing results, which are modules included in the album editing application 1400 of this embodiment.

[0205] The editing result reflecting unit 2201 reflects the contents of the editing results for a specified album in other albums of the specified album. Specifically, based on the edited spread in the specified album, the editing result reflecting unit 2201 creates or adds a spread in the other albums that is similar to the edited spread and appropriate for each album. However, in this embodiment as in the first embodiment, such reflection is not performed on at least some of the other albums.

[0206] FIG. 22(b) is a detailed software block diagram of the edit result reflecting unit 2201 when the process of reflecting edited content is a process of creating and adding a similar spread.

[0207] The image acquisition unit 220101 is a processing unit similar to the image acquisition unit 203 used in creating an album (see FIG. 2(a)). If there is no user specification when the image acquisition unit 220101 acquires an image group, or if the specification remains the same as when the album was created and the data acquired by the image acquisition unit 203 is still held, the image acquisition process by the image acquisition unit 220101 may be skipped. In this case, the data acquired by the image acquisition unit 203 may be used as is. However, if the data in the HDD 104 has been updated, the image group is acquired from the HDD 104.

[0208] The used image determination unit 220102 determines whether the image acquired by the image acquisition unit 220101 is used in the album based on the image usage information included in the album information received by the album information receiving unit 1902. This determination is made using the image usage information for all albums. In this embodiment, information about an image determined to be a used image by the used image determination unit 220102 is treated as used image information along with information about which album and which spread the image is used in. If an image determined to be a used image is placed in an album, image overlap will occur within one of the albums. Therefore, a recommendation recommending the use of a used image will not be created. A flag indicating whether the image is used or not (used image flag), an image label, or the like may be used as a means for preventing used images from being used in additional spreads. For images determined to be used by the used image determination unit 220102, the following processing may not be performed, or the image may be excluded from selection during image selection. In this embodiment, the used image flag is set.

[0209] The similar image determination unit 220103 performs image similarity determination to determine whether images are similar. In this embodiment, similarity determination is performed between images with a used image flag set and other unused images. Images determined to be similar to used images in this similarity determination may be similar to the image in the spread and may be better images, so they may be used to create the spread. In addition, similarity labels are set in the same number as the number of images in the spread, and if there are multiple unused images determined to be similar to the same image in the spread, they are managed with the same similarity label. A similar spread can be created by selecting one image from each group of images with the same similarity label and creating a spread.

[0210] An example of image similarity determination is determination using Structure Similarity (SSIM). Specifically, the SSIM values ​​of the edited spread image and the image acquired by the image acquisition unit 220101 are calculated. If the calculated SSIM value is equal to or greater than a threshold value of 0.8, the images may be determined to be similar. Note that the threshold value of 0.8 is merely an example, and other values ​​may be used. Furthermore, PSNR may be used instead of SSIM. When PSNR is used, the threshold may be freely set, such as 40 dB. This threshold may be determined in advance or may be selected by the user. When the user selects the threshold, the threshold may not be directly set, but may be indirectly selected by operating a slider to increase or decrease the threshold. If the images to be determined to be similar differ in size, the unused image to be determined is resized to the size of the used image before similar image determination is performed. For example, if the used image is smaller than the unused image, the unused image is trimmed to the size of the used image before similar image determination is performed. Conversely, if the used image is larger than the unused image, the unused image is enlarged to the same size as the used image before similar image determination is performed. Other examples of similar image determination include methods for classifying images based on the content they contain, such as classification based on the main subject of the image, or methods for detecting objects within an image and classifying images that contain many identical objects as similar images.

[0211] The main subject setting unit 220104 sets a main subject to be emphasized (prioritized) for each album. The main subject setting unit 220104 is a processing unit similar to the main subject setting unit 205. In this embodiment, the main subject of the common spread when editing an album is set to two different subjects, that is, the main subject of the individual spread when creating the album, and the main subject of the common spread when creating the album. This allows the main subject of the common spread when editing an album to differ for each album, making it possible to select similar images suitable for each user.

[0212] The image selection parameter generation unit 220105 generates parameters (referred to as image selection parameters) for selecting images based on the main subject set by the main subject setting unit 220104 and the analysis information obtained by the image analysis unit 1703. The image selection parameters here are set values ​​that determine the image selection criteria. An example of such parameters is a weight assigned to the score obtained by the image scoring unit 1710.

[0213] The image selection unit 220106 selects images using parameters generated by the image selection parameter generation unit 220105. In this embodiment, the similar image determination unit 220103 calculates a final image score for each image in the group of images determined to be similar to the image arranged in the spread, and the image selection unit 220106 selects the image with the highest score. By replacing the image selected by the image selection unit 220106 with the image that was arranged, a similar spread is created in each album that reflects the content of the editing results.

[0214] As described above, the editing result reflecting unit 2201 creates a similar spread that takes into consideration the user (that is, the main subject) associated with the album that reflects the editing contents, so that editing suitable for each album can be performed.

[0215] FIG. 23 is a flowchart showing the process of reflecting the edit results.

[0216] In S2301, the image acquisition unit 220101 reads the candidate image group from the HDD 104 and expands it in the RAM 103, using the information on the image group used during editing received by the album information receiving unit 1902. Note that if the candidate image group has already been expanded in the RAM 103, the processing of this step does not need to be performed.

[0217] In S2302, the used image determination unit 220102 determines whether the image acquired by the image acquisition unit 220101 is an image that has already been used in the album. This determination is made based on the image usage information included in the album information received by the album information receiving unit 1902 and the criteria determined by the spread criteria determination unit 1703. For example, if the spread criteria determination unit 1703 determines the individual criteria as the spread criteria, the used image determination unit 220102 may determine whether the image is a used image using the image usage information in the album to be edited. On the other hand, if the spread criteria determination unit 1703 determines the common criteria as the spread criteria, the used image determination unit 220102 may determine whether the image is a used image using the image usage information in all albums.

[0218] In S2303, the similar image determination unit 220103 determines similar images using the determination result of the used image determination unit 220102 between used images and other images.

[0219] In S2304, the image analysis unit 1706 analyzes the image data expanded in the RAM area 103. The image analysis process has the same processing flow as the process described in FIG.

[0220] In S2305, the image analysis unit 1706 determines whether the processes from S2301 to S2304 have been completed for all images in the image group in the HDD 104 that may be acquired by the image acquisition unit 220101. If the determination result in this step is true, the process proceeds to S2306. On the other hand, if the determination result is false, the process returns to S2301.

[0221] In S2306, the scene division unit 1707 performs scene division on the group of images acquired by the image acquisition unit 220101. S2306 is the same processing as S505.

[0222] In S2307, the scene classification unit 1708 performs scene classification on the group of sub-images divided by the scene division unit 1707. S2307 is the same processing as S506.

[0223] In S2308, the main subject setting unit 220104 sets a main subject for each album. In the setting of this step, the main subject setting unit 220104 acquires information indicating the main subject for each album based on the determination result by the edit spread determination unit 1903, and uses the acquired information.

[0224] In S2309, the image scoring unit 1710 scores each image. S2309 is the same process as S508.

[0225] In S2310, the image scoring unit 1710 determines whether the scoring in S2309 has been completed for all images analyzed by the image analysis unit 1706. If the determination result in this step is true, the process proceeds to S2311, whereas if the determination result is false, the process returns to S2309.

[0226] In S2311, the image selection parameter generation unit 220105 generates image selection parameters using information about the main subject of the edited spread received from the main subject setting unit 220104. S2311 is a process similar to S510.

[0227] In S2312, the image selection unit 220106 creates a similar spread by selecting images using the image selection parameters generated in S2311 and the results of the similar image determination in S2303.

[0228] In S2313, the album information output unit 1714 outputs the album information.

[0229] <<Effects of this embodiment>> The above is an explanation of how the editing results of a specific album are reflected in other albums in this embodiment. Specifically, in the other albums, spreads with layouts similar to the common spread edited in the specific album are created. However, this is not reflected in at least some albums. This allows the layout of each individual album to be adjusted.

[0230] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The contents of the above-described embodiments may also be combined as appropriate.

Claims

1. Computer, an editing means for editing the layout of a plurality of albums having, as attributes of a spread, an attribute of a common spread common to all the albums and an attribute of an individual spread not common to all the albums; a first determination means for determining whether an edited spread has an attribute of being a common spread for a predetermined edited album; a second determination means for determining whether or not to reflect the content of the editing result for the predetermined album for each of the albums excluding the predetermined album among the plurality of albums when the determination result of the first determination means is true; a reflecting means for reflecting the content of the editing result of the predetermined album to an album determined to reflect the content of the editing result of the predetermined album among the albums excluding the predetermined album from the plurality of albums; It functions as Control is performed so that the editing results in the specified album are reflected in the first album, based on the fact that by reflecting the content of the editing results in the specified album in a first album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the first album will be higher, and control is performed so that the editing results in the specified album are not reflected in the second album, based on the fact that by reflecting the content of the editing results in the specified album in a second album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the second album will be lower. A program characterized by:

2. The computer further functioning as an album setting means for setting which album the content should be reflected in; the second determination means makes a determination based on the setting by the album setting means; 2. The program according to claim 1 .

3. The albums whose contents are reflected by the reflecting means are not all of the albums among the plurality of albums except for the predetermined album, 3. The program according to claim 2.

4. The computer and further functioning as a third determination means for determining whether the editing has been performed for each of the plurality of albums before the determination by the first determination means.

4. The program according to claim 1, wherein the program is a program for executing a program for executing a program for executing a program.

5. The determination by the first determination means is made for an album for which the determination result by the third determination means is true.

5. The program according to claim 4.

6. The computer and further functioning as a parameter generating means for generating an image selection parameter for selecting an image similar to an image used in a common spread added to the predetermined album.

6. The program according to claim 1, wherein the program is a program for executing a program for executing a program for executing a program.

7. The computer It also functions as a setting tool for setting the main subject for each album, the setting means sets two main subjects of the common spread when editing the album: a main subject of the individual spread when creating the album, and a main subject of the common spread when creating the album; 7. The program according to claim 6.

8. the parameter generating means generates the image selection parameters based on the main subject set by the setting means.

8. The program according to claim 7,

9. The plurality of albums edited by the editing means are albums automatically generated for each user.

9. The program according to claim 1, wherein the program is a program for executing a program for executing a program for executing a program.

10. an editing means for editing the layout of a plurality of albums having, as attributes of a spread, an attribute of a common spread common to all the albums and an attribute of an individual spread not common to all the albums; a first determination means for determining whether an edited spread has an attribute of being a common spread for a predetermined edited album; a second determination means for determining whether or not to reflect the content of the editing result for the predetermined album for each of the albums excluding the predetermined album among the plurality of albums when the determination result of the first determination means is true; a reflecting means for reflecting the content of the editing result of the predetermined album to an album determined to reflect the content of the editing result of the predetermined album among the albums excluding the predetermined album from the plurality of albums; and Control is performed so that the editing results in the specified album are reflected in the first album, based on the fact that by reflecting the content of the editing results in the specified album in a first album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the first album will be higher, and control is performed so that the editing results in the specified album are not reflected in the second album, based on the fact that by reflecting the content of the editing results in the specified album in a second album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the second album will be lower.

1. An image processing device comprising:

11. editing the layout of a plurality of albums having, as attributes of spreads, an attribute of common spreads common to all the albums and an attribute of individual spreads not common to all the albums; a first determination step of determining whether the edited spread has an attribute of being a common spread for a predetermined album in which editing has been performed; a second determination step of determining whether or not to reflect the content of the editing result for the predetermined album for each of the albums excluding the predetermined album among the plurality of albums when the determination result of the first determination step is true; a reflecting step of reflecting the content of the editing result of the predetermined album in an album determined to reflect the content of the editing result of the predetermined album among the plurality of albums excluding the predetermined album; and Control is performed so that the editing results in the specified album are reflected in the first album, based on the fact that by reflecting the content of the editing results in the specified album in a first album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the first album will be higher, and control is performed so that the editing results in the specified album are not reflected in the second album, based on the fact that by reflecting the content of the editing results in the specified album in a second album among the multiple albums that is not the specified album, the score based on the main subject corresponding to the second album will be lower. An image processing method comprising:

Images