PROGRAM, IMAGE PROCESSING APPARATUS, AND IMAGE PROCESSING METHOD

Abstract

To determine a reference for creating a double spread to be added and perform an automatic creation and addition of the double spread, when the addition of the double spread is performed in editing of a plurality of albums.SOLUTION: A program allows a computer to function as editing means for editing layouts of a plurality of albums having as an attribute of a double spread: an attribute of a common double spread that is common between individual albums; and an attribute of an individual double spread for each of the albums. The editing means automatically creates the double spread with a reference in accordance with the attribute and adds the created double spread to one album of the plurality of albums.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The present invention relates to a technique for creating an album and editing the created album. [Background technology]

[0002] The number of photos taken by users is rapidly increasing due to the widespread use of digital cameras, the widespread use of smart devices, and improvements in the camera performance of smart devices. Patent Document 1 proposes a method for multiple users to jointly create an album.

[0003] It is also known that when creating multiple albums, album data is automatically generated that includes a double-page spread created according to general standards common to multiple albums (referred to as a common double-page spread) and a double-page spread created according to individual standards specialized for each album (referred to as an individual double-page spread). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-66328 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when adding a double-page spread in editing a plurality of albums that have already been created, unless a criterion for creating the additional double-page spread is determined, the double-page spread cannot be automatically created and added.

[0006] In view of the above problems, one embodiment of the present invention aims to automatically create and add a double-page spread by determining criteria for creating the additional double-page spread when adding a double-page spread when editing multiple albums. [Means for solving the problem]

[0007] In one embodiment of the present invention, a computer is configured as an editing unit for editing the layout of a plurality of albums having a common spread attribute common to each album and an individual spread attribute of each album as spread attributes. a position determining means for determining a position where a double-page spread is to be added; The editing means automatically creates a spread based on a criterion corresponding to the attribute, and adds the created spread to one of the plurality of albums. automatically creating a spread that is identical or similar to the spread added to the one album, and adding the created identical or similar spread to each of one or more albums among the plurality of albums except for the one album, wherein the position where the spread is added in the one album is the same as the position where the spread is added in the one or more albums. The program is characterized by: Effect of the Invention

[0008] According to one embodiment of the present invention, when adding a double-page spread in editing a plurality of albums, a criterion for creating the additional double-page spread can be determined, and the double-page spread can be automatically created and added. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram of hardware capable of executing an application according to the present invention. [Diagram 2] FIG. 2 is a software block diagram of an album creation application. [Diagram 3] FIG. 2 is a diagram illustrating an image layout. [Figure 4] FIG. 2 is a diagram of a display screen displayed by an application. [Diagram 5] FIG. 13 is a process flow diagram of automatic layout. [Figure 6] FIG. 13 is a diagram illustrating image group division. [Figure 7] FIG. 1 is a diagram illustrating scene classification. [Figure 8] FIG. 13 is a diagram illustrating image selection. [Figure 9] FIG. 2 is a block diagram of template selection. [Figure 10] FIG. 11 is a process flow diagram of template selection. [Figure 11] FIG. 2 is a diagram illustrating an image layout. [Figure 12] FIG. 1 is a block diagram of a double-page spread binding. [Figure 13] FIG. 13 is a process flow diagram for merging double-page spreads. [Figure 14] FIG. 2 is a software block diagram of an album editing application. [Figure 15A] FIG. 13 is a diagram illustrating an example of an album editing screen. [Figure 15B] FIG. 13 is a diagram illustrating an example of an album editing screen. [Figure 16] FIG. 13 is a diagram showing an example of adding a double-page spread. [Figure 17] 2 is a block diagram showing details of an album editing section. FIG. [Figure 18] FIG. 13 shows a double-page spread adding UI. [Figure 19] FIG. 2 is a software block diagram of an album editing application. [Figure 20] FIG. 11 is a flow diagram of a process for executing a double-page spread addition. [Figure 21] FIG. 13 is a flow diagram showing a process of reflecting an editing result of a given album in another album. [Figure 22] FIG. 2 is a software block diagram of an album editing application. [Figure 23] FIG. 13 is a flow diagram showing a process of reflecting an editing result of a given album in another album. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0011] (Embodiment 1) In this embodiment, an application for creating an album (hereinafter, also referred to as "application" or simply "app") is operated in an image processing device, and album data in which multiple photo images are automatically laid out is generated on the application. By printing this album data, multiple albums can be created as printed matter. In this embodiment, a spread common to at least two albums and an individual spread for each album are created. Each album has an individual spread with a different configuration for each subject (referred to as a main subject) that is particularly emphasized in the layout, for example. The total number of spreads constituting each album (referred to as the total number of spreads) is the same for all albums, according to the designation of the user operating the app. However, the respective numbers of common spreads and individual spreads for each album change depending on the image group and settings. The spreads and the images arranged within the spreads are arranged in order of shooting time.

[0012] By using individual spreads, the multiple albums created are albums that are partially specialized for each main subject. In other words, it is assumed that the multiple albums created in this embodiment will not be completely identical albums in which the same images are arranged in the same layout on all spreads. As a result, some of the multiple albums created may be completely identical albums.

[0013] In addition, in this embodiment, the user can edit the album data of an album that is created to include a common spread and an individual spread. In the following description, the process is broadly divided into a process for automatically generating album data and a process for a user to edit the generated album data. The process for a user to edit is also performed automatically. First, the configuration of the process for automatically generating album data will be described. After that, the process for a user to edit the generated album data will be described.

[0014] <<Album data generation process>> <Hardware configuration of image processing device> Hereinafter, the hardware configuration of the image processing device in this embodiment will be described with reference to Fig. 1. Note that, although the following describes a case where the image processing device is an information processing device (PC), other devices such as a smartphone may also be adopted as the image processing device.

[0015] 1, the image processing device 100 includes a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display 105, a keyboard 106, a mouse 107, and a data communication device 108. These components are connected by a data bus 109, and can transmit and receive data between each other.

[0016] The CPU 101 controls the entire image processing device 100. The CPU 101 also executes the image processing method described in this embodiment according to a program. Note that, although the image processing device in FIG. 1 has one CPU, the image processing device may have multiple CPUs.

[0017] The ROM 102 stores programs executed by the CPU 101. The RAM 103 provides a memory for temporarily storing various information when the CPU 101 executes the programs. The HDD 104 stores a database that holds image files or processing results such as image analysis, and in this embodiment, the HDD 104 stores an application program for creating an album. This application program is sometimes called an album creation application, and will be described later with reference to FIG. 2 and the like.

[0018] The display 105 is a device for displaying the user interface (hereinafter referred to as UI) of this embodiment or the layout results of images and presenting them to the user. The display 105 may have a touch sensor function. The keyboard 106 is one of the input devices possessed by the image processing device 100, and is used, for example, to input predetermined information on the GUI displayed on the display 105. In this embodiment, the user inputs the number of double pages of the album via the keyboard 106. The mouse 107 is one of the input devices possessed by the image processing device 100, and is used, for example, to click and press a button on the GUI displayed on the display 105.

[0019] The data communication device 108 is a device for communicating with an external device such as a printer or a server. For example, album data resulting from automatic layout is transmitted to a printer or a server connected to the image processing device 100 via the data communication device 108. The data bus 109 connects the above-mentioned components with the CPU 101. The above is the content of the hardware configuration of the image processing device in this embodiment.

[0020] <About the software configuration of the image processing device> The software configuration of the image processing device in this embodiment, in other words, the functional configuration realized by an album creation application installed in the image processing device, will be described below with reference to Fig. 2. The album creation application is started when the user double-clicks the application icon displayed on the display 105 using the mouse 107. The album creation application has various functions, but the following mainly describes the automatic layout function provided by the automatic layout processing unit 202.

[0021] 2, this application has an album creation condition setting section 201, an automatic layout processing section 202, an album display section 214, and an album information storage section 215. The "automatic layout function" is a function that classifies and selects images of photographs taken based on their contents and attributes, and then lays them out to generate album data. The generated album data is displayed on the display 105.

[0022] The album creation condition setting unit 201 sets the album creation conditions in the automatic layout processing unit 202 according to the user's mouse operation. In this embodiment, the album creation conditions include a group of images to be used in the album, the total number of double-page spreads per album, the commercial material of the album, the number of albums, and the main subject of each album. The image group may be set using, for example, accompanying information or attribute information of individual image data such as the shooting date and time, or may be set based on the structure of the file system in which the image data is stored, such as the designation of a device or directory. This image group may be a storage destination that stores image data from each user who can be each main subject, or may be a storage destination that stores image data collected by a specific user. Note that a "spread" corresponds to, for example, one display window in a display, and corresponds to a pair of adjacent pages (i.e., two pages) that a user can view at once when opening a book in a printed matter. Note that the two pages in a spread may be formed by binding pages printed on different sheets of paper adjacent to each other, or by folding a single sheet of paper in the middle to form a spread. Here, the explanation will be given using the concept of a spread, but pages may also be used. The main subject is a subject that is laid out with particular emphasis in the album. Examples include a specific individual, a category of objects such as a dog, a cat, a flower, or food, or the color of an object or the entire image. For the sake of explanation, a person will be used as the main subject here.

[0023] Here, the album created in this embodiment will be described with reference to Fig. 3. In this embodiment, albums of different variations are created based on the number of albums and the main subject of each album set by the album creation condition setting unit 201. In the example of Fig. 3, album 1, album 2, and album 3 are created as albums of different variations.

[0024] Each album is composed of a "common spread" in which the same images are laid out in the same arrangement across multiple different variations of albums, and "individual spreads" in which images or arrangements are different from those in other albums. In the example of Fig. 3, albums 1 to 3 each include a common spread 302. Meanwhile, album 1 includes an individual spread 303, album 2 includes an individual spread 304, and album 3 includes an individual spread 305.

[0025] Image 301 is an image arranged on a spread, and characters in image 301 represent the main subject appearing in the image. As shown in FIG. 3, common spread 302 includes an image showing multiple main subjects. Meanwhile, individual spread 303 includes an image showing only main subject A, and so on, and individual spreads 303-305 each include an image showing only a specific main subject. In this way, when multiple albums are created in this embodiment, the multiple albums each include spreads created according to different standards. That is, by creating the albums according to different standards, the resulting spreads are also different.

[0026] The automatic layout processing unit 202 has an image acquisition unit 203, an image analysis unit 204, a main subject setting unit 205, a scene division unit 206, a scene classification unit 207, an image scoring unit 208, a spread creation parameter generation unit 209, and a spread creation unit 210. It also has a spread combination unit 211, a layout information output unit 212, and an album information output unit 213.

[0027] The image acquisition unit 203 acquires a group of images that satisfy the album creation conditions set by the album creation condition setting unit 201 from among the images stored in the HDD 104. The image group here refers to a group of images that become layout candidates when creating an album. For example, when January 1, XX year to December 31, XX year are specified as the shooting date and time, all images taken from January 1, XX year to December 31, XX year correspond to the group of images of the layout candidates. The images stored in the HDD 104 include still images, cut-out images cut out from a video, and the like. The still images or cut-out images are acquired from an imaging device such as a digital camera or a smart device. The imaging device may be included in the image processing device 100, or may be included in an external device of the image processing device 100. In addition, when the imaging device is an external device, the image acquisition unit 203 acquires images via the data communication device 108. In addition, the still images or cut-out images may be images acquired from an external network or server via the data communication device 108. An example of an image acquired from a network or server is a social networking service image (hereinafter, SNS image). By executing an OS program, CPU 101 analyzes data accompanying image data for each image and determines the storage source from which the image is acquired. However, by acquiring an image from an SNS via an application, the acquisition destination of the image may be managed within the application. The images acquired by image acquisition unit 203 are not limited to those described above, and may be other types of images.

[0028] The image analysis unit 204 analyzes the image data acquired by the image acquisition unit 203. In this embodiment, image features are derived, and object determination within the image, face detection, facial expression recognition of the detected face, and personal recognition of the detected face are performed. Furthermore, the image analysis unit 204 refers to data (e.g., Exif information) attached to the image data acquired from the HDD 104 to acquire information on the shooting date and time. Note that information obtained as a result of the image analysis unit 204 analyzing the image data is referred to as "analysis information."

[0029] Main subject setting section 205 sets a main subject to be emphasized for each album. In this embodiment, main subject setting section 205 acquires information on a main subject to be emphasized for each album sent from album creation condition setting section 201.

[0030] The image scoring section 208 scores each image based on the album creation conditions set by the album creation condition setting section 201 and the analysis information derived by the image analysis section 204. In this embodiment, the image is scored so that an image suitable for the layout will have a high score. When scoring so that an image suitable for the layout will have a high score, image analysis information, image classification results, image information, album creation conditions, etc. are used. However, other information may be used additionally or alternatively. An image suitable for the layout may be, for example, an image with high aesthetic quality such as high contrast or sharp edges, or an image in which a subject (so-called main subject) that is the main subject, such as a person, animal, or building, is shown large. Furthermore, the image scoring section 208 also scores each main subject set by the main subject setting section 205. For example, an image with very high aesthetic quality but without the set main subject is given a low score, whereas an image in poor shooting conditions such as backlight is given a high score in which the set main subject is shown large.

[0031] The scene division unit 206 divides the images acquired by the image acquisition unit 203 into scenes by using the analysis information derived by the image analysis unit 204. Here, a "scene" refers to a collection of images (sub-images) grouped according to the analysis information. For example, with respect to two images captured at different times, the scene division unit 206 determines that images whose capture times differ by less than a predetermined threshold are from the same scene, or determines that images that include the same people or background are from the same scene.

[0032] The scene classification unit 207 determines a category for each scene obtained by the scene division unit 206. Examples of scene categories include situations such as indoors or outdoors, or events at the time the image was taken, such as travel or a wedding.

[0033] The spread creation parameter generation unit 209 generates a plurality of parameters for creating a spread based on the main subject set by the main subject setting unit 205 and the analysis information derived by the image analysis unit 204. The plurality of parameters includes, for example, a parameter for a common spread and a parameter for an individual spread. The parameters here are set values ​​that determine the creation criteria for the spread. For example, such parameters include a weight or a threshold value that determines the priority of the score obtained by the image score unit 208, which is used when selecting an image to be used in the spread. This is a set value that selects whether to prioritize an image in which only a specific main subject is shown, or an image in which two or more main subjects are shown, among images in which the main subject set by the main subject setting unit 205 is shown. By using a plurality of parameters, the creation criteria change depending on the spread for each album. The spread creation parameters also include a parameter that determines a score that takes into account the relationship within the spread when the image is arranged in the spread.

[0034] Here, for convenience, a parameter for calculating the score of an image alone when selecting an image to be used in a spread is referred to as a first parameter, and a parameter for calculating a score taking into consideration the relationship when the image is arranged in the spread is referred to as a second parameter. In this case, the spread creation unit 210 described later first selects candidate images to be used in the spread according to the first parameter. Then, the spread creation unit 210 determines images and templates suitable for the spread from among the candidate images according to the second parameter. Here, for convenience, the parameters are divided into the first parameter and the second parameter, but the parameters that take these into consideration comprehensively are collectively referred to as the spread creation parameters.

[0035] The spread creation unit 210 creates spread data by selecting and arranging images to be used in the spread according to the parameters generated by the spread creation parameter generation unit 209. For example, the spread creation unit 210 selects candidate images according to the first parameters. As described above, the first parameters are parameters that change the weight for each set main subject, and are also parameters that give weights according to a predetermined priority. As an example of the priority, a priority according to the presence or absence of trimming may be considered. For example, since the main subjects in a common spread are often multiple people, an image of a size that does not require trimming may be prioritized so that the main subject is not omitted. Since the main subject in an individual spread is one person, it is considered that trimming may be performed so that the main subject is close up, so there is no need to lower the priority of the image to be trimmed.

[0036] Such priorities may be determined in advance or may be appropriately set by the user. By preparing weights or thresholds according to priorities, the user can change the parameters by changing the priorities of various pieces of information. Furthermore, such priorities may be priorities that take into consideration various pieces of information in addition to the above-mentioned information. Examples of the various pieces of information include information on the scene of the image, information on the number of objects other than the main subject, the number of people other than the main subject, resolution, or background.

[0037] The spread creation unit 210 then applies a second parameter to the scores of the candidate images thus determined, and creates a spread that will have a higher total score for the images to be arranged in the spread. The total score for the images to be arranged in the spread is called the spread score. The second parameter is a parameter that takes into consideration the relationship within the spread, and can, for example, prioritize the use of images at a certain time interval within the same spread, or assign weights according to the slot size.

[0038] As described above, the spread creation parameters (first and second parameters) are different for common spreads and individual spreads. This allows image selection to be performed based on different spread criteria, resulting in different spreads being created for common spreads and individual spreads.

[0039] The spread combining unit 211 combines the spread data created by the spread creating unit 210. Specifically, the spread combining unit 211 determines the order of the spreads created by the spread creating unit 210, thereby creating the layout of the entire album.

[0040] The layout information output unit 212 outputs the album data according to the layout of the entire album created by the spread combining unit 211. The album data is, for example, image data in a bitmap format in which each image is arranged according to a predetermined layout.

[0041] The album information output unit 213 outputs album information used in generating the album. The album information includes, for example, album configuration information and album generation information. The album configuration information is configuration information that configures an album, such as used image information, image arrangement information, template information, slot shape and size information, background color information, trimming information, text or stamp editing information, and common / individual page information. The album generation information is generation information used when generating album data, such as album creation conditions, image analysis information, score information, or scene information. Note that when an album is generated using this album generation information, the same album can be generated. The album information may include the album data itself. This information is output on an album-by-album basis.

[0042] The album display section 214 performs display control for displaying the bitmap data output by the layout information output section 212 on the display 105 .

[0043] The album information holding unit 215 receives and holds the album information output by the album information output unit 213. When the album is updated due to album editing or the like, the album information holding unit receives and holds the album information adjusted to the update. The album information here includes, in addition to the album configuration information and the album generation information described above, album change information. Examples of the album change information may be the user operation history of album editing or the difference information changed by album editing. These pieces of information are held in units of albums.

[0044] When the program of the album creation application in the present embodiment is installed in the image processing apparatus 100, a startup icon of this application is displayed on the top screen (desktop) of the OS (operating system) operating on the image processing apparatus 100. When the user double-clicks the startup icon on the desktop displayed on the display 105 with the mouse 107, the program of the album creation application stored in the HDD 104 is loaded into the RAM 103. Then, the program loaded into the RAM 103 is executed by the CPU 101, and the album creation application is started. The above is the content regarding the software configuration of the image processing apparatus in the present embodiment. Note that the application may be in another form, for example, a web application that displays a screen or the like within a browser operating on the image processing apparatus 100.

[0045] <Regarding the GUI screen> Hereinafter, the GUI screen of the album creation application in the present embodiment will be described with reference to FIG. 4. FIG. 4 is a diagram showing a GUI screen 401 provided by the started album creation application and displayed on the display 105. The user can set the creation conditions of the album via the GUI screen 401.

[0046] The GUI screen 401 has a path box 402 and a folder selection button 403 as setting sections for photographic images to be included in the album. The path box 402 is a box for indicating the storage location (path) in HDD 104 of the group of images to be the subject of album creation. The folder selection button 403 is a button for selecting a folder that contains the group of images to be created in the album. When the user clicks the folder selection button 403 with the mouse 107, a tree made up of multiple folders is displayed. Then, when the user selects a folder that contains the group of images to be created in the album, the folder path of the selected folder is displayed in the path box 402.

[0047] The number of spreads box 404 is a box for specifying the total number of spreads in one album. The user can directly input a number into the number of spreads box 404 using the keyboard 106, or can input a number into the number of spreads box 404 from a list using the mouse 107. The number of albums box 410 is a box for specifying the number of albums to be created.

[0048] Main subject designation section 405 is an element for designating a main subject to be emphasized for each of the albums of the number inputted in album number box 410. For example, the user can designate a main subject by using mouse 107 to drag and drop an image in which the main subject appears into main subject designation section 405. For example, Mr. A, Mr. B, etc., mentioned above in FIG. 3, are designated.

[0049] The spread photo number setting section 406 is a slider bar for setting the number of images to be arranged on each spread of the album to be created. When the slider is moved to the "More" side, the number of images to be arranged on each spread increases. On the other hand, when the slider is moved to the "Less" side, the number of images to be arranged on each spread decreases.

[0050] The product designation section 407 is an element for setting the product of the album to be created. As specific items related to the product to be set, the size of the album or the type of paper for the album can be adopted. The type of cover or the type of binding section can also be set.

[0051] The OK button 408 is a button for deciding the selected conditions as the album creation conditions. When the user clicks the OK button 408 with the mouse 107, the album creation conditions are finalized, and the album creation conditions are communicated to the automatic layout processing unit 202 via the album creation condition setting unit 201. To be more specific, path information input in the path box 402 is communicated to the image acquisition unit 203. Furthermore, the value of the total number of spreads input in the number of spreads box 404 is communicated to the spread creation unit 210. The value of the number of albums input in the album number box 410 and information on the main subject specified in the main subject designation unit 405 are communicated to the spread creation parameter generation unit 209.

[0052] A reset button 409 is a button for resetting each setting content on the display screen. The above is the content regarding the GUI screen of the album creating application in this embodiment.

[0053] <About automatic layout processing> The automatic layout process in this embodiment will be described below with reference to Fig. 5(a). Fig. 5(a) is a flowchart of a process for executing the automatic layout of the album creation application according to this embodiment. The flowcharts shown in Figs. 5(a) to (c) are realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or the RAM 103 and executing it.

[0054] In step S501, the album creation condition setting unit 201 sets album creation conditions. In the following, "step S~" is simply abbreviated as "S~". In this embodiment, as the album creation conditions, the image group to be used in the album, the number of spreads, the number of albums, the main subject of each album, the number of images to be arranged on each spread, and the commercial material for creating the album are set (see FIG. 4).

[0055] In S502, the image acquisition unit 203 reads images that satisfy the conditions related to the image group to be used in the album set in S501 from the HDD 104. Then, the image acquisition unit 203 develops the read images in the RAM 103.

[0056] In S503, the image analysis unit 204 analyzes the image expanded in S502 in the RAM 103. Here, the image analysis in this step will be described with reference to FIG.

[0057] In S50301, the image analysis unit 204 acquires information on the shooting date and time corresponding to the image data acquired by the image acquisition unit 203. In this embodiment, the information on the shooting date and time is acquired based on the Exif information accompanying each image data.

[0058] In S50302, the image analysis unit 204 derives image quality features for the image data acquired by the image acquisition unit 203. For example, the image quality features include the amount of focus. An edge detection method can be used as a method for determining the amount of focus, and a known Sobel filter can be used as the edge detection method. The edge of the image is detected by the Sobel filter, and the brightness difference between the start point and the end point of the edge is divided by the distance between the start point and the end point to calculate the brightness gradient, that is, the slope of the edge. The average slope of the edges in the image is calculated, and an image with a large average slope can be considered to be in focus better than an image with a small average slope. In this embodiment, multiple thresholds are set to measure the magnitude of the calculated average slope of the edges, and the calculated slope of the edges is determined to be equal to or greater than a threshold to determine whether the amount of focus is acceptable. Specifically, a first slope threshold and a second slope threshold (note that the first slope threshold>the second slope threshold) are set as two different slope thresholds, and the amount of focus is determined in three stages of ○, △, and ×. If the average gradient of the edges in the image is equal to or greater than the first threshold, the amount of focus is deemed to be suitable (denoted by a circle), if the average gradient of the edges in the image is less than the first threshold and equal to or greater than the second threshold, the amount of focus is deemed to be acceptable (denoted by a triangle), and if the average gradient of the edges in the image is less than the second threshold, the amount of focus is deemed to be unacceptable (denoted by an x).

[0059] In S50303, the image analysis unit 204 performs object detection and classification of the detected object on the image data acquired by the image acquisition unit 203. In this embodiment, a face is detected as an object. Any known method can be adopted as a face detection method, and one such known method is AdaBoost, which creates a strong classifier from a plurality of weak classifiers prepared. In this embodiment, a human face is detected using a strong classifier created by AdaBoost. In S50303, a face is detected and the top left coordinate value and the bottom right coordinate value of the area of ​​the detected face in the image are obtained. By obtaining these two coordinate values, the position and size of the face can be specified. Furthermore, by simultaneously executing AdaBoost, which detects animals such as dogs or cats and food, as well as human faces, it is possible to detect objects of people, animals, and food and at the same time classify what the objects in the image are. Note that the objects to be detected are not limited to those mentioned above, and may be flowers, buildings, ornaments, etc. Also, although a case has been described here in which objects are detected (classified) using AdaBoost, object detection (classification) may also be performed using a trained model such as a neural network.

[0060] In S50304, the image analysis unit 204 performs personal recognition on the face detected by the image analysis unit 204. First, the similarity between the extracted face image and the representative face image stored in the face dictionary database for each personal ID is derived. Then, the derived similarity is equal to or greater than a predetermined threshold and the personal ID with the highest similarity is determined as the ID corresponding to the extracted face image. In other words, the person corresponding to the personal ID with the similarity equal to or greater than a predetermined threshold and the highest similarity is specified as the person of the extracted face image. The image analysis unit 204 acquires the similarity between each of the representative face images and the face image in the image to be processed. Note that if all of the similarities derived for each personal ID are less than a predetermined threshold, the person of the extracted face image is regarded as a new person, and a new personal ID is assigned and registered in the face dictionary database. The face dictionary database is stored in, for example, the HDD 104.

[0061] Returning to the explanation of Fig. 5(a), in S504, the image analysis unit 204 judges whether the processes from S502 to S503 have been completed for all images in the image group that satisfies the condition set in S501. If the judgment result in this step is true, the process proceeds to S505. On the other hand, if the judgment result is false, the process returns to S502.

[0062] In S505, the scene splitting unit 206 executes scene splitting on the image group acquired by the image acquisition unit 203. Scene splitting refers to dividing the acquired image group into a plurality of sub-image groups using image information. Specifically, the image group is split into a plurality of groups based on the time difference between the shooting dates and times of the images, using the information on the shooting date and time already acquired in S50301 as image information. An example of actual splitting is as follows.

[0063] Among the image group, the image with the oldest (or newest) shooting time is first focused on, and the time difference between the image with the next oldest (or newest) shooting time is calculated, and it is determined whether the calculated time difference is equal to or greater than a predetermined threshold. This process is performed for all images while sequentially replacing the image of interest with the image with the newer (or older) shooting time. Note that "division" in this embodiment means dividing the image group into the newer and older images with the boundary between the two images. In this embodiment, if the difference between the shooting time of the image of interest and the shooting time of the next oldest (or newest) image is 16 hours or more, the image group is divided so that these images belong to different sub-image groups. Note that the threshold value of the time difference used in the division is not limited to this. FIG. 6 shows the result of dividing the image group using the above-mentioned scene division method.

[0064] In this embodiment, the scene division is performed using the information on the shooting date and time, but the information used is not limited to this. For example, the image group may be divided using the information on the shooting location so that images taken at close locations belong to the same scene. As another example, the result of personal recognition may be used. For example, in a graduation album, images of students belonging to a certain group (class or club activities, etc.) may be registered in the application in advance, and a group of images in which the students belonging to the group appear may be treated as one scene (sub-image group). As another example, information other than image information may be used. Here, a method of grouping similar events into one scene will be described. The name of the folder in which the images are saved, or tag information added by a social networking service, etc., is acquired as meta information associated with the images. For example, a group of images having meta information including a search word such as "athletic meet" or "school trip" is treated as one scene. Regarding the designation of the search word, the user may select from among search words pre-installed in the application via the album creation condition setting unit 201, or may input the search word in a text box.

[0065] In S506, the scene classification unit 207 performs scene classification for each of the sub-image groups obtained in S505. Note that, in the following, an example will be described in which each sub-image group (each scene) is classified into one of three categories, "travel," "daily life," and "ceremony," but the items to be classified are not limited to these.

[0066] First, a plurality of sub-image groups that are previously determined to be classified into one of travel, daily life, and ceremony are acquired, and the shooting feature amount is acquired for each of the acquired plurality of sub-image groups. The shooting feature amount acquired here is, for example, the shooting period, the number of shots, and the number of people photographed. The shooting period is the time difference between the shooting time of the oldest image and the shooting time of the newest image among the images included in the sub-image group. The number of shots is the number of images included in the sub-image group (i.e., the number of photos). The number of people photographed is the number of faces in an image that shows a face, that is, the number of faces included in one image. Then, for the plurality of sub-image groups, the average value and standard deviation of the shooting period, the average value and standard deviation of the number of shots, and the average value and standard deviation of the number of people per image are obtained. In this embodiment, the number of faces per image is the number of people per image.

[0067] FIG. 7 shows an example of the average and standard deviation of the shooting period (time), the average and standard deviation of the number of shots, and the average and standard deviation of the number of people per image, which are obtained for a plurality of sub-image groups. These obtained values ​​are incorporated in advance into the program of the album creation application. That is, at the design stage of the application, parameters are generated by learning using images collected in advance, and the generated parameters are incorporated in the program. After the album creation application is started, the average values ​​of the shooting period, the number of shots, and the number of people per image are calculated for each sub-image group obtained as a result of dividing the image group specified by the user via the pass box 402 in S505. Then, for each scene, the score is calculated according to the following formula, using the parameters previously incorporated in the application (specifically, the average values ​​and standard deviations of the feature quantities of the shooting period, the number of shots, and the number of people for each sub-image group). Score for each scene and feature = 50 - |10 × (average value for each scene and feature - feature for each sub-image group) / standard deviation for each scene and feature | Equation (1) Average score per scene = (Score of shooting period per scene + Score of number of shots per scene + Score of number of people shooting per scene) / Number of feature items ... Equation (2)

[0068] Using formula (1), a score is obtained for each scene and each feature of the sub-image group of interest. For example, for a travel scene, scores are obtained for the shooting period, the number of shots, and the number of people photographed. These scores are averaged using formula (2) to obtain a score for the travel scene. Similarly, scores are obtained for other scenes, such as everyday life and ceremonies. Using the method described above, the average score for travel scenes, the average score for everyday life scenes, and the average score for ceremony scenes are calculated for each sub-image group. Note that the number of feature items in formula (2) is three.

[0069] The scene with the highest score among the scores calculated for each scene for each sub-image group is classified as the scene of that sub-image group. If two or more scenes have the same score, the scenes are classified according to a default priority. For example, in this embodiment, the order of priority is predetermined as daily life > ceremony > travel, with daily life being given the highest priority. Note that the order of priority is not limited to this, and the user may be allowed to change the order of priority.

[0070] The sub-image groups (1) to (8) in FIG. 6 show sub-image groups obtained by scene division of the image groups. For the sub-image group (5) among these sub-image groups, the shooting period is 36 hours, the number of shots is 300, and the number of people photographed is 1.7. The average score for travel is 45.32, the average score for daily life is 18.38, and the average score for ceremony is -29.92. In this case, the scene of the sub-image group (5) is classified as travel. The classified sub-image group is managed by associating it with a scene ID so that the scene can be identified. Note that, in this embodiment, the shooting period, the number of shots, and the number of people photographed are described as the feature amounts of the shooting, but the feature amounts of the shooting are not limited to these. For example, in order to identify whether it is indoors or outdoors, a flag such as whether the sky is photographed or not may be adopted as a feature amount by image segmentation using machine learning. Alternatively, the ceremony may be further classified into finer categories. For example, when classifying a scene into either a wedding ceremony or a wedding reception, the bride and groom are the two people who appear most frequently in all images using the personal recognition results. Scenes with many images showing only the bride and groom may be classified as weddings, and scenes with many images showing people other than the bride and groom may be classified as wedding receptions. Furthermore, the feature amount may be derived by machine learning. For example, scene classification may be performed by preparing a scene to be classified and a group of images representing that scene, and using a Convolution Neural Network (CNN) to learn images as input and output as scene classification results.

[0071] In S507, main subject setting section 205 sets a main subject for each album. In this embodiment, main subject setting section 205 acquires information on the main subject for each album transmitted from album creation condition setting section 201. In this embodiment, an example in which three types of albums are generally created is described, but here, for the sake of simplicity, an example in which main subject A is set in album 1 and main subject B is set in album 2 for two types of albums will be described.

[0072] In S508, the image score unit 208 assigns a score to each image. This process is called image scoring process. The "score" derived in this step is a parameter for evaluating suitability for use in an album, and a score evaluated from a viewpoint described later is assigned to each image data, and is referred to when selecting an image, described later. In this embodiment, in addition to the quality of image quality, a score is assigned for each main subject set in S507. First, a score is assigned according to the feature of image quality derived in S50302. Here, if the focus amount is ○ (suitable), 1 point is assigned, if it is △ (acceptable), 0.5 points is assigned, and if it is × (unacceptable), 0 points is assigned. This allows the score of an image that is in focus to be increased. Note that in this embodiment, the focus amount is adopted as the feature of image quality, but the feature of image quality is not limited to this. For example, the image size may be used, or shooting information such as lens information used at the time of shooting may be used, or the compression format of the image input to the application may be used.

[0073] After giving a score according to the feature of the image quality, the image scoring unit 208 gives a score for each set main subject. That is, in this example, three types of scores are given to one image, specifically, scores evaluated from three viewpoints: whether the image quality is good, whether the main subject A is included, and whether the main subject B is included. The face size derived in S50303 and the individual recognition result in S50304 can be used for the scoring of each main subject. If the main subject is not included in the image, a score of 0 is given, and if the main subject is included, the ratio of the face of the individual set as the main subject to the image size is set as the score for each main subject. Note that in this embodiment, the scoring of each main subject is performed using the face size, but something other than the face size may be used. For example, the facial expression of the person who is the main subject may be determined, and if the person is smiling, a score may be added. Also, if a person other than a person is set as the main subject, the scoring is performed similarly according to the object size by using the results of object detection and classification in S50303. As a result of the above processing, for example, a score for image quality, a score for main subject A, and a score for main subject B are calculated for the first image. For example, for an image that shows main subject A but not main subject B, scores are calculated for each aspect, such as 1 point for image quality, 1 point for main subject A, and 0 point for main subject B.

[0074] In S509, the image scoring unit 208 determines whether the scoring in S508 has been completed for all images in the image group acquired by the image acquisition unit 203. If the determination result in this step is true, the process proceeds to S510, whereas if the determination result is false, the process returns to S508.

[0075] In S510, the spread creation parameter generating unit 209 receives the album creation conditions from the album creation condition setting unit 201 and also receives information on the main subject for each album from the main subject setting unit 205 to generate spread creation parameters. As described above, the spread creation parameters are set values ​​used when selecting and laying out images that compose a spread. For example, when selecting an image to be used in a spread based on the scores given to the images in S508, a weight that controls which score is to be prioritized between the score according to image quality and the score of each main subject, or a threshold value that is a criterion for image selection, is generated. The spread creation parameter generating unit 209 generates, as the spread creation parameters, "common parameters" that are common to all albums and "individual parameters" that are different for each album. That is, the spread creation parameter generating unit 209 generates (the number of albums + 1) spread creation parameters. This makes it possible to generate a "common spread" in which the images and arrangement that compose the spread are the same appear in multiple albums, and an "individual spread" in which the images and arrangement that compose the spread appear only in a specific album. Then, using such spread creation parameters (first parameters), the spread creation unit 210 finds a score representing the image from the scores (three types of scores in this example) assigned to the image in S509. That is, for one image, scores are found for each of the spread creation parameters. Then, using the scores thus found, an image to be placed on the spread is selected. For example, when creating a spread, for a certain image X, the score obtained using the common parameters will be different from the score obtained using the individual parameters. Below, the spread creation parameters and an example of deriving the scores of images when creating a spread using the spread creation parameters will be described.

[0076] In this embodiment, the common parameters are weighted so that the scores given to the images are evaluated equally. i The score representing an image I when using I score S is expressed by the following equation (3).

[0077]

number

[0078] Where: I score i is the score given to image I by the ith score, and w i represents the weight for each score. In this embodiment, w i =1 / 3, the image quality score, main subject A score, and main subject B score are evaluated equally. This gives priority to images with good image quality and high scores that include many people. Here, the image quality score reflects the score according to priority, such as the presence or absence of cropping, as described above.

[0079] On the other hand, for the individual parameters, weights are set to give priority to images that include the main subject for each album. o The score representing an image I when using I score I is expressed by the following equation (4).

[0080]

number

[0081] Where: I score q is the quality score given to image I, I score o is the score of the main subject O given to image I, and w q is the weight for the image quality score, w o represents the weight for the main subject O. In this embodiment, w q =w o By setting =1, only images that include the main subject are evaluated, and the image to be prioritized is determined based on the image quality and the size of the main subject. In this embodiment, the weighting of the score is used as the spread creation parameter, but the spread parameters are not limited to this. I score S andI score I In both cases, the score is calculated so that it is high when the main subject is included, but this is not limited to the above. For an image that does not include a main subject at all, such as a landscape, the score may be calculated based on a separate image quality score or image information. For example, for an image that is shot in a short time different from an image with a high score, the image quality score may be used as the representative score. This allows images that are likely to be related to images that include a main subject to be given a high score. Furthermore, the score may be calculated by specifying these relationships as weights.

[0082] In S511, the spread creation unit 210 creates the spread data of the album using the parameters generated in S510. The spread creation unit 210 creates the spread data so that the score of the spread is high. As described above, the score of the spread is the sum of the scores of each image in the spread. That is, it is the sum of the scores obtained by multiplying the scores of each image obtained according to the first parameter by the second parameter. As described above, the score of the spread is, for example, the sum of the results of multiplying the scores of the images by a weight according to the size of the slot in which the image is placed. In this example, first, the candidate images are determined, and then the layout including the slot and the placement is performed. As a result, a spread with a high score as a spread is created. Here, the spread creation process of this step will be described with reference to FIG. 2(b) and FIG. 5(c). FIG. 2(b) is a detailed block diagram of the spread creation unit 210. FIG. 5(c) is a detailed flowchart of the spread creation in S511.

[0083] In S51101, the spread creation parameter setting unit 21001 reads one of the parameters generated in S510. In this embodiment, the individual parameters of album 1, the individual parameters of album 2, and the common parameters are read in this order.

[0084] In S51102, the spread candidate image acquisition unit 21002 acquires candidate images to be used for the spread from among all the images acquired by the image acquisition unit 203, based on the scores assigned in S508 and the spread creation parameters set in S51101. In this embodiment, based on the scores calculated in S508 and the spread creation parameters set in S51101, as described above, scores are assigned again to each image according to formula (3) or formula (4) (image re-scoring). Then, an image with a score obtained as a result of the re-scoring that is equal to or greater than a predetermined threshold is acquired as a candidate image. That is, in the first S51102 in this example, each image is re-scoring when the individual parameters of the album 1 are applied. Then, an image with a score equal to or greater than a predetermined threshold when the individual parameters of the album 1 are applied is acquired as a candidate image. If there is no candidate image, the image with the highest score is set as the candidate image. Here, the candidate images are acquired from the entire image group, but this is not limited to this. Images already selected in the image selection process described later may be excluded.

[0085] In S51103, the number of spreads setting unit 21003 determines whether the spread creation parameters set in the most recent S51101 are individual parameters. If the determination result in this step is true, the process proceeds to S51104, whereas if the determination result is false, the process proceeds to S51105.

[0086] If the answer is YES in S51103 (i.e., if individual parameters were set in the most recent S51101), proceed to S51104. In S51104, the spread number setting unit 21003 sets the number of spreads (i.e., the number of individual spreads corresponding to the individual parameters set in S51101) based on the scene division results performed in S505 and the candidate images acquired in S51102. In this embodiment, the number of spreads is set according to the following formulas (5) and (6). Number of spreads per scene = ceil(number of candidate images in a scene ÷ maximum number of images per spread) Equation (5) Number of individual spreads = ΣNumber of spreads per scene... Equation (6)

[0087] The number of spreads created by the individual parameters is obtained by formula (5) and formula (6). The maximum number of images per spread is set based on a user input via the spread photo number setting unit 406. For example, when the scene is divided as shown in FIG. 6, the number of spreads is determined according to the number of candidate images in each scene. Each scene always includes an image acquired by the image acquisition unit 203, but does not necessarily include a candidate image (a candidate image acquired as a result of rescoring based on the individual parameters). Naturally, in this process (processing according to the individual parameters), the number of spreads is 0 for a scene that does not include a candidate image. On the other hand, a suitable number of spreads is assigned to a scene that includes many images of a main subject corresponding to the individual parameters. If the number of individual spreads is equal to or greater than the total number of spreads set in S501, the following formula (7) is followed. Number of individual spreads = total number of spreads - 1 Formula (7)

[0088] Formula (7) allows spreads created with different parameters, that is, individual parameters and common parameters, to be mixed in one album. This allows the number of individual spreads to be set according to the number of images in which the main subject is captured in high quality for each scene. Note that an upper and lower limit for the number of individual spreads may be set.

[0089] On the other hand, if NO in S51103 (that is, if the common parameters were set in the immediately preceding S51101), in S51105, the spread number setting unit 21003 sets the number of spreads according to the following formula (8). Number of common spreads = total number of spreads - MIN (total number of individual spreads for each album) ··· Equation (8)

[0090] Using formula (8), it is possible to set the common number of spreads according to both the number of individual spreads and the total number of spreads. Note that, using formula (8), the common number of spreads is calculated according to the number of individual spreads in the album with the fewest number of individual spreads. Therefore, for other albums, adding the common number of spreads in formula (8) to the number of individual spreads in that album will exceed the set total number of spreads. In this case, adjustments will be made in the process of combining spreads, which will be described later.

[0091] Here, the case where the number of spreads is set according to the number of candidate images has been described, but the method of setting the number of spreads is not limited to this. For example, the spread creation parameter generation unit 209 may set the number of spreads in the parameters, and the number of spreads setting unit 21003 may read the number of spreads set in the parameters. For example, the spread creation parameter generation unit 209 may set the number of spreads pre-installed in the program of the album creation application, or may calculate the number of spreads based on the ratio of individual spreads and common spreads specified by the user. The number of individual spreads and the number of common spreads have been described above. Below, the processing for each parameter set in S51101 will be described.

[0092] In S51106, the spread allocation unit 21004 divides and allocates the candidate images acquired in S51102 (i.e., candidate images according to the spread generation parameters to be processed) into sub-image groups for the number of spreads set in S51104 or S51105. In this embodiment, the candidate images are arranged in the order of the shooting times acquired in S50301, and are divided at the location where the shooting time difference between adjacent images is large. This process is performed until the image is divided into the number of spreads set in S51104 or S51105. In other words, the division is performed (number of spreads-1). This makes it possible to create an album in which images are arranged in the order of the shooting times. In other words, the division is performed as described in FIG. 6, but the example described in FIG. 6 differs in that the image groups to be divided at this time are candidate image groups according to the spread generation parameters to be processed acquired in S51102. The process of S51106 may be performed on a scene-by-scene basis. For example, the spread allocation unit 21004 divides the images to which a scene "travel" is assigned, by arranging them in candidate time order among the candidate images. Next, the spread allocation unit 21004 divides the images to which a scene "daily life" is assigned, by arranging them in candidate time order. As described above, the spread allocation unit 21004 may perform division processing while switching scenes.

[0093] In S51107, the spread image selection unit 21005 selects images to be used for the layout from the group of candidate images assigned to the spread to be processed in S51106. At this point, the number of candidate images assigned to the spread to be processed may be greater than the maximum number of images per spread. Of course, there may be cases where the number is the same or less. In other words, the images to be used for the layout have not been determined, so in S51107, a process of selecting images to be used for the layout is performed. Below, a case where four images are selected from the group of candidate images assigned to a certain spread will be described as an example with reference to FIG. 8.

[0094] FIG. 8(a) shows the time difference between the shooting dates and times of the first and last images included in the candidate image group assigned to a spread, in other words, the shooting period of the candidate image group. Using FIG. 8(b), a method for selecting the first image when selecting four images will be described. Of all images shot during the shooting period of the candidate image group shown in FIG. 8(b), the image with the highest score re-scored in S51102 is selected as the first image. Then, in the selection of the second and subsequent images, the shooting period of the candidate image group is subdivided and images are selected so that the selected images are not concentrated in a part of the shooting period of the candidate image group. First, as shown in FIG. 8(c), the shooting period of the candidate image group is divided into two to create two shooting sections (periods). In other words, the shooting period is divided into two equal parts to create two shooting sections (groups). Next, as shown in FIG. 8(d), the image with the highest score is selected as the second image from among the images belonging to the shooting period (section indicated by the solid line) in which the first image is not selected. Next, as shown in Fig. 8(e), each shooting section in Fig. 8(d) is divided into two equal parts. In Fig. 8(f), the image with the highest score among the images shot in the shooting sections shown by solid lines in Fig. 8(e), i.e., the two shooting sections in which the first and second images are not selected (the images corresponding to the two shooting sections), is selected as the third image.

[0095] Next, a case where an image cannot be selected because there is no image in the shooting section in the range where an image is selected will be described by taking the selection of the fourth image as an example. In this embodiment, the shooting period is divided according to time regardless of the number of images. Therefore, there is a case where an image does not exist in the shooting section obtained by dividing. For example, as shown in FIG. 8(g), it is desired to select a fourth image from a shooting section in which an image has not yet been selected (shooting section shown with diagonal lines), but there is no image in this shooting section. In such a case, as shown in FIG. 8(h), each shooting section in which an image has already been selected is divided into two. Next, as shown in FIG. 8(i), the image with the highest score is selected as the fourth image from among the images shot in the shooting section in which the first to third images shown by solid lines in the figure have not been selected. Note that even if the number of images to be selected increases or decreases from four, images are selected in the same manner as described above.

[0096] Returning to the explanation of Fig. 5(c), in S51108, the template setting unit 21006 selectively sets a plurality of templates to be used for layout from among the templates stored in the HDD 104. In this step, a template with a design that satisfies the album creation conditions set in S501 is set.

[0097] In S51109, the spread layout unit 21007 determines the image layout for the spread to be processed. Specifically, the spread layout unit 21007 determines a template suitable for laying out the images selected in S51107 from among the multiple templates set in S51108. Here, the method of determining the template in this step will be described with reference to Fig. 9. Fig. 9 is a functional block diagram related to the template determination.

[0098] The selected image information acquisition unit 901 acquires the number of images selected by the spread image selection unit 21005 and image information related to the images. The acquired image information is the width and height of the image, shooting date and time information, and the score calculated by the image score unit 208. The template selection unit 902 selects a template having a number of slots that matches the number of selected images from among the templates set by the template setting unit 21006. The selected image sorting unit 903 sorts the selected images in order of shooting time. The template selection unit 904 selects a template from among the templates selected by the template selection unit 902, in which the main slots in which the main images are placed match the order of the main images arranged in order of shooting time. Here, the "main image" is the image with the highest score obtained as a result of re-scoring in S51102 among the multiple images selected by the spread image selection unit 21005. On the other hand, an image that is not a main image among the multiple images is called a "sub-image". When selecting a template, in addition to the order, the selection condition is that the aspect ratio of the main image and the aspect ratio of the slot in which the main image is placed also match. In this embodiment, the image with the older shooting time is laid out in the upper left of the template, and the image with the newer shooting time is laid out in the lower right. From among the templates selected by the template selection unit 904, the template selection unit 905 selects a template in which the slot in which the sub-images are placed matches the order of the sub-images arranged in order of shooting time, and the aspect ratio of the image and the slot match. Figure 10 is a flowchart of the template determination process.

[0099] In S1001, the selected image information acquisition unit 901 acquires selected image information. The selected image information acquired in this step includes the number of selected images. For the sake of explanation, it is assumed here that the number of selected images is 3.

[0100] In S1002, the template selection unit 902 selects a template whose number of selected images and number of slots match. Here, it is assumed that the number of selected images acquired in S1001 is 3, and a template whose number of slots is 3 is selected. Below, an example will be described in which templates (1-1) to (4-4) shown in Fig. 11 are selected in this step.

[0101] In S1003, the selected image sorting unit 903 arranges the selected images in order of their shooting time. Here, it is assumed that the relationship shown in Fig. 11A is obtained as a result of arranging the selected images in order of their shooting time. Also, it is assumed that an image 1105 is a main image, and images 1106 and 1107 are sub-images.

[0102] In S1004, the template selection unit 904 selects a template in which the position of the main image matches the position of the slot for the main image (main slot) when the images are arranged in the shooting order. Furthermore, a template in which the aspect ratio of the main image matches the aspect ratio of the main slot is selected. Here, image 1105 is the most recently shot image, is an image for the main slot, and has a portrait aspect ratio, so templates (3-1) to (3-4) in FIG. 11 are candidates.

[0103] In S1005, the template selection unit 905 selects a template in which the aspect ratio of the sub-image matches that of the slot for the sub-image (referred to as a sub-slot). In the example of FIG. 11, the older image 1106 for the sub-slot, i.e., the image to be placed in the upper left corner, is a portrait image, and the newer image 1107 for the sub-slot is a landscape image. Therefore, the template (3-2) is determined as the template most suitable for the selected three images. In this way, in S51109, the template to be used for layout for each spread and information that can identify which image is to be laid out in which slot of the template are determined. The information determined here is called spread data.

[0104] Returning to the explanation of Fig. 5(c), in S51110, the spread layout section 21007 judges whether the processes of S51107 to S51109 have been completed for all spreads to be processed. If the judgment result of this step is true, the process proceeds to S51111. On the other hand, if the judgment result is false, the process returns to S51107.

[0105] In S51111, the spread creation parameter setting unit 21001 determines whether the processes of S51101 to S51110 have been completed for all parameters generated in S510. If the determination result in this step is true, the process proceeds to S512. On the other hand, if the determination result is false, the process returns to S51101.

[0106] Returning to the explanation of Fig. 5(a), in S512, the spread combining unit 211 determines the spread number in the album for each spread created by the spread creating unit 210, and creates the layout of the entire album. In this embodiment, the spreads are arranged based on the shooting times of the images arranged on the spreads, so that the images are arranged in the order of their shooting times.

[0107] The spread combining in S512 will be described in detail below with reference to Fig. 12 and Fig. 13. Fig. 12 is a block diagram showing a detailed configuration of the spread combining unit 211 in this embodiment. Fig. 13 is a flowchart of the process executed by the spread combining unit 211.

[0108] In S1301, the spread acquisition unit 1201 acquires spread data for one album from among the spreads created by the spread creation unit 210. In this embodiment, for example, the spread data for album 1 acquired in this step includes spread data created with common parameters and spread data created with individual parameters for album 1.

[0109] In S1302, the sorting reference value calculation unit 1202 calculates a sorting reference value for each spread data acquired by the spread acquisition unit 1201. In this embodiment, the sorting reference value is the average shooting time of the images arranged in the spread data. The sorting reference value is not limited to the shooting time. For example, it may be the average shooting position (latitude and longitude) of the images arranged in the spread, the number of images, or image analysis information such as a color histogram.

[0110] In S1303, the sorting unit 1203 sorts the spread data acquired by the spread acquisition unit 1201 based on the sorting reference value calculated by the sorting reference value calculation unit 1202. In this embodiment, the spreads are sorted in ascending order starting from the spread with the earliest average shooting time.

[0111] In S1304, the number of spreads adjustment unit 1204 determines whether the number of spreads data in the album to be processed is the same as the total number of spreads set in S501. If the result of the determination in this step is true, the process proceeds to S1306. On the other hand, if the result of the determination in this step is false, that is, if the number of spreads in the album to be processed was set to be greater than or equal to the total number of spreads in S51104 and the number of spreads in the album exceeds the total number of spreads, the process proceeds to S1305.

[0112] In S1305, the number of spreads adjustment unit 1204 deletes unnecessary spread data from the spread data acquired by the spread acquisition unit 1201, thereby adjusting the number of spreads in the album to approach the total number of spreads set in S501. In this embodiment, the number of spreads is adjusted by deleting common spread data. Specifically, from the common spread data in the album, common spread data whose shooting time calculated by the sorting standard value calculation unit 1202 is closest to the shooting time of the individual spread data is deleted. After deletion, the process proceeds to S1304. Note that in this embodiment, the common spread data is deleted, but this is not limited to this. The total number of spreads may be adjusted by deleting individual spread data.

[0113] In S1306, the spread acquisition unit 1201 judges whether the processes of S1301 to S1305 have been completed for all albums. If the judgment result of this step is true, the process proceeds to S513, whereas if the judgment result is false, the process returns to S1301. This allows images to be arranged in the order of shooting time throughout the entire album. The above is the content of the automatic layout process in this embodiment. In this embodiment, the number and positions of common spreads and individual spreads can be automatically set for each album. It is possible to automatically lay out multiple albums with different main subjects, each having the same common spread in multiple albums and different individual spreads for each album.

[0114] <<Editing the album data>> <Editing process overview> So far, a process for automatically generating album data having common spreads and individual spreads as spread attributes has been described. Below, functions and processes of an album editing application that allows a user to edit the generated album data will be described. In this embodiment, the editing user will be described assuming that the editing user is the same user as the user who operates the album creation application, but it does not have to be the same user. Below, first, a general outline of the album data editing process will be described, and then the detailed process of this embodiment will be described.

[0115] 14 is a software block diagram of an album editing application 1400 in this embodiment. In this embodiment, the album editing application 1400 is described as being part of the functions of an album creation application stored in the HDD 104, but it may be provided as an application separate from the album creation application. Furthermore, the image processing device on which the album editing application is started may be the same device as the image processing device 100 that generated the album data, or may be a different image processing device.

[0116] 14, the album editing application 1400 includes an edit mode instruction unit 1401, an album information receiving unit 1402, an album editing unit 1403, an album display unit 1404, and the album information holding unit 215. It is to be noted that the album information holding unit 215 does not necessarily have to be included in the album editing application 1400.

[0117] The edit mode instruction unit 1401 transitions the album edit application 1400 to the edit mode when it detects that the user has clicked on the edit mode panel 1508 (see FIG. 15A) by UI operation using the mouse 107, for example. The edit mode is a mode in which operations for changing the contents of the album, such as adding a spread, trimming, replacing images, changing the template, changing the size, changing the layout, inserting text / stamps, or deleting a spread, can be performed. Here, an example of transitioning to the edit mode when a click on the edit mode panel is detected has been described, but the edit mode may always be transitioned to after the album data is generated. Alternatively, the edit mode may be transitioned to when any album is selected.

[0118] When the edit mode is entered by the edit mode instructing section 1401, the album information receiving section 1402 receives album information from the album information holding section 215. As described above, the album information is information including album configuration information, album creation information, and album modification information. The album information receiving section 1402 receives the album information on an album-by-album basis.

[0119] The album editing unit 1403 edits the album in response to a UI operation by the user using the mouse 107 or keyboard, using the album information received by the album information receiving unit 1402 from the album information holding unit 215. As described above, the album information includes album configuration information and album generation information, and by using the album information, album data similar to album data created by an album creation application can be obtained. When the album editing unit 1403 updates the album, the album editing unit 1403 outputs the album information to the album information holding unit 215 to update the album information. In addition, the album editing unit 1403 can create a recommendation (also called a recommendation) to be displayed to the user when editing the album. Examples of the recommendation include a recommended image to be replaced in a spread that the user is editing, or a recommended spread including a template or arrangement. When the album editing unit 1403 transitions to the edit mode by the edit mode instruction unit 1401, the album editing unit 1403 can create various recommendations and output the created recommendations to the album display unit 1404.

[0120] The album display unit 1404 displays the bitmap data output by the album editing unit 1403 on the display 105. The album display unit 1404 also displays an editing screen currently being edited by the album editing unit 1403 on the display 105. The album display unit 1404 also displays recommendations created by the album editing unit 1403.

[0121] The album data obtained as a result of editing by the album editing application 1400 may be uploaded to a server via the Internet based on a user instruction. Based on the uploaded album data, printing and bookbinding processes are executed, an album (photo book) is produced as a booklet, and the book is delivered to a designated user.

[0122] 15A and 15B are diagrams showing an example of a UI display of an album editing screen in this embodiment. FIG. 15A is an example of a screen displayed on the display 105 after the album is generated. As described above, the album data in this embodiment is data of three albums, namely, album 1 of person A, album 2 of person B, and album 3 of person C, whose album subjects (main subjects) are as described above. Image 1501 is an example of an image arranged on a spread, and the characters in image 1501 represent the main subjects appearing in the image. Common spread 1502 is an example of a layout centered on an image in which multiple main subjects appear. On the other hand, individual spreads 1503, 1504, and 1505 are respectively laid out centered on images in which main subjects A, B, and C appear. As described above, in this embodiment, multiple albums are created that are composed of spreads created according to different standards. Edit mode panel 1508 is a panel that transitions to edit mode when clicked with mouse 107. Characters 1506 and 1507 are characters that respectively indicate that it is a common spread and an individual spread on the display 105. Although it is indicated by characters here, the background color of the spread may be different between a common spread and an individual spread.

[0123] FIG. 15B(1) is a screen displayed on the display 105 by the album editing application 1400 after clicking the edit mode panel 1508 with the mouse 107. That is, FIG. 15B(1) shows an example of the edit screen. FIG. 15B(1) shows an edit screen in album units. That is, each spread of each album is displayed. If the album or spread does not fit within the screen, the screen may be configured to allow the whole to be confirmed by scrolling, or the size of the spread to be displayed may be changed to display all spreads on the screen. In this embodiment, different processing is performed depending on whether the spread to be edited is a common spread or an individual spread. Details will be described later. In the example of the edit screen in FIG. 15B(1), the "spread to be edited" refers to each spread of each album as the "spread to be edited". That is, in the example of FIG. 15B(1), a total of 12 spreads are the "spread to be edited".

[0124] The image insertion panel 1509 is in a state where an image is inserted when the mouse 107 is clicked. For example, all images in the image acquisition section at the time of creating the album may be displayed to determine an image to be inserted, and the image may be selected. Instead of all images, an image not used in the album may be selected. The text insertion panel 1510 is in a state where text is inserted when the mouse 107 is clicked. For example, the mouse cursor may be changed to a text insertion cursor in response to a click of the mouse 107 on the text insertion panel 1510, so that text can be input when the mouse 107 is clicked inside the album. At that time, the text size, font, or color of the text may be set. The stamp insertion panel 1511 is in a state where a stamp is inserted when the mouse 107 is clicked. For example, stamps may be displayed in response to a click of the mouse 107 on the stamp insertion panel, so that the user can select any stamp from among them and add the stamp to the album. The two-page spread addition panel 1512 is a panel that displays a two-page spread addition UI when the mouse 107 is clicked. The plus mark 1513 is an icon which, when clicked with the mouse 107, fixes the position at which a spread is added for insertion to the plus mark, thereby adding a spread.

[0125] FIG. 15B(2) shows an edit mode in which an arbitrary spread of an arbitrary album is focused. FIG. 15B(2) shows an example in which spread 1 of album 1 in FIG. 15B(1) is focused. FIG. 15B(2) shows an edit screen for editing in spread units. In the edit screen in FIG. 15B(2), the "spread to be edited" is the spread that is focused. That is, in the example in FIG. 15B(2), the "spread to be edited" is spread 1 of album 1. To enter the spread-unit edit mode as shown in FIG. 15B(2), for example, when any spread in FIG. 15B(2) is clicked with the mouse 107, the mode may be shifted to a spread-unit edit mode with the spread focused on that spread. Note that when spread 1 of album 2, which is displayed small in FIG. 15B(2), is clicked, an edit screen is displayed with the focus on spread 1 of the clicked album 2.

[0126] In the examples so far, it has been assumed that various panels are left-clicked with the mouse 107. However, for example, a menu may be displayed by right-clicking the mouse 107, and the desired editing state may be achieved by left-clicking the mouse 107 on that menu.

[0127] The editing mode instruction unit 1401 also sets the timing for reflecting the editing results. Reflecting the editing results is a process of applying the editing performed on one album to other albums based on the change information due to the editing. The timing for reflecting the editing results may be when some editing operation is performed by the user, when the user saves the album, when the edit target spread is changed, or when the edit target album is changed. The timing may be specified by user settings, or the timing may be determined in advance on the software side and used. When such various timing conditions are satisfied, the album editing unit 1403 performs a process for reflecting the editing results. In this embodiment, the timing for reflecting the editing results is when some editing operation is performed by the user. This concludes the general description of the album editing process.

[0128] Fig. 16 is a diagram showing an example of adding a spread. Fig. 16 shows an example in which a spread is added to album 1 in the album configuration shown in Fig. 15A, and an edited spread reflected in response to this edit operation of adding a spread is added to album 2, and the result is displayed on display 105. Note that in this example, the edited spread is not added to album 3.

[0129] Fig. 16 shows an example in which a common spread has been added. Specifically, a common spread has been added to Album 1. The added common spread is added at the same spread position as Spread 2, and the subsequent spread numbers are moved down.

[0130] Also, the same common spread is added to album 2 as album spread 2, in the same way as album 1. This makes the number of spreads in album 1 and the number of spreads in album 2 the same, eliminating any sense of unfairness. In this example, since the additional spread 1601 is a common spread, a spread with the same layout is added to album 2 as the editing-reflected spread 1602. Note that the additional spread 1601 and the editing-reflected spread 1602 are either matching spreads with no difference at all, or similar spreads.

[0131] Also, since the editing content of Album 1 is not reflected in Album 3, there is one less spread compared to Albums 1 and 2.

[0132] 17 is a block diagram showing details of the album editing unit 1403. The album editing unit 1403 may include various functions, but in this embodiment, the description will be centered on blocks related to the function of adding a spread. The album editing unit 1403 has a spread addition condition setting unit 1701, a spread addition position determination unit 1702, a spread reference determination unit 1703, an image acquisition unit 1704, a used image determination unit 1705, an image analysis unit 1706, a scene division unit 1707, and a scene classification unit 1708. It also has a main subject setting unit 1709, an image score unit 1710, a spread creation parameter generation unit 1711, a spread creation unit 1712, a spread addition unit 1713, and an album information output unit 1714.

[0133] The spread addition condition setting unit 1701 sets conditions for starting the spread addition process. This spread addition process is a process of adding a spread automatically by the album editing application 1400. In this embodiment, the spread addition process includes a process of reflecting the editing result of the added spread in other albums, that is, a process of adding a spread to other albums. The spread addition conditions are set when the user specifies the spread addition conditions by a UI operation using the mouse 107 or the keyboard and instructs to execute the specified conditions. The spread addition conditions include a group of images to be used in the album, the number of spreads to be added, and the album to which the spreads are to be added.

[0134] The number of spreads to be added may be specified by the user, or a fixed number of spreads may always be added. The number of spreads to be added may also be determined from the number of images not actually used among the images used in the album. The image scores of images not used in the album may also be used. Specifically, an image score threshold may be set, and the number of images with image scores equal to or greater than the image score threshold may be calculated to determine the number of spreads to be added to the album. The quotient of the result of dividing the number of images equal to or greater than the image score threshold by a fixed value may also be used as the number of spreads to be added.

[0135] As another example, a threshold value for the spread score may be determined and used. The spread score is the sum of scores obtained by multiplying the images included in the spread by a predetermined weight. The weight here may be, for example, a weight using time information in the spread, the size of the slot in the spread in which the image is included, and arrangement information. Alternatively, a weight based on the size of the slot in which each image is arranged may be used. The image arranged in a larger slot is the more important the image in the spread, so this is to strengthen the element of such an image. In this way, the sum of images in the spread multiplied by a predetermined weight becomes the score of the spread. For example, a spread may be created using images whose image scores are equal to or greater than a threshold, and as many spreads as are required to create a spread whose spread score is equal to or greater than the spread score threshold may be added. As another example, the maximum number of spreads that does not increase the price of the album may be added.

[0136] The album to which spreads are added may be specified by the user, or spreads may be added to the album currently being edited. The album to be added set in the spread addition conditions is, so to speak, the primary addition album. When spreads are added to this primary addition album, the same number of spreads as those added to the primary addition album will also be added to other albums (which may also be called secondary addition albums).

[0137] 18 shows a display screen 1801 provided by the album creating application for adding a spread, which is displayed on the display 105. The user can specify conditions for creating an additional spread via the display screen 1801. The display screen 1801 is a spread adding UI that is displayed when, for example, the spread adding panel 1512 in FIG. 15B(1) is clicked.

[0138] A path box 1802 is a box for indicating the storage location (path) of the image group for which an additional spread is to be created in the HDD 104. The storage location may be the same as the storage location when the album was created, or it may be different.

[0139] The folder selection button 1803 is a button for selecting a folder containing a group of images for which an additional spread is to be created. When the user clicks the folder selection button 1803 with the mouse 107, a tree made up of multiple folders is displayed. Then, when the user selects a folder containing a group of images for which an album is to be created, the folder path of the selected folder is displayed in the path box 1802.

[0140] The number of double-page spreads to be added box 1804 is a box for specifying the number of double-page spreads to be added to the album. The user can directly input a number into the number of double-page spreads box 1804 using the keyboard 106, or can input a number into the number of double-page spreads box 1804 from a list using the mouse 107. In this embodiment, an example of adding one double-page spread to one album will be described. Note that in this double-page spread addition UI, an example has been described in which the user specifies the number of double-page spreads to be added, but as described above, the number of additional double-page spreads may be determined automatically. Also, if no number is input into the number of double-page spreads box 1804, the number of additional double-page spreads may be determined automatically.

[0141] The additional spread position box 1805 is a box for specifying a spread number as the position of a spread to be added when adding a spread to an album. The user can directly input a number into the additional spread position box 1805 using the keyboard 106, or can input a number into the additional spread position box 1805 from a list using the mouse 107. Note that if the additional spread panel 1512 is clicked after clicking the plus mark 1513 in Fig. 15B(1), the position of the plus mark 1513 may be set as the position for adding the spread.

[0142] The additional spread attribute check box 1806 is a box for specifying attributes when creating an additional spread. The user can specify the check box using the keyboard 106 or mouse 107. A spread with the selected attributes is automatically generated. If the check box is not checked, the attributes may be automatically determined, or an automatic determination check box may be added.

[0143] The spread image number setting section 1807 is a slider bar for setting the number of images to be arranged on the additional spread to be created. When the slider is moved to the "More" side, the number of images to be arranged on each spread increases. On the other hand, when the slider is moved to the "Less" side, the number of images to be arranged on each spread decreases.

[0144] The album selection button 1808 is a button for selecting an album into which an additional spread is to be inserted. When the user clicks the album selection button 1808 with the mouse 107, a list of currently created albums is displayed. When the user specifies one or more albums from the list, the specified albums are selected as the albums into which an additional spread is to be inserted. As described above, the album currently being edited may be automatically determined as the album into which an additional spread is to be inserted, or the addition UI may be configured so that the album currently being edited is selected by default. In this embodiment, spreads are also added to albums other than the album selected with the album selection button 1808. However, spreads are not added to all albums.

[0145] The OK button 1809 is a button for deciding the selected conditions as the additional spread creation conditions. When the user clicks the OK button 1809 with the mouse 107, the additional spread creation conditions are decided, and the additional spread conditions are communicated to each section of the album editing section 1403 via the additional spread condition setting section 1701. The reset button 1810 is a button for resetting each setting information on the display screen.

[0146] Returning to the description of FIG. 17, the spread addition position determining unit 1702 determines where in the album the additional spread is to be inserted. When the additional spread position is specified by the spread addition condition setting unit 1701, the spread addition position determining unit 1702 can set the position as the spread addition position. Alternatively, the spread addition position determining unit 1702 may automatically determine the spread addition position. An example of automatically determining the spread addition position is adding the spread to the spread next to the spread currently being edited by the user. As another example, the spread may always be added to the end of the spread. When the positions of the common spread group and the individual spread group are divided in half in the album, the additional spread may be inserted as the last spread of the common spread group or the individual spread group according to the spread criteria specified in the spread addition condition. Also, after creating the layout of the additional spread, the additional spread may be determined to a position that matches the time axis of the layout, or may be determined to a position that gives the highest score for the entire album. The score for the entire album may be determined, for example, by the arrangement of the spreads in the spread combining section 211 in accordance with a combining rule (eg, a time axis, etc.).

[0147] As an example of automatically determining the position for adding a spread after creating the spread, the position for adding the spread may be determined at a position where the time axis matches the reference in the album, using time axis information of the created spread and information on the spread reference. In the case of an album in which the positions of the common spread group and the individual spread group are divided in half, the position for adding the spread may be determined at a position where the time axis matches, after determining which group it is by giving priority to the spread reference. In the case of an album in which the positions of the common spread group and the individual spread group are not divided in half, the position for adding the additional spread may be determined at a position where the time axis matches. Note that a recommended position for adding the spread may be determined and displayed to the user.

[0148] The spread criterion determination unit 1703 determines the spread creation criterion for the added spread. Specifically, it determines which criterion to use among common spread, individual spread, or other spread. The other spread is a spread creation criterion that is neither common nor individual and is not used when automatically creating an album. For example, it may be a criterion that uses only the image score without determining a main subject, or a criterion that is a weighted sum of the common spread criterion and the individual spread criterion.

[0149] When an additional spread standard is set in the spread addition condition setting unit 1701, the spread standard determination unit 1703 determines the spread standard according to the setting. When an additional spread standard is not set or in other cases, the spread standard determination unit 1703 may automatically determine the spread standard. An example of automatically determining the spread standard may be a determination according to the position of the spread to be added. For example, the spread standard used in the spreads on both sides of the spread to be added may be used. When the spread standards used in the spreads on both sides are different, the number of images suitable for a common spread and the number of images suitable for an individual spread may be compared among images not used in the album, and the standard with the larger number of images may be used. Note that an image suitable for a common spread or an image suitable for an individual spread is an image with a high score compared with each spread standard. When comparing the scores, a threshold may be set, and an image suitable for a high score standard may be determined when the difference between the score for the common spread and the score for the individual spread is equal to or greater than the threshold. When an image is newly added during editing and the image score has not been calculated, the image score may be calculated here.

[0150] As another example of automatically determining the spread criterion, the spread criterion for the additional spread may be determined based on the balance between the number of common spreads and the number of individual spreads in the album. As an example of determining the criterion using the balance between the number of common spreads and the number of individual spreads in the album, the criterion may be determined so that the ratio between the number of common spreads and the number of individual spreads approaches 5:5. In addition, when the user selects an image to be used for a spread and instructs to add a spread, the spread criterion suitable for the selected image may be determined. In addition, each spread may be automatically created according to a plurality of creation criteria, and the criterion with the highest spread score may be adopted as the spread criterion. One of the examples described above may be used, or a combination of a plurality of examples may be used. The spreads automatically created according to a plurality of creation criteria may be displayed on the display 105, and the criterion for the spread selected by the user may be determined as the spread criterion.

[0151] The image acquisition unit 1704 acquires the image group set by the spread addition condition setting unit 1701. The image acquisition unit 1704 is a processing unit similar to the image acquisition unit 203 of the album creation application. If the user does not specify an image group, or if the user specifies the same as when the album was created and the data acquired by the image acquisition unit 203 when the album was created is held, the acquired images may be used as is without image acquisition. If the storage location of the image group set by the spread addition condition setting unit 1701 is different from the folder acquired by the image acquisition unit 203 for album creation, or if the HDD 104 is updated and the number of images has changed, the set image group is acquired from the HDD 104.

[0152] The used image determination unit 1705 determines whether the image acquired by the image acquisition unit 1704 is an image used in the album from the image use information included in the album information received by the album information receiving unit 1402. The determination of whether the image is used may be made differently depending on the determination by the spread criterion determination unit 1703. For example, when the spread criterion is determined to be an individual criterion by the spread criterion determination unit 1703, the used image determination unit 1705 may determine whether the image is used using the image use information in the album to be edited. On the other hand, when the spread criterion is determined to be a common criterion by the spread criterion determination unit 1703, the used image determination unit 1705 may determine whether the image is used using the image use information in all albums. If an image determined to be a used image is placed in an album, duplication of images will occur in one of the albums. For this reason, the used image is not used for creating an additional spread. As an example of a method for not using the image for creating an additional spread, the acquired image data may be discarded. Also, a used image flag or a used label may be used to mark the image as a used image, so that the processing described below is not performed, or the image is not selected as an image for creating a double-page spread. In this embodiment, a used image flag is attached to a used image, and processing is performed so that the image is not selected as an image to be used for creating an additional double-page spread.

[0153] An image analysis unit 1706 performs an analysis process on the image data acquired by the image acquisition unit 1704. The image analysis unit 1706 is a processing unit similar to the image analysis unit 204. For images acquired by the image acquisition unit 1704 that are the same as those at the time of creating the album, the analysis results at the time of creating the album may be used.

[0154] The scene dividing unit 1707 divides the images acquired by the image acquiring unit 1704 into scenes based on the analysis information of the image analyzing unit 1706. The scene dividing unit 1707 is a processing unit similar to the scene dividing unit 206. When the images acquired by the image acquiring unit 203 include the same images as those at the time of creating the album, the division result at the time of creating the album may be used.

[0155] The scene classification unit 1708 determines the category of the images in the scenes obtained by the scene division unit 1707. The scene classification unit 1708 is a processing unit similar to the scene classification unit 207. When using the same images as when the album was created, the classification result when the album was created may be used.

[0156] Main subject setting unit 1709 sets a main subject to be emphasized for each album. Main subject setting unit 1709 is a processing unit similar to main subject setting unit 205. Main subject setting unit 1709 sets a main subject corresponding to the double-page spread standard determined by double-page spread standard determination unit 1703. Note that it may be set that there is no main subject.

[0157] Image scoring section 1710 assigns a score to each image for each main subject set by main subject setting section 1709, based on the analysis information obtained by image analysis section 1706. Image scoring section 1710 is a processing section similar to image scoring section 208. In the case of the same image and the same subject as when the album was created, the score at the time of album creation may be used without assigning a new score.

[0158] The spread creation parameter generating unit 1711 generates parameters for creating a spread based on the main subject set by the main subject setting unit 1709 and the analysis information obtained by the image analysis unit 1706. The spread creation parameter generating unit 1711 is a processing unit similar to the spread creation parameter generating unit 209. The spread creation parameter generating unit 1711 sets parameters for creating a spread corresponding to the spread standard determined by the spread standard determining unit 1703. An example of this spread creation parameter is a weight given to the score obtained by the image scoring unit 1710. An example of information used for calculating the weight is various information described in the spread creation parameter generating unit 209. In addition, the weight may be determined using the image information used, the time information in the spread including the image, the size of the slot in the spread including the image, and the arrangement information as the spread creation parameters. For example, as described in the spread creation parameter generating unit 209, when placing an image in a slot, the weighting may be changed depending on whether the image has been trimmed and placed or not, and whether the spread in which the image is used is a common spread or an individual spread. Also, the score of a user-used image that is additionally inserted by the user may be weighted, and the score may be adjusted to be larger. Also, the weighting may be reduced so that images that have already been used in an album are not selected, in order to avoid duplicate use.

[0159] The spread creation parameters also include a parameter (second parameter) that derives the score for the spread. In the example of FIG. 15B(2), the score is the sum of the scores obtained by multiplying the scores of each image in spread 1 of album 1 by a weighting factor based on the size of the slot in which each image is placed. The larger the slot in which an image is placed, the more important the image is in that spread, and this is to enhance the element of such an image. Note that, although an example of summing the scores of each image weighted by the slot size has been described here, weighting is not necessary, and the score for the spread may be the average of each image.

[0160] The spread creation unit 1712 creates a spread using the result of the image score unit 1710, the used image information from the used image determination unit 1705, and the spread creation parameters generated by the spread creation parameter generation unit 1711. The spread creation unit 1712 selects the image with the highest final image score as the image to be used in the spread, and creates a spread by arranging it. The spread creation unit 1712 is a processing unit similar to the spread creation unit 210. The spread creation unit 1712 creates the number of spreads set by the spread addition condition setting unit 1701. Furthermore, in this embodiment, the spread creation unit 1712 performs processing to create additional spreads in other albums in response to creating an additional spread in the album to which the spread is to be added, as set by the spread addition conditions.

[0161] A spread adding section 1713 adds the additional spreads created by the spread creating section 1712 to the album in accordance with the result of the spread adding position determining section 1702, determines the spread numbers of the spreads in the album, and creates the layout of the entire album.

[0162] When the album is updated, the album information output unit 1714 outputs the album information to the album information holding unit 215 based on the editing results.

[0163] In this embodiment, the album editing process has been described mainly with an example of automatically adding a two-page spread, but the album editing process may be performed in response to a user operation. For example, as described above, the album editing unit 1403 can edit the album based on a user input even in the case of trimming, image replacement, template change, size change, layout change, insertion of text / stamp, or deletion of a two-page spread.

[0164] FIG. 19 is a software block diagram showing a group of modules related to reflecting the editing results, which are modules included in the album editing application 1400 in this embodiment.

[0165] An edit result reflection timing determination unit 1901 determines whether or not the edit result reflection timing condition specified by the edit mode instruction unit 1401 is satisfied, that is, whether or not it is time to reflect the results of editing performed on one album in another album.

[0166] The album information receiving unit 1902 is a processing unit similar to the album information receiving unit 1402, and receives album information on an album-by-album basis from the album information holding unit 215. Specifically, this album information is album configuration information, album creation information, and album change information. The album change information includes information on changes due to editing. The information on changes due to editing may be difference information changed by editing the album, or may be the operation history of the user who edited the album.

[0167] The edited spread determination unit 1903 determines whether the edited spread is a common spread, using album change information included in the album information received by the album information receiving unit 1902. In this example, it is determined whether the edited spread is a common spread, based on whether the criterion used when the spread was automatically generated is a common criterion. However, the method of determining whether it is a common spread is not limited to this. For example, when the same spread is used between albums, the same spread may be determined as a common spread, common spread management may be performed using a flag, or a spread with a certain fixed spread number may be determined as a common spread.

[0168] The editing result reflection condition setting unit 1904 sets conditions for reflecting the editing results of a spread determined to be a common spread by the edit spread determination unit 1903 to other albums. Specifically, the conditions may be layout information such as background color or template, or editing contents such as text insertion or stamps, or the conditions may be a spread number or a position such as a certain area of ​​the album. As another example, the editing results may always be reflected without setting the reflection condition settings. In this embodiment, the editing results are reflected for edits other than text insertion and stamp insertion.

[0169] The editing result reflection album setting unit 1905 sets which album the editing result is to be reflected in. In this embodiment, the editing result is set not to be reflected in at least one album. As an example of a method of determining an album in which the editing result is to be reflected in a specific album, the user may group the albums in advance and reflect the result in each group, or the editing result may be reflected in an album that has a high score for the spread when the editing result is reflected. Alternatively, for image editing, an album to be reflected in may be set based on the image score. As an example of setting an album to be reflected in based on the image score, the editing content is evaluated based on the image score using the main subject of the individual spread, and the editing content is reflected in an album with a high score. In this way, in this embodiment, the editing content is reflected in an album with a high score, and is not reflected in an album that has a low score for the spread when the editing result is reflected. Also, when the score is high for the spread of all albums as a result of reflecting the editing result, the editing content is not reflected in the album with the smallest increase in score. This allows appropriate editing content to be reflected for each target user linked to the album.

[0170] An editing results reflecting album determination unit 1906 determines whether or not to reflect the editing results for each album, based on the setting of whether or not the album reflects the editing results, set by the editing results reflecting album setting unit 1905. For albums determined by the editing results reflecting album determination unit not to reflect the editing results, the following processing units do not perform processing.

[0171] The editing result reflection area determination unit 1907 determines whether the target area satisfies the conditions set by the editing result reflection condition setting unit 1904. In this embodiment, areas that satisfy the editing result reflection conditions and areas that do not are managed by using the determination results of the editing result reflection area determination unit 1907. As an example of a method of managing the determination results, the result reflection may be managed using a flag indicating whether or not the editing results are reflected in each spread area, or only the user operation history of the edit that satisfies the editing result reflection condition may be passed to the editing result reflection unit 1908.

[0172] The editing result reflecting unit 1908 reflects the editing made to the spread determined as a common spread by the edited spread determining unit 1903 in other albums using the result of the determination by the editing result reflecting area determining unit 1907 and the information on changes due to editing received by the album information receiving unit 1902. The editing reflecting process in this example applies the editing contents made to a certain album to other albums as they are, specifically, adds the same spread as the spread added to a certain album to other albums. As a method of creating the same spread, a method of performing the same operation as the operation described in the album change information after creating an album at the time of automatic generation using the album generating information is considered. According to this method, the same spread with a completely identical layout and arrangement can be created. Alternatively, the same spread with a completely identical layout and arrangement may be created using the album configuration information received by the album information receiving unit. The editing result reflecting unit 1908 process can make the layout of the common spread the same between the edited album and a part of the albums excluding the edited album.

[0173] When the album is updated, the album information output unit 1909 outputs album information based on the editing results to the album information holding unit 215. Specifically, this album information includes album configuration information, album creation information, and album modification information.

[0174] In this embodiment, the above-mentioned configuration allows editing performed in a specific album, specifically, a change in the common spread, to be reflected in other albums of the specific album, but not reflected in at least some of the albums, thereby allowing the layout of each album to be adjusted.

[0175] Fig. 20 is a flowchart showing a process of executing a spread addition in the album editing application 1400 according to this embodiment. The flowchart shown in Fig. 20 is realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or RAM 103 and executing it.

[0176] In S2001, the spread addition condition setting unit 1701 sets the spread addition conditions. In this embodiment, the spread addition conditions are set as the image group to be used in the album, the number of spreads to be added, and the album to which the spread addition is to be performed. Note that here, the user sets the target album to which the spread addition is to be performed, but in the processing of this embodiment, the spread addition is also performed for albums for which the album addition setting has not been performed here.

[0177] In S2002, the spread addition position determination unit 1702 determines the spread addition position. In S2003, the spread reference determination unit 1703 determines which reference to use for the added spread. Specifically, as described above, it determines which spread reference to use from among the common spread, the individual spread, or other spread.

[0178] In S2004, the image acquisition unit 1704 reads images included in the image group set by the spread addition condition setting unit 1701 from the HDD 104, and loads them in the RAM 103. If the images have already been loaded in the RAM 103, no processing is performed.

[0179] In S2005, used image determination unit 1705 determines whether the acquired image is a used image based on the album information (used image information) acquired by album information receiving unit 1402 and the spread criteria determined by spread criteria determination unit 1703. If the additional spreads are a common criteria, used image information of all albums is used. If the additional spreads are an individual criteria or other criteria, used image information of the album currently being edited is used. The flowchart will be explained below using an example where the additional spreads are a common criteria.

[0180] In S2006, the image analysis unit 1706 analyzes the image data expanded in the RAM 103. The image analysis process has a processing flow similar to that described in FIG.

[0181] In S2007, the image analysis unit 1706 determines whether the processes of S2005 and S2006 have been completed for all images in the image group in the HDD 104 specified by the double-page spread addition condition setting unit 1701. If not, the process returns to S2004. If completed, the process proceeds to S2008.

[0182] In S2008, the scene division unit 1707 performs scene division on the group of images acquired by the image acquisition unit 1704. S2008 is the same process as S505. In S2009, the scene classification unit 1708 performs scene classification on the group of sub-images divided by the scene division unit 1707. S2009 is the same process as S506.

[0183] In S2010, main subject setting unit 1709 sets a main subject for each album. Main subject setting unit 1709 sets a main subject for the album to be added based on the album information received by album information receiving unit 1402, the album to which a spread is to be added that was set in S2001, and the spread criteria determined in S2003. In this example, it is assumed that the additional spread process is to be performed on album 1. Also, in this example, a common spread criteria is used. Therefore, main subjects ABC are set in album 1.

[0184] In S2011, the image scoring unit 1710 scores each image. S2011 is the same process as S508. In S2012, the image scoring unit 1710 determines whether the image scoring in S2011 has been completed for all images. If not, the process returns to S2011. If completed, the process proceeds to S2013.

[0185] In S2013, the spread creation parameter generating unit 1711 generates spread creation parameters based on the spread addition conditions set in S2001 and the main subject of the additional spread set in S2010. S2013 is the same process as S510.

[0186] In S2014, the spread creating unit 1712 creates an additional spread for the album to be added, using the spread creation parameters generated in S2013. The spread creation process is the same as that in S511.

[0187] In S2015, the album editing unit 1403 judges whether or not the number of spreads created in S2014 has reached the number of additional spreads set in the spread addition condition setting unit 1701. If the number of additional spreads has not been reached, the process returns to S2002. If the number of additional spreads has been reached, the process proceeds to S2016.

[0188] In S2016, the spread adding unit 1713 inserts the spread created by the spread creating unit 1712 into the determined position of the additional spread, combines the albums, and creates a layout for the entire album.

[0189] In S2017, the album information output unit 1714 outputs the album information.

[0190] Next, a process of reflecting the editing results included in the automatic layout process in the album creation application according to this embodiment will be described with reference to Fig. 21. Fig. 21 is a flowchart showing a process of reflecting the editing results by the album editing application 1400 according to this embodiment. The flowchart shown in Fig. 21 is realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the ROM 102 or RAM 103 and executing it.

[0191] In S2101, the edit result reflection timing determination unit 1901 determines whether or not the edit result reflection timing condition specified by the edit mode instruction unit 1401 is met. If it is the edit result reflection timing, the process proceeds to S2102. If it is not the edit result reflection timing, the process ends.

[0192] In 2102, the album information receiving section 1902 receives album information including information on the editing content performed on a specific album from the album information holding section 215. The album information received in this step is expanded in the RAM 103 and held therein.

[0193] In S2103, the edited spread determination unit 1903 determines whether the album to be determined (first target album) is an album that has been edited. If the determination result in this step is true, the process proceeds to S2104, whereas if the determination result is false, the process proceeds to S2111.

[0194] In S2104, the edited spread determination unit 1903 determines whether the edited spread in the edited album (hereinafter referred to as the edited spread) is a common spread. If the result of the determination in this step is true, the process proceeds to S2105, whereas if the result of the determination is false, the process proceeds to S2111.

[0195] In S2105, the edit result reflection condition setting unit 1904 sets a condition for reflecting the edit result (hereinafter referred to as an edit result reflection condition).

[0196] In S2106, the editing result reflection album setting unit 1905 sets a target album in which the editing result is reflected.

[0197] In S2107, the editing result reflection album determination unit 1906 determines whether the album to be determined (second target album) is a target album in which the editing results are reflected, based on the setting in S2106. If the determination result in this step is true, the process proceeds to S2108, whereas if the determination result is false, the process proceeds to 2110.

[0198] In S2108, the edit result reflection area determination unit 1907 determines whether the target area satisfies the condition set by the edit result reflection condition setting unit 1904. If the determination result in this step is true, the process proceeds to S2109, whereas if the determination result is false, the process proceeds to S2110.

[0199] In S2109, the editing result reflecting unit 1908 reflects the contents of the editing results made in the predetermined album to the target area of ​​the album for which both the determinations in S2107 and S2108 are true.

[0200] In S2110, the editing result reflecting unit 1908 judges whether the processes from S2107 to S2109 have been completed for all albums except for the specified album in which editing was performed. If the judgment result in this step is true, the process proceeds to S2111, whereas if the judgment result is false, the second focused album is changed to an undetermined one, and the process returns to S2107.

[0201] In S2111, the editing result reflecting unit 1908 judges whether the processes from S2103 to S2110 have been completed for all albums. If the judgment result in this step is true, the series of processes ends. On the other hand, if the judgment result in this step is false, the first focused album is changed to an undetermined one, and the process returns to S2103.

[0202] <<Effects of this embodiment>> The above is an explanation of the automatic generation of additional spreads in this embodiment, and the reflection of the editing results of the common spread in a given album in other albums. In this embodiment, the automatic generation of additional spreads can be performed in response to an instruction for additional spreads from a user. In addition, the editing results of the common spread can be applied to each album, allowing editing suitable for each album to be performed.

[0203] (Embodiment 2) In the first embodiment, when the editing results of a specified album are reflected in other albums of the specified album, the change information due to the editing corresponding to the specified album is used. In contrast, in the present embodiment, a spread similar to the edited spread in the specified album is created based on the edited spread in the specified album, and the created spread is added to the other album. Note that in the present embodiment, the same reference numerals and names are used for the same components as in the first embodiment, and the description is omitted as appropriate.

[0204] The functional configuration of this embodiment will be described below with reference to Fig. 22. Fig. 22(a) is a software block diagram showing a group of modules related to reflection of editing results, as modules included in the album editing application 1400 of this embodiment.

[0205] The editing result reflecting unit 2201 reflects the contents of the editing results for a specified album in other albums of the specified album. Specifically, the editing result reflecting unit 2201 creates or adds a spread in the other albums that is similar to the edited spread in the specified album and appropriate for each album, based on the edited spread in the specified album. However, in this embodiment, as in the first embodiment, such reflection is not performed on at least some of the other albums.

[0206] FIG. 22B is a detailed software block diagram of the edit result reflecting unit 2201 when the process of reflecting edited contents is a process of creating and adding a similar spread.

[0207] The image acquisition unit 220101 is a processing unit similar to the image acquisition unit 203 used in creating an album (see FIG. 2(a)). If there is no user specification when the image acquisition unit 220101 acquires the image group, or if the specification has not changed since the album was created and the data acquired by the image acquisition unit 203 is retained, the image group acquisition process by the image acquisition unit 220101 may be skipped. In this case, the data acquired by the image acquisition unit 203 may be used as is. However, if the data in the HDD 104 has been updated, the image group is acquired from the HDD 104.

[0208] The used image determination unit 220102 determines whether the image acquired by the image acquisition unit 220101 is used in the album from the image use information included in the album information received by the album information receiving unit 1902. This determination is performed using the image use information of all albums. In this embodiment, the information of the image determined to be a used image by the used image determination unit 220102 is treated as used image information together with information on which spread of which album it is used in. If an image determined to be a used image is placed in an album, image overlap occurs in one of the albums. Therefore, a recommendation recommending the use of a used image is not created. As a means for not using a used image in an additional spread, a flag (used image flag) indicating whether the image is used or not, an image label, or the like may be adopted. For an image determined to be used by the used image determination unit 220102, the following process may not be performed, or the image may be excluded from selection during image selection. In this embodiment, the used image flag is set.

[0209] The similar image determination unit 220103 performs image similarity determination to determine whether images are similar. In this embodiment, similarity determination is performed between an image with a used image flag set and other unused images. An image determined to be similar to a used image in this similarity determination may be an image similar to an image in a spread and may be a better image, so it may be used to create a spread. In addition, similarity labels are provided in the same number as the number of images in a spread, and when there are multiple unused images determined to be similar to the same image in a spread, they are managed by attaching the same similarity label. A similar spread can be created by selecting one image from each group of images with the same similarity label and creating a spread.

[0210] An example of image similarity determination is a determination using Structure Similarity (SSIM). Specifically, the SSIM value is calculated between the edited double-page spread image and the image acquired by the image acquisition unit 220101. If the calculated SSIM value is equal to or greater than a threshold value of 0.8, the images may be determined to be similar. Note that the threshold value of 0.8 is an example, and other values ​​may be adopted. Moreover, PSNR may be used instead of SSIM. When PSNR is used, the threshold value may be freely set, for example, 40 dB. The threshold value may be determined in advance, or may be selected by the user. When the user selects the threshold value, the threshold value may not be directly determined, but may be indirectly selected by operating a slider to increase or decrease the threshold value. When the sizes of the images to be determined as similar images are different, the size of the unused image to be determined is adjusted to the size of the used image, and then the similar image determination is performed. For example, when the used image is smaller than the unused image, the unused image is trimmed to the size of the used image, and then the similar image determination is performed. Conversely, when the used image is larger than the unused image, the unused image is enlarged to the same size as the used image, and then the similar image determination is performed. Other examples of similar image determination include methods for classifying images according to the content they contain, such as a method for classifying images according to the main subject of the image, or a method for classifying images that contain many identical objects by performing object detection within the image as similar images.

[0211] Main subject setting unit 220104 sets a main subject to be emphasized (prioritized) for each album. Main subject setting unit 220104 is a processing unit similar to main subject setting unit 205. In this embodiment, the main subject of the common spread when editing an album is set to two different subjects, the main subject of the individual spread when creating the album, and the main subject of the common spread when creating the album. This allows the main subject of the common spread when editing an album to differ for each album, making it possible to select a similar image suitable for each user.

[0212] The image selection parameter generating unit 220105 generates parameters (referred to as image selection parameters) for selecting an image based on the main subject set by the main subject setting unit 220104 and the analysis information obtained by the image analysis unit 1703. The image selection parameters here are set values ​​that determine the image selection criteria. An example of such a parameter is a weighting that is assigned to the score obtained by the image score unit 1710.

[0213] The image selection unit 220106 selects images using parameters generated by the image selection parameter generation unit 220105. In this embodiment, the similar image determination unit 220103 calculates a final image score for each image of the image group determined to be similar to the image arranged in the spread, and the image selection unit 220106 selects the image with the highest score. By replacing the image selected by the image selection unit 220106 with the image that was arranged, a similar spread that reflects the contents of the editing results is created in each album.

[0214] As described above, the edit result reflecting unit 2201 creates a similar spread that takes into consideration the user (that is, the main subject) associated with the album in which the edit contents are reflected, so that editing suited to each album can be performed.

[0215] FIG. 23 is a flowchart showing a process for reflecting the editing results.

[0216] In S2301, the image obtaining unit 220101 reads the candidate image group from the HDD 104, using the information on the image group used during editing received by the album information receiving unit 1902, and loads the candidate image group in the RAM 103. Note that if the candidate image group has already been loaded in the RAM 103, the processing of this step does not need to be performed.

[0217] In S2302, the used image determination unit 220102 determines whether the image acquired by the image acquisition unit 220101 is an image that has already been used in the album. This determination is made based on image usage information included in the album information received by the album information receiving unit 1902 and the criteria determined by the spread criteria determination unit 1703. For example, if the spread criteria determination unit 1703 determines the spread criteria to be individual criteria, the used image determination unit 220102 may determine whether the image is a used image using image usage information in the album to be edited. On the other hand, if the spread criteria determination unit 1703 determines the spread criteria to be common criteria, the used image determination unit 220102 may determine whether the image is a used image using image usage information in all albums.

[0218] In S2303, the similar image determination unit 220103 performs similar image determination using the determination result by the used image determination unit 220102 between used images and other images.

[0219] In S2304, the image analysis unit 1706 analyzes the image data expanded in the RAM area 103. The image analysis process has a process flow similar to that described in FIG.

[0220] In S2305, the image analysis unit 1706 determines whether the processes from S2301 to S2304 have been completed for all images in the image group in the HDD 104 that may be acquired by the image acquisition unit 220101. If the determination result in this step is true, the process proceeds to S2306. On the other hand, if the determination result is false, the process returns to S2301.

[0221] In S2306, the scene splitting unit 1707 executes scene splitting for the image group acquired by the image acquisition unit 220101. S2306 is the same processing as S505.

[0222] In S2307, the scene classification unit 1708 executes scene classification for the group of sub-images divided by the scene division unit 1707. S2307 is the same process as S506.

[0223] In S2308, the main subject setting unit 220104 sets a main subject for each album. In the setting of this step, the main subject setting unit 220104 acquires information indicating the main subject for each album based on the determination result by the edited spread determination unit 1903, and uses the acquired information.

[0224] In S2309, the image scoring unit 1710 scores each image. S2309 is the same process as S508.

[0225] In S2310, the image scoring unit 1710 determines whether the scoring in S2309 has been completed for all images analyzed by the image analysis unit 1706. If the determination result in this step is true, the process proceeds to S2311, whereas if the determination result is false, the process returns to S2309.

[0226] In S2311, the image selection parameter generation unit 220105 generates image selection parameters using information on the main subject of the edited spread received from the main subject setting unit 220104. S2311 is a process similar to S510.

[0227] In S2312, the image selection unit 220106 creates a similar spread by selecting images using the image selection parameters generated in S2311 and the results of the similar image determination in S2303.

[0228] In S2313, the album information output unit 1714 outputs the album information.

[0229] <<Effects of this embodiment>> The above is an explanation of how the contents of the editing results for a specific album in this embodiment are reflected in other albums from the specific album. Specifically, in other albums, spreads with layouts similar to the common spread edited in the specific album are created. However, such reflection is not performed in at least some albums. This allows the layout of each album to be adjusted.

[0230] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The contents of the above-mentioned embodiments may be combined as appropriate.

Claims

1. making the computer function as an editing means for editing the layout of a plurality of albums having a common spread attribute common to each album and an individual spread attribute of each album as spread attributes, and as a position determining means for determining a position where a spread is to be added; the editing means automatically creates a spread based on a criterion according to the attribute, adds the created spread to one of the plurality of albums, automatically creates a spread identical or similar to the spread added to the one album, and adds the created identical or similar spread to each of one or more albums among the plurality of albums excluding the one album; a position where a spread is added in the one album is the same as a position where a spread is added in the one or more albums; A program characterized by:

2. the number of spreads in the one album to which spreads have been added by the editing means is equal to the number of spreads in the one or more albums to which spreads have been added by the editing means; 2. The program according to claim 1 .

3. the position determination means determines a position where the double-page spread is to be added based on a user's designation; 3. The program according to claim 1 or 2.

4. The computer is further caused to function as a criterion determination means for determining a criterion for creating an additional spread.

4. The program according to claim 1, wherein the program is a program for executing a program for executing a program.

5. the standard determination means determines one of a common spread standard, an individual spread standard, and another standard; 5. The program according to claim 4.

6. The computer is further caused to function as a setting unit for setting a main subject for each album.

6. The program according to claim 4 or 5.

7. causing the computer to further function as a parameter generating means for generating a double-page spread creation parameter for creating a double-page spread; 7. The program according to claim 6 .

8. the parameter generating means generates the double-page spread creating parameters based on the criterion determined by the criterion determining means and the main subject set by the setting means.

8. The program according to claim 7 .

9. The plurality of albums edited by the editing means are albums created for each user.

9. The program according to claim 1, wherein the program is a program for executing a program for executing a program.

10. and causing the computer to function as an automatic creation unit that automatically creates the plurality of albums edited by the editing unit.

10. The program according to claim 9 .

11. an editing means for editing the layout of a plurality of albums having a common spread attribute common to each album and an individual spread attribute of each album as spread attributes; a position determining means for determining a position where a double-page spread is to be added; having the editing means automatically creates a spread based on a criterion according to the attribute, adds the created spread to one of the plurality of albums, automatically creates a spread identical or similar to the spread added to the one album, and adds the created identical or similar spread to each of one or more albums among the plurality of albums excluding the one album; a position where a spread is added in the one album is the same as a position where a spread is added in the one or more albums; 13. An image processing device comprising:

12. an editing step of editing the layout of a plurality of albums having a common spread attribute common to each album and an individual spread attribute of each album as spread attributes; a position determining step for determining a position where a spread is to be added; having In the editing step, a spread is automatically created based on a criterion according to the attribute, the created spread is added to one of the plurality of albums, a spread identical or similar to the spread added to the one album is automatically created, and the created identical or similar spread is added to each of one or more albums among the plurality of albums except for the one album; a position where a spread is added in the one album is the same as a position where a spread is added in the one or more albums; 13. An image processing method comprising:

Images