Information processing device and its control method

Abstract

To allow for appropriately generating a poster that matches a user-intended impression while matching the color scheme of an image and the entire poster.SOLUTION: An information processing device disclosed herein is configured to appropriately generate a poster expressing a user-intended impression while matching the color scheme of an image provided by a user and the entire poster.SELECTED DRAWING: Figure 2

Description

Technical Field

[0006] , , , ,

[0005] , , , The invention is characterized by comprising: an image acquisition means for acquiring an image; a receiving means for receiving a target impression from a user; a determination means for determining the main color of an image based on the color values ​​of the pixels constituting the image acquired in the image acquisition step; a selection means for selecting a color scheme pattern based on the determined main color and the target impression; and a poster generation means for generating a poster based on the image and the color scheme pattern selected by the selection means. ,

[0007] , ,

[0008] ,

[0001] The present invention relates to a technique for generating a poster.

Background Art

[0002] Conventionally, a method has been proposed in which a template for storing information such as the shape and arrangement of images, characters, graphics, etc. that make up a poster is prepared, and a poster is generated by automatically arranging images, characters, graphics, etc. according to the template.

[0003] In Patent Document 1, an apparatus for specifying color matching using both the representative color of an image and a word indicating sensibility has been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, although color matching suitable for the sensibility word and the image is specified, whether the impression of the entire poster created by them represents the impression intended by the user has not been considered at all.

[0006] An object of the present invention is to appropriately generate a poster that matches the impression intended by the user while matching the color scheme of the image and the entire poster.

Means for Solving the Problems

[0007] In order to solve the above problems, the information processing apparatus of the present invention The invention is characterized by comprising: an image acquisition means for acquiring an image; a receiving means for receiving a target impression from a user; a determination means for determining the main color of an image based on the color values ​​of the pixels constituting the image acquired in the image acquisition step; a selection means for selecting a color scheme pattern based on the determined main color and the target impression; and a poster generation means for generating a poster based on the image and the color scheme pattern selected by the selection means.

Effects of the Invention

[0008] According to the present invention, it is possible to appropriately generate a poster that matches the impression intended by the user while matching the color scheme of the image and the entire poster. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the hardware configuration of an information processing device. [Figure 2] This is a software block diagram for a poster creation application. [Figure 3] This is a diagram explaining the skeleton. [Figure 4] This is a diagram illustrating color scheme patterns. [Figure 5] This figure shows the display screen provided by a poster creation application. [Figure 6] This figure shows the display screen provided by a poster creation application. [Figure 7] This flowchart shows the poster generation process, which involves quantifying the impression given by a poster. [Figure 8] This is a diagram illustrating the subjective evaluation of posters. [Figure 9] This is a flowchart showing the poster generation process. [Figure 10] This is a diagram explaining how to select a skeleton. [Figure 11] This diagram illustrates how to select color schemes and fonts. [Figure 12] This is a software block diagram that explains the layout section in detail. [Figure 13] This is a flowchart showing the layout process. [Figure 14] This is a diagram illustrating the input for the layout section. [Figure 15] This is a diagram illustrating the operation of the layout section. [Figure 16] This figure shows an example of a section for specifying the target impression. [Figure 17] This is a software block diagram for a poster creation application. [Figure 18]It is a flowchart showing the poster generation process. [Figure 19] It is a diagram for explaining the combination generation unit. [Figure 20] It is a diagram for explaining the combination generation unit. [Figure 21] It is a software block diagram of the poster creation application. [Figure 22] It is a flowchart showing the poster generation process. [Figure 23] It is a diagram for explaining the process of extracting the main color. [Figure 24] It is a diagram for explaining the color scheme assignment process. [Figure 25] It is a software block diagram of the poster creation application. [Figure 26] It is a flowchart showing the poster generation process. [Figure 27] It is a software block diagram for explaining the layout section in detail. [Figure 28] It is a flowchart showing the layout process. [Figure 29] It is a diagram for explaining the image selection process.

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention according to the claims, and not all combinations of the features described in the present embodiment are essential for the solution means of the present invention. In addition, the same reference numerals are assigned to the same components, and the description thereof is omitted.

[0011] <<First Embodiment>> In the present embodiment, an example of a method for automatically generating a poster by operating an application for creating a poster (hereinafter also referred to as an "app") in an information processing apparatus will be described. In the following description, the "image" includes a still image and a frame image extracted from a moving image, unless otherwise specified.

[0012] Figure 1 is a block diagram showing the hardware configuration of an information processing device. The information processing device 100 can be, for example, a personal computer (hereinafter referred to as a PC) or a smartphone. In this embodiment, the information processing device will be described as a PC. The information processing device 100 includes a CPU 101, ROM 102, RAM 103, HDD 104, display 105, keyboard 106, pointing device 107, and data communication unit 108.

[0013] The CPU (Central Processing Unit / Processor) 101 comprehensively controls the information processing device 100 and, for example, reads a program stored in the ROM 102 into the RAM 103 and executes it, thereby realizing the operation of this embodiment. In Figure 1, there is one CPU, but it may be composed of multiple CPUs. The ROM 102 is a general-purpose ROM and stores, for example, programs executed by the CPU 101. The RAM 103 is a general-purpose RAM and is used, for example, as working memory to temporarily store various information when the CPU 101 executes a program. The HDD (Hard Disk Drive) 104 is a storage medium (storage unit) for storing image files, a database that holds processing results such as image analysis, and skeletons used by poster creation applications.

[0014] The display 105 is a display unit that displays the user interface (UI) and the resulting layout of image data (hereinafter also referred to as "image") to the user. Although not shown, it also has a display control unit for displaying on the display unit. The keyboard 106 and pointing device 107 receive instructions from the user. The display 105 may also have a touch sensor function. The keyboard 106 is used, for example, when the user inputs the number of double-page spreads of the poster they want to create on the UI displayed on the display 105. The pointing device 107 is used, for example, when the user clicks a button on the UI displayed on the display 105.

[0015] The data communication unit 108 communicates with external devices via a wired or wireless network. For example, the data communication unit 108 transmits data laid out by the automatic layout function to a printer or server that can communicate with the information processing device 100. The data bus 109 connects each block in Figure 1 so that they can communicate with each other.

[0016] The configuration shown in Figure 1 is merely an example and is not limited to it. For example, the information processing device 100 does not have a display 105, and the UI may be displayed on an external display.

[0017] In this embodiment, the poster creation application is stored on the HDD 104. As will be described later, the application is launched by the user selecting the application icon displayed on the display 105 using the pointing device 107 and clicking or double-clicking it.

[0018] Figure 2 is a software block diagram of a poster creation application. The poster creation application includes a poster creation condition specification unit 201, an image specification unit 202, a text specification unit 203, a target impression specification unit 204, a poster display unit 205, and a poster generation unit 210. The poster generation unit 210 includes an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a skeleton selection unit 214, a color scheme pattern selection unit 215, a font selection unit 216, a layout unit 217, an impression estimation unit 218, and a poster selection unit 219.

[0019] Each program module corresponding to each component shown in Figure 2 is included in the poster creation application described above. The CPU 101 then executes each program module, causing the CPU 101 to function as each component shown in Figure 2. Hereafter, each component shown in Figure 2 will be described as performing various processes. Figure 2 also specifically shows a software block diagram related to the poster generation unit 210, which performs the automatic poster creation function.

[0020] The poster creation condition specification unit 201 specifies poster creation conditions to the poster generation unit 210 in response to UI operations performed by the pointing device 107. In this embodiment, the poster creation conditions specify the poster size, the number of posters to be created, and the usage category. The poster size may be specified as the actual width and height, or as a paper size such as A1 or A2. The usage category indicates what purpose the poster will be used for, such as restaurants, school events, or sales.

[0021] The text specification unit 202 specifies the text information to be placed on the poster through UI operations using the keyboard 106. The text information to be placed on the poster refers to strings of characters such as titles, dates and times, and locations. The text specification unit 203 identifies whether each piece of text information is of a specific type, such as title, date and time, or location, and then outputs it to the skeleton acquisition unit 213 and the layout unit 217.

[0022] The image selection unit 203 specifies a group of images stored on the HDD 104 to be placed on the poster. The image group may be specified based on the structure of the file system containing the images, such as the device and directory, or it may be specified by accompanying information of individual images, such as the date and time of shooting, or by attribute information. The image selection unit 203 outputs the file paths of the specified images to the image acquisition unit 211.

[0023] The target impression specification unit 204 specifies the target impression of the poster to be created. The target impression is the impression that the poster should ultimately convey. In this embodiment, the degree to which a word representing an impression should be emphasized is specified by UI operation using the pointing device 107. Details about impressions will be described later.

[0024] The image acquisition unit 211 acquires the image group specified by the image selection unit 203 from the HDD 104. The image acquisition unit 211 outputs the acquired image data to the image analysis unit 212. It also outputs the number of acquired images to the skeleton acquisition unit 213. Images stored in the HDD 104 include still images and frame images extracted from videos. Still images and frame images are acquired from imaging devices such as digital cameras and smart devices. The imaging device may be provided by the information processing device 100 or by an external device. If the imaging device is an external device, the images are acquired via the data communication unit 108. As another example, still images may be illustrations created with image editing software or CG images created with CG production software. Still images and extracted images may be images acquired from a network or server via the data communication unit 108. Examples of images acquired from a network or server include social networking service images (hereinafter referred to as "SNS images"). Furthermore, the program executed by CPU 101 analyzes the data associated with each image to determine its source. SNS images may be obtained from SNS via the application, and the source may be managed within the application. The images are not limited to those described above; other types of images may also be used.

[0025] The image analysis unit 212 analyzes the image data acquired from the image acquisition unit 211 using a method described later to obtain image features. The analysis process involves object recognition and extraction of major colors to obtain these image features. The image analysis unit 212 associates the acquired image features with the image data and outputs them to the color scheme pattern selection unit 215 and the layout unit 217.

[0026] The skeleton acquisition unit 213 acquires a group of skeletons from the HDD 104 that match the conditions specified by the poster creation condition specification unit 201, the text specification unit 202, and the image acquisition unit 211. In this embodiment, a skeleton is information that represents the arrangement of strings, images, shapes, etc. to be placed on a poster.

[0027] Figure 3 shows an example of a skeleton. On the skeleton 301 in Figure 3(a), four graphic objects 302-305, one image object 306, and three text objects 307-309 are placed. Each object records not only its position, size, and angle, which indicate where it will be placed, but also metadata necessary to generate the poster. Figure 3(b) shows an example of metadata. For example, text objects 307-309 hold the type of text information to be placed as metadata. Here, text object 307 indicates that the title will be placed, text object 308 that the subtitle will be placed, and text object 309 that the body text will be placed. In addition, graphic objects 302-305 hold the shape of the graphic as metadata, and graphic objects 304 and 305 also hold a group ID. Here, graphic objects 302, 304, and 305 indicate that they are rectangles, and graphic object 303 indicates that it is an ellipse. Also, graphic object 304 is assigned group ID 1. Here, the group ID is information referenced when assigning colors, as described later, and indicates that the same color is assigned to the same group ID. Note that the object type and metadata are not limited to these. For example, there may be a map object for placing maps, or a barcode object for placing QR codes (registered trademarks) or barcodes. Also, metadata for text objects may include metadata that indicates the line spacing and character spacing. The metadata may include the purpose of the skeleton and be used to control whether or not the skeleton can be used depending on the purpose. Note that the target impression of the skeleton changes depending on the placement of images, text, graphics, fonts, color schemes, graphics, etc.

[0028] The skeletons may be saved on HDD104 in CSV format, for example, or in a database format such as SQL. The skeleton acquisition unit 213 outputs the skeletons acquired from HDD104 to the skeleton selection unit 214.

[0029] The skeleton selection unit 214 selects a group of skeletons from those acquired from the skeleton acquisition unit 213 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 217. Since the overall layout of the poster is determined by the skeleton, the variations of the generated poster can be increased by preparing various types of skeletons in advance.

[0030] The color scheme pattern selection unit 215 retrieves a group of color schemes from the HDD 104 that match the target impression specified by the target impression specification unit 204, and selects a candidate group of color schemes based on the main colors of the image obtained from the image analysis unit 212. The color scheme pattern selection unit 215 outputs the selected group of color schemes to the layout unit 217. A color scheme pattern is a combination of colors used for a poster.

[0031] Figure 4 shows an example of a color scheme pattern table. In this embodiment, the color scheme pattern is represented as a combination of four colors. The color scheme ID column in Figure 4 is an ID for uniquely identifying the color scheme. Columns 1 to 4 represent the colors in RGB order, with each color value ranging from 0 to 255. Although a four-color scheme pattern is used in this embodiment, other numbers of colors may be used, or multiple numbers of colors may be mixed.

[0032] The font selection unit 216 retrieves a set of fonts from the HDD 104 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 217.

[0033] The layout unit 217 generates poster data by laying out various data for each skeleton acquired from the skeleton selection unit 214. The layout unit 217 places text acquired from the text specification unit 202 and image data acquired from the image analysis unit 212 for each skeleton, and applies a color scheme acquired from the color scheme pattern selection unit 216. The layout unit 217 outputs the generated poster data to the impression estimation unit 218.

[0034] The impression estimation unit 218 estimates the impression of each poster data obtained from the layout unit 217 and associates it with each poster data. The impression estimation unit 218 outputs the estimated impression associated with each poster data to the poster selection unit 219.

[0035] The poster selection unit 219 compares the target impression specified by the target impression specification unit 204 with the estimated impression obtained from the impression estimation unit 218, and selects poster data in which the distance between the estimated impression and the target impression is less than or equal to a threshold. The threshold may be pre-programmed into the poster creation application, or it may be specified by the user through the poster creation condition specification unit 201. The selection results are saved in the HDD 104. The poster selection unit 219 outputs the selected poster data to the poster display unit 205.

[0036] The poster display unit 205 outputs a poster image for display on the display 105 according to the poster data acquired from the poster selection unit 219. The poster image is, for example, bitmap data. The poster display unit 205 displays the poster image on the display 105.

[0037] When the poster creation application is installed on the information processing device 100, a launch icon appears on the top screen (desktop) of the OS (operating system) running on the information processing device 100. When the user double-clicks the launch icon displayed on the display 105 with the pointing device 107, the application program stored on the HDD 104 is loaded into the RAM 103 and launched by being executed by the CPU 101.

[0038] Although not shown in the illustration, the poster creation application may also include a function that, after displaying the generated result on the poster display unit 205, allows the user to further edit the arrangement, color, and shape of images, text, and graphics (shapes, illustrations, photographs, etc.) to modify the design to the user's specifications.

[0039] Furthermore, if the system includes a function to print poster data stored on the HDD104 using a printer according to the conditions specified in the poster creation condition specification unit 201, the user will be able to obtain a printed copy of the poster they created.

[0040] <Example of display screen> Figure 5 shows an example of an application screen 501 provided by the poster creation application. The application screen 501 (first screen) is displayed on the display 105 when the application is launched and is a screen for receiving the target impression from the user. The user sets the poster creation conditions, text, and images, which will be described later, via the application screen 501, and the poster creation condition specification unit 201, image specification unit 202, and text specification unit (character acquisition unit) 203 acquire the settings from the user through this UI screen.

[0041] The title box 502, subtitle box 503, and body box 504 accept the specification of text information to be placed on the poster. In this embodiment, three types of text information are accepted, but this is not limited to these. For example, additional text information such as location and date and time may be accepted. Also, it is not necessary for all specifications to be completed, and blank fields may be left.

[0042] The image selection area 505 is the area that displays the image to be placed on the poster. Image 506 represents a thumbnail of the selected image. The image add button 507 is a button for adding an image to be placed on the poster. When the user presses the image add button 507, the image selection unit 202 displays a dialog screen for selecting a file stored on the HDD 104 and accepts the user's selection of an image file. The thumbnail of the selected image is added to the image selection area 507.

[0043] The impression sliders 508-511 are used to set the target impression of the poster you are creating. For example, slider 508 sets the target impression of luxury, with the right side indicating a higher level of luxury and the left side indicating a lower level of luxury (cheaper-looking poster). For example, setting impression slider 508 to the right and impression slider 511 to the left, meaning a poster with a high level of luxury and low level of weightiness, will have an elegant impression. On the other hand, keeping impression slider 508 on the right and setting impression slider 511 to the right as well, meaning a poster with a high level of both luxury and weightiness, will have a gorgeous impression. In this way, by combining multiple impression sliders, it is possible to set different directions of impression, even with the same level of luxury.

[0044] Radio buttons 512 can be used to control the on / off state of each target impression. Figure 5 shows the state where luxury and approachability are on, and dynamism and gravitas are off. Selecting off with radio button 512 excludes that impression from impression control. For example, a user who wants to create a calm poster with low dynamism but has no specific requirements for other impressions can turn off all radio buttons 512 except for dynamism to generate a poster that specializes in low dynamism. This allows for highly flexible control, such as using all target impressions or using only some target impressions in poster generation.

[0045] The size list box 513 is a list box for setting the size of the poster to be created. A user can click on it using the pointing device 107 to display a list of available poster sizes, from which they can select one.

[0046] The "Number of creations" box (514) allows you to set the number of candidate posters to create.

[0047] Category list box 515 allows you to set the intended use category for the poster you are creating.

[0048] The reset button 516 is used to reset the various settings information on the app screen 501.

[0049] When the user presses the OK button 517, the poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 output the content set on the application screen 501 to the poster generation unit 210. At this time, the poster creation condition specification unit 201 obtains the size of the poster to be created from the size list box 513, the number of posters to be created from the number box 514, and the purpose category of the poster to be created from the category list box 515. The text specification unit 202 obtains the text information to be placed on the poster from the title box 503, subtitle box 503, and body box 504. The image specification unit 203 obtains the image file path to be placed on the poster from the image specification area 505. The target impression specification unit 204 obtains the target impression of the poster to be created from the impression sliders 508~511 and radio buttons 512. The poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 may process the values ​​set on the application screen 501. For example, the text specification unit 202 may remove unnecessary whitespace characters at the beginning or end of the input text information. The target impression specification unit 204 may also format the values ​​specified by the impression sliders 508 to 511. In this embodiment, the leftmost setting of the slider is set to -2, and the rightmost setting is set to +2, and the values ​​are formatted to integer values ​​from -2 to +2. This corresponds to the impression being low (-2), slightly low (-1), neutral (0), slightly high (+1), and high (+2). The reason for formatting to -2 to +2 is to match the scale with the estimated impression described later and facilitate distance calculation, but it is not limited to this and may be normalized to 0 to 1, for example.

[0050] Figure 6 shows an example of a poster preview screen where the poster image generated by the poster display unit 205 is displayed on the display 105. When the OK button 309 on the application screen 501 is pressed and poster generation is complete, the screen displayed on the display 105 transitions to the poster preview screen 601.

[0051] Poster image 602 is a poster image output by the poster display unit 205. The poster generation unit 210 generates the number of posters specified by the poster creation condition specification unit 201, so the created posters are also displayed as a list in poster image 602. The user can select a poster by clicking on it with the pointing device 107.

[0052] The edit button 603 allows you to edit the selected poster through a UI that provides editing functions not shown in the image.

[0053] The print button 604 allows you to print the selected poster via the printer's control UI (not shown).

[0054] <Quantifying the impression of a poster> This section explains a method for quantifying the impression of a poster, which is necessary for the poster generation process described later. This poster impression quantification involves quantifying the impressions people have of various posters. Simultaneously, it derives a correspondence between poster images and the impressions they convey. This allows us to estimate the impression of a poster from the poster image we will create. Once the impression is estimated, it becomes possible to control the poster's impression by modifying it, or to search for a poster that possesses a specific target impression. Note that the poster impression quantification process is performed, for example, by running an impression learning application in an information processing device before the poster generation process to learn the poster's impression.

[0055] Figure 7 is a flowchart showing the process of quantifying the impression of a poster. The flowchart shown in Figure 7 is implemented, for example, by the CPU 101 reading a program stored in the HDD 104 into the RAM 103 and executing it. The process of quantifying the impression of a poster will be explained with reference to Figure 7. In the description of each process, the symbol "S" indicates a step in the flowchart (the same applies hereafter in this specification).

[0056] In S701, the subjective evaluation acquisition method performs subjective evaluations of impressions of posters. Figure 8 illustrates an example of a method for subjective evaluation of impressions of posters. The subjective evaluation acquisition method presents posters to subjects and obtains subjective evaluations of the impressions they receive from the posters. Measurement methods such as the Semantic Differential (SD) method and the Likert scale can be used. Figure 8 shows an example of a questionnaire using the SD method, where multiple evaluators are presented with pairs of adjectives that express impressions, and scores are assigned to the pairs of adjectives that are recalled from the target poster. After obtaining subjective evaluation results from multiple subjects for multiple posters, the subjective evaluation acquisition method calculates the average value of the responses for each pair of adjectives to obtain a representative score for that pair. Note that the subjective evaluation method for impressions does not have to be the SD method; it is sufficient to have words that express impressions and corresponding scores.

[0057] In S702, the factor analysis means performs factor analysis on the subjective evaluation results obtained by the subjective evaluation acquisition means. If the subjective evaluation results are left as they are, the number of adjective pairs becomes a dimensionality, making control complex. Therefore, it is desirable to reduce the dimensionality to an efficient level using analytical methods such as principal component analysis or factor analysis. In this embodiment, we will explain the process assuming that the dimensionality has been reduced to four factors by factor analysis. Naturally, this number varies depending on the selection of adjective pairs in the subjective evaluation and the factor analysis method used. Furthermore, the output of the factor analysis is assumed to be standardized. That is, each factor is scaled so that the mean is 0 and the variance is 1 in the poster used for the analysis. This allows the -2, -1, 0, +1, and +2 of the impressions specified in the target impression specification unit 204 to directly correspond to -2σ, -1σ, mean, +1σ, and +2σ in each impression, facilitating the calculation of the distance between the target impression and the estimated impression, as described later. In this embodiment, the four factors are referred to as a sense of luxury, familiarity, dynamism, and gravitas, as shown in Figure 5. However, these are names conveniently assigned to convey an impression to the user through the user interface, and each factor is composed of multiple adjective pairs influencing each other.

[0058] In S703, the impression learning method associates poster images with impressions. While it is possible to quantify the impressions of posters that have been subjectively evaluated using the method described above, it is necessary to estimate impressions of posters to be created without subjective evaluation. Associating poster images with impressions can be achieved by training a model that estimates impressions from poster images using deep learning methods such as Convolutional Neural Networks (CNNs) or machine learning methods using decision trees. In this embodiment, the impression learning method takes poster images as input and four factors as outputs, and performs supervised deep learning using a CNN. That is, it learns subjectively evaluated poster images and their corresponding impressions as ground truth to create a deep learning model, and then estimates impressions by inputting unknown poster images into that trained model.

[0059] The deep learning model created above is stored, for example, on HDD104, and the impression estimation unit 218 loads the deep learning model stored on HDD104 into RAM103 and executes it. The impression estimation unit 218 converts the poster data acquired from the layout unit 217 into an image and estimates the impression of the poster by running the deep learning model loaded into RAM103 on CPU101 or GPU109. In this embodiment, a deep learning method is used, but it is not limited to this. For example, if a machine learning method such as a decision tree is used, feature quantities such as the average brightness and edge quantity of the poster image may be extracted by image analysis, and a machine learning model that estimates the impression based on those feature quantities may be created.

[0060] <Processing flow> Figure 9 is a flowchart of the poster generation process of a poster creation application. The flowchart shown in Figure 9 is realized, for example, when the CPU 101 reads a program stored in the HDD 104 into the RAM 103 and executes it. In the explanation of Figure 9, each component shown in Figure 2 is described as performing a process, which is activated when the CPU 101 executes the poster creation application. The poster generation process will be explained with reference to Figure 9. In the explanation of each process, the symbol "S" indicates a step in the flowchart (the same applies hereafter in this specification).

[0061] In the S901, the poster creation application displays the application screen 501 on the display 105. The user inputs various settings through the UI screen of the application screen 501 using the keyboard 106 or pointing device 107.

[0062] In S902, the poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 obtain settings from the application screen 501.

[0063] In S903, the skeleton selection unit 214, the color scheme pattern selection unit 215, and the font selection unit 216 determine the number of skeletons, color scheme patterns, and fonts to select according to the number of posters to be created specified in the poster creation condition specification unit 201. In this embodiment, the layout unit 217 generates poster data with a number of skeletons × number of color scheme patterns × number of fonts by a method described later. At this time, the number of skeletons, color scheme patterns, and fonts to select is determined so that the number of posters generated exceeds the number to be created. In this embodiment, the number of skeletons, color scheme patterns, and fonts are determined according to formula M2.

[0064]

number

[0065] For example, if the number of posters to be created is 6, the number of posters to be selected is 3, resulting in 27 poster data generated by the layout unit 217, from which the poster selection unit 219 selects 6 poster data. This allows the poster selection unit 219 to select posters from among the generated poster data that better match the target impression in terms of overall impression.

[0066] In S904, the image acquisition unit 211 acquires image data. Specifically, the image acquisition unit 211 reads the image file in the HDD 104 specified by the image selection unit 203 into the RAM 103.

[0067] In S905, the image analysis unit 212 performs object recognition and extraction of major colors on the image acquired in S904. Here, known methods can be used for the object recognition process. In this embodiment, objects are recognized by a discriminator created by Deep Learning. The discriminator outputs a likelihood of 0 to 1 for whether a certain pixel constituting the image is a pixel constituting each object, and recognizes that there are objects in the image that exceed a certain threshold. By recognizing the object image, the image analysis unit 212 can obtain the type and location of objects such as faces, pets such as dogs or cats, flowers, food, buildings, ornaments, and landmarks.

[0068] In the primary color extraction process, the image analysis unit 212 extracts the dominant color from the image. In this embodiment, the primary color is identified by determining the color with the highest frequency of appearance in the image. Figure 23 is a diagram illustrating primary color extraction in more detail. The graph in Figure 23 is a histogram schematically representing the frequency of appearance of colors in the image. Here, the horizontal axis represents the color of a pixel in three channels (R, G, B), and the vertical axis represents the number of pixels belonging to the range of ±8 before and after each R, G, and B channel. For example, the bar graph (8, 8, 24) represents the number of pixels where the R channel is 0 to 16, the G channel is 0 to 16, and the B channel is 16 to 32. That is, the histogram consists of 16 gradations per channel and a total of 4096 bins. In this embodiment, (248, 248, 216), which is the color with the highest frequency of appearance (2301), is extracted as the primary color. In this embodiment, the histogram was created by dividing the RGB color space into 16 gradations per channel, but this is not the only method. Other tonal ranges are also acceptable, as are other color spaces such as Lab or HSV.

[0069] In S906, the skeleton acquisition unit 213 acquires skeletons that meet various setting conditions. In this embodiment, it is assumed that each skeleton is written to a separate file and stored in the HDD 104. The skeleton acquisition unit 213 sequentially reads the skeleton files from the HDD 104 into the RAM 103, keeping the skeletons that meet the setting conditions in the RAM 103 and deleting the skeletons that do not meet the conditions from the RAM 103. The skeleton acquisition unit 213 first determines whether the size of the skeleton matches the poster size specified in the poster creation condition specification unit 201. Note that while the size matches here, it is also acceptable if only the aspect ratio matches. In that case, the skeleton acquisition unit 213 expands or shrinks the coordinate system of the read skeleton to acquire a skeleton that matches the poster size specified in the poster creation condition specification unit 201. Next, it determines whether the category of the skeleton matches the usage category specified in the poster creation condition specification unit 201. Skeletons used only for specific purposes have their purpose category recorded in the skeleton file, and are not retrieved unless the corresponding purpose category is selected. This prevents the skeleton from being used in other purpose categories if it is designed specifically for a particular purpose, such as if it depicts a school image using graphics or a sports equipment pattern. Next, the skeleton retrieval unit 213 determines whether the number of image objects in the loaded skeleton matches the number of images retrieved by the image retrieval unit 211. Finally, the skeleton retrieval unit 213 determines whether the character objects in the loaded skeleton match the character information specified in the text specification unit 202. More specifically, it determines whether the type of character information specified in the text specification unit 202 exists within the skeleton. For example, suppose a string is specified in the title box 502 and body box 504 on the app screen 501, and a blank is specified in the subtitle box 503.In this case, the skeleton is searched for all character objects within the skeleton, and is considered a match if both a character object with "Title" set as the type of character information in the metadata and a character object with "Body" specified are found; otherwise, it is considered a failure. As described above, the skeleton acquisition unit 213 stores on RAM 103 skeletons in which the skeleton size, the number of image objects, and the type of character objects all match the set conditions. In this embodiment, the skeleton acquisition unit 213 checked all skeleton files on HDD 104, but is not limited to this. For example, a poster creation application may store a database on HDD 104 that associates the file paths of skeleton files with search conditions (skeleton size, number of image objects, and type of character objects) in advance. In that case, the skeleton acquisition unit 213 can acquire skeleton files at high speed by reading only the skeleton files that match the search results from the database from HDD 104 to RAM 103.

[0070] In S907, the skeleton selection unit 214 selects a skeleton from among the skeletons acquired in S906 that matches the target impression specified in the target impression specification unit 204. Here, Figure 10 is a diagram illustrating how the skeleton selection unit 214 selects a skeleton. Figure 10(a) is a diagram showing an example of a table that links skeletons to impressions. The skeleton name column in Figure 10(a) contains the file name of the skeleton, and the columns for luxury, familiarity, dynamism, and solidity indicate how much the skeleton contributes to each impression. First, the skeleton selection unit 214 calculates the distance between the target impression acquired from the target impression specification unit 204 and the skeleton impression table in Figure 10(a). For example, if the target impression is "luxury +1, familiarity -1, dynamism -2, solidity +2", the distance calculated by the skeleton selection unit 214 will be as shown in Figure 10(b). In this embodiment, Euclidean distance is used as the distance. Next, the skeleton selection unit 214 selects the top N skeletons with the smallest distance in Figure 8(b). In this embodiment, the skeleton selection unit 214 selects the top two skeletons, namely skeleton 1 and skeleton 4.

[0071] Here, the setting method for N may be a fixed value, or it may be variable depending on the conditions specified in the poster creation condition specification unit 201. For example, if the number of posters to be created is specified as 6 in the number of posters box 514 on the app screen 501, the poster generation unit 210 will generate 6 posters. In the layout unit 217, which will be described later, the poster is generated by combining the skeleton, color scheme, and font selected in the skeleton selection unit 214, the color scheme selection unit 215, and the font selection unit 216. For example, by selecting 2 skeletons, 2 color schemes, and 2 fonts, 2 x 2 x 2 = 8 posters can be generated, thus satisfying the condition of 6 posters to be created. In this way, the number of skeletons to be selected may be determined according to the conditions specified in the poster creation condition specification unit 201.

[0072] Furthermore, the range of values ​​for each impression in the skeleton impression table in Figure 10(a) does not need to be the same as the range of values ​​for the impression specified by the target impression specification unit 204. In this embodiment, the range of values ​​for the impression specified by the target impression specification unit 204 is -2 to +2, but the range of values ​​for the impressions in the skeleton impression table may be different. In that case, the distance calculation is performed after scaling the range of values ​​in the skeleton impression table to match the range of values ​​for the target impression. Also, the distance calculated by the skeleton selection unit 214 is not limited to Euclidean distance; it is acceptable as long as it can calculate the distance between vectors, such as Manhattan distance or cosine similarity. In addition, if the target impression is set to off using radio button 517, it is excluded from the distance calculation.

[0073] A skeleton impression table can be created by fixing elements such as color schemes, fonts, and images and text information placed on the skeleton, generating posters based on each skeleton, and estimating their impressions. In other words, by estimating the impressions of posters that use the same colors and images but have different arrangements, the relative characteristics of each skeleton can be compiled into a table. At that time, it is desirable to perform a process to cancel out the impression caused by the color schemes and images used, such as standardizing the overall estimated impression or averaging the impressions of multiple posters generated from a single skeleton using multiple color schemes and images. This allows for the creation of a table showing the influence of arrangement on the impression, such as how a skeleton with a small image will have its impression determined by elements such as graphics and text rather than the image, or how a tilted arrangement of images and text will create a strong sense of dynamism. Figure 10(c) is an example of skeletons corresponding to skeletons 1 to 4 in Figure 10(a). For example, in skeleton 1, image objects and text objects are arranged regularly, and the image area is small, resulting in a low sense of dynamism. Skeleton 2 has a high sense of familiarity and a low sense of weight due to the circular shape and image objects. Skeleton 3 has a high sense of dynamism due to the large placement of image objects and the placement of tilted shape objects overlapping the image objects. Skeleton 4 has a high sense of weight and a low sense of dynamism because images are placed throughout the entire skeleton and text objects are kept to a minimum. In this way, the impression changes depending on the placement of images, shapes, and text. Note that the method of creating the skeleton impression table is not limited to these methods; it may also be estimated from the characteristics of the placement information itself, such as the area and coordinates of images and title strings, or it may be adjusted manually. The skeleton impression table is stored in HDD 104, and the skeleton selection unit 214 reads the skeleton impression table from HDD 104 into RAM 103 and references it.

[0074] In S908, the color scheme selection unit 215 selects a color scheme from the colors contained in the image acquired in S904 and the target impression entered by the user in S901. In other words, it selects a color scheme that matches the main color extracted by the image analysis unit 212 and the target impression specified by the target impression specification unit 204. First, the color scheme selection unit 215 selects a color scheme that includes the main color acquired from the image analysis unit 212. That is, it calculates the color difference between colors 1 to 4 of the color scheme shown in Figure 4 and the main color, and selects a color scheme in which the color difference of at least one of colors 1 to 4 is below a threshold. As a result, the colors assigned to the poster will be the same as the colors contained in the image, preventing the colors of the image and the overall poster from clashing and maintaining a sense of unity throughout the poster. Next, the color scheme selection unit 215, similar to S906, refers to the impression table corresponding to the color scheme and further selects a color scheme from the color scheme selected by the main color according to the target impression. Figure 11(a) shows an example of a color scheme impression table that links color schemes with impressions. The color scheme selection unit 215 calculates the distance between the "luxury" column to the "heavy" column in Figure 11 and the target impression, and selects the top N color schemes with the smallest distances. In this embodiment, the top two color schemes are selected. This makes it possible to select a color scheme that is suitable for the target impression from among color schemes that can maintain the overall unity of the poster. The color scheme impression table, like the skeleton impression table, is created by fixing the skeleton and images other than the color scheme, and then creating posters with different color schemes and estimating the impressions, thereby creating a table of the impression trends of the color schemes. In this embodiment, the most frequently occurring color is used as the main color, but this is not limited to this, and multiple colors may be extracted. For example, when extracting the second main color, the image analysis unit 212 considers the histogram in Figure 23 in a three-dimensional space of (R, G, B) and selects the second largest color among the colors that are the local maximums as the second main color.A local maximum occurs when, for example, when focusing on the color (24, 24, 24), the number of pixels of neighboring colors in the RGB space—(24, 24, 8), (24, 24, 40), (24, 8, 24), (24, 40, 24), (8, 24, 24), and (40, 24, 24)—is all smaller than the number of pixels of (24, 24, 24). Note that six or more adjacent colors may be used as neighboring colors. This allows for the extraction of not only the most frequently occurring color, but also prominent colors that act as accent colors in the image. Furthermore, the main colors may be thresholded by the number of pixels. For example, by extracting only colors with a certain number of pixels or more as the main color, such as "the local maximum of the histogram and with a pixel count of 5% or more of the total number of pixels in the image," locally noticeable colors like noise can be excluded. As another example, the image analysis unit 212 may extract colors with a pixel count above a threshold as the main color. This allows the system to extract not just the most recent color, but all similar colors as the primary color, even in images where similar colors are spread over a wide area, such as in a gradient. Furthermore, if multiple images are specified in the image selection unit 203, the image analysis unit 212 may extract multiple primary colors by calculating the primary color for each image. If two or more primary colors are extracted, the color scheme pattern selection unit 215 selects a color scheme pattern that includes at least one of the primary colors.

[0075] In S909, the font selection unit 216 selects a font that matches the target impression specified in the target impression specification unit 204. Similar to S906, the font selection unit 216 refers to the impression table corresponding to the font and selects a font according to the target impression. Figure 11(b) shows an example of a font impression table that links fonts to impressions. Note that, similar to the skeleton impression table, the font impression table can be created by fixing the skeleton other than the font, the color scheme, and the image, and then creating posters with different fonts and estimating the impressions, thereby creating a table of font impression trends.

[0076] In S910, the layout unit 217 sets text information, images, color schemes, and fonts for the skeleton selected by the skeleton selection unit 214 to generate a poster.

[0077] S910 and the layout unit 217 will be explained in detail using Figures 12, 13, 14, and 15. Figure 12 is an example of a software block diagram that explains the layout unit 217 in detail. Figure 13 is a flowchart that explains S910 in detail. Figure 14 is a diagram that explains the information input to the layout unit 217. Figure 14(a) is a table summarizing the character information specified by the text specification unit 202 and the image specified by the image specification unit 203. Figure 14(b) is an example of a color scheme pattern obtained from the color scheme pattern selection unit 215, and Figure 14(c) is an example of a font obtained from the font selection unit 216. Figure 15 is a diagram that explains the processing process of the layout unit 217.

[0078] In S1301, the layout unit 217 enumerates all possible combinations of skeletons obtained from the skeleton selection unit 214, color patterns obtained from the color pattern selection unit 215, and fonts obtained from the font selection unit 216. The layout unit 217 then generates poster data for each combination in order through subsequent layout processing. For example, if the number of skeletons obtained from the skeleton selection unit 214 is 3, the number of color patterns obtained from the color pattern selection unit 215 is 2, and the number of fonts obtained from the font selection unit 216 is 2, the layout unit 217 generates 3 × 2 × 2 = 12 poster data.

[0079] In S1302, the color assignment unit 1201 assigns each color pattern obtained from the color pattern selection unit 215 to each skeleton obtained from the skeleton selection unit 214. Figure 15(a) shows an example of a skeleton. In this embodiment, an example of assigning the color pattern of color ID 1 in Figure 14(b) to the skeleton 1501 in Figure 15(a) will be described. The skeleton 1501 in Figure 15(a) consists of three graphic objects 1502 to 1504, one image object 1505, and three character objects 1506 to 1508. First, the color assignment unit 1201 assigns colors to the graphic objects 1502 to 1504. In this embodiment, the color assignment unit 1201 assigns colors so that the areas of color 1, color 2, and color 3 of the color pattern approach a ratio of 70:25:5. It is known that a well-balanced design is easily achieved by using a color scheme ratio of 70% (base color), 25% (main color), and 5% (accent color), and this embodiment follows that principle. In this embodiment, color 4 of the color scheme pattern is assigned to the title text object and is not used for the color scheme of the graphic objects. This is to make the text stand out by using the accent color for the text objects. More specifically, first, the area ratio of all graphic objects is calculated. That is, the area of ​​each graphic object is divided by the total area of ​​all graphic objects. At this time, areas where image objects or graphic objects overlap and obscure the graphic objects are excluded from the area calculation. Figure 15(b) shows the area ratio of each graphic object placed on skeleton 1501. When colors 1 to 3 are assigned to these graphic objects, the area ratio of each color is calculated. Figure 15(c) is an example of a table showing the color scheme, the area ratio in that color scheme, and the error from the target color scheme ratio of 70:25:5. The color scheme and area ratio are calculated for all possible combinations as shown in Figure 15(c), and the color scheme is applied using the combination that minimizes the area error. In this embodiment, color 1 is assigned to graphic object 1502, and color 2 is assigned to graphic objects 1503 and 1504.

[0080] Next, the last color of the color scheme is assigned to the text object whose metadata type is "Title". In this embodiment, color 4 is assigned to text object 1506. Next, the text color is set based on the brightness of the background of the text object for text objects whose metadata type is other than "Title". In this embodiment, if the brightness of the background of the text object is below a threshold, the text color is set to white; otherwise, the text color is set to black. Figure 15(d) shows the state of the skeleton 1509 after the above color scheme assignment process. The color scheme assignment unit 1201 outputs the colored skeleton data to the image placement unit 1202.

[0081] In S1303, the image placement unit 1202 places the image data obtained from the image analysis unit 212 onto the skeleton data obtained from the color assignment unit 1201, based on the accompanying analysis information. In this embodiment, the image placement unit 1202 assigns the image data 1401 to the image object 1504 within the skeleton. Furthermore, if the aspect ratio of the image object 1504 and the image data 1401 are different, the image placement unit 1202 performs trimming to match the aspect ratio of the image data 1401 to that of the image object 1504. More specifically, based on the face position and object position obtained by the image analysis unit 202 analyzing the image data 1401, the image placement unit 1202 performs trimming to minimize the reduction in the face area and object area due to trimming. Note that the trimming method is not limited to this, and other trimming methods may be used, such as trimming the center of the image or devising a composition so that the face position forms a triangular composition. The image placement unit 1202 outputs the skeleton data with image assignments to the image correction unit 1203.

[0082] In S1304, the image correction unit 1203 obtains the image-assigned skeleton data from the image placement unit 1202 and performs corrections on the images placed on the skeleton. In this embodiment, if the image resolution is insufficient, upsampling processing is performed using super-resolution processing. First, the image correction unit 1203 checks whether the images placed on the skeleton meet a certain resolution. For example, suppose a 1600px × 1200px image is assigned to a 200mm × 150mm area on the skeleton. In this case, the print resolution of the image can be calculated using equation M1.

[0083]

number

[0084] Next, the image correction unit 1203 compensates for the perceived resolution by performing super-resolution processing if the print resolution of the image is below a threshold, and conversely, does not perform any image correction if the resolution is sufficient. In this embodiment, super-resolution processing is performed when the resolution is less than 300 dpi.

[0085] In S1305, the font setting unit 1204 sets the font obtained from the font selection unit 216 to the image-corrected skeleton data obtained from the image correction unit 1203. Figure 10(c) shows an example of the font selected by the font selection unit 216. In this embodiment, the font is set for the character objects 1505, 1506, and 1507 of the skeleton 1508. In posters, from the perspective of attracting attention, it is common to set a font that stands out for the title and a font that is easy to read for the other characters. Therefore, in this embodiment, the font selection unit 216 selects two types of fonts: a title font and a body font. The font setting unit 1204 sets the title font for the character object 1505, which is the title, and the body font for the other character objects 1506 and 1507. The font setting unit 1204 outputs the skeleton data with the font set to the text placement unit 1205. In this embodiment, the font selection unit 216 selected two types of fonts, but this is not limited to this; for example, only the title font may be selected. In that case, the font setting unit 1204 uses the font corresponding to the title font as the body font. That is, if the title is a Gothic font, then a representative Gothic font with high readability should be chosen for the other text as well; if the title is a Mincho font, then a representative Mincho font should be chosen for the other text as well, and so on. The body font should be set to match the type of title font. Of course, the title font and body font can also be the same. Alternatively, the font can be used differently depending on the degree to which you want to make something stand out, such as using the title font for the title and subtitle text objects and the body font for other text objects, or using the title font for fonts above a certain size.

[0086] In S1306, the text placement unit 1205 places the text specified by the text specification unit 202 onto the font-configured skeleton data obtained from the font-configured unit 1204. In this embodiment, each text shown in 14(a) is assigned by referring to the metadata of the skeleton's character objects. That is, the title "Summer Grand Sale" is assigned to character object 1505, and the subtitle "Beat the Summer Heat" is assigned to character object 1506. Since no body text is set, nothing is assigned to character object 1507. Figure 15(c) shows skeleton 1509, which is an example of skeleton data after processing by the text placement unit 1205. The text placement unit 1205 outputs the skeleton data with the placed text to the text decoration unit 1206.

[0087] In S1307, the text decoration unit 1206 applies decoration to the character objects within the pre-placed text skeleton obtained from the text placement unit 1205. In this embodiment, if the color difference between the title character and its background area is below a threshold, an outline is added to the title character. This improves the readability of the title. The text decoration unit 1206 outputs the decorated skeleton data, i.e., the poster data with the layout completed, to the impression estimation unit 218.

[0088] In S1308, the layout unit 217 determines whether all poster data has been generated. If poster data has been generated for all combinations of skeletons, color schemes, and fonts, the layout process ends and the program proceeds to S911. If not all poster data has been generated, the program returns to S1301 and generates poster data for the remaining combinations.

[0089] This concludes the explanation of S910. Return to Figure 9.

[0090] In S911, the impression estimation unit 218 renders each poster data acquired from the layout unit 217, estimates the impression of the rendered poster image, and links the estimated impression to the poster data. This allows for the evaluation of the impression of the final poster, including images and text, as well as the impression of individual elements of the poster, such as color scheme and arrangement. For example, even with the same color scheme, the arrangement will change if the skeleton is different, so the actual area occupied by each color will differ. Therefore, it is necessary to evaluate not only the impression trends of individual color schemes and skeletons, but also the impression of the final poster.

[0091] In S912, the poster selection unit 219 selects posters to present to the user from the poster data and estimated impression obtained from the impression estimation unit 218. In this embodiment, the poster selection unit 219 selects posters in which the distance between the target impression and the estimated impression of the poster is less than or equal to a predetermined threshold. If the number of selected posters is less than the number of posters to be created specified in the poster creation condition specification unit 201, the poster selection unit 219 selects the remaining posters in ascending order of the distance between the target impression and the estimated impression of the poster. In this embodiment, the poster selection unit 219 also selects the remaining posters, but this is not limited to this. For example, if the number of posters selected by the poster selection unit 219 is less than the number to be created, the preview screen 601 may display a message indicating that there are insufficient posters. Alternatively, the poster selection unit 219 may select the remaining posters and then display on the preview screen 601 in a way that distinguishes between posters where the distance between the target impression and the estimated impression is less than or equal to the threshold and posters where the distance is greater than or equal to the threshold. As another example, if there are insufficient selected posters, the process returns to S903, and the number of skeletons, color patterns, and fonts to be selected may be increased.

[0092] In S913, the poster display unit 205 renders the poster data selected by the poster selection unit 219 and outputs the poster image to the display 105. That is, it displays the preview screen 601 shown in Figure 6.

[0093] The above is a description of the poster generation process flow, which generates a poster based on the user's specified impression.

[0094] As described above, this embodiment makes it possible to generate posters that express the impression desired by the user. More specifically, in this embodiment, multiple variations of poster candidates can be generated according to the target impression by selecting and combining each element constituting the poster, such as the skeleton, color scheme, and font, based on the target impression. At that time, by using a color scheme that has colors similar to the main color of the image input by the user, the overall unity of the poster can be maintained. Furthermore, by estimating the overall impression of the poster and selecting a poster that is close to the target impression from the group of poster candidates, it is possible to generate a poster in which not only the individual elements but also the overall impression aligns with the user's intentions.

[0095] <<Modification 1 of the First Embodiment>> In the first embodiment, the target impression was set using the impression setting slider bars 508-511 on the app screen 501, but the method of setting the target impression is not limited to this.

[0096] Figure 16 shows an example of a UI for setting target impressions. Figure 16(a) shows an example of setting target impressions using a UI on a radar chart. By manipulating the handle 1601 on the radar chart in Figure 16(a), the target impression for each axis can be set. The target impression specification unit 204 acquires the target impression in the UI such that it is -2 when the handle 1601 is in the center and +2 when it is outside. In Figure 19(a), the target impressions are luxury +0.8, familiarity +1.1, dynamism -0.1, and solidity -0.7. Thus, target impressions can also be decimal numbers. Furthermore, the radar chart in Figure 16(b) shows an example of turning off some target impressions. For example, the user can turn off the target impression for the axis corresponding to the handle by double-clicking the handle 1601 with the pointing device 107. The user can then turn on the target impression again by clicking the axis 1602 on the radar chart with the pointing device 107. Figure 16(b) has the same target impression as Figure 16(a), except for the sense of dynamism, but the sense of dynamism is turned off.

[0097] Figure 16(c) shows an example of a UI where the target impression is set from an image rather than a word. The sample poster display area 1603 displays poster images 1604 to 1607, each representing a larger or smaller impression. A checkbox 1608 is displayed next to each poster image. The user can select a poster by clicking the checkbox 1608 with the pointing device 107, which indicates the poster closest to the desired poster. The target impression specification unit 204 determines the target impression by referring to the impressions corresponding to the selected poster images. Figure 16(d) is a table showing the impressions corresponding to the poster images 1604 to 1607 in Figure 16(c) and the final target impression. For example, suppose poster images 1604 and 1607 are selected as shown in Figure 16(c). In this case, the target impression specification unit 204 determines the combined impression 1613, which is an example of impressions 1609 and 1612 respectively, as the target impression. In this example, the target impression is determined by taking the value with the maximum absolute value among the impressions corresponding to the selected poster images. While the example shown presents the poster image that maximizes each impression, this is not the only approach. Multiple poster images that enhance different impressions may be used, or more poster images than the number of impressions may be presented. This allows users to intuitively specify their target impression based on the actual poster rather than words.

[0098] <<Modification 2 of the First Embodiment>> In the first embodiment, the color assignment unit 1201 assigned colors according to the area ratio, but the method of assigning colors is not limited to this. For example, colors may be assigned so that adjacent graphic objects do not have the same color as much as possible. Figure 24 is a diagram illustrating an example of how the color assignment unit 1201 assigns colors to a skeleton. Figure 24(a) is a skeleton 2401 used to explain the color scheme. The skeleton 2401 consists of five graphic objects 2402 to 2406, one image object 2407, and three text objects 2408 to 2410.

[0099] First, the color assignment unit 1201 calculates the adjacency relationships between graphic objects. Figure 24(b) is a schematic representation of the adjacency relationships of graphic objects using nodes and links. Nodes represent graphic objects, and links are established between adjacent graphic objects. The numbers listed on the nodes are color numbers assigned using the method described later. Here, node 2411 corresponds to graphic object 2402, node 2412 to graphic objects 2403 and 2404, node 2413 to graphic object 2406, and node 2414 to graphic object 2405. Note that in Figure 24(a), 2403 and 2404 have the same metadata group ID. The color assignment unit 1201 controls the system to assign the same color to objects with the same group ID. Therefore, in Figure 24(b), 2403 and 2404 are represented as being included in a single node 2412.

[0100] Next, the color assignment unit 1201 assigns a color number to each node. Figure 24(c) is a table summarizing the information about the nodes shown in Figure 24(b) and the assigned color numbers. The color assignment unit 1201 assigns color numbers to nodes in order of the number of links, and if the number of links is the same, in order of the area. At that time, it ensures that the same color is not assigned to an adjacent node that has already been assigned a color. For example, in Figure 24(c), nodes 2112 and 2113 have the same number of links, so the color number is assigned to node 2112 first because it has a larger area. At this time, since the adjacent node 2111 has already been assigned color number 1, it is assigned the next number, color number 2. Node 2113 has been assigned color number 1 to the adjacent node 2111 and color number 2 to node 2112, so it is assigned the next number, color number 3.

[0101] Finally, the color assignment unit 1201 assigns colors to the graphic objects according to the color scheme numbers. Specifically, color 1 is assigned to graphic object 2402, color 2 to 2403, 2404, and 2405, and color 3 to 2406. Figure 24(d) shows the result of applying color scheme ID 1 from Figure 14(b) to the skeleton 2401 using the method described above. This assigns different colors to adjacent graphic objects, thus preserving the original design intent. However, in this embodiment, the title of the text object 2108 is assigned color 4, but it is not shown in Figure 24(d).

[0102] Note that in complex skeletons, the number of colors in the color scheme may be insufficient. For example, Figure 24(e) shows an example of the relationship between nodes and links in a skeleton with more than 3 adjacent shape objects. In Figure 24(e), if you want to assign a different color to all adjacent shape objects, 4 colors are needed. Figure 24(f) shows an example of assigning color scheme numbers to the nodes and links in Figure 24(e). Node 2418 is assigned color scheme number 4 to avoid the color scheme number of the adjacent node. In this case, for example, the adjacency length between shape objects (the length of the edge touching the other shape object) is calculated and assigned to the link. Then, starting with the link with the smallest adjacency length, links are cut until there is no shortage of colors. Figure 24(g) shows an example of the relationship between nodes and links when link 2415 is cut in Figure 24(e). Figure 24(h) shows an example of the color scheme numbers in this case, where 3 colors are used to assign different color scheme numbers to all adjacent nodes. This allows for color schemes that minimize the impact of adjacent shapes having the same color even when there is a shortage of colors. Furthermore, various color schemes with different numbers of colors may be provided. For example, the color scheme assignment unit 1201 may select and apply only color schemes from those obtained from the color scheme pattern selection unit 215 that have a number of colors equal to or greater than the maximum color scheme number assigned to the skeleton. This allows for the selection of a color scheme suitable for the skeleton. In this embodiment, links were cut based on the adjacent lengths between graphic objects, but this is not limited to this. For example, the overlapping area between graphic objects or the ratio of adjacent lengths (the ratio of adjacent lengths to the outer perimeter length of graphic objects) may be used.

[0103] <<Second Embodiment>> In the first embodiment, an example was described in which a skeleton, color scheme, and font are selected based on an image and target impression specified by the user to generate a poster. In the second embodiment, an example is described in which the image selection unit selects an image from among a plurality of candidate images specified by the user based on the target impression and color scheme. More specifically, first the color scheme selection unit selects a color scheme that matches the target impression, and then the image selection unit selects an image that matches the selected color scheme. As a result, an image that matches both the target impression and the color scheme is selected, so that the overall consistency of the poster is maintained without the user having to uniquely decide on the image to use.

[0104] Figure 25 is a software block diagram of the poster creation application in this embodiment. Configurations numbered the same as in Figure 2 perform the same processing as described in the first embodiment, so their explanation is omitted here. Also, the flowchart showing the processing of the poster generation unit 2501 of the poster creation application in this embodiment is the same flowchart as in Figure 9, so its explanation is omitted here.

[0105] In this embodiment, the poster creation condition specification unit 2502 specifies the poster size, number of posters to be created, and usage category, as well as the number of images to be placed on the poster, as poster creation conditions. The number of images can be entered by the user, for example, by adding an image count input field to the application screen 501. The number of images may be a fixed value or a specified range.

[0106] The candidate image selection unit 2503 selects a group of images from the image group stored in the HDD 104 to be placed on the poster. The image group is specified, for example, by the user selecting multiple images in the image selection area 505 on the application screen 501. When the user presses the add image button 507, the image selection unit 505 displays a dialog screen for selecting files stored in the HDD 104 and accepts the user's selection of image files. The dialog screen for selecting files may allow the user to specify multiple image files, or it may be specified based on the structure of the file system containing the images, such as a directory, or it may be specified by accompanying information of individual images such as the date and time of shooting, or attribute information. The candidate image selection unit 2503 outputs the file paths of the specified image group to the image acquisition unit 211.

[0107] The skeleton acquisition unit 2504 sequentially reads skeleton files from the HDD 104 into the RAM 103, retaining skeletons that meet the set conditions in the RAM 103 and deleting skeletons that do not meet the conditions from the RAM 103. Similar to the skeleton acquisition unit 213, the skeleton acquisition unit 2504 determines whether the size, usage category, and text information of the skeleton match the specifications of the poster creation condition specification unit 2502 and the text specification unit 202. Furthermore, the skeleton acquisition unit 2504 determines whether the number of images specified in the poster creation condition specification unit 2502 matches the number of image objects in the skeleton. The skeleton acquisition unit 2504 retains skeletons that meet all conditions in the RAM 103.

[0108] The color scheme pattern selection unit 2505 retrieves a group of color scheme patterns from the HDD 104 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 2506. Similar to the processing of S905, the color scheme pattern selection unit 2505 selects a color scheme pattern according to the target impression by referring to a color scheme pattern impression table as shown in Figure 28.

[0109] The layout unit 2506 generates poster data by laying out various data for each skeleton acquired from the skeleton selection unit 214. The layout unit 2506 outputs the generated poster data set to the impression estimation unit 218.

[0110] Figure 26 is a flowchart showing the poster generation process of the poster creation application in this embodiment. Note that processes numbered the same as those in Figure 9 perform the same processes as those described in the first embodiment, and therefore their explanation is omitted here.

[0111] In S2601, the poster creation condition specification unit 2502, the text specification unit 202, the candidate image specification unit 2503, and the target impression specification unit 204 obtain settings from the application screen 501.

[0112] In S2602, the skeleton acquisition unit 2504 acquires the skeleton from HDD104 and expands it into RAM103.

[0113] In S2603, the color scheme pattern selection unit 2505 obtains a group of color scheme patterns from the HDD 104 that match the target impression specified by the target impression specification unit 204.

[0114] In S2604, the layout unit 2506 generates poster data by laying out various data for each skeleton acquired from the skeleton selection unit 214.

[0115] Figure 27 is a software block diagram illustrating the layout unit 2506 in detail in this embodiment. Figure 28 is a flowchart illustrating the layout processing in detail in this embodiment, and is a detailed flowchart of S2604. Note that processes numbered the same as those in Figures 12 and 13 perform the same processing as described in the first embodiment, so their explanation is omitted here.

[0116] In S2801, the image selection unit 2701 sorts the image data acquired from the image analysis unit 212 according to the color scheme pattern. The image selection unit 2701 compares the color scheme pattern with the main colors of the images and sorts the images in order of those with the closest color to the color scheme pattern. Figure 29 is a diagram illustrating the image selection process. Figure 29(a) is an example of a color scheme pattern. Figure 29(b) is a diagram illustrating the distance between the image and the color scheme. The main color column in Figure 29(b) represents the main colors acquired from the image analysis unit 212. The distance column to color 1 to the distance column to color 4 represents the distance between the color scheme pattern shown in Figure 29(a) and the main colors. In this embodiment, the Euclidean distance in RGB space is calculated. The minimum distance column represents the distance to the color among color 1 to color 4 that is closest to the main color. The image selection unit 2701 sorts the minimum distance column in ascending order.

[0117] In S2802, the image selection unit 2701 selects images to be used for the skeleton based on the images sorted in S2801. First, the image selection unit 2701 counts the number of image objects in the skeleton and determines the number of images to select. Next, the image selection unit 2701 selects the required number of images from the sorted images. For example, if the skeleton has one image object, it selects image ID1 in Figure 29(b), and if there are two image objects, it selects image ID1 and image ID2. This allows for the selection of images with colors similar to the colors in the color scheme, thus maintaining a sense of unity throughout the poster. In this embodiment, the minimum distance between the main color and each color in the color scheme was used as the distance between the main color and the color scheme, but this is not limited to this; for example, the average distance may be used. The distance may also be calculated using other color spaces such as Lab or HSV. Furthermore, if there are multiple main colors, the distance between all main colors and the color scheme can be calculated, and the minimum distance can be determined.

[0118] In this embodiment, posters were generated for all color schemes, but this is not limited to this. For example, if the image selection unit 2701 does not have a sufficient number of images for which the distance between the main color of the image and the color scheme is below a threshold, it may skip generating posters using those color schemes. This allows for a greater sense of uniformity in the posters by using only color schemes that match the images specified by the user.

[0119] As described above, according to this embodiment, the color scheme pattern to be used for the poster can be selected to match the target impression, and the image to be used for the poster can be selected to match the selected color scheme pattern. This makes it possible to generate a poster that closely matches the target impression while maintaining overall consistency, without the user having to uniquely decide on the image to be used for the poster.

[0120] <<Third Embodiment>> In the first embodiment, an example was described in which a poster is generated by selecting the components of the poster—the skeleton, color scheme, and font—based on the target impression. In the second embodiment, the combination generation unit searches for combinations of poster components that make the overall impression of the poster close to the target impression, based on a genetic algorithm. This allows for more flexible selection of poster components that are optimal for the target impression without the need for pre-calculations such as skeleton impression tables, color scheme impression tables, and font impression tables.

[0121] Figure 17 is a software block diagram of the poster creation application in the third embodiment. Configurations numbered the same as in Figure 2 perform the same processing as described in the first embodiment, and therefore their explanation is omitted here.

[0122] The combination generation unit 1701 obtains a group of skeletons from the skeleton acquisition unit 213, poster data and estimated poster impressions from the impression estimation unit 218, a target impression from the target impression specification unit 204, and the main colors of the image from the image analysis unit 212. The combination generation unit 1701 also obtains a list of color schemes and fonts from the HDD 104. The combination generation unit 1701 generates combinations of poster components (skeletons, color schemes, fonts) to be used for poster generation. The combination generation unit 1701 outputs the generated combinations of poster components to the layout unit 217.

[0123] The poster selection unit 1702 selects posters from the poster data obtained from the impression estimation unit 218 in which the distance between the estimated impression of the poster and the target impression specified by the target impression specification unit 204 is less than or equal to a threshold. The poster selection unit 1702 also determines whether the number of selected posters has reached the production quantity specified in the production quantity box 514.

[0124] Figure 18 is a flowchart showing the poster generation process of the poster creation application in this embodiment. Note that processes numbered the same as those in Figure 9 perform the same processes as those described in the first embodiment, and therefore their explanation is omitted here.

[0125] In S1801, the operation during the first execution and the operation from the second loop onward will be explained separately. First, during the first execution of S1801, the combination generation unit 1701 acquires the skeleton, color scheme, and font tables to be used for poster generation. Figure 19 is a diagram illustrating the tables used by the combination generation unit 1701. Figure 19(a) shows a list of skeletons acquired by the combination generation unit 1701 from the skeleton acquisition unit 213. Figures 19(b) and 19(c) show a list of fonts and a list of color scheme patterns acquired by the combination generation unit 1701 from the HDD 104. The combination generation unit 1701 generates random combinations from the above three tables. At that time, the combination generation unit 1701 uses only color scheme patterns that match the main colors extracted by the image analysis unit 212 for the color scheme table. Specifically, the color difference between colors 1 to 4 of the color scheme and the main color is calculated, and only color schemes in which the color difference of at least one of colors 1 to 4 is below a threshold are used. This ensures that the colors assigned to the poster are the same as the colors included in the image, thus maintaining a sense of unity throughout the poster. In this embodiment, 100 combinations are generated. Figure 19(d) shows the combination table generated in this embodiment.

[0126] Next, in the second loop and beyond of S1801, the combination generation unit 1701 calculates the distance between the estimated poster impression obtained from the impression estimation unit 218 and the target impression, and links it to the combination table. Figure 20 is a diagram illustrating the operation of S1801 from the second loop onward. Figure 20(a) shows the result of linking the distance between the estimated poster impression and the target impression to Figure 19(d). More specifically, the layout unit 217 generates posters based on Figure 19(d), and the impression estimation unit 218 estimates an impression for each generated poster. The distance column in Figure 20(a) represents the distance between the estimated impression and the target impression of the poster generated by the combination in the corresponding row. The combination generation unit 1701 generates a new combination table from Figure 20(a). Figure 20(b) is the newly generated combination table. In this embodiment, new combinations are generated using tournament selection and uniform crossover in a genetic algorithm. First, N combinations are randomly selected from the table in Figure 20(a). Here, for example, let N=3. Next, the top two combinations with the smallest distance (closest to the target impression) are selected from the chosen combinations. Finally, new combinations are generated by randomly swapping each element (skeleton ID, color scheme ID, font ID) in the two selected combinations. For example, combinations ID 1 and 2 in Figure 20(b) are the result of generating combinations ID 1 and 3 in Figure 16(a), where the color scheme ID has been swapped. Figure 20(b) is the result of repeating the above procedure to generate 100 new combinations.

[0127] This allows for efficient combination searching based on the distance between the target impression and the estimated impression. In this embodiment, 100 combinations were generated, but this is not limited to this. Furthermore, while tournament selection and uniform crossover were used, other methods such as ranking selection, roulette selection, or single-point crossover may also be used. Incorporating mutations can also reduce the likelihood of falling into local optima. In addition, while a skeleton (arrangement), color scheme, and font were used as the components of the poster to be searched, other components may be used. For example, multiple patterns could be prepared to be inserted into the background of the poster, and the search could determine which pattern to use or not use. By increasing the number of components to be searched, more varied posters can be generated, expanding the range of impression expression.

[0128] In S1802, the poster selection unit 1702 calculates the distance between the estimated poster impression and the target impression, similar to S1801, and creates the same table as in Figure 20(a). The poster selection unit 1702 stores poster data whose distance from the target impression is below a threshold in the RAM 103.

[0129] In S1803, the poster selection unit 1702 determines whether the poster data held in S1802 has reached the number of posters to be created specified in the number of posters to be created box 514. If the number of posters to be created has been reached, the poster generation process ends. If the number of posters to be created has not been reached, the process proceeds to S1801.

[0130] In this embodiment, a genetic algorithm was used to search for combinations of poster components, but the search method is not limited to this, and other search methods such as nearest neighbor search or tab search may be used.

[0131] As described above, according to this embodiment, the combination of elements to be used in the poster is searched for while selecting a color scheme pattern to match the main color of the image. This makes it possible to generate a poster that maintains a sense of unity throughout the poster while the overall impression of the generated poster is close to the target impression. This is particularly effective when generating a poster based on images and text information input by the user. For example, consider a case where the image has a dynamic impression, but the goal is to generate a poster with a calm impression overall. In this embodiment, the overall impression of the poster can be evaluated and a combination of skeleton, color scheme, and font that approaches the target impression can be searched for. Therefore, the elements of the poster can be controlled to match the image, such as using a skeleton with a small image area to suppress the impression of the image, or using fonts and color schemes that are more subdued. According to this embodiment, the optimal combination of elements for the overall impression of the poster can be found flexibly, and posters that are close to the target impression can be created in various variations.

[0132] <<Fourth Embodiment>> In the first, second, and third embodiments, examples were described in which a poster is generated by controlling the components of the poster based on the target impression. In the fourth embodiment, an example is described in which a template combining a skeleton, color scheme, and font is prepared in advance, and the layout unit generates a poster simply by setting the image and text information. This makes it possible to generate a poster that matches the target impression with simpler processing.

[0133] Figure 21 is a software block diagram of the poster creation application in the third embodiment. Configurations numbered the same as in Figure 2 perform the same processing as described in the first embodiment, and therefore their explanation is omitted here.

[0134] The template acquisition unit 2101 acquires a set of templates from the HDD 104 that match the conditions specified by the poster creation condition specification unit 201, the text specification unit 202, and the image acquisition unit 211. In this embodiment, a template refers to a skeleton with pre-set color schemes and fonts. The template acquisition unit 2101 outputs the acquired set of templates to the template selection unit 2102.

[0135] The template selection unit 2102 selects a template from among those acquired from the template acquisition unit 2101 that uses a color similar to the main color acquired from the image analysis unit 212, and outputs it to the layout unit 2103.

[0136] The layout unit 2103 generates poster data by laying out images obtained from the image analysis unit 212 and text obtained from the text specification unit 203 for each template obtained from the template selection unit 2102. The layout unit 2103 outputs the generated poster data set to the impression estimation unit 218.

[0137] Figure 22 is a flowchart showing the poster generation process of the poster creation application in this embodiment. Note that processes numbered the same as those in Figure 9 perform the same processes as those described in the first embodiment, and therefore their explanation is omitted here.

[0138] In S1801, the template acquisition unit 2101 acquires templates that match various setting conditions. In this embodiment, it is assumed that each template is written in one file and stored in the HDD 104. Similar to the skeleton acquisition unit 213, the template acquisition unit 2101 sequentially reads template files from the HDD 104 into the RAM 103, keeps templates that match the setting conditions in the RAM 103, and deletes templates that do not match the conditions from the RAM 103.

[0139] In S1802, the template selection unit 2102 selects a template from among those acquired from the template acquisition unit 2101 that uses a color similar to the main color acquired from the image analysis unit 212. More specifically, it calculates the color difference between the color set in the graphic objects within the template and the main color, and selects a template that has graphic objects whose color difference is below a threshold.

[0140] In S1803, the layout unit 2103 sets image and text information for all templates obtained from the template selection unit 2102 and generates a poster. Note that the image setting is the same as that of the image placement unit 902 and the text information setting is the same as that of the text placement unit 905, so the explanation is omitted.

[0141] As explained above, according to this embodiment, by preparing templates with various color schemes and fonts in advance, it is possible to generate posters that closely match the target impression with simple processing.

[0142] <<Other Embodiments>> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0143] Some features of the embodiment can be summarized as follows: [1] Image acquisition means for acquiring images, A means of receiving target impressions from the user, A selection means for selecting a color scheme pattern from the colors contained in the aforementioned image and the aforementioned target impression, An information processing apparatus characterized by having a poster generation means for generating a poster based on the aforementioned image and the aforementioned color scheme pattern.

[0144] [2] The information processing apparatus according to [1], wherein the selection means identifies the main color contained in the image and selects a color scheme pattern from the identified main color and the target impression.

[0145] [3] It has a display control means that displays a screen for receiving target impressions, The information processing apparatus according to [1] or [2], characterized in that the receiving means receives a target impression via the screen.

[0146] [4] The poster generation means is characterized by generating a poster such that the difference between the impression the poster has and the target impression is less than or equal to a predetermined threshold. [1] to [3] The information processing device according to any one of these three items.

[0147] [5] It further has a means for obtaining characters, The poster generation means is characterized in that it generates a poster based on the image, the characters and the target impression, as described in any one of [1] to [4].

[0148] [6] The poster generation means is characterized by generating a poster by changing the arrangement of any one of the images, characters, or graphics that the poster has based on the target impression, as described in any one of the information processing devices in any one of the [1] to [5].

[0149] [7] A method for controlling an information processing device, The image acquisition process involves obtaining an image, A reception process in which the target impression is received from the user, A selection means for selecting a color scheme pattern from the colors contained in the aforementioned image and the aforementioned target impression, A control method for an information processing apparatus, characterized by having a poster generation step that generates a poster based on the aforementioned image and the aforementioned color scheme pattern.

[0150] [8] A program readable by a computer for causing the computer to function as an information processing device as described in any one of paragraphs [1] to [7].

Claims

1. Image acquisition means for acquiring images, A means of receiving target impressions from the user, A determination means for determining the main color of an image based on the color values ​​of the pixels constituting the image acquired by the image acquisition means, A selection means for selecting a color scheme pattern based on the determined main color and the target impression, An information processing apparatus characterized by having a poster generation means for generating a poster based on the image and the color scheme pattern selected by the selection means.

2. It has a display control means that displays a screen for receiving target impressions, The information processing apparatus according to claim 1, characterized in that the receiving means receives a target impression via the screen.

3. The information processing apparatus according to claim 1, characterized in that the poster generation means generates a poster such that the difference between the impression the poster has and the target impression is less than or equal to a predetermined threshold.

4. It further has a means for obtaining characters, The information processing apparatus according to claim 1, characterized in that the poster generation means generates a poster based on the image, the text, and the target impression.

5. The information processing apparatus according to claim 1, characterized in that the poster generation means generates a poster by changing the arrangement of any of the images, characters, or graphics that the poster has based on the target impression.

6. The information processing apparatus according to claim 1, characterized in that the main color is the color that appears most frequently in the acquired image.

7. The color scheme pattern is composed of multiple colors, The information processing apparatus according to claim 1, characterized in that the selection means includes at least one color whose color difference from the main color is within a predetermined threshold.

8. The system further comprises a storage means for storing multiple color schemes in association with their respective impression information, The information processing apparatus according to claim 7, wherein the selection means selects a color scheme pattern that includes a color whose color difference from the main color is within the predetermined threshold, and the difference between the impression information stored in the storage means and the target impression is within the threshold.

9. The information processing apparatus according to claim 5, characterized in that the poster generation means generates a poster that includes the characters in the colors included in the color scheme selected by the selection means.

10. The information processing apparatus according to claim 3, characterized in that the receiving means receives the setting of a target impression using a slider bar.

11. The information processing apparatus according to claim 1, characterized in that the image acquisition means acquires an image selected by the user.

12. A method for controlling an information processing device, The image acquisition process involves obtaining an image, A reception process in which the target impression is received from the user, A determination step in which the main color of the image is determined based on the color values ​​of the pixels constituting the image acquired in the image acquisition step, A selection step of selecting a color scheme pattern based on the determined main color and the target impression, A control method for an information processing apparatus, characterized by having a poster generation step that generates a poster based on the image and the color scheme selected in the selection step.

13. A program readable by a computer for causing the computer to function as an information processing device according to claim 1.

Images