Information processing apparatus, method, and program

Abstract

To provide an information processing apparatus for generating and displaying a plurality of commercial materials that give a sense of unity and give the impression of a brand intended by a user.SOLUTION: An information processing apparatus is configured to: generate multiple pieces of commercial material data corresponding to a first commercial material type, based on content of a commercial material, a brand design, a designation of an impression, and conditions of the commercial material; generate multiple pieces of commercial material data corresponding to a second commercial material type, based on content of a commercial material, a brand design, a designation of an impression, and conditions of the commercial material; and display a commercial material image on the basis of the first commercial material data included in the multiple pieces of commercial material data corresponding to the first commercial material type, and the second commercial material data included in the multiple pieces of commercial material data corresponding to the second commercial material type. The first and second commercial material data are commercial material data of a combination in which similarity of content is highest, and similarity of the brand design is highest.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, method, and program. [Background technology]

[0002] It has been known in the past to prepare a template that stores information such as the shape and arrangement of images, characters, graphics, etc. that constitute a poster, and to generate a poster by automatically arranging the images, characters, graphics, etc. according to the template. Patent Document 1 proposes generating a poster by selecting templates in ascending order of difference between the impression evaluation value of the template and the impression evaluation value of the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-59123 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, no consideration is given to generating a commercial product that expresses the impression intended by the user, and further, no consideration is given to generating multiple commercial products that can be recognized as having a unified design when different images are used not only for one type of commercial product but also for multiple types of commercial products.

[0005] The present invention has an object to provide an information processing device, method, and program for generating and displaying a plurality of commercial materials that have a sense of unity and express the impression of a brand intended by a user. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to the present invention includes a first reception means for receiving content of a product, a second reception means for receiving a brand design, a third reception means for receiving a designation of an impression that the product will give to a user, a fourth reception means for receiving conditions of the product, a first generation means for generating a plurality of product data corresponding to a first product type based on the content received by the first reception means, the brand design received by the second reception means, the designation of the impression received by the third reception means, and the conditions of the product received by the fourth reception means, and The system comprises a second generation means for generating multiple product data corresponding to a second product type based on the product conditions accepted by the fourth acceptance means, and a display means for displaying product images based on first product data included in the multiple product data generated by the first generation means and second product data included in the multiple product data generated by the second generation means, wherein the first product data and the second product data displayed by the display means are product data representing a combination having the highest similarity in content and the highest similarity in brand design among multiple combinations obtained from the multiple product data generated by the first generation means and the multiple product data generated by the second generation means. Effect of the Invention

[0007] According to the present invention, it is possible to generate and display a plurality of commercial products that have a sense of unity and express the impression of a brand that a user intends. [Brief description of the drawings]

[0008] [Figure 1] 2 is a block diagram showing a hardware configuration of a multiple product generation device. FIG. [Diagram 2] FIG. 2 is a software block diagram of a multiple product creation application. [Diagram 3]FIG. 2 is a diagram showing a skeleton. [Figure 4] FIG. 13 is a diagram showing a table of color scheme patterns. [Diagram 5] FIG. 13 illustrates an application start screen. [Figure 6] FIG. 13 illustrates a multiple product preview screen. [Figure 7] 13 is a flowchart showing an impression quantification process. [Figure 8] FIG. 13 is a diagram illustrating a subjective evaluation method for impressions. [Figure 9] 13 is a flowchart showing a multiple product generation process. [Figure 10] FIG. 13 is a diagram illustrating skeleton selection. [Figure 11] FIG. 13 is a diagram showing a color scheme pattern impression table. [Figure 12] FIG. 2 is a software block diagram of a layout unit. [Figure 13] 13 is a flowchart showing the process of S911. [Figure 14] FIG. 4 is a diagram for explaining information input to a layout unit. [Figure 15] FIG. 13 is a diagram for explaining the process of processing by a layout unit. [Figure 16] FIG. 13 is a diagram showing a UI for setting a target impression and setting brand information. [Figure 17] FIG. 2 is a software block diagram of a multiple product creation application. [Figure 18] 13 is a flowchart showing a multiple product generation process. [Figure 19] 13 is a diagram for explaining a table used by the combination generation unit. FIG. [Figure 20] FIG. 13 is a diagram for explaining the operation of S1802 from the second loop onward. [Figure 21] FIG. 13 is a diagram illustrating a subjective evaluation for determining weights between design elements. [Figure 22] FIG. 13 is a diagram illustrating a subjective evaluation for determining weights between design elements. [Diagram 23] FIG. 13 illustrates an application start screen. [Figure 24] This is a flowchart when the "Reflect saved products" button is pressed. [Diagram 25] FIG. 13 illustrates an application start screen. [Figure 26] FIG. 13 illustrates a multiple product preview screen. [Figure 27] 13 is a flowchart showing a multiple product generation process. [Figure 28] FIG. 13 is a diagram showing a combination of product sets. [Figure 29] FIG. 13 is a diagram showing a table in which target impressions are associated with image feature amounts. [Diagram 30] FIG. 4 is a diagram showing a gamma curve when performing gamma correction. [Diagram 31] FIG. 11 is a diagram showing RGB magnifications when color temperature correction is performed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] [First embodiment] In this embodiment, the "product" refers to printed matter such as posters, pamphlets, store menus, and postcards, and is, for example, the design of advertising media for users. In this embodiment, the "brand" refers to a design that expresses the identity (corporate philosophy, vision, code of conduct, and characteristics) of a company or store. In order to have a brand widely recognized by users, it is necessary to convey a message to users with a design that has a consistent worldview. What is important for a brand to be recognized is a unified design with a consistent worldview. To achieve this, it is necessary to have a unified design for product packages, store designs, and PR products (websites, pamphlets, posters, nouns, postcards), which are points of contact with users. In order to have a unified design, it is necessary to include similar design elements between multiple products. Examples of design elements include logos (symbol marks), fonts, patterns, and colors. If these design elements can be included between multiple products, the design will be felt to be unified, and users will be able to recognize a consistent worldview. In this embodiment, it is possible to automatically generate a design that has a unified feel and expresses the impression of the brand intended by the user for multiple products. This allows even a user with no design knowledge to generate a plurality of commercial materials for raising awareness of a brand.

[0011] In this embodiment, a multiple product creation application (hereinafter, also referred to as an "app") for creating multiple products is operated in a multiple product creation device, and a process for automatically generating multiple products is described as an example. In the following description, "image" includes still images and frame images extracted from videos, unless otherwise specified.

[0012] 1 is a block diagram showing a hardware configuration of a multiple product generation device 100. The multiple product generation device 100 is an information processing device, such as a personal computer (hereinafter referred to as a PC) or a smartphone. In this embodiment, the multiple product generation device is described as a PC. The multiple product generation device 100 has a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display 105, a keyboard 106, a pointing device 107, a data communication unit 108, and a GPU 109.

[0013] The CPU (Central Processing Unit / Processor) 101 comprehensively controls the multiple product generation device 100, and for example, reads out a program stored in the ROM 102 into the RAM 103 and executes it to realize the operation of this embodiment. In FIG. 1, there is one CPU, but the device may be configured with multiple CPUs. The ROM 102 is a general-purpose ROM, and for example, a program executed by the CPU 101 is stored in the ROM 102. The RAM 103 is a general-purpose RAM, and is used as a working memory for temporarily storing various information when the CPU 101 executes a program. The HDD (Hard Disk) 104 is a storage medium (storage unit) for storing image files, a database for holding processing results such as image analysis, and skeletons used by the multiple product generation application. The skeletons will be described later.

[0014] The display 105 is a display unit that displays the user interface (UI) of this embodiment and the electronic multiple products as the layout result of image data (hereinafter also referred to as "images") to the user. The keyboard 106 and the pointing device 107 receive instruction operations from the user. The display 105 may have a touch sensor function. The keyboard 106 is used, for example, when the user inputs the generation conditions for the multiple products that he / she wishes to create on the user interface (UI) displayed on the display 105. The pointing device 107 is used, for example, when the user clicks a button on the UI displayed on the display 105.

[0015] The data communication unit 108 communicates with an external device via a wired or wireless network. For example, the data communication unit 108 transmits data laid out by an automatic layout function to a printer or a server capable of communicating with the multiple product generation device 100. The GPU 109 is a graphics processing unit, and can perform efficient calculations by processing more data in parallel. The GPU 109 is used, for example, when learning is performed multiple times using a learning model such as deep learning. The data bus 109 connects the blocks in FIG. 1 to each other so that they can communicate with each other.

[0016] 1 is merely an example, and is not limited to this. For example, the multiple product generation device 100 may display a UI on an external display.

[0017] The multiple product creation application in this embodiment is stored in HDD 104. The multiple product creation application is started when the user executes an operation such as clicking or double-clicking the application icon displayed on display 105 with pointing device 107.

[0018] 2 is a software block diagram of a multiple product creation application. The multiple product creation application has a multiple product creation condition designation unit 201, a text designation unit 202, an image designation unit 203, a target impression designation unit 204, a multiple product display unit 205, and a multiple product generation unit 210. Furthermore, the multiple product generation unit 210 has an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a skeleton selection unit 214, a color scheme pattern selection unit 215, a font selection unit 216, an image correction parameter selection unit 221, a layout unit 217, an impression estimation unit 218, a design similarity calculation unit 219, an image similarity calculation unit 222, and a multiple product selection unit 220.

[0019] When the multiple product creation application is installed in multiple product generation device 100, a startup icon is displayed on the top screen (desktop) of the OS (operating system) running on multiple product generation device 100. The user operates (e.g., double-clicks) the startup icon displayed on display 105 with pointing device 107. When operated in this manner, the program of the multiple product creation application stored in HDD 104 is loaded into RAM 103 and executed by CPU 101. This starts up the multiple product creation application.

[0020] The multiple product creation application includes program modules corresponding to the components shown in Fig. 2. CPU 101 executes each program module, causing CPU 101 to function as each component shown in Fig. 2. Hereinafter, each component shown in Fig. 2 will be described assuming that each component executes various processes. Fig. 2 also shows a software block diagram relating to multiple product generation unit 210 that executes an automatic multiple product creation function.

[0021] The multiple product creation condition designation unit 201 designates multiple product creation conditions to the multiple product generation unit 210 in response to a UI operation by the pointing device 107. In this embodiment, the type of multiple products to be created and the purpose category are designated as the multiple product creation conditions. The size of the multiple products is linked to the type of multiple products. Furthermore, some products may have multiple sizes. In that case, the actual values ​​of the width and height may be designated, or the paper size such as A1 or A2 may be designated. The purpose category is a category that indicates what purpose the multiple products are used for, such as a restaurant, a school event, a sale, etc.

[0022] The text designation unit 202 designates character information to be placed in the commercial material to be generated by a UI operation using the keyboard 106. The character information to be placed represents, for example, a character string representing a title, date and time, a location, and the like. The type of character information may change depending on the type of commercial material to be created selected by the multiple commercial material creation condition designation unit 201. For example, when a poster is selected, the title, subtitle, and main text are designated. For a postcard, the title, address, and contact information are displayed. When multiple commercial materials are selected, the overlapping information may be designated separately or may be combined into one. Furthermore, the text designation unit 202 outputs each piece of character information to the skeleton acquisition unit 213 and the layout unit 217 after linking it to determine whether it is a title, date and time, location, or other type of information.

[0023] The image designation unit 203 designates one or more image data stored in the HDD 104 to be placed on the multiple commercial materials. The designated images may be designated individually according to the type of commercial material to be created selected in the multiple commercial material creation condition designation unit 201, or a common image may be designated. The image data may be designated based on the structure of a file system including the image data, such as a device and a directory, or may be designated by associated information for identifying the image, such as the shooting date and time, or attribute information. The image designation unit 203 outputs the file path of the designated image to the image acquisition unit 211.

[0024] The target impression designation unit 204 designates one target impression for the multiple commercial materials to be created. The target impression is an impression that the multiple commercial materials to be created are ultimately required to maintain. In this embodiment, the strength indicating the degree to which the impression is to be given to a word expressing an impression is designated by a UI operation using the pointing device 107. Information indicating the target impression designated by the target impression designation unit 204 is shared by the skeleton selection unit 214, the color scheme pattern selection unit 215, the font selection unit 216, the image correction parameter selection unit 221, and the multiple commercial materials selection unit 220. Details regarding impressions will be described later.

[0025] Next, a detailed description will be given of the configuration of the multiple product generation unit 210. The difference between products can be realized by the skeleton acquisition unit 213 selecting a skeleton of the target product.

[0026] The image acquisition unit 211 acquires image data designated by the image designation unit 203 from the HDD 104. The image acquisition unit 211 outputs the acquired image data to the image analysis unit 212 and the layout unit 217. The image acquisition unit 211 also outputs the number of acquired images to the skeleton acquisition unit 213. The images stored in the HDD 104 include still images and frame images cut out from a video. The still images and frame images are acquired from an imaging device such as a digital camera or a smart device. The imaging device may be provided in the multiple product generation device 100 or may be provided in an external device. In addition, when the imaging device is an external device, the images are acquired via the data communication unit 108. As another example, the still images may be illustration images created by image editing software or CG images created by Computer Graphics (CG) production software. The still images and cut-out images may be images acquired from a network or a server via the data communication unit 108. As an example of images acquired from a network or a server, social networking service images (hereinafter referred to as "SNS images") may be mentioned. In addition, the program executed by the CPU 101 analyzes data attached to each image to determine the storage source. For example, the SNS image may be acquired from the SNS via an application, and the acquisition destination may be managed within the application. Note that the image is not limited to the above-mentioned images, and may be other types of images.

[0027] The image analysis unit 212 executes an analysis process of the image data acquired from the image acquisition unit 211 by a method described later, and acquires information indicating an image feature amount described later. Specifically, for example, the image analysis unit 212 executes an object recognition process described later, and acquires information indicating an image feature amount of the image data. The image analysis unit 212 also links the acquired information indicating the image feature amount to the image data and outputs the image data to the layout unit 217.

[0028] The skeleton acquisition unit 213 acquires one or more skeletons that meet the conditions specified by the multiple product creation condition designation unit 201, the text designation unit 202, and the image acquisition unit 211 from the HDD 104. In this embodiment, a skeleton is information that indicates the arrangement of character strings, images, figures, etc. to be arranged in the multiple products. In this embodiment, the skeleton is one of the product components that make up the product, along with the color scheme pattern and font.

[0029] FIG. 3 is a diagram showing an example of a skeleton for a poster among the commercial materials. On a skeleton 301 in FIG. 3(a), three graphic objects 302, 303, and 304, one image object 305, and four character objects 306, 307, 308, and 309, which are objects on which characters are arranged, are arranged. In addition to the position, size, and angle of each object, metadata required for generating a poster is also recorded. FIG. 3(b) is a diagram showing an example of metadata. For example, character objects 306 to 309 hold, as metadata attributes, what type of character information is arranged. Here, character object 306 represents the title, character object 307 represents the subtitle, and character objects 307 and 308 represent the main text. In addition, graphic objects 302 to 304 hold, as metadata attributes, color scheme numbers (color scheme IDs) indicating the shape of the graphic and the color scheme pattern. Here, the attributes of graphic objects 302 and 303 represent a rectangle, and the attribute of graphic object 304 represents an ellipse. Also, it is assumed that color scheme number 1 is assigned to the graphic object 302, and color scheme number 2 is assigned to the graphic objects 303 and 304. Here, the color scheme number is information to be referenced when coloring, which will be described later, and indicates that different colors are assigned to different color scheme numbers. Also, the graphic objects may be drawn by painting them with a uniform color. Also, the graphic objects may be drawn by cutting out the pattern of a background illustration in the shape of a graphic object. Note that the types of objects and metadata are not limited to the above. For example, there may be a map object for placing a map, or a barcode object for placing a QR code (registered trademark) or a barcode. Also, as metadata for a character object, there may be metadata that indicates the width of the line space or the width of the character space. The metadata may include the purpose of the skeleton and be used to control whether or not to use the skeleton depending on the purpose.

[0030] Skeletons may be managed separately for multiple products. For example, there are skeletons for multiple products, skeletons for menus, skeletons for postcards, skeletons for tri-fold leaflets, skeletons for calendars, skeletons for banners, etc. Furthermore, skeletons may be managed as groups across multiple products based on relationships such as placement. For example, skeletons in group 1 are a skeleton group that gives off a sense of luxury, and the same skeleton group ID is stored in the metadata of each skeleton. This makes it possible to determine the skeleton to be applied to other products based on the skeleton used to create one product. As a result, even when designs are created for multiple products, a unified design can be generated across multiple products. The skeletons may be managed separately according to the ratio of width to height. This makes it possible to acquire a skeleton with a width to height ratio that matches the size specified by the multiple product creation condition specification unit 201. The skeletons may be stored in HDD 104 in, for example, a CSV format, or in a DB format such as SQL. The skeleton acquisition unit 213 outputs one or more skeletons acquired from HDD 104 to the skeleton selection unit 214.

[0031] The skeleton selection unit 214 selects, from the skeletons acquired from the skeleton acquisition unit 213, a skeleton that matches the type of product designated by the multiple product creation condition designation unit 201 and the target impression designated by the target impression designation unit 204. Then, it outputs the skeleton to the layout unit 217. One or more skeletons are selected for one type of product, and one or more skeletons that match the target impression are selected for each product type. Since the overall layout of each product is determined by the skeleton, by preparing various types of skeletons in advance, it is possible to increase the variation of each product after generation.

[0032] The color scheme selection unit 215 obtains from HDD 104 one or more color scheme patterns that match the desired impression specified by the desired impression specification unit 204, and outputs to the layout unit 217. A color scheme pattern is a combination of colors used for a product. Furthermore, if a color is specified by the multiple product creation condition specification unit 201, a color scheme pattern that includes that color is also obtained from HDD 104, and outputs to the layout unit 217.

[0033] FIG. 4 is a diagram showing an example of a color scheme table. In this embodiment, a color scheme pattern is expressed as a combination of four colors. The color scheme ID column in FIG. 4 is an ID for uniquely identifying a color scheme pattern. The color 1 to color 4 columns express colors in RGB order, with each RGB color value ranging from 0 to 255 ((R, G, B)=(0 to 255, 0 to 255, 0 to 255)). Note that, although this embodiment uses a color scheme pattern consisting of a combination of four colors, other numbers of colors may be used, or a mixture of multiple colors may be used.

[0034] The font selection unit 216 selects one or more font patterns that match the desired impression specified by the desired impression specification unit 204, obtains them from the HDD 104, and outputs them to the layout unit 217. A font pattern is a combination of at least one of a title font, a subtitle font, and a body font. If a font is specified by the multiple product creation condition specification unit 201, a font pattern including that font is also obtained from the HDD 104 and output to the layout unit 217.

[0035] The image correction parameter selection unit 221 selects one or more image correction parameters according to the image feature amount acquired by the image analysis unit 212 and the target impression specified by the target impression specification unit 204. The image correction parameter selection unit 221 links the selected image correction parameters to the image data and outputs the image data to the layout unit 217.

[0036] The layout unit 217 lays out various data for each of the one or more skeletons acquired from the skeleton selection unit 214, thereby creating one or more pieces of product data, the number of which is equal to or greater than the number of types designated by the multiple product creation condition designation unit 201. The layout unit 217 places text acquired from the text designation unit 202 for each skeleton. The layout unit 217 also performs image correction on the image data acquired from the image correction parameter selection unit 217 and then places the image on the skeleton. Then, the layout unit 217 applies a color scheme pattern acquired from the color scheme pattern selection unit 215, and applies a font pattern acquired from the font selection unit. Furthermore, if a pattern is designated by the multiple product creation condition designation unit 201, the layout unit 217 places a pattern in a graphic area for each skeleton. Furthermore, if a logo is designated by the multiple product creation condition designation unit 201, the layout unit 217 places a logo for each skeleton. The layout unit 217 outputs the generated one or more pieces of product data to the impression estimation unit 218, the design similarity calculation unit 219, and the image similarity calculation unit 222.

[0037] The impression estimation unit 218 estimates an impression of each of the multiple product data acquired from the layout unit 217, and links the estimated impression (estimated impression) to each product data. Then, the impression estimation unit 218 outputs one or more pieces of product data linked with the estimated impression to the multiple product selection unit 220.

[0038] The design similarity calculation unit 219 calculates the design similarity between different products among the multiple product data acquired from the layout unit 217, and links the calculated design similarity to each product data. The design similarity calculation unit 219 then outputs one or more product data linked with the design similarity to the multiple product selection unit 220.

[0039] The image similarity calculation unit 222 calculates the similarity of image data arranged between different commercial materials among the multiple commercial material data acquired from the layout unit 217, and links the calculated image similarity to each commercial material data. The image similarity calculation unit 222 then outputs the one or multiple commercial material data linked with the image similarity to the multiple commercial material selection unit 220.

[0040] The multiple product selection unit 220 calculates an impression match degree by comparing the target impression specified by the target impression specification unit 204 with the estimated impression of each of the multiple product data linked to the estimated impression acquired from the impression estimation unit 218. The multiple product selection unit 220 determines a combination of products based on the impression match degree, the design similarity between different products acquired by the design similarity 219, and the image similarity between different products acquired by the image similarity calculation unit 222. Which of the impression match degree, the design similarity, and the image similarity is to be emphasized in the determination varies depending on the reflection degree of the brand information designated by the multiple product creation condition specification unit 201. The determined combination of products is stored in the HDD 104. The multiple product selection unit 220 outputs each selected product data to the multiple product display unit 205.

[0041] The multiple product display unit 205 outputs a multiple product image to be displayed on the display 105 according to the combination of product data acquired from the multiple product selection unit 220. The multiple product image is, for example, bitmap data. The multiple product display unit 205 displays the multiple product image on the display 105.

[0042] The multiple product creation application may be provided with a function for allowing the user to edit the layout, color, shape, and the like of images, text, and graphics through additional operations and further change the design to the user's desire after the generation results are displayed in multiple product display unit 205. Furthermore, if the application is provided with a function for printing the multiple product data stored in HDD 104 using a printer according to conditions specified in multiple product creation condition specification unit 201, the user will be able to obtain printed copies of the multiple products that he or she has created.

[0043] <Example of display screen> 5 is a diagram showing an example of an application start-up screen 501 provided by the multiple product creation application. The application start-up screen 501 is a setting screen displayed on the display 105. A user sets multiple product creation conditions, text, and images, which will be described later, via the application start-up screen 501, and the multiple product creation condition designation unit 201, the image designation unit 202, and the text designation unit 202 acquire the settings from the user via this UI screen.

[0044] The title box 502, the subtitle box 503, and the text box 504 accept the specification of text information to be placed on multiple commercial materials. In this embodiment, three types of text information are accepted as an example, but the present invention is not limited to this. For example, additional text information such as location and date and time may be accepted. In addition, it is not necessary to complete all the specifications, and some boxes may be blank. In addition, the display may be changed according to the specification result of the product creation specification area 513 described later. For example, when a poster is selected, boxes for specifying the title, subtitle, and text are displayed. In the case of a postcard, boxes for specifying the title, address, and contact information are displayed. When multiple commercial materials are selected, the overlapping boxes may be specified separately, or may be combined into one.

[0045] The image designation area 505 is an area for displaying an image to be placed in a plurality of commercial materials. The image 506 represents a thumbnail of the designated image, and the commercial material in which the image is used is selected. The image addition button 507 is a button for adding an image to be placed. When the image addition button 507 is pressed by the user, the image designation unit 202 displays a dialogue screen for selecting a file stored in the HDD 104 and accepts the image file selection by the user. Then, a thumbnail of the selected image is added to the image designation area 505. Note that, as an example, in the present embodiment, the image 506 allows the user to designate a commercial material in which the image is used, but it is not necessarily required that the commercial material in which the image is used be designated. In that case, for example, the layout unit 217 may select an image to be placed in the skeleton of the image data.

[0046] The impression sliders (impression slider bars, or impression setting sliders) 508 to 511 are objects that set one target impression factor for each product to be created. For example, the slider 508 is a slider that sets a target impression factor related to luxury, and the target impression is set such that the more the slider is slid to the right, the more luxurious the product, and the more the slider is slid to the left, the less luxurious (cheaper) the product. In addition, by combining the target impression factors set by each slider, a target impression is set that reflects not only the target impression factor set by one slider, but also the target impression factors set by other sliders. For example, assume that a user operation is performed on the screen of the multiple product generation application, and the impression slider 508 is set to the right of the center of the slider, and the impression slider 511 is set to the left of the center of the slider. In that case, a product with an elegant impression, such as a high sense of luxury and a low sense of solidity, is generated. For example, when the user operates the impression slider 508 to the right of the center of the slider and the impression slider 511 to the right of the center of the slider, a product with a gorgeous impression, such as a high sense of luxury and dignity, is generated. In this way, by combining factors of the target impression indicated by a plurality of impression sliders, it is possible to set target impressions with different directions, such as a target impression of "elegant" and a target impression of "gorgeous", even if a common target impression factor of "luxury" is set. In other words, the target impression is composed and determined by a plurality of factors indicating an impression, but may be determined by one factor indicating an impression. In this embodiment, the state in which the slider is set to the leftmost position is set to -2, and the state in which the slider is set to the rightmost position is set to +2, and the value is corrected to an integer value between -2 and +2. These numerical values ​​indicate that the impression is low, -1 is slightly low, 0 is neither, +1 is slightly high, and +2 is high, with -2 being low, -1 being slightly low, 0 being neutral, +1 being slightly high, and +2 being high. The purpose of correcting to -2 to +2 is to match the scale with the estimated impression and to facilitate distance calculation, which will be described later, but the normalization is not limited to this and may be performed using values ​​from 0 to 1.

[0047] The impression valid radio button 512 is a button that can execute control of enabling or disabling the setting of each target impression. The user can set whether to enable or disable the setting of each target impression by pressing the impression valid radio button 512 and setting on / off. For example, when off is selected with the impression valid radio button 512, the impression is excluded from the impression control. For example, a user who wants to create a calm product with low dynamism but has no particular designation for other impressions can generate a product specialized in low dynamism by turning off the impression valid radio buttons 512 other than dynamism. Note that FIG. 5 shows a state in which the sense of luxury and the sense of familiarity are selected on, and the sense of dynamism and the sense of solidity are selected off. This enables highly flexible control, such as using all the target impressions for product generation, or using only some of the target impressions for product generation. Note that the impression valid radio button 512 may be omitted in a configuration in which the leftmost setting of each slider is the same as when each target impression is not set (for example, when the slider 508 is set to the leftmost setting, the sense of luxury is 0). In this case, when the user wishes to disable the setting of each target impression, the user can disable the setting of each target impression by setting each slider to the leftmost position.

[0048] The product creation specification area 513 is composed of multiple check boxes that determine the type of product to be created. The user can enable or disable the check box of the product to be created by clicking with the pointing device 107. The category list box 514 allows the user to set the usage category of the multiple products to be created.

[0049] In addition to the above conditions, this embodiment has a configuration that allows brand information (brand design) to be input on the UI screen. By inputting brand information, the user can specify that the same design elements be used across different products. This can improve the sense of unity in the designs across different products. Below, as an example of design elements, a box that allows the setting of key colors, patterns, logos, and fonts will be described. This is just one example, and other design-related items may also be used.

[0050] In the key color specification box 515, information on a color that is common between different commercial materials is input. In this embodiment, a list for specifying a color is displayed by a click operation with the pointing device 107, and the color can be specified. Also, a UI (not shown) for specifying a color by a click operation with the pointing device 107 may be further displayed and the color may be specified. For example, a color palette listing multiple colors may be displayed as the UI for selection.

[0051] The design specification box 516 is used to input information about a design that is common to different commercial materials. In this embodiment, a list for specifying a design is displayed and the design can be specified by clicking the pointing device 107. The design may also be specified by selecting a file in which the design is saved by clicking the pointing device 107. The file may be an image file (JPEG, bitmap) or vector data (PDF).

[0052] The logo designation box 517 is used to input information about a logo that is common to different commercial products. In this embodiment, a list for designating a logo is displayed and the logo can be designated by a click operation with the pointing device 107. Alternatively, the logo may be designated by selecting a file in which the logo is saved by a click operation with the pointing device 107. The file may be an image file (JPEG, bitmap) or vector data (PDF).

[0053] The font specification box 518 is used to input information about a font that is common to different commercial materials. In this embodiment, a list for specifying a font is displayed and the font can be specified by clicking the pointing device 107. Alternatively, the font may be specified by selecting a file in which the font is saved by clicking the pointing device 107.

[0054] The reflection degree slider bar 520 sets the weight of how much the brand information set above is reflected as the design similarity. The leftmost position is a weight of 0, which means that the input brand information is ignored. The rightmost position is a weight of 1, which means that the input brand information is always used. For example, when the reflection degree is specified as shown in FIG. 5, the reflection degree is 0.4, and in the above-mentioned multiple product selection unit 220, the weight by the design similarity is 0.4 and the weight by the impression value is 0.6. The brand information enable radio button 519 is a button that can execute control of enabling or disabling the setting of each brand information. The user can set whether to enable or disable the setting of each brand information by pressing the brand information enable radio button 519 and setting it to on / off. FIG. 5 shows a state in which the key color and the pattern are enabled.

[0055] The reset button 521 is a button for resetting each setting information on the application startup screen 501. When the user presses the OK button 522, the multiple product creation condition designation unit 201, the text designation unit 202, the image designation unit 203, and the target impression designation unit 204 output the contents set on the application startup screen 501 to the multiple product generation unit 210. At that time, the multiple product creation condition designation unit 201 acquires the type of product to be created from the created product designation area 513, and the use category of the multiple products to be created from the category list box 514. Furthermore, the multiple product creation condition designation unit 201 acquires a color from the key color designation box 515, a design from the design designation box 516, a logo from the logo designation box 517, and a font from the font designation box 518. Furthermore, the reflection degree of the brand information is acquired from the reflection degree slider bar 520, and whether or not the brand information is valid is acquired from the brand information valid radio button 519.

[0056] The text designation unit 202 acquires character information to be placed on the multiple commercial materials from the title box 503, the subtitle box 503, and the body box 504. The image designation unit 203 acquires an image file path to be placed on the multiple commercial materials from the image designation area 505. The target impression designation unit 204 acquires a target impression of the multiple commercial materials to be created from the impression sliders 508 to 511 and the radio button 512. Note that the multiple commercial material creation condition designation unit 201, the text designation unit 202, the image designation unit 203, and the target impression designation unit 204 may process the value set on the application startup screen 501. For example, the text designation unit 202 may remove unnecessary blank characters at the beginning or end from the input character information. Also, the target impression designation unit 204 may correct the value of the target impression designated by the impression sliders 508 to 511.

[0057] 6 is a diagram showing an example of a multiple product preview screen in which a multiple product image generated by multiple product display unit 205 is displayed on display 105. When OK button 522 on application start-up screen 501 is pressed and multiple product generation is completed, the screen displayed on display 105 transitions to multiple product preview screen 601.

[0058] Multiple product images 602 are images of each product output by multiple product display unit 205. Multiple product generation unit 210 generates the types of products specified by multiple product creation condition specification unit 201, and therefore displays a list of the generated multiple types of products as multiple product images 602 on multiple product preview screen 601. For example, when a poster is clicked on by the user with pointing device 107, the poster is placed in a selected state.

[0059] The edit button 603 allows multiple selected products to be edited through a UI that provides an editing function (not shown). The print button 604 allows multiple types of displayed products to be printed through a printer control UI (not shown). The save button 605 allows setting information related to multiple types of displayed products to be saved in a predetermined format in a re-editable form in the HDD 104. The predetermined format may be CSV format or JSON format. The saved setting information is the setting information on the application start screen 501 in FIG. 5, and includes the estimated impression of the products and brand information (logo, design, key color, font).

[0060] <Quantifying impressions of multiple products> Here, a method of quantifying the impressions of multiple products, which is a pre-processing required for the multiple product generation process to execute the impression estimation process described later in S912 of FIG. 9(a), will be described. In this embodiment, a process for quantifying the impression of a poster will be described as an example. The impressions of other products can be quantified by performing a similar process. Furthermore, the quantification results of the impressions may be reused between products that are similar in size or use.

[0061] The process of quantifying the impression of a poster is performed by a vendor developing a poster creation application at the development stage of the poster creation application. The process of quantifying the impression of a poster may be executed by the poster generation device 100, or may be executed by an information processing device different from the poster generation device 100. When the process is executed by an information processing device different from the poster generation device 100, the process is executed by the CPU of that information processing device.

[0062] In the process of quantifying the impression of a poster, the impression that a person has of various posters is quantified. At the same time, a correspondence between the poster image and the impression of the poster is derived. This makes it possible to estimate the impression of the poster from the generated poster image. If the impression can be estimated, it becomes possible to control the impression of the poster by modifying the poster image, or to search for a poster image that has a certain target impression. The process of quantifying the impression of a poster is executed, for example, in a poster generation device by running an impression learning application for learning the impression of the poster image in advance before the poster generation process.

[0063] Fig. 7 is a flowchart showing the poster impression quantification process. The flowchart shown in Fig. 7 is realized, for example, by CPU 101 reading a program stored in HDD 104 into RAM 103 and executing it. The poster impression quantification process will be described with reference to Fig. 7. The process in Fig. 7 is executed. Note that the symbol "S" in the description of each process indicates a step in the flowchart (the same applies below).

[0064] In S701, the CPU 101 executes acquisition of a subjective evaluation of the impression of the poster. FIG. 8 is a diagram for explaining an example of a subjective evaluation method of the impression of the poster. The CPU 101 presents a poster to a subject, and acquires a subjective evaluation of the impression of the poster from the subject. In this case, a measurement method such as the Semantic Differential (SD) method or the Likert scale method can be used. FIG. 8 shows an example of a questionnaire using the SD method, in which adjective pairs expressing impressions are presented to a plurality of evaluators, and the adjective pairs that are recalled from the target poster are scored. After acquiring subjective evaluation results from a plurality of subjects for a plurality of posters, the CPU 101 calculates an average value of the responses to each adjective pair, and sets the average value as a representative score value for the corresponding adjective pair. Note that the subjective evaluation method of the impression may be other than the SD method, as long as words expressing impressions and corresponding scores are determined.

[0065] In S702, the CPU 101 executes a factor analysis of the acquired subjective evaluation results. If the subjective evaluation results are used as is, the number of adjective pairs will be the number of dimensions, which will complicate the control, so it is desirable to reduce the number of dimensions to an efficient number by an analysis method such as principal component analysis or factor analysis. In this embodiment, the description will be given assuming that the dimensions are reduced to four factors by factor analysis. Naturally, this number varies depending on the selection of adjective pairs in the subjective evaluation and the factor analysis method. In addition, the output of the factor analysis is assumed to be standardized. That is, each factor is scaled so that the average is 0 and the variance is 1 in the poster used in the analysis. As a result, the impressions of -2, -1, 0, +1, and +2 specified by the target impression specification unit 204 can be directly corresponding to -2σ, -1σ, the average value, +1σ, and +2σ in each impression, which makes it easier to calculate the distance between the target impression and the estimated impression, which will be described later. In this embodiment, the four factors are luxury, familiarity, dynamism, and solidity as shown in FIG. 5. However, these are names given for the sake of convenience in order to convey impressions to the user through the user interface, and each factor is composed of multiple adjective pairs that influence each other.

[0066] In S703, the CPU 101 associates the poster image with the impression. Although it is possible to quantify the poster that has been subjectively evaluated by the above-mentioned method, it is necessary to estimate the impression of the poster to be created without subjective evaluation. The association between the poster image and the impression can be realized by learning a model that estimates the impression from the poster image, for example, using a deep learning method using a Convolution Neural Network (CNN) or a machine learning method using a decision tree. In this embodiment, in the impression learning, a poster image is input, and four factors are output, and supervised deep learning using CNN is performed. That is, the subjectively evaluated poster image and the corresponding impression are learned as the correct answer to create a deep learning model, and an unknown poster image is input to the learning model to estimate the impression.

[0067] The deep learning model created above is stored in, for example, HDD 104, and the impression estimation unit 218 loads the deep learning model stored in HDD 104 in RAM 103 and executes it. The impression estimation unit 218 images the poster data acquired from the layout unit 217, and estimates the impression of the poster by operating the deep learning model loaded in RAM 103 with the CPU 101 or the GPU 109. Note that, although the deep learning method is used in this embodiment, the present invention is not limited to this. For example, when using a machine learning method such as a decision tree, a machine learning model may be created that extracts feature amounts such as the average brightness value and edge amount of the poster image by image analysis, and estimates the impression based on the feature amounts.

[0068] <Image correction processing> The image feature amount acquired by the image analysis unit 212, the image correction parameter selected by the image correction parameter selection unit 221, and the image correction process performed by the layout unit 217 will be described. FIG. 29 shows a table for deriving the image feature amount from the target impression. The table for deriving the image feature amount is created by performing impression estimation on many images in advance and statistically analyzing the image feature amount corresponding to each impression value. In this embodiment, the impression estimation for the image is performed by applying the impression estimation process for the poster described above to the image. FIG. 29 shows the impression on the vertical axis and the image feature amount on the horizontal axis. For example, in order to give a sense of familiarity, the image feature amount is represented as medium saturation and number of colors, and small edge amount. In order to give a sense of dynamism, the image feature amount is represented as high saturation, a hue from red to orange, and a large number of colors and edge amount. In order to give a sense of solidity, the image feature amount is represented as low saturation and brightness. In addition, there are impression values ​​that cannot be controlled by the image feature amount, such as a sense of luxury. In the case of an impression that cannot be controlled by the image feature amount, no correction process is performed. The image analysis unit 212 acquires the image feature amounts shown in FIG.

[0069] Next, the image correction process will be described. For example, a correction process is performed to make the saturation of the image data medium. The correction process for the image feature amount shown in FIG. 29 is shown as an example. The process of correcting the lightness or saturation includes gamma correction. FIG. 30 is a diagram showing a gamma curve when performing gamma correction. The horizontal axis represents the input value, and the vertical axis represents the output value. When the image data is RGB data, it is converted to the LCH color space by a known technique, and gamma correction is performed on the lightness L or the saturation C. When gamma correction is performed by the upwardly convex gamma table 3001 in FIG. 30, the lightness L or the saturation C becomes high. When gamma correction is performed by the downwardly convex gamma table 3002 in FIG. 30, the lightness L or the saturation C becomes low. After performing gamma correction, an inverse conversion is performed from the LCH color space to RGB data by a known technique. In this embodiment, the saturation and lightness are corrected using a gamma table, but more detailed correction may be performed by using, for example, a Bezier curve. As a process for correcting the hue, there is a method for correcting the hue H in the LCH color space, but it is not possible to add color near the white point. In addition, simply changing the hue changes the color of the subject in the image, which results in a process that gives a sense of incongruity. Therefore, a color temperature correction process will be described as an example in this embodiment. The color temperature is a value that represents the color of a light source, and color temperature correction is a process that corrects the image data as if it was taken under a light source different from the environment in which it was taken. By color temperature correction, the color near the white point can also be changed, and even if the color of the subject changes, no significant sense of incongruity is felt. FIG. 31 shows an example of RGB magnification when performing color temperature correction. Table 3101 shows the magnification of R, table 3102 shows the magnification of G, and table 3103 shows the magnification of B. For example, in order to correct image data to a color temperature of 4000K, R of the RGB data is multiplied by about 1.7 and B is multiplied by about 0.6, resulting in a reddish correction. In addition, when correcting the color temperature accurately, the color temperature in the environment in which the image data was taken is necessary, but in this embodiment, since it is used to change the hue, the color temperature of the shooting environment is not necessarily required. The process for correcting the edge amount includes a sharpening filter process or a blurring filter process.The amount of edges can be increased by applying a sharpening filter, and decreased by applying a blurring filter. A sharpening filter can be implemented, for example, by applying a Laplacian filter to image data and adding the result to the original image data. A blurring filter can be implemented, for example, by using a Gaussian filter. Image correction processing can change the impression of an image by changing the image features in this way.

[0070] Next, the image correction parameters will be described. The image correction parameters are a group of parameters necessary for performing each of the above image correction processes. For example, in the case of gamma correction, the gamma value for determining the gamma curve corresponds to the image correction parameters, and in the case of color temperature correction, the color temperature for determining the correction coefficient corresponds to the image correction parameters. In addition, in the case of a sharpening filter, the addition coefficient of the Laplacian filter corresponds to the image correction parameters, and in the case of a blurring filter, the σ value for adjusting the blurring amount corresponds to the image correction parameters.

[0071] Fig. 9 is a flowchart showing a multiple product generation process by multiple product generation unit 210 of the multiple product creation application. As described above, the flowchart shown in Fig. 9 is started when the user sets various setting items on the multiple product creation application and presses OK button 522.

[0072] The flowchart shown in FIG. 9(a) is realized, for example, by CPU 101 reading a program stored in HDD 104 into RAM 103 and executing it. In this embodiment, the components shown in FIG. 2 that function when CPU 101 executes a multiple-item creation application will be described as executing the processes. The multiple-item generation process will be described with reference to FIG. 9. Note that the symbol "S" in the description of each process indicates a step in the flowchart (the same applies hereinafter in this specification).

[0073] In S901, the multiple product creation application displays the application start screen 501 on the display 105. The user inputs each setting via the UI screen of the application start screen 501 using the keyboard 106 or the pointing device 107.

[0074] In S902, the multiple product creation condition designation unit 201, the text designation unit 202, the image designation unit 203, and the target impression designation unit 204 acquire corresponding settings from the application start screen 501. The multiple product creation condition designation unit 201 acquires the product type, category, and brand information to be created. The product type to be created is acquired from the product creation designation area 513. In FIG. 5, as an example, three types of product are shown: poster, banner, and postcard. The category is acquired from the category list box 514. In FIG. 5, as an example, food and drink is shown. The brand information acquires the key color, the pattern, the logo, the font, and the reflection degree. The key color acquires the color designated in the key color designation box 515. Furthermore, whether or not the key color is to be used for product generation is acquired by the brand information valid radio button 519. In FIG. 5, as an example, the key color is a color expressed in light gray, and it is shown that it is used for product generation. The picture is acquired from the picture designation box 516. Furthermore, whether or not the picture is to be used for product generation is acquired from the brand information valid radio button 519. In FIG. 5, as an example, the picture is a picture of tilted bricks, which indicates that it is to be used for product generation. The logo is acquired from the logo designation box 517. Furthermore, whether or not the logo is to be used for product generation is acquired from the brand information valid radio button 519. In FIG. 5, as an example, no logo is designated, which indicates that it is not to be used for product generation. The font is acquired from the font designation box 518. Furthermore, whether or not the font is to be used for product generation is acquired from the brand information valid radio button 519. In FIG. 5, as an example, a Gothic font is designated, which indicates that it is not to be used for product generation.

[0075] The text designation unit 202 acquires character information to be placed on multiple commercial materials from a title box 503, a subtitle box 503, and a body box 504. As an example, in FIG. 5, the title indicates that TitleTittleTittleTittle is acquired. As the subtitle, a space is acquired. As the body, four lines of TextTextText are acquired. The image designation unit 203 acquires an image designated in an image designation area 505. As an example, in FIG. 5, an image of an upward sloping line is acquired.

[0076] The target impression designation unit 204 acquires factors of the target impression from impression sliders 508 to 511. Furthermore, whether or not to use as the target impression is acquired from impression validity radio button 512. As an example, Fig. 5 shows that luxury is -1, familiarity is +1, dynamism is -0.8, and profoundness is 0. Furthermore, it shows that luxury and familiarity are used as the target impressions, and dynamism and profoundness are not used as the target impressions.

[0077] In S903, the image acquisition unit 211 acquires image data. Specifically, the image acquisition unit 211 identifies a corresponding image file for the image in the settings acquired in S902. Then, the image acquisition unit 211 reads the identified image file from the HDD 104 to the RAM 103.

[0078] In S904, the image analysis unit 212 executes an analysis process on the image data acquired in S903, and acquires information indicating a feature amount. In this embodiment, the image feature amounts shown in FIG. 29 are acquired.

[0079] In S905, the product type to be created and the number of products to be created for each product type are determined. The product type to be created is the number of product types acquired in S902. For example, in FIG. 5, it is three. Furthermore, the number of products to be created is set to a predetermined number for each product type. In this embodiment, five is used as an example. That is, in FIG. 5, it is determined to create a total of 15 products, including five posters, five postcards, and five banners. In this embodiment, a combination of products in which one product is selected for each product type is called a "product set." For example, in FIG. 5, a combination of three products, including one poster, one postcard, and one banner, is represented as one product set. In this embodiment, the number of products to be created is five, but other numbers may be used. If the number of products to be created is large, it is easier to generate a product set that is close to the target impression and has a high degree of design similarity between the products. If the number of products to be created is small, the generation process can be performed at high speed.

[0080] Steps S906 to S914, which will be described later, are repeated for the number of types of products to be created, and steps S907 to S913 are repeated for the number of products to be created. In other words, steps S907 to S913 are repeated for the number of types of products to be created multiplied by the number of products to be created. In this embodiment, there are three types of products to be created, and five products to be created, so steps S907 to S913 are repeated 15 times.

[0081] In S906, the skeleton acquisition unit 213 acquires skeletons that meet various set conditions for the product type to be processed. In this embodiment, it is assumed that each skeleton is described in one file and stored in the HDD 104. The skeleton acquisition unit 213 sequentially reads skeleton files from the HDD 104 to the RAM 103, leaves skeletons that meet the set conditions on the RAM 103, and deletes skeletons that do not meet the conditions from the RAM 103. Here, FIG. 9(b) is a flowchart of the condition determination process performed by the skeleton acquisition unit 213. The condition determination process of the skeleton acquisition unit 213 will be described in detail with reference to FIG. 9(b). S921 to S927 are subflows of S906.

[0082] In S921, the skeleton acquisition unit 213 determines whether the size of the skeleton read into the RAM 103 matches the size preset for the product type to be processed. For example, in FIG. 5, the poster is A2, the postcard is 100 mm×148 mm, and the banner is 360 mm×45 mm. Note that although it is confirmed that the sizes match here, it is also sufficient that the aspect ratios match. In this case, the skeleton acquisition unit 213 enlarges or reduces the coordinate system of the read skeleton to acquire a skeleton that matches the size of the product to be processed.

[0083] In S922, skeleton acquisition unit 213 determines whether the use category specified by multiple product creation condition specification unit 201 matches the skeleton category. For skeletons used only for specific purposes, the use category is written in the skeleton file, and the skeleton is not acquired unless the corresponding use category is selected. This can prevent a skeleton designed specifically for a specific purpose, such as a graphic pattern that evokes a school or a sporting goods pattern, from being used in other use categories. Note that if a use category is not set in application startup screen 501, S922 is skipped.

[0084] In S923, the skeleton acquisition unit 213 determines whether the number of image objects of the loaded skeleton matches the number of images acquired by the image acquisition unit 211.

[0085] In S924, skeleton acquisition unit 213 determines whether the character object of the loaded skeleton matches the character information specified by text designation unit 202. More specifically, for example, it determines whether the type of character information specified by text designation unit 202 is in the skeleton. For example, assume that character strings are designated in title box 502 and body box 504 on application startup screen 501, and a blank is designated in subtitle box 503. In this case, all character objects in the skeleton are searched, and if both a character object with "title" set as the type of character information in metadata and a character object with "body" set are found, it is determined that "there is" the type of character information in metadata, and otherwise it is determined that "there is" the type of character information in metadata.

[0086] In S925, the skeleton acquisition unit 213 determines whether or not a graphic object exists in the loaded skeleton. If a pattern is specified in the multiple product creation condition specification unit 201, a graphic object must exist to draw the pattern. If a pattern is not specified in the multiple product creation condition specification unit 201, S925 is skipped.

[0087] In S926, the skeleton acquisition unit 213 determines whether or not a logo object exists in the loaded skeleton. If a logo is specified in the multiple product creation condition specification unit 201, an object must exist to draw the logo. If a logo is not specified in the multiple product creation condition specification unit 201, S926 is skipped.

[0088] In S927, as a result of performing the processes from S921 to S926, skeletons that match all the determination processes remain in the RAM 103. The skeleton acquisition unit 213 selects the skeletons remaining in the RAM 103 as skeletons to be used for generating a poster.

[0089] As described above, the skeleton acquisition unit 213 stores in the RAM 103 a skeleton whose skeleton size, usage category, number of image objects, type of character objects, number of graphic objects, and number of logo objects all match the set conditions. In this embodiment, the skeleton acquisition unit 213 judges all skeleton files on the HDD 104, but this is not limited to the above. For example, the multiple product creation application may store in the HDD 104 a database that associates file paths of skeleton files with search conditions in advance. In this case, the skeleton acquisition unit 213 can quickly acquire skeleton files by reading only the matching skeleton files from the HDD 104 to the RAM 103 as a result of searching the database. In addition, the search conditions are, for example, the skeleton size, the number of image objects, the type of character objects, the number of graphic objects, and the number of logo objects.

[0090] This concludes the explanation of S906. Let us return to the explanation of FIG.

[0091] In S907, the skeleton selection unit 214 selects a skeleton that matches the target impression specified by the target impression specification unit 204 from among the skeletons acquired in S906. Here, FIG. 10 is a diagram for explaining a method in which the skeleton selection unit 214 selects a skeleton for a poster skeleton as an example. The same process is performed for other types of products. FIG. 10(a) is a diagram showing an example of a table that links skeletons to impressions. The skeleton name column in FIG. 10(a) lists the skeleton file names, and the columns for luxury, familiarity, dynamism, and solidity show numbers (numeric values) that indicate the degree to which each skeleton affects each impression. These numerical values ​​indicate that -2 is low, -1 is slightly low, 0 is neither, +1 is slightly high, and +2 is high, respectively, for that impression. First, the skeleton selection unit 214 calculates the distance between the target impression acquired from the target impression designation unit 204 and the impression of each skeleton shown in the skeleton impression table of FIG. 10(a). For example, if the target impression is "luxury +1, familiarity -1, dynamism -2, solidity +2", the distance calculated by the skeleton selection unit 214 is as shown in FIG. 10(b). In this embodiment, for example, Euclidean distance is used as the distance (hereinafter, simple distance is referred to as Euclidean distance). The smaller the value indicated by the Euclidean distance, the closer the target impression and the impression of the skeleton are. Next, the skeleton selection unit 214 selects the top N skeletons with the smallest distance values ​​in FIG. 10(b). In this embodiment, the skeleton selection unit 214 selects the top two skeletons. That is, the skeleton selection unit 214 selects skeleton 1 and skeleton 4. Here, N may be set to an integer of 1 or more.

[0092] Also, the value range of each impression in the skeleton impression table in FIG. 10(a) does not need to be the same as the value range of the impression specified by the target impression designation unit 204. In this embodiment, the value range of the impression specified by the target impression designation unit 204 is -2 to +2, but the value range of the impression in the skeleton impression table may be different from this. In that case, the value range of the skeleton impression table is scaled to match the value range of the target impression, and then the distance calculation is performed. Also, the distance calculated by the skeleton selection unit 214 is not limited to the Euclidean distance, and any distance between vectors can be calculated, such as Manhattan distance or cosine similarity. Also, impressions for which the target impression is set to off by the radio button 517 are excluded from the distance calculation.

[0093] The skeleton impression table is created in advance by, for example, fixing the color scheme, font, image and character data to be arranged on the skeleton, generating a poster image based on each skeleton, and estimating the impression, and is stored in the HDD 104. That is, the impression of poster images that use the same color and image, but have different arrangements of characters and images, is estimated, and the relative characteristics with other skeletons are tabulated. At this time, the estimated impression may be standardized as a whole, or the impressions of multiple poster images generated from one skeleton using multiple color scheme patterns and images may be averaged, thereby canceling the impression due to the color scheme pattern, image, etc. used. This allows the influence of the arrangement on the impression to be tabulated, such as the impression of a skeleton with a small image being determined by elements such as graphics and characters regardless of the image, or the impression of a strong sense of dynamism being felt when the image or character is arranged in a slanted manner.

[0094] FIG. 10(c) is an example of a skeleton corresponding to skeleton 1 to skeleton 4 in FIG. 10(a). For example, skeleton 1 has image objects and text objects regularly arranged and the image area is small, so it has a low sense of dynamism. Skeleton 2 has circular graphic objects and image objects, so it has a high sense of familiarity and a low sense of solidity. Skeleton 3 has a high sense of dynamism by arranging image objects large and arranging inclined graphic objects overlapping image objects. Skeleton 4 has an image arranged over the entire skeleton and a minimum number of text objects, so it has a high sense of solidity and a low sense of dynamism. In this way, when a poster image includes text or images, poster images with different target impressions are generated depending on the method of arranging the text or images. Note that the method of creating the skeleton impression table is not limited to this, and may be estimated from the characteristics of the arrangement information itself, such as the area and coordinates of the image or title string, or may be adjusted manually. The skeleton impression table is stored in the HDD 104, and the skeleton selection unit 214 reads the skeleton impression table from the HDD 104 into the RAM 103 for reference.

[0095] In S908, the color scheme pattern selection unit 215 selects a color scheme pattern that matches the target impression specified by the target impression specification unit 204. Furthermore, when a color is specified by the multiple product creation condition specification unit 201, a color scheme pattern that matches the specified color is selected. The method of selecting a color scheme pattern that matches the target impression is the same as that of S906, and refers to an impression table corresponding to the color scheme pattern, and selects a color scheme pattern according to the target impression. FIG. 11(a) shows an example of a color scheme pattern impression table that links a color scheme pattern with an impression. The color scheme pattern selection unit 215 calculates the distance value of the impression indicated by the column of luxury to column of solidity in FIG. 11(a) and the distance value between the target impression, and selects the top N color scheme patterns with the smallest distance value. In this embodiment, the top two color scheme patterns are selected. Note that, like the skeleton impression table, the color scheme pattern impression table can tabulate the tendency of the impression of the color scheme pattern by creating and estimating the impression after fixing the skeleton, font, and image other than the color scheme pattern. The method for selecting a color scheme pattern that matches a specified color is to select a color close to a color contained in the color scheme pattern. For example, if a color is included in the CIE L*A*B* color space with a distance ΔE of 2.0 or less, it is selected. Another criterion may be that the distance ΔRGB in the RGB color space is 1.0 or less. Specifically, for example, if the specified color is (R,G,B)=(0,67,69), color 1 with color scheme ID 1 in Figure 11(a) has a ΔRGB of 1.0 or less, and is therefore selected.

[0096] In S909, the font selection unit 216 selects a font combination that matches the target impression specified by the target impression specification unit 204. Furthermore, when a font is specified by the multiple product creation condition specification unit 201, a font combination that matches the specified font is selected. The method of selecting a font combination that matches the target impression is the same as that of S906, and refers to an impression table corresponding to the font, and selects a font according to the target impression. FIG. 11(b) shows an example of a font impression table that links fonts to impressions. The font selection unit 216 calculates the distance value between the impression indicated by the column of luxury to the column of profoundness in FIG. 11(b) and the target impression, and selects the top N fonts with the smallest distance value. Note that, like the skeleton impression table, the font impression table can tabulate the tendency of font impressions by fixing the skeleton, color scheme pattern, and image other than the font, and then creating and estimating the impression of a poster with a different font. A method of selecting a font combination that matches a specified font is, for example, to select a font combination that includes the specified font. Furthermore, a font combination close to the font impression value of the font combination that includes the specified font may be further selected. This allows you to select a font that gives a similar impression to the specified font.

[0097] In S910, the image correction parameter selection unit 221 selects image correction parameters according to the image feature amount acquired by the image analysis unit 212 and the target impression designated by the target impression designation unit 204. An example of a method for selecting image correction parameters that match the target impression will be described. The image feature amount acquired by the image analysis unit 212 is the image feature amount shown in FIG. 29. For example, if the familiarity of the target impression is a positive value, image correction parameters are selected such that the saturation of the corrected image is medium and the edge amount is small. As described above, in this embodiment, gamma correction is used as the saturation correction, and a blurring filter is used as the edge amount correction. That is, if the saturation of the image feature amount acquired by the image analysis unit 212 is high, a gamma value of 2 is selected to apply a gamma table with a downward convex shape, and if the saturation is low, a gamma value of 0.5 is selected to apply a gamma table with an upward convex shape. Note that if the saturation is medium, a gamma value of 1 is selected and no correction is applied. Also, if the edge amount of the image feature amount acquired by the image analysis unit 212 is high, a blurring filter is applied to reduce the edge amount. In this embodiment, a Gaussian filter is used as the blurring filter, so the larger the correction strength σ, the greater the blurring amount, and the closer to 0, the smaller the blurring amount. For example, if the edge amount acquired by the image analysis unit 212 is large, σ is selected as 2, and if the edge amount is medium, σ is selected as 1.5, thereby correcting the edge amount. In this way, by selecting image correction parameters according to the image analysis results and the target impression, the corrected image is more likely to match the impression of the entire design. Note that multiple image correction parameters are selected for the image correction parameters, such as the color scheme selected in S908 and the font selected in S909. For example, the target impressions are sorted in descending order, and image correction parameters that match the top N are selected.

[0098] In S911, the layout unit 217 generates a poster by setting character information, an image, a color scheme, a font, a design, and a logo for the skeleton selected by the skeleton selection unit 214. The processing of S911 and the layout unit 217 will be described in detail with reference to Figs. 12, 13, 14, and 15.

[0099] FIG. 12 is an example of a software block diagram for explaining the layout unit 217 in detail. The layout unit 217 is composed of a color scheme allocation unit 1201, an image arrangement unit 1202, an image correction unit 1203, a font setting unit 1204, a text arrangement unit 1205, a text decoration unit 1206, a pattern setting unit 1207, and a logo arrangement unit 1208. FIG. 13 is a flowchart for explaining S911 in detail. FIG. 14 is a diagram for explaining an example of information input to the layout unit 217. FIG. 14(a) is a table summarizing character information specified in the text designation unit 202, an image specified in the image designation unit 203, and a pattern and a logo specified in the multiple product creation condition designation unit 201. FIG. 14(b) is an example of a table showing a color scheme pattern acquired from the color scheme pattern selection unit 215, and FIG. 14(c) is an example of a table showing a font acquired from the font selection unit 216. FIG. 15 is a diagram for explaining the process of the layout unit 217.

[0100] S911 will be described in detail with reference to Figure 13. S1301 to S1310 are sub-flows of S911.

[0101] In S1301, the layout unit 217 lists all combinations of skeletons acquired from the skeleton selection unit 214, color patterns acquired from the color pattern selection unit 215, and fonts acquired from the font selection unit 216. The layout unit 217 generates poster data for each combination in order by the following layout process. For example, if the number of skeletons acquired from the skeleton selection unit 214 is 3, the number of color patterns acquired from the color pattern selection unit 215 is 2, and the number of fonts acquired from the font selection unit 216 is 2, the layout unit 217 generates 3×2×2=12 poster data. Next, in S1301, the layout unit 217 selects one of the listed combinations, and executes the processes of S1302 to S1309.

[0102] In S1302, the color scheme allocation unit 1201 allocates the color scheme pattern acquired from the color scheme pattern selection unit 215 to the skeleton acquired from the skeleton selection unit 214. FIG. 15(a) is a diagram showing an example of a skeleton. In this embodiment, an example is described in which a color scheme pattern with a color scheme ID of 1 in FIG. 14(b) is allocated to a skeleton 1501 in FIG. 15(a). The skeleton 1501 in FIG. 15(a) includes two graphic objects 1502 and 1503, one image object 1504, and three character objects 1505, 1506, and 1507. First, the color scheme allocation unit 1201 allocates colors to the graphic objects 1502 and 1503. Specifically, the color scheme allocation unit 1201 allocates a corresponding color from the color scheme pattern based on a color scheme number, which is metadata described in the graphic object. Next, the color allocation unit 1201 allocates, for example, the last color of the color scheme pattern to the character objects whose metadata is "type" and whose attribute is "title". That is, in this example, color 4 is allocated to the character arranged in the character object 1505. Next, for the character arranged in the character object whose metadata is "type" and whose attribute is other than "title", the color is set based on the brightness of the background of the character object. In this example, if the brightness of the background of the character object is equal to or lower than a threshold, the character color is set to white, and if not, the character color is set to black. FIG. 15(b) is a diagram showing the state of the skeleton 1508 after the above color allocation process. The color allocation unit 1201 outputs the skeleton data with the color allocation to the image arrangement unit 1202.

[0103] In S1303, the image arrangement unit 1202 arranges the image data acquired from the image analysis unit 212 on the skeleton data acquired from the color allocation unit 1201 based on the associated analysis information. In this example, the image arrangement unit 1202 allocates the image data 1401 to the image object 1504 in the skeleton. Furthermore, when the aspect ratios of the image object 1504 and the image data 1401 are different, the image arrangement unit 1202 performs trimming so that the aspect ratio of the image data 1401 matches the aspect ratio of the image object 1504. More specifically, based on the position of the object obtained by the image analysis unit 212 analyzing the image data 1401, trimming is performed so that the object area reduced by trimming is minimized. Note that the trimming method is not limited to this, and other trimming methods may be used, such as trimming the center of the image or devising a composition so that the face position is in a triangular composition. The image arrangement unit 1202 outputs the skeleton data to which the images have been assigned to the image correction unit 1203 .

[0104] In S1304, the image correction unit 1203 obtains skeleton data to which images have been assigned from the image arrangement unit 1202, and performs correction on the images arranged on the skeleton. In this embodiment, the image correction unit 1203 performs image correction processing on the image data arranged on the skeleton in accordance with the image correction parameters associated with the image data. That is, the image correction unit 1203 performs gamma correction processing of brightness and saturation, color temperature correction processing, sharpening filter processing, and blurring filter processing using the image correction parameters selected by the image correction parameter selection unit 221.

[0105] In S1305, the font setting unit 1204 sets the font acquired from the font selection unit 216 to the skeleton data after image correction acquired from the image correction unit 1203. FIG. 14C shows an example of a combination of fonts selected by the font selection unit 216. In this example, an example in which the font with font ID "2" in FIG. 14C is assigned to the skeleton data after image correction is described. In this example, fonts are set to the character objects 1505, 1506, and 1507 of the skeleton 1508. Note that in posters, a font that stands out is often set for the title from the viewpoint of eye-catchingness, and a font that is easy to read from the viewpoint of visibility is often set for the other characters. Therefore, in this example, the font selection unit 216 selects two types of fonts, a title font and a body font. The font setting unit 1204 sets the title font to the character object 1505 whose attribute is "title", and sets the body font to the other character objects 1506 and 1507. The font setting unit 1204 outputs skeleton data with font setting to the text arrangement unit 1205. In this example, the font selection unit 216 selects two types of fonts, but this is not limiting, and for example, only the title font may be selected. In this case, the font setting unit 1204 uses a font corresponding to the title font as the body font. In other words, if the title is a Gothic font, a typical Gothic font with high readability is selected for the rest, and if the title is a Mincho font, a typical Mincho font is selected for the rest, and so on, to set a body font that matches the type of title font. The title font and the body font may be the same. The font may be changed depending on the degree of prominence, such as using a title font for the character objects of the title and subtitle and a body font for the other character objects, or using a title font for a certain font size or larger.

[0106] In S1306, the text arrangement unit 1205 arranges the text specified by the text designation unit 202 in the skeleton data with font set obtained from the font setting unit 1204. In this example, each text shown in FIG. 14(a) is assigned by referring to the metadata attributes of the skeleton character objects. That is, "Summer Big Thank You Sale" with the attribute "Title" is assigned to the character object 1505, and "Blow Away the Midsummer Heat" with the attribute "Subtitle" is assigned to the character object 1506. Since "Body" is not set, nothing is assigned to the character object 1507. FIG. 15(c) shows a skeleton 1509 which is an example of skeleton data after processing by the text arrangement unit 1205. The text arrangement unit 1205 outputs the skeleton data with text arrangement to the text decoration unit 1206.

[0107] In S1307, the text decoration unit 1206 applies decoration to the character objects in the skeleton with text layout obtained from the text layout unit 1205. In this example, if the color difference between the title character and its background area is equal to or less than a threshold, a process of adding a border to the title character is performed. This improves the readability of the title. The text decoration unit 1206 outputs the skeleton data with the decoration to the picture layout unit 1207. In S1308, the pattern setting unit 1207 sets a pattern to the graphic object determined to be a background area in the text decorated skeleton data acquired from the text decoration unit 1206. Here, the graphic object determined to be a background area is the graphic object with the color scheme number 1. In FIG. 15(b), a pattern is set to the background area 1508. When setting a pattern, the color is changed to the color scheme assigned by the color scheme assignment unit 1201, and then the pattern is set to the background area. FIG. 15(d) is an example of a pattern set. When a pattern is not set by the multiple product creation condition designation unit 201, the pattern set in the skeleton data may be set. The pattern setting unit 1207 outputs the skeleton data with the pattern set to the logo placement unit 1208.

[0108] In S1309, the logo placement unit 1208 places the logo on the logo object in the skeleton data with the image set therein obtained from the image setting unit 1207. Fig. 15(d) is an example of a logo placed. If no logo is set in the multiple product creation condition designation unit 201, S1309 is skipped. The logo placement unit 1208 outputs the skeleton data with the logo placed therein, that is, the product data with all layout completed, to the impression estimation unit 218.

[0109] In S1310, the layout unit 217 determines whether all product data has been generated. If the layout unit 217 determines that product data has been generated for all skeletons, color patterns, font combinations, pictures, and logos for the product to be processed, it ends the layout process and transitions to S912. If it determines that all product data has not been generated, it returns to S1301 and generates product data for combinations that have not been generated yet.

[0110] This concludes the explanation of S911. Let us return to the explanation of FIG.

[0111] In S912, the impression estimation unit 218 estimates an impression for the product data generated in S911 and links it to the product data. Specifically, a rendering process is performed on the product data generated in S911. Next, an impression of the rendered product image is estimated. Finally, the estimated impression is linked to the product data. The rendering process is a process of converting the product data into image data. For example, even if the color scheme is the same, if the skeleton is different, the layout will change, so which color is actually used in what area will differ. Therefore, since it is necessary to evaluate not only the tendency of the individual impressions of the color scheme pattern and the skeleton, but also the final impression of the product, this process is executed at this timing. This makes it possible to evaluate not only the impression of the individual elements of the product, such as the color scheme and layout, but also the impression of the final product laid out including images and characters.

[0112] In S913, it is determined whether all product data has been generated for the number of products to be created for one product type determined in S905. In this example, it is five. If data has not been generated for the number of products to be created, the process returns to S907 and new product data is generated. If data has been generated for the number of products to be created, the process transitions to S915.

[0113] In S914, it is determined whether all product data has been generated for the product types to be created determined in S905. In this example, five pieces of product data are generated for each of the three product types. In other words, 15 pieces of product data are generated. If the number of products to be created for all product types has not been generated, the target product type is changed, and the process returns to S906 to generate product data for the newly set product type for the number of products to be created. If product data has been generated for all product types, the generation process ends and the process transitions to S915.

[0114] In S915 , the design similarity calculation unit 219 calculates the design similarity for all patterns of product sets combined with all the product data acquired from the layout unit 217 .

[0115] S915 will be described in detail with reference to Fig. 9(c). S931 to S937 are sub-flows of S915.

[0116] In S931, the design similarity calculation unit 219 performs the processes from S906 to S914 and determines all possible combinations of the product sets from the product data generated. Specifically, in this example, there are 15 product data in total, including five posters, five postcards, and five banners. The five poster product data are designated Pn (n=an integer from 1 to 5), the five postcard product data are designated Cn (n=an integer from 1 to 5), and the five banner product data are designated Bn (n=an integer from 1 to 5). In this case, there are 5×5×5=125 patterns of product set combinations. FIG. 28 is a table showing an example of a product set combination.

[0117] Steps S932 to S936 are repeated for the number of product sets determined in step S931. For simplicity's sake, the following description will be given assuming that the product set contains two products, product 1 and product 2. For example, product 1 is a poster and product 2 is a postcard.

[0118] In S932, the design similarity calculation unit 219 calculates the similarity of the color scheme. The similarity of the color scheme is calculated by calculating the degree to which the colors used are similar. The smaller the similarity of the color scheme, the more similar the value. Specifically, the similarity of the color scheme is calculated by the distance ΔE in the CIE L*A*B* color space of the combination of the color scheme patterns used. Assume that the color scheme patterns used in the product 1 are color scheme 1 and color scheme 2, and the color scheme patterns used in the product 2 are color scheme 3 and color scheme 4. In this case, the sum of the distances ΔE in the CIEL*A*B* color space of the color schemes 1 and 3, color scheme 1 and color scheme 4, color scheme 2 and color scheme 3, and color scheme 2 and color scheme 4 is set as the similarity of the color schemes. In addition, the similarity of the color schemes may be calculated by using the area ratio to the total area of ​​the products with each color scheme as a weight. In this case, a rendering process is performed on the product 1 and the product 2, and the area is calculated using the rendered products 1 and 2. In this case, the formula for calculating the color scheme similarity Mc is as follows: color scheme 1's ratio to product 1's total area is Wc1, color scheme 2's ratio to product 1's total area is Wc2, color scheme 3's ratio to product 2's total area is Wc3, and color scheme 4's ratio to product 2's total area is Wc4. If ΔE between color scheme 1 and color scheme 3 is ΔE13, ΔE between color scheme 1 and color scheme 4 is ΔE14, ΔE between color scheme 2 and color scheme 3 is ΔE23, and ΔE between color scheme 2 and color scheme 4 is ΔE14, then the formula is as follows:

[0119] Mc = Wc1×Wc3×ΔE13+Wc1×Wc4×ΔE14+Wc2×Wc3×ΔE23+Wc2×Wc4×ΔE24 For example, the ratio of color scheme 1 to the total area of ​​product 1 is 0.5, color scheme 2 to the total area of ​​product 1 is 0.1, color scheme 3 to the total area of ​​product 2 is 0.3, and color scheme 4 to the total area of ​​product 2 is 0.2. The ΔE between color scheme 1 and color scheme 3 is 1, the ΔE between color scheme 1 and color scheme 4 is 2, the ΔE between color scheme 2 and color scheme 3 is 3, and the ΔE between color scheme 2 and color scheme 4 is 4. The similarity of the color schemes in this case is 0.52.

[0120] In S933, the design similarity calculation unit 219 calculates the similarity of the patterns. The similarity of the patterns used in the commercial materials is calculated by calculating the degree to which the shapes of the patterns are similar. The smaller the similarity of the patterns, the more similar the value is. Specifically, for example, the similarity of the patterns is calculated from the similarity of the outlines of the patterns. The calculation method may use an existing method. For example, an outline image is generated by applying an edge extraction filter to pattern images cut out to the same size. The extracted outline images are superimposed, and the number of pixels in which both patterns are edge pixels is calculated. A value indicating the similarity of the patterns is calculated based on the ratio of the number of pixels in which both patterns are edge pixels to the total number of pixels in the pattern images. If the pattern is not used in either commercial material, the value is the maximum value, and it is determined that they do not match. If the pattern is used in both commercial materials, the value is zero, and it is determined that they match. For example, a value obtained by adding the square of the difference in pixel values ​​of the outline images is used as the similarity.

[0121] In S934, the design similarity calculation unit 219 calculates the similarity of the logos. The similarity of the logos used in the commercial materials is calculated, for example, by calculating the degree of similarity between the shape and color of the logos. The similarity of the logo shapes is calculated by carrying out the same process as in S932. The similarity of the logo colors is calculated from the color difference ΔE between the colors with the largest area in the logos. The similarity of the logos is calculated as the sum of the similarity of the logo shapes and the similarity of the logo colors. The smaller the logo similarity value, the more similar it is.

[0122] In S935, the design similarity calculation unit 219 calculates the similarity of fonts. The similarity of fonts used in the commercial material is calculated by calculating the degree of similarity between the shape and color of the font. The similarity of fonts is calculated by performing the same process as in S932 on character images rendered with the same text. The smaller the value of the similarity of fonts, the more similar the value. The similarity of fonts may also be calculated from the result of classification by font type. For example, font types for alphabets include serif and sans serif. If the fonts used in the commercial material are serifs or sans serifs, the similarity of fonts is set to zero and they are considered to match. If the fonts used in the commercial material are different between serif and sans serif, the similarity of fonts is set to the maximum value and they are considered to be mismatched. The similarity may also be calculated by the difference in font thickness. For example, the similarity of fonts with thickness 1 and thickness 2 is 1.

[0123] In S936, the design similarity calculation unit 219 performs an overall evaluation from the similarities calculated in S932 to S935, and calculates an overall similarity. The calculated overall similarity is linked to each product set and stored in the RAM 103. Since each similarity has a different value range, all product sets are standardized as a whole. The overall similarity is calculated by adding up the standardized similarities. In addition, each similarity may be weighted and added up. This is because it is known that when designs are recognized as similar, the influence of color and shape is strong and the influence of font and logo is low. Therefore, by increasing the weight of the color scheme similarity and the pattern similarity and decreasing the weight of the logo similarity and the font similarity, it is possible to imitate the evaluation that a person recognizes that they are the same design. The weight can be determined by subjective evaluation.

[0124] FIG. 21 and FIG. 22 are diagrams for explaining a subjective evaluation that determines the weight between design elements of the design similarity. For example, a reference product as shown in FIG. 21(a) is created. A product in which only one design element is changed from the reference product is created. For example, a product in which only the logo is changed as shown in FIG. 21(b), a product in which only the color scheme is changed as shown in FIG. 21(c), a product in which only the font is changed as shown in FIG. 21(d), and a product in which only the picture is changed as shown in FIG. 21(e). The reference product is compared with each of the changed products, and a subjective evaluation is performed to determine how similar they can be determined. For example, the evaluation value as shown in FIG. 22 is set to 5 (very different) to 0 (very similar), and a subjective evaluation is performed for multiple subjects to compare the reference product with products with different color schemes. The same evaluation is performed for different pictures, logos, and fonts. The average value of the multiple subjects is calculated. Since a large average value indicates that the products are not similar, it can be estimated that even a small difference is likely to cause the designs to be determined to be not similar. Therefore, when calculating the design similarity, the weight of the design element is increased. On the other hand, since a small average value indicates similarity, it can be assumed that designs are easily judged to be similar even if there is only a small difference. Therefore, when calculating the design similarity, the weight of the design element is decreased. Specifically, the weight is calculated by setting the sum of the average values ​​of each design element to 1, and setting the ratio of the average value of each design element to the weight. This makes it possible to appropriately generate multiple products with a unified design among multiple products. In the above explanation, since the product set includes two products, there is only one design similarity, but if the product set includes three or more products, there are multiple combinations of two products, so multiple design similarities are calculated. Specifically, if the product set includes product 1, product 2, and product 3, three design similarities are calculated: product 1 and product 2, product 2 and product 3, and product 3 and product 2. When the product set includes three or more products, the design similarity calculation unit 219 calculates the sum of the design similarities as the design similarity of the product set. The design similarity calculation unit 219 associates the calculated design similarity with the product set and stores it.

[0125] This concludes the explanation of S915. Let us return to the explanation of FIG.

[0126] In S916, the image similarity calculation unit 222 calculates image similarities for all patterns of product sets combined with all product data acquired from the layout unit 217 in the same manner as in S915.

[0127] S916 will be described in detail with reference to Fig. 9(d). S941 to S945 are sub-flows of S916.

[0128] In S941, the image similarity calculation unit 222 executes the processes from S906 to S914 to determine all possible combinations of the product set from the product data generated. The specific process is the same as S931, so a description thereof will be omitted here.

[0129] Steps S942 to S945 are repeated for the number of product sets determined in step S941. For the sake of simplicity, the following description will be given assuming that the product set contains two products, product 1 and product 2. For example, product 1 is a poster and product 2 is a postcard.

[0130] In S942, the image similarity calculation unit 222 extracts an image area from the product data. The image arranged on the skeleton may be trimmed during the arrangement process, or may be hidden by other objects arranged on top of the image. The image similarity calculation unit 222 specifies the visible area of ​​the image data arranged on the skeleton and extracts pixel data.

[0131] In S943, the image similarity calculation section 222 extracts the color that appears most frequently from the pixel data extracted in S942 as the primary color.

[0132] In S944, the image similarity calculation unit 222 calculates the similarity between the main color of the product 1 extracted in S943 and the main color of the product 2. Here, the color similarity is calculated by the color difference ΔE. In this embodiment, the similarity is calculated by assuming the main color as one color, but the top N main colors may be extracted and the sum of their similarities may be calculated. In addition, the image similarity may be calculated using other indices such as the average luminance value of the image and the edge amount. For example, the intermediate feature of an image recognizer using machine learning, which is a known technology, is used as an index, and the cosine similarity of two intermediate feature values ​​is calculated to calculate the similarity that takes into account the composition other than color and the content appearing in the image. The image similarity calculation unit 222 associates the calculated image similarity with the product set and stores it.

[0133] This concludes the explanation of S916. Let us return to the explanation of FIG.

[0134] In S917, the multiple product selection unit 220 selects a product set to be output to the display 105 (presented to the user) from the estimated impression acquired from the impression estimation unit 218, the design similarity acquired from the design similarity calculation unit 219, and the image similarity acquired from the image similarity 222. Specifically, the multiple product selection unit 220 first calculates the impression distance between the estimated impression and the target impression linked to the product data. The calculated impression distance is linked to the product data and stored. In this example, since 15 product data are generated, 15 impression distances are calculated. The Euclidean distance is used as the impression distance. The smaller the value indicated by the Euclidean distance, the closer the target impression and the estimated impression are. In addition, the distance calculated by the poster selection unit 219 is not limited to the Euclidean distance, and any distance between vectors such as Manhattan distance and cosine similarity may be used. Next, the multiple product selection unit 220 calculates the total impression distance for each product set. In this example, one product set includes three products: a poster, a postcard, and a banner. The total impression distance is calculated by adding up the three calculated impression distances. The calculated total impression distance is linked to the product set and saved.

[0135] Next, the multiple product selection unit 220 calculates the sum of the total impression distance, the design similarity, and the image similarity, which are linked to the product set. The multiple product selection unit 220 selects the product set from the calculated sum. In this example, since one product set is displayed on the preview screen 601 of FIG. 6, the product set with the smallest sum is selected. Also, when multiple product sets are displayed on the preview screen, the top of the product sets in order of the smallest sum value for the number of products displayed are selected. Also, a product set with a sum value equal to or smaller than a predetermined threshold value may be selected. In this case, if it is not possible to select the number of product sets displayed on the preview screen, the process may return to S907 and generate different product data. Which of the impression distance, the design similarity, and the image similarity is to be prioritized is determined based on the reflection degree of the brand information specified by the multiple product creation condition specification unit 201. It can be determined based on the reflection degree by applying weights to the total impression distance, the design similarity, and the image similarity when they are added together. For example, when the reflection degree is 0.4 as shown in Fig. 5, the weight of the impression distance is 0.6, and the weights of the design similarity and image similarity are each 0.2, and the combined value is calculated. This allows the user to adjust the priority of the impression matching degree and the brand matching degree using the reflection degree slider bar 520 on the application start-up screen 501, and makes it possible to automatically generate merchandise that better matches the user's intentions.

[0136] In S918, the multiple product display unit 205 displays the preview screen 601 of Fig. 6. Specifically, the multiple product data included in the product set selected by the multiple product selection unit 220 in S917 is rendered and output to the display 105.

[0137] The above is an explanation of the multiple product generation process for generating multiple products by the user specifying an impression.

[0138] As described above, according to this embodiment, it is possible to automatically generate commercial materials that have a sense of unity and express the impression of the brand intended by the user even if different images are arranged for multiple commercial materials. More specifically, in this embodiment, in order to generate a design that expresses the impression of the brand intended by the user, each element constituting the commercial material, such as a skeleton, a color scheme pattern, a font, and an image correction parameter, is combined based on the target impression for each commercial material. Furthermore, by estimating the overall impression of the commercial material and selecting a commercial material that is close to the target impression from one or more poster candidates, it is possible to generate a commercial material that has an overall impression in line with the user's intention, not just individual elements. Furthermore, in order to generate a design with a sense of unity, it is possible to select and display a design with a high degree of similarity in design elements by calculating the design similarity and image similarity between the commercial materials.

[0139] <Modification of the first embodiment> In the first embodiment, the preview screen 601 displays the generated product set, but the target impression and brand information may also be set on the preview screen.

[0140] FIG. 16 is a diagram showing an example of a UI for setting a target impression and setting brand information on a preview screen 1601. The components with the same numbers as those in FIG. 5 perform the same operations as those in FIG. 5. The components with the same numbers as those in FIG. 6 perform the same operations as those in FIG. 6. The multiple product display unit 205 displays the preview screen 1601 in S918. The preview screen 1601 has the impression sliders 508 to 511 and impression valid radio button 512 of the application start screen 501 arranged as UIs for setting a target impression. Furthermore, the preview screen 1601 has the key color designation box 515, the picture designation box 516, the logo designation box 517, the font designation box 518, and the reflection degree slider bar 520 arranged as UIs for setting brand information. Furthermore, the preview screen 1601 has a reflection button 1602 for reflecting the set information. The user changes the target impression and brand information while checking the preview image 601. Then, by pressing the reflect button 1601, the process proceeds to S902 in Fig. 9, and the processes from S902 to S918 are executed. As a result, the multiple product display unit 205 displays a preview image 601 that reflects the target impression and brand information settings set on the preview screen 1601. This allows the target impression and brand information to be set while checking the results of the product set generation. As a result, a design that has a sense of unity and expresses the brand impression intended by the user can be automatically generated without cumbersome screen transitions.

[0141] In the first embodiment, the target impression and brand information are set using the start screen 501. However, for a user who does not know about impressions and brands, it is difficult to set the impression and brand information using the start screen 501. The user has to repeatedly reset the settings while viewing the product generation result, which is a cumbersome operation. In this embodiment, the target impression and brand information may be reflected from the generation result of one product to another product. Specifically, for one product type, multiple products reflecting various impressions and brands are displayed and the user is allowed to select one of them. Then, the target impression and brand information are obtained from the selected product, and a product of another product type is generated. This allows even a user who does not know about impressions and brands to automatically generate a design that has a sense of unity and expresses the user's intended brand impression without repeatedly resetting the settings.

[0142] FIG. 25 is a diagram showing an example of an application start screen 2501 provided by a multiple product creation application in this embodiment. The components with the same numbers as those in FIG. 5 perform the same operations as those in FIG. 5. As shown in FIG. 25, compared to the application start screen 501, there is no UI for setting impressions and brand information. When the OK button 2502 is pressed, various variations of products with different target impressions and brand information are generated for the product type at the top left of the product types selected in the product creation designation area 513. When the products are generated, the multiple product display unit 219 displays the generated products on the preview screen 2601 in FIG. 26. As an example, in this embodiment, the generation results of various variations of poster products are displayed. Next, the user selects a poster that matches his / her intention from the poster products displayed in the product image 2602. After the selection, when the OK button 2403 is pressed, the target impression and brand information are obtained from the selected poster product, and products of other product types are generated. When other product types are generated, the screen transitions to the preview screen 601 in FIG. 6. In the present embodiment, as an example, the product set displayed as a preview in FIG. 6 indicates that the rightmost poster displayed on the preview screen 2601 in FIG. 26 has been selected.

[0143] Fig. 27 is a flowchart showing a process for generating multiple products in this embodiment. Note that, among the processes in this flowchart, the processes with the same numbers as those in the flowchart in Fig. 9 are the same as those described in the first embodiment, so their explanations are omitted here. Note that, in the process of this flowchart, S901, S902, S905, S915, S916, and S917 shown in Fig. 9 are omitted.

[0144] In S2701, the multiple product creation application displays the application start-up screen 2501 on the display 105. The user inputs each setting via the UI screen of the application start-up screen 2501 using the keyboard 106 or the pointing device 107.

[0145] In S2702, multiple product creation condition specification unit 201, text specification unit 202, and image specification unit 203 acquire corresponding settings from application startup screen 501. Text specification unit 202 and image specification unit 203 acquire the same settings as in S902. Multiple product creation condition specification unit 201 acquires the product type and category to be created by performing the same processing as in S902.

[0146] After S904 in FIG. 27, in S2703, the first product type to be generated in various variations displayed on the preview screen 2601 and the target impression to be generated are determined. The first product type is selected from the product types displayed at the top left among the products selected by the multiple product creation condition designation unit 201. In this example, a poster is selected. Also, the product type may be changed to one for which the user can confirm the impression and brand information. For example, a product that includes a logo, font, color scheme, and pattern, which are examples of brand information, as design elements may be selected. Next, the target impression designation unit 204 determines multiple target impression patterns discretely to provide various variations. As an example, three patterns of dynamism +2 (other impressions are zero), dynamism -2 (other impressions are zero), and luxury +2 (other impressions are zero) may be used. In order to provide various variations, it is preferable that the target impression values ​​are separated in the impression space. Furthermore, a predetermined number is determined as the number of products to be created for one target impression pattern. In this embodiment, as an example, five is used. That is, in this embodiment, five poster materials are generated for three target impression patterns. In this example, five poster materials are generated for one target impression pattern by performing the processes from S907 to S913 in FIG. 27.

[0147] After S913, in S2704, the multiple product selection unit 220 selects one product from the multiple products created for one target impression pattern, based on the estimated impression acquired from the impression estimation unit 218. Specifically, first, the multiple product selection unit 220 calculates the impression distance between the estimated impression linked to the product data and the target impression, similar to S917 in FIG. 9. Next, the product with the smallest impression distance is selected. The selected product is stored in the RAM 103. The selected product is displayed on the preview screen 2601 in S2706, which will be described later.

[0148] In S2705, it is determined whether the number of products corresponding to the number of target impressions determined in S2703 has been selected in S2704. In this example, it is three. If the number of products corresponding to the number of target impressions has not been selected, the target impression designation unit 204 changes the target impression pattern to the next target impression pattern, and then proceeds to S907. If the number of products corresponding to the number of target impressions has been selected, the process proceeds to S2705.

[0149] In S2706, the products selected in S2704 are displayed on the preview screen 2601, and the user is prompted to select one. The product data selected by the user is stored in the RAM 103.

[0150] In S2707, a target impression and brand information are obtained from the product data selected by the user in S2706. The target impression to be obtained is an estimated impression linked to the product data selected by the user. The brand information to be obtained is at least one of the logo, color scheme, font, and design included in the product data selected by the user, and image data included in the product data. If there is product data that does not include a logo, color scheme, font, or design, the brand information is not obtained. In this way, at least one of the logo, color scheme, font, and design can be made the same for skeletons of different products.

[0151] In S2708, a product type different from the first product type is determined as the product type to be generated based on the settings acquired in S2702 and the target impression and brand information acquired in S2707. As an example, in FIG. 25, there are two products, namely, postcards and banners. Furthermore, a predetermined number is determined as the number of products to be generated for one product type. In this example, five products are generated. By performing the processes from S906-2 to S913-2, five products are generated for one product type. S906-2, S907-2, S911-2, S912-2, and S913-2 are the same as the explanations in S906, S907, S911, S912, and S913.

[0152] After S907-2, in S2709, the color scheme selection unit 215 selects a color scheme that matches the target impression and brand information acquired in S2707. First, the color scheme selection unit 215 narrows down all color schemes to only those that are close to the color scheme acquired in S2707. The distance between two color schemes can be calculated by the total color difference ΔE of the colors included in each color scheme. In this example, the color scheme has four colors as shown in FIG. 11(a), so the color scheme is narrowed down to a color scheme with a total color difference ΔE of 3.0 or less for the color scheme acquired in S2707 as a close color scheme. Next, the color scheme selection unit 215 selects a color scheme that matches the target impression from the narrowed down color schemes in the same manner as in S908. Note that in this example, the total color difference of the colors in the color scheme is used to calculate the distance between the color schemes, but for example, only the color difference of color 1 in FIG. 11(a) may be used. This allows for variation in color schemes while maintaining similarity of the designs by matching the base colors of the designs.

[0153] In S2710, the font selection unit 216 selects a font combination that matches the target impression and brand information acquired in S2707. First, the font selection unit 216 narrows down all fonts to only those in the same font family as the font acquired in S2707. A font family refers to the type of font, and fonts in the same font family have similar designs but different thicknesses. Next, the font selection unit 216 selects a font combination that matches the target impression from the narrowed down fonts in the same manner as in S909.

[0154] In S2711, the image correction parameter selection unit 221 selects image correction parameters that match the target impression and brand information acquired in S2707. The image corrections in this example are gamma correction of brightness and saturation, color temperature correction, and edge amount correction.

[0155] FIG. 27(b) is a sub-flow explaining S2711 in detail.

[0156] In S2721, the image correction parameter selection unit 221 extracts the primary color (reference primary color) of the image data acquired in S2707. Note that the primary color is extracted in the same manner as in S941 of FIG.

[0157] In S2722, the image correction parameter selection unit 221 extracts the main color (adjusted main color) of the image data to be placed in the commercial material. Note that since the image data has not yet been placed in the skeleton, the main color in the entire image data is extracted.

[0158] In S2723, the image correction parameter selection unit 221 narrows down the value range of the image correction parameters from the reference primary color extracted in S2721 and the adjusted primary color extracted in S2722. Here, an example will be described in which the reference primary color is RGB(220, 110, 200) and the adjusted primary color is RGB(140, 0, 140). In this example, according to the RGB-HSL conversion, the luminance value L of the reference primary color is 62, and the luminance value L of the adjusted primary color is 37. Therefore, for example, in the gamma correction of the brightness, the value range is narrowed down so that the gamma correction is performed so that the luminance value of the adjusted primary color approaches the luminance value of the reference primary color. Since the calculation formula of the gamma correction is as follows, a correction of γ≒0.73 is required to make the luminance value of the adjusted primary color the luminance value of the reference primary color.

[0159] L'=255×(L÷255)^γ L': luminance value of the reference primary color, L: luminance value of the adjusted primary color In this example, the range of the image correction parameters is set to ±0.1 to set the luminance value around the reference primary color. That is, here, 0.63<γ<0.83 is the range of the luminance correction parameters. Note that the range may be set to upper and lower limits so that image quality is not compromised by extreme correction. For gamma correction of saturation, the above luminance value is replaced with saturation and similar processing is performed to narrow the range. For color temperature correction, the color temperature that minimizes the color difference ΔE between the adjusted primary color and the reference primary color according to the RGB magnification shown in Figure 31 is found, and the range is set to ±1000K around that.

[0160] In S2724, the image correction parameter selection unit 221 selects image correction parameters that match the target impression, from within the range of the image correction parameter values ​​narrowed down in S2723, in the same manner as in S910.

[0161] This concludes the explanation of S2711. In this way, by limiting each image correction parameter so as to be closer to the image of the first product, while allowing a certain degree of latitude, the product can be selected in a later step based on the image similarity, design similarity, and impression match in the final design. Return to the explanation of FIG. 27(a).

[0162] After S916, in S2712, the multiple product selection unit 220 selects one product from the multiple (five in this example) created products for one product type based on the estimated impression acquired from the impression estimation unit 218, the design similarity acquired by the design similarity calculation unit 219, and the image similarity acquired by the image similarity calculation unit 222. Specifically, first, the multiple product selection unit 220 calculates the impression distance between the estimated impression linked to the product data and the target impression, similar to S917. Next, it calculates the design similarity between the first product and the created product, similar to S915. Next, it calculates the image similarity between the first product and the created product, similar to S916. Finally, it selects the product with the smallest total value of the impression distance, the design similarity, and the image similarity. The selected product is stored in the RAM 103. The selected product is displayed on the preview screen 601 in S918.

[0163] This allows even users who are unfamiliar with impressions and brands to automatically set impressions and brand information by selecting the generated product. As a result, a design that has a sense of unity and expresses the brand impression intended by the user can be automatically generated without the need for cumbersome repetitive operations.

[0164] [Second embodiment] The second embodiment will be described below with respect to the differences from the first embodiment. In the first embodiment, a skeleton, color scheme, and font, which are components of a commodity, are selected based on a target impression to generate a commodity set consisting of different types of commodity, and further, a design with a high degree of similarity in design elements between different commodity materials is generated by calculating the design similarity and image similarity between the commodity materials. In the second embodiment, the combination generating unit 1702 searches for a combination of components of a commodity that makes the impression of the entire poster close to the target impression, and a combination of design elements that increases the degree of design similarity, based on a genetic algorithm. This makes it possible to more flexibly select the components of a commodity that are optimal for the target impression and the design elements that have a high degree of design similarity, without creating a skeleton impression table, a color scheme impression table, a font impression table, an image correction parameter table, etc. in advance.

[0165] Fig. 17 is a software block diagram of a multiple product creation application in the second embodiment. In the configuration of the block diagram shown in Fig. 17, a multiple skeleton acquisition unit 1701 and a combination generation unit 1702 are configured instead of the skeleton selection unit 214, the color scheme pattern selection unit 215, the font selection unit 216, and the image correction parameter selection unit 221 in Fig. 2. Note that components with the same numbers as those in Fig. 2 perform the same processes as those described in the first embodiment, and therefore will not be described here.

[0166] The multiple skeleton acquisition unit 1701 acquires skeletons for the number of product types specified in the multiple product creation condition designation unit 201. In order to use a search algorithm, all skeletons for the number of specified product types are acquired and stored in the RAM 103.

[0167] The combination generating unit 1702 acquires one or more skeletons for each product type from the multiple skeleton acquiring unit 1701, and acquires product data and product estimated impressions from the impression estimating unit 218. The combination generating unit 1702 acquires a target impression from the target impression designating unit 204, acquires design similarity between products from the design similarity calculating unit 219, and acquires image similarity between products from the image similarity calculating unit 222. Furthermore, the combination generating unit 1702 acquires a list of color scheme patterns and fonts, and a list of designs from the HDD 104. Furthermore, the combination generating unit 1702 generates combinations of components (skeleton, color scheme pattern, font, image correction parameter) and design elements (logo, design) of the product used for product generation, for each product type. As an example, in this embodiment, three combinations of posters, postcards, and banners are generated. One combination for each product type is defined as one product set combination. The combination generating unit 1702 outputs the generated product set combination to the layout unit 217.

[0168] Fig. 18 is a flowchart showing the processing of the multiple product generation unit 210 of the multiple product creation application in this embodiment. Note that, among the processing of this flowchart, the processing with the same numbers as those in the flowchart of Fig. 9 is the same as the processing described in the first embodiment, so the description will be omitted here. Note that, in the processing of this flowchart, S906 to S910 and S913 shown in Fig. 9 are omitted.

[0169] After S905, in S1801, the multiple skeleton acquisition unit 1701 acquires skeletons for the number of product types specified in the multiple product creation condition designation unit 201. Specifically, the processes from S921 to S926 in FIG. 9(b) are repeated for the number of product types specified. In order to use a search algorithm, all skeletons for the number of product types specified are acquired. As an example, in FIG. 5, all skeletons for posters, postcards, and banners are acquired.

[0170] In the description of S1802, the operation at the first execution and the operation after the second loop will be described separately. First, at the first execution of S1802, the combination generating unit 1702 acquires each table of skeleton, color scheme, font, logo, and pattern used for generating the product. FIG. 19 is a diagram for explaining the tables used by the combination generating unit 1702. FIG. 19(a) shows a list of skeletons acquired by the combination generating unit 1701 from the multiple skeleton acquiring unit 1801. FIG. 19(b), FIG. 19(c), FIG. 19(d), and FIG. 19(e) show a list of fonts, a list of color schemes, a list of logos, and a list of patterns acquired by the combination generating unit 1702 from the HDD 104, respectively. The combination generating unit 1702 generates the specified products by random combination from the above five tables for the number of product types. In this example, 100 product set combinations are generated. In addition, the image correction parameters are also randomly combined. In this embodiment, the luminance gamma correction and chroma gamma correction are set to 0.7 to 1.3, the color temperature to 4000K to 9000K, and the edge correction amount to a random combination of values ​​within the range of -2 to +2.

[0171] FIG. 19(f) shows an example of a product set combination table generated in this embodiment. Note that the edge correction in FIG. 19(f) shows the manner and degree of edge correction, with + being a sharpening filter and - being a blurring filter. Also, +1 and +2 show the additive coefficients of the Laplacian filter being 0.5 and 1, respectively, -1 and -2 show the σ values ​​of the Gaussian filter being 1.5 and 2, respectively, and 0 shows that edge correction is not performed. Also, when brand information is specified in the multiple product creation condition specification unit 201, settings are made so that the IDs are the same between the products. FIG. 19(f) shows a combination table when a pattern is specified as the brand information. The same pattern ID is set for product 1 and product 2. Furthermore, when the reflection degree of brand information is specified in the multiple product creation condition specification unit 201, the ratio of product 1 and product 2 having the same ID is determined based on the set value. Specifically, for example, if the reflection degree of brand information is specified as 0.3, 30 of the 100 combinations will have the same ID, and the remaining 70 will be random combinations. After that, the combination generation unit 1702 executes the processes of S911 (layout), S912 (impression estimation), S915 (design similarity estimation), and S1803 for all the generated combinations.

[0172] Next, in the second loop of S1802 and thereafter, the combination generation unit 1702 calculates an evaluation value from the estimated impression acquired from the impression estimation unit 218, the design similarity acquired from the design similarity calculation unit 219, and the image similarity acquired from the image similarity calculation unit 222, and links the evaluation value to the product set combination table. The method of calculating the evaluation value is the same as the method of calculating the combined value of the total impression distance, the design similarity, and the image similarity in S917 of FIG.

[0173] FIG. 20 is a diagram for explaining the operation of S1802 after the second loop. FIG. 20(a) is a table in which the above evaluation values ​​are linked to FIG. 19(d). The evaluation value column in FIG. 20(a) represents the evaluation value of the product set generated by the combination of the corresponding row. The combination generating unit 1701 generates a new combination table from FIG. 20(a). FIG. 20(b) is a newly generated combination table. In this embodiment, new combinations are generated using tournament selection and uniform crossover in a genetic algorithm. First, N combinations are randomly selected from the table in FIG. 20(a). Here, for example, N=3. Next, from the selected combinations, the top two combinations with the smallest evaluation values ​​are selected. Finally, in the two selected combinations, each element of the combination (skeleton ID, color scheme ID, font ID, logo ID, picture ID, image correction parameters) is randomly replaced to generate a new combination. For example, combination IDs 1 and 2 in FIG. 20(b) show the results generated from combination IDs 1 and 3 in FIG. 20(a), in which the color scheme IDs are replaced. The above procedure was repeated to generate 100 new combinations, as shown in Figure 20(b).

[0174] This allows efficient combination search based on the evaluation values ​​of the total impression distance, design similarity, and image similarity. In this embodiment, 100 combinations are generated, but this is not limited to this. In addition, tournament selection and uniform crossover are used, but this is not limited to this, and other methods such as ranking selection, roulette selection, and one-point crossover may be used. Mutation may be incorporated to make it difficult to fall into a local optimum solution. In addition, skeletons (layouts), color scheme patterns, fonts, logos, and patterns are used as components of the commercial material to be searched, but other components may be used. By increasing the number of components to be searched, posters with more variety can be generated, and the range of impression expression can be increased.

[0175] In S1803, the multiple product selection unit 1703 calculates the evaluation values ​​of the total impression distance and the design similarity in the same manner as in S1802, and creates a table similar to that shown in Fig. 20(a). The multiple product selection unit 1703 stores in the RAM 103 a product set whose evaluation values ​​of the total impression distance, the design similarity, and the image similarity are equal to or less than predetermined thresholds.

[0176] In S1803, the multiple product selection unit 1703 further determines whether the number of product sets stored in the RAM 103 reaches a predetermined number of combinations. If the multiple product selection unit 1703 determines that the number of combinations has reached the predetermined number, it proceeds to S918. If it determines that the number of combinations has not reached the predetermined number, it returns to S1802. That is, the process of S1802 in the second loop described above is executed, and the processes of S1802 to S1803 are repeatedly executed until the number of product sets stored in the RAM 103, whose evaluation values ​​of the total impression distance, the design similarity, and the image similarity are equal to or less than the threshold, reaches a predetermined number. Note that if the number of product sets whose evaluation values ​​are equal to or less than the threshold are stored is equal to or more than the predetermined number, the multiple product selection unit 1703 may compare the evaluation values ​​of the respective stored product sets, and finally store in the RAM 103 only the product set with the smaller evaluation value. Then, the product set determined to have a larger evaluation value based on the comparison result may be deleted from the RAM 103.

[0177] In this embodiment, a genetic algorithm is used to search for combinations of components and design elements to be used in a commercial product, but the search method is not limited to this, and other search methods such as a local search method or a Tabu search method may also be used.

[0178] As described above, according to this embodiment, by searching for a combination of components and design elements to be used in the products, a product set with a design that is close to the target impression and has a unified feel can be generated for multiple products. This is particularly effective when generating a product set according to images and text information input by a user. For example, consider a case in which an image has a dynamic impression, but a product set with a calm impression as a whole is to be generated. In this embodiment, a combination of skeletons, color patterns, fonts, logos, designs, and image correction parameters that are close to the target impression and have high design similarity and image similarity can be searched for by evaluating the impression of the entire product. Therefore, in order to suppress the impression of the image, the components of the product can be controlled according to the image, such as using a skeleton with a small image area, using a more calm font or color scheme, or suppressing saturation with image correction parameters. Furthermore, even if the impression intended by the user differs from the impression of the brand information, a skeleton with a small area in which the brand information is placed can be used. According to this embodiment, it is possible to flexibly find a combination of components that is optimal for the overall impression of the product set and design elements that create a unified look, and to create a variety of product sets that have a unified look and express the brand impression intended by the user.

[0179] [Third embodiment] In the first and second embodiments, examples of generating a commercial material based on the setting values ​​set on the start screen 501 are described. In the third embodiment, an example of generating a new commercial material based on saved commercial material data is described. A commercial material is not created once and then finished; commercial materials for seasonal events and occasions are repeatedly created. In this case, in order to increase brand recognition, it is necessary to repeatedly create the same design and impression in the commercial material, rather than using a different design each time. In this embodiment, by reflecting the impression information and brand information of the previously created commercial material on the start screen, a commercial material that expresses the impression of the brand intended by the user can be generated even after the passage of time.

[0180] Fig. 23 is a diagram showing an example of an application start screen 2301 provided by a multiple product creation application in the third embodiment. The application start screen 2301 is displayed in the process of S901 in Fig. 9. The components with the same numbers as those in Fig. 5 perform the same processes as those described in the first embodiment, and therefore the description thereof will be omitted here.

[0181] The "reflect saved product" button 2302 can reflect the product data saved by the save button 605 on the preview screen 601 in Fig. 6 in the settings of the application launch screen 2301. Fig. 24 is a flowchart when the "reflect saved product" button 2302 is pressed.

[0182] In S2401, the multiple product creation application displays a UI (not shown) that allows the user to select previously saved product data, and accepts user selection of product data to be reflected. The selected product data is saved in the RAM 103.

[0183] In S2402, the multiple product creation application acquires an impression setting value from the selected product data and reflects it in the settings of the application launch screen 2301. Specifically, for example, a target impression is acquired from the selected product data and reflected in impression slider bars 508 to 511 and impression valid radio button 512 of the application launch screen 2301. When reflecting, if there is no setting for valid impression in the application launch screen 2301, impressions with a large absolute value of the impression are made valid, and impressions with a small absolute value are made invalid. This is because it can be presumed that an impression with a large absolute value of the impression is an impression that represents a brand. Specifically, for example, an impression with an absolute value greater than 1 is made valid. Also, impressions may be compared with each other, and the top two with the largest absolute values ​​of the impression may be made valid. This allows the user to set impression values ​​that repeatedly result in the same impression without being particularly conscious of it.

[0184] In S2403, the multiple product creation application acquires brand information from the selected product data and reflects it in the settings of the application startup screen 2301. Specifically, for example, the brand information is acquired from the selected product data and reflected in the key color designation box 515, the picture designation box 516, the logo designation box 517, the font designation box 518 and the reflection degree slider bar 520 of the startup screen 2301. This allows the user to set impression values ​​that will repeatedly become the same design elements without being particularly conscious of it.

[0185] After the impression information and brand information of the previously created product are reflected, the user can generate multiple products by inputting content and selecting products and categories. The method of generating multiple products is the same as in the first and second embodiments.

[0186] This allows the user to repeatedly generate products with the same impression and design elements without being particularly aware of it.Furthermore, by inputting the content to be placed, it is possible to generate products that are optimal for the input content, the impression of the previously created product, and the brand information.

[0187] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0188] The disclosure of the present embodiment includes the following information processing device, method, and program. (Item 1) A first reception means for receiving content of a commodity; a second means for receiving the brand design; A third reception means for receiving a designation of an impression that the product gives to the user; A fourth reception means for receiving conditions of the product; a first generating means for generating a plurality of product data corresponding to a first product type based on the content received by the first receiving means, the brand design received by the second receiving means, the impression designation received by the third receiving means, and the product conditions received by the fourth receiving means; a second generating means for generating a plurality of product data corresponding to a second product type based on the content received by the first receiving means, the brand design received by the second receiving means, the impression designation received by the third receiving means, and the product conditions received by the fourth receiving means; a display means for displaying a commercial material image based on first commercial material data included in the plurality of commercial material data generated by the first generating means and second commercial material data included in the plurality of commercial material data generated by the second generating means; Equipped with The first product data and the second product data displayed by the display means are product data of a combination having the highest similarity in content and the highest similarity in brand design among a plurality of combinations obtained from the plurality of product data generated by the first generation means and the plurality of product data generated by the second generation means. 23. An information processing apparatus comprising: (Item 2) 2. The information processing device according to item 1, further comprising a first acquisition means for acquiring a similarity of the content for each of the plurality of combinations. (Item 3) The information processing device described in item 2, characterized in that the first acquisition means acquires, as the similarity of the content, a similarity between a color contained in an image as the content of the first product data and a color contained in an image as the content of the second product data. (Item 4) 4. The information processing device according to item 2 or 3, further comprising a second acquisition means for acquiring the similarity of the brand design for each of the plurality of combinations. (Item 5) The information processing device according to item 4, wherein the second acquisition means acquires a similarity between a color scheme represented by the first product data and a color scheme represented by the second product data as the similarity of the brand design. (Item 6) The information processing device described in item 4 or 5, characterized in that the second acquisition means acquires, as the similarity of the brand design, the similarity between the background pattern represented by the first product data and the background pattern represented by the second product data. (Item 7) The information processing device described in any one of items 4 to 6, characterized in that the second acquisition means acquires the similarity between the logo represented by the first product data and the logo represented by the second product data as the similarity of the brand design. (Item 8) The information processing device described in any one of items 4 to 7, characterized in that the second acquisition means acquires, as the similarity of the brand design, a similarity between a font specified in text as the content of the first product data and a font specified in text as the content of the second product data. (Item 9) a selection unit that selects a combination to be displayed by the display unit based on the similarity of the content acquired by the first acquisition unit and the similarity of the brand design acquired by the second acquisition unit, The display means displays the first product data and the second product data of the combination selected by the selection means. 9. The information processing device according to any one of items 4 to 8, (Item 10) A first estimation means for estimating an impression of each of the plurality of product data generated by the first generation means; A second estimation means for estimating an impression of each of the plurality of product data generated by the second generation means; Equipped with The selection means selects a combination to be displayed by the display means based on the impression estimated for each of the plurality of product data generated by the first generation means and the impression estimated for each of the plurality of product data generated by the second generation means. 10. The information processing device according to item 9, (Item 11) A fifth reception means for receiving a reflection degree of the brand design in the display by the display means, The selection means selects a combination to be displayed by the display means based on the reflection degree accepted by the fifth acceptance means. Item 11. An information processing device according to item 10. (Item 12) A second acquisition means for acquiring a plurality of product components corresponding to the first product type based on the designation of the impression accepted by the third acceptance means; a third acquisition means for acquiring a plurality of product components corresponding to the second product type based on the designation of the impression accepted by the third acceptance means, The first generation means generates the plurality of product data corresponding to the plurality of product components acquired by the second acquisition means, The second generation means generates the plurality of product data corresponding to the plurality of product components acquired by the third acquisition means. 12. The information processing device according to any one of items 1 to 11, (Item 13) The product further includes a storage means for storing a table in which a plurality of the product components are defined, Item 13. The information processing device according to item 12, wherein in the table, each of the plurality of product components is associated with an impression. (Item 14) The second acquisition means acquires from the table a plurality of product components close to the designated impression accepted by the third acceptance means, The third acquisition means acquires from the table a plurality of product components close to the designated impression accepted by the third acceptance means. 14. The information processing device according to item 13, (Item 15) The information processing device described in item 12, characterized in that the acquisition of the multiple product components corresponding to the first product type by the second acquisition means, and the acquisition of the multiple product components corresponding to the second product type by the third acquisition means are performed using a genetic algorithm. (Item 16) 16. The information processing device according to any one of items 12 to 15, wherein the product components include a skeleton for determining an arrangement of the content. (Item 17) 17. The information processing device according to any one of items 12 to 16, wherein the product component includes a color pattern. (Item 18) 18. The information processing device according to item 17, wherein the color scheme pattern is a combination of multiple colors. (Item 19) 19. The information processing device according to any one of items 12 to 18, wherein the product components include a font pattern. (Item 20) 20. The information processing device according to item 19, wherein the font pattern is a combination of a title font and a body font. (Item 21) The product component includes a parameter used for image correction, The image correction is performed in the generation of a plurality of product data corresponding to the first product type by the first generation means and in the generation of a plurality of product data corresponding to the second product type by the second generation means. 21. The information processing device according to any one of items 12 to 20, (Item 22) 22. The information processing device according to item 21, wherein the image correction includes at least one of a brightness or saturation correction, a hue correction, and an edge amount correction. (Item 23) 23. An information processing device according to any one of items 1 to 22, characterized in that a first product image represented by the first product data and a second product image represented by the second product data are displayed so as to be selectable by a user. (Item 24) The information processing device described in item 23, further comprising a storage means for storing product data of a product image selected by the user from the first product image and the second product image, together with information regarding the impression of the selected product data and information regarding the brand design. (Item 25) Item 25. The information processing device described in item 24, further comprising a second display means for displaying a setting screen including the first receiving means, the second receiving means, the third receiving means, and the fourth receiving means based on the information stored by the storage means. (Item 26) 26. The information processing apparatus according to claim 1, wherein the conditions for the product include a type of the product. (Item 27) 27. The information processing device according to any one of items 1 to 26, wherein the commercial materials include at least one of posters, banners, and postcards. (Item 28) A method executed in an information processing device, comprising: a first reception step for receiving content for a commodity; A second reception process for receiving brand designs; a third reception Step of receiving a designation of the impression that the product gives to the user; A fourth receiving step of receiving conditions of the product; a first generation step of generating a plurality of product data corresponding to a first product type based on the content received in the first reception step, the brand design received in the second reception step, the impression designation received in the third reception step, and the product conditions received in the fourth reception step; a second generation step of generating a plurality of product data corresponding to a second product type based on the content received in the first reception step, the brand design received in the second reception step, the impression designation received in the third reception step, and the product conditions received in the fourth reception step; a display step of displaying a commercial material image based on first commercial material data included in the plurality of commercial material data generated in the first generation step and second commercial material data included in the plurality of commercial material data generated in the second generation step; having The first product data and the second product data displayed in the display process are product data of a combination having the highest similarity in the content and the highest similarity in the brand design among a plurality of combinations obtained from the plurality of product data generated in the first generation process and the plurality of product data generated in the second generation process. A method comprising: (Item 29) 28. A program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 27.

[0189] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0190] 100 Information processing device: 101 CPU: 102 ROM: 103 RAM

Claims

1. A first reception means for receiving content of a product; A second reception means for receiving a brand design; A third reception means for receiving a designation of an impression that the product gives to a user; A fourth reception means for receiving conditions of the product; a first generating means for generating a plurality of product data corresponding to a first product type based on the content received by the first receiving means, the brand design received by the second receiving means, the impression designation received by the third receiving means, and the product conditions received by the fourth receiving means; a second generating means for generating a plurality of product data corresponding to a second product type based on the content received by the first receiving means, the brand design received by the second receiving means, the impression designation received by the third receiving means, and the product conditions received by the fourth receiving means; a display means for displaying a commercial material image based on first commercial material data included in the plurality of commercial material data generated by the first generating means and second commercial material data included in the plurality of commercial material data generated by the second generating means; Equipped with the first product data and the second product data displayed by the display means are product data of a combination having the highest similarity in the content and the highest similarity in the brand design among a plurality of combinations obtained from the plurality of product data generated by the first generation means and the plurality of product data generated by the second generation means; 23. An information processing apparatus comprising:

2. 2 . The information processing apparatus according to claim 1 , further comprising a first acquisition unit that acquires a similarity of the content for each of the plurality of combinations.

3. The information processing device according to claim 2, characterized in that the first acquisition means acquires, as the similarity of the content, a similarity between a color contained in an image as the content of the first product data and a color contained in an image as the content of the second product data.

4. The information processing apparatus according to claim 2 , further comprising a second acquisition means for acquiring a similarity of the brand design for each of the plurality of combinations.

5. The information processing device according to claim 4 , wherein the second acquisition means acquires, as the similarity of the brand design, a similarity between a color scheme represented by the first product data and a color scheme represented by the second product data.

6. The information processing device according to claim 4, characterized in that the second acquisition means acquires, as the similarity of the brand design, a similarity between a background pattern represented by the first product data and a background pattern represented by the second product data.

7. 5 . The information processing apparatus according to claim 4 , wherein the second acquisition means acquires, as the similarity of the brand design, a similarity between a logo represented by the first product data and a logo represented by the second product data.

8. The information processing device according to claim 4, characterized in that the second acquisition means acquires, as the similarity of the brand design, a similarity between a font specified in text as the content of the first product data and a font specified in text as the content of the second product data.

9. a selection unit that selects a combination to be displayed by the display unit based on the similarity of the content acquired by the first acquisition unit and the similarity of the brand design acquired by the second acquisition unit, The display means displays the first product data and the second product data of the combination selected by the selection means.

5. The information processing apparatus according to claim 4.

10. A first estimation means for estimating an impression of each of the plurality of pieces of product data generated by the first generation means; A second estimation means for estimating an impression of each of the plurality of pieces of product data generated by the second generation means; Equipped with The selection means selects a combination to be displayed by the display means based on the impression estimated for each of the plurality of product data generated by the first generation means and the impression estimated for each of the plurality of product data generated by the second generation means.

10. The information processing apparatus according to claim 9,

11. A fifth reception means for receiving a reflection degree of the brand design in the display by the display means, The selection means selects a combination to be displayed by the display means based on the reflection degree accepted by the fifth acceptance means. The information processing device according to claim 10.

12. A second acquisition means for acquiring a plurality of product components corresponding to the first product type based on the designation of the impression accepted by the third acceptance means; a third acquisition means for acquiring a plurality of product components corresponding to the second product type based on the designation of the impression accepted by the third acceptance means, The first generation means generates the plurality of product data corresponding to the plurality of product components acquired by the second acquisition means, The second generation means generates the plurality of product data corresponding to the plurality of product components acquired by the third acquisition means.

2. The information processing apparatus according to claim 1,

13. The product further includes a storage means for storing a table in which a plurality of the product components are defined, The information processing apparatus according to claim 12 , wherein in the table, each of the plurality of product components is associated with an impression.

14. The second acquisition means acquires from the table a plurality of product components close to the designated impression accepted by the third acceptance means, The third acquisition means acquires from the table a plurality of product components close to the designated impression accepted by the third acceptance means.

14. The information processing apparatus according to claim 13,

15. The information processing device according to claim 12, characterized in that the acquisition of the plurality of product components corresponding to the first product type by the second acquisition means, and the acquisition of the plurality of product components corresponding to the second product type by the third acquisition means are performed using a genetic algorithm.

16. The information processing apparatus according to claim 12 , wherein the product components include a skeleton for determining a layout of the content.

17. The information processing device according to claim 12 , wherein the commercial material components include a color pattern.

18. 18. The information processing apparatus according to claim 17, wherein the color scheme pattern is a combination of a plurality of colors.

19. The information processing apparatus according to claim 12 , wherein the commercial material components include a font pattern.

20. 20. The information processing apparatus according to claim 19, wherein the font pattern is a combination of a title font and a body font.

21. The product component includes a parameter used for image correction, the image correction is performed in the generation of a plurality of pieces of product data corresponding to the first product type by the first generation means and in the generation of a plurality of pieces of product data corresponding to the second product type by the second generation means; 13. The information processing apparatus according to claim 12.

22. 22. The information processing apparatus according to claim 21, wherein the image correction includes at least one of a brightness or chroma correction, a hue correction, and an edge amount correction.

23. The information processing apparatus according to claim 1 , wherein the first product image represented by the first product data and the second product image represented by the second product data are displayed so as to be selectable by a user.

24. The information processing device according to claim 23, further comprising a storage means for storing product data of a product image selected by the user from the first product image and the second product image, together with information regarding an impression of the selected product data and information regarding a brand design.

25. 25. The information processing device according to claim 24, further comprising a second display means for displaying a setting screen including the first reception means, the second reception means, the third reception means, and the fourth reception means, based on the information stored by the storage means.

26. The information processing apparatus according to claim 1 , wherein the conditions for the product include a type of the product.

27. 2. The information processing apparatus according to claim 1, wherein the commercial material includes at least one of a poster, a banner, and a postcard.

28. A method executed in an information processing device, comprising: A first reception step of receiving content of a commodity; a second reception step for receiving a brand design; A third reception step of receiving a designation of an impression that the product gives to the user; A fourth reception step of receiving conditions of the product; a first generation step of generating a plurality of product data corresponding to a first product type based on the content received in the first reception step, the brand design received in the second reception step, the impression designation received in the third reception step, and the product conditions received in the fourth reception step; a second generation step of generating a plurality of product data corresponding to a second product type based on the content received in the first reception step, the brand design received in the second reception step, the impression designation received in the third reception step, and the product conditions received in the fourth reception step; a display step of displaying a commercial material image based on first commercial material data included in the plurality of commercial material data generated in the first generation step and second commercial material data included in the plurality of commercial material data generated in the second generation step; having the first product data and the second product data displayed in the display step are product data of a combination having the highest similarity in the content and the highest similarity in the brand design among a plurality of combinations obtained from the plurality of product data generated in the first generation step and the plurality of product data generated in the second generation step; A method comprising:

29. 28. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 27.

Images