Information processing apparatus, information processing method, and program

Abstract

To provide an information processing apparatus for generating data of a product such as a poster, which allows even a user who does not know a content to be used to easily obtain a product with an intended design.SOLUTION: According to an embodiment of the present disclosure, there is provided an information processing apparatus that generates data of a product, the information processing apparatus including a receiving unit that receives designation of a desired impression from a user, the desired impression being an impression that the product is required to finally hold; A first content selected by the selection means when the reception means receives designation of a first desired impression is different from a second content selected by the selection means when the reception means receives designation of a second desired impression different from the first desired impression.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] One method for creating poster design data using an information processing device such as a PC or smartphone is to use a template in which the shapes and arrangements of images, text, graphics, etc. to be placed on the poster are predetermined. Patent Document 1 also discloses a program that, when a user specifies the impression of the poster (target impression) and the images and text to be placed on the poster (hereinafter, both are also referred to as "content"), automatically generates poster data in which that content is arranged. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-004399 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, the user can freely specify the target impression of the poster and the content to be used on the poster. Therefore, it was difficult for users with poor design skills to determine what specifications they needed to make to achieve the intended design. In addition, the user had to clearly specify the content they wanted to use in advance.

[0005] The present disclosure aims to enable an information processing device that generates data for a product such as a poster to easily obtain a product with an intended design even for a user who is not clear about the content they want to use. [Means for solving the problem]

[0006] The present disclosure provides an information processing device that generates data for a work, comprising: a receiving means that receives from a user a designation of a target impression, which is the impression that the work is ultimately required to retain; and a selection means that selects content to be placed in the work from a content group based on the target impression received by the receiving means, wherein the first content selected by the selection means when the receiving means receives a designation of a first target impression is different from the second content selected by the selection means when the receiving means receives a designation of a second target impression that is different from the first target impression. [Effects of the Invention]

[0007] According to the present disclosure, even users who are not clear about the content they want to use can easily obtain a product with the design they intend. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of the poster generation device. [Figure 2] FIG. 1 is a software block diagram of a poster creation application. [Figure 3] FIG. 2 is a software block diagram of a content selection unit. [Figure 4] FIG. 1 is a diagram illustrating a skeleton. [Figure 5] FIG. 10 is a diagram illustrating a color scheme pattern. [Figure 6] FIG. 10 is a diagram showing a mode selection screen provided by a poster creation application. [Figure 7] FIG. 10 is a diagram showing a generation condition setting screen provided by a poster creation application. [Figure 8] FIG. 10 is a diagram showing an image designation screen provided by a poster creation application. [Figure 9] FIG. 10 is a diagram showing a preview screen provided by a poster creation application. [Figure 10] 10 is a flowchart showing a process for quantifying a poster impression. [Figure 11] FIG. 10 is a diagram illustrating a subjective evaluation of a poster. [Figure 12] 10 is a flowchart showing a content impression quantification process. [Figure 13] 10A and 10B are diagrams comparing differences in poster generation results depending on operation modes. [Figure 14] 10 is a flowchart showing an operation mode switching process. [Figure 15] 10 is a flowchart showing a poster generation process in impression-priority mode. [Figure 16] 10 is a flowchart showing a content selection process. [Figure 17] FIG. 10 is a diagram illustrating a skeleton selection method. [Figure 18] 10A and 10B are diagrams illustrating a method for selecting a color scheme pattern and a font. [Figure 19] FIG. 2 is a software block diagram illustrating the layout unit in detail. [Figure 20] 10 is a flowchart showing a layout process. [Figure 21] FIG. 10 is a diagram for explaining input to a layout section. [Figure 22] FIG. 10 is a diagram illustrating the operation of a layout unit. [Figure 23] 10 is a flowchart showing a poster generation process in a content priority mode. [Figure 24] FIG. 10 is a diagram showing a modified example of a UI for setting a target impression. [Figure 25] FIG. 10 is a diagram showing a modified example of a UI for setting a target impression. [Figure 26] FIG. 10 is a software block diagram of a poster creation application according to a second embodiment. [Figure 27] 10 is a flowchart showing a poster generation process in impression-priority mode according to the second embodiment. [Figure 28] 10 is a data table used by the combination generation unit. [Figure 29] FIG. 10 is a diagram illustrating the processing steps of a combination generation unit. [Figure 30]FIG. 10 is a software block diagram of a poster creation application according to a third embodiment. [Figure 31] FIG. 10 is a diagram showing a content selection screen provided by a poster creation application. [Figure 32] 11 is a flowchart showing a poster generation process according to a third embodiment. [Figure 33] 11 is a flowchart showing a content selection process according to the third embodiment. [Figure 34] FIG. 10 is a software block diagram of a poster creation application according to a fourth embodiment. [Figure 35] FIG. 10 is a diagram showing a generation condition setting screen provided by a poster creation application. [Figure 36] 13 is a flowchart showing a poster generation process according to the fourth embodiment. [Figure 37] FIG. 2 is a software block diagram illustrating the content extraction unit in detail. [Figure 38] 10 is a flowchart showing a content extraction process. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claims, and not all of the combinations of features described in the embodiments are necessarily essential to the present disclosure. Note that the same components are given the same reference numerals and descriptions thereof will be omitted.

[0010] <<First Embodiment>> In the following embodiments, a method for automatically generating designed poster data by running a poster creation application (hereinafter also referred to as "app") on a poster generation device will be described as an example. Conventional poster creation applications allow users to freely specify the target impression of the poster and the content to be placed on the poster. Therefore, even if the desired content is not clear, the user must specify the content. In such a situation, if content that does not match the specified target impression is specified, the poster generated by the poster creation application will also be output as a result that does not match the target impression. Therefore, the user must repeatedly search for content to specify before obtaining a poster with the intended design. Therefore, the poster creation application of the first embodiment selects content that is suitable for the target impression specified by the user from a content group. This makes it easier to generate a poster in which content suitable for the target impression is placed.

[0011] In the following description, unless otherwise specified, the term "image" includes still images and frame images extracted from videos. In the following embodiments, posters are described as examples of creative works, but the creative works are not limited to posters. The image can be used for any creative work that includes at least one of image content and text content, such as flyers, menus, banners, calendars, photo collages, certificates, business cards, shop cards, postcards, invitations, membership cards, and more. These creative works can be used not only in print but also as electronic content on websites, social networking sites, virtual spaces, and the like.

[0012] <System configuration> 1 is a block diagram showing the hardware configuration of a poster generation device. The poster generation device 100 is an information processing device, such as a personal computer (hereinafter referred to as a PC) or a smartphone. In this embodiment, the poster generation device is described as a PC. The poster generation device 100 includes a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display 105, a keyboard 106, a pointing device 107, a data communication unit 108, and a GPU 109.

[0013] A CPU (Central Processing Unit / Processor) 101 comprehensively controls the poster generation device 100, and realizes the operation of this embodiment by, for example, reading a program stored in a ROM 102 into a RAM 103 and executing it. Although FIG. 1 shows one CPU, the device may be configured with multiple CPUs.

[0014] The ROM 102 is a general-purpose ROM and stores, for example, a program executed by the CPU 101. The RAM 103 is a general-purpose RAM and is used, for example, as a working memory for temporarily storing various pieces of information when the CPU 101 executes a program.

[0015] The HDD (hard disk) 104 is a storage medium (storage unit) for storing image files, a database for holding processing results such as image analysis, and skeletons and the like used by the poster creation application.

[0016] The display 105 is a display unit that displays the user interface (UI) of this embodiment and an electronic poster as a layout result of image data (hereinafter also referred to as "image") to the user. The keyboard 106 and pointing device 107 accept instructions and operations from the user. The display 105 may also have a touch sensor function.

[0017] The keyboard 106 is used, for example, when the user inputs the number of double-page spreads of the poster that he or she wants to create on the UI displayed on the display 105 .

[0018] The pointing device 107 is used, for example, when the user clicks a button on the UI displayed on the display 105 .

[0019] The data communication unit 108 communicates with external devices via a wired or wireless network, etc. For example, the data communication unit 108 transmits data laid out by the automatic layout function to a printer or server that can communicate with the poster generation device 100.

[0020] The GPU 109 is a processor that performs image processing in response to instructions from the CPU 101. The GPU 109 performs, for example, analysis of the image to be placed on the poster, estimation of the impression of the image or text, estimation of the impression of the poster, layout and color arrangement of the image, text, etc. for the skeleton, and generates a poster image.

[0021] The data bus 110 connects the blocks in FIG. 1 so that they can communicate with each other. Note that the configuration shown in FIG. 1 is merely an example and is not limited to this. For example, the poster generation device 100 may not have the display 105 and may display the UI on an external display.

[0022] The poster creation application in this embodiment is stored in HDD 104. The poster creation application is started by the user performing an operation such as clicking or double-clicking the application icon displayed on display 105 with pointing device 107.

[0023] <Software block diagram> FIG. 2 is an example software block diagram of a poster creation application. The poster creation application includes a poster creation condition specification unit 201, a text specification unit 202, an image specification unit 203, a target impression specification unit 204, a poster display unit 205, a poster generation unit 210, and an operation mode specification unit 230. The poster generation unit 210 includes an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a skeleton selection unit 214, a color scheme selection unit 215, a font selection unit 216, a layout unit 217, a poster impression estimation unit 218, a poster selection unit 219, a content selection unit 220, an image impression estimation unit 221, and a UI modification unit 222. FIG. 2 also shows a software block diagram particularly related to the poster generation unit 210, which executes the automatic poster creation function.

[0024] When the poster creation application is installed in poster generation device 100, a startup icon is displayed on display 105 on the top screen (desktop) of the OS (operating system) running on poster generation device 100. When the startup icon is operated (e.g., double-clicked) with pointing device 107, the program of the poster creation application stored in HDD 104 is loaded into RAM 103 and executed by CPU 101. This starts the poster creation application.

[0025] The poster creation application described above includes program modules corresponding to the components shown in Fig. 2. CPU 101 executes each program module, causing CPU 101 to function as each component shown in Fig. 2. Hereinafter, each component shown in Fig. 2 will be described as executing various processes.

[0026] The operation mode designation unit 230 designates an operation mode in response to a UI operation performed by the pointing device 107. In this embodiment, there are two operation modes: an impression-priority mode and a content-priority mode. Details of each operation mode will be described later. The operation mode designation unit 230 outputs the designated operation mode to the poster generation unit 210.

[0027] The poster creation condition specification unit 201 specifies poster creation conditions to the poster generation unit 210 in response to UI operations using the pointing device 107. In this embodiment, the poster creation conditions specified include the poster size, number of posters to be created, number of images, and purpose category. The poster size may be specified as actual dimensions of width and height, or a paper size such as A1 or A2. The number of images is the number of images to be arranged on the poster. The purpose category is a category that indicates the purpose of the poster, such as restaurant, school event, sale, or awareness campaign. The creation conditions specified by the poster creation condition specification unit 201 are input to the skeleton acquisition unit 213, skeleton selection unit 214, color scheme pattern selection unit 215, font selection unit 216, poster selection unit 219, and content selection unit 220.

[0028] The text designation unit 202 accepts designation by the user of one or more pieces of character information that are candidates for character information to be placed on a poster through UI operations using the keyboard 106. The character information to be placed on a poster represents, for example, a character string that indicates a title, date and time, location, etc. The text designation unit 202 also outputs the candidate character information to the content selection unit 220 after linking each piece of character information with information (tag or attribute information) that indicates the type of character information, such as whether it is a title, or information indicating a date and time or location.

[0029] The image designation unit 203 accepts user designation of one or more image data (hereinafter also referred to as an image group) that are candidates for images to be arranged on a poster. For example, if the image data is stored on the HDD 104, the image group can be designated based on the structure of a file system containing the image data, such as a device and a directory. The image group may also be designated based on attribute information or accompanying information for identifying the images, such as the shooting date and time. The image designation unit 203 may also designate image data included in the poster creation application and provided as materials (hereinafter also referred to as "application material images") as the image group. The image designation unit 203 may also designate image data held by an external image providing service linked to the poster creation application (hereinafter also referred to as "linked material images") as the image group. The image designation unit 203 may also designate image data generated by an image generation AI as the image group. A generation AI is a machine learning model that generates new data based on learned data, and an image generation AI is a generation AI that generates images. Specifically, image generation AI is AI that can generate images from images or text using a diffusion model, a GAN model, etc. The image designation unit 203 outputs the file paths of the designated image group to the image acquisition unit 211.

[0030] The target impression designation unit 204 accepts designation by the user of a target impression for the poster to be created. The target impression is the impression that the poster to be created is ultimately required to retain, and is set so as to be given to people who view the created poster (production). In this embodiment, the intensity indicating the degree to which the impression is to be given to a word expressing an impression is designated by UI operation using the pointing device 107. Information indicating the target impression designated by the target impression designation unit 204 is shared by the content selection unit 220, skeleton selection unit 214, color scheme pattern selection unit 215, font selection unit 216, and poster selection unit 219. Details of impressions will be described later.

[0031] Poster generation unit 210 executes a poster generation process according to the operation mode designated by operation mode designation unit 230. The operation mode will be described later.

[0032] Next, the software configuration of the poster generator 210 will be described in detail.

[0033] The image acquisition unit 211 acquires a group of images specified by the user in the image designation unit 203 from a specified acquisition source. The image acquisition unit 211 outputs image data included in the acquired group of images to the content selection unit 220 and the image impression estimation unit 221. Image acquisition sources include the HDD 104, a storage area on a network, a PC on which the image generation AI runs, and the like. Examples of images to be acquired include still images, frame images extracted from videos, material images created in advance for this application (hereinafter referred to as application material images), material images provided by an image providing service (hereinafter referred to as linked material images), and images generated by the generation AI. The still images and frame images are acquired from an imaging device such as a digital camera or a smart device. The imaging device may be included in the poster generation device 100 or an external device. If the imaging device is an external device, the images are acquired via the data communication unit 108. As another example, the still images may be illustration images created using image editing software or CG images created using CG production software. The still images and cropped images may be images acquired from a network or a server via the data communication unit 108. Examples of images acquired from a network or a server include social networking service images (hereinafter referred to as "SNS images"), linked material images, and images generated using an externally provided image generation AI. The program executed by the CPU 101 analyzes data attached to each image to determine the storage source. For example, SNS images may be acquired from an SNS via an application, and the acquisition source may be managed within the application. Note that the images are not limited to the above-described images, and other types of images may also be used.

[0034] The content selection unit 220 acquires the number of images from the poster creation condition designation unit 201, one or more candidate character information from the text designation unit 202, image data included in the image group from the image acquisition unit 211, and the designated target impression from the target impression designation unit 204. The content selection unit 220 selects image data to be used for the poster from the image data included in the image group based on the target impression and the number of images. The content selection unit 220 also selects character information to be used for the poster from the candidate character information based on the target impression. In this embodiment, the content selection unit 220 selects character information to be used as a title. The content selection unit 220 outputs the selected image data to the image analysis unit 212 or the layout unit 217, and outputs the selected character information to the skeleton acquisition unit 213 and the layout unit 217.

[0035] 3 is a software block diagram of the content selection unit 220. As shown in FIG. 3, the content selection unit 220 includes a determination unit 301, a content impression estimation unit 302, a content evaluation unit 303, and a content selection unit 304.

[0036] The determination unit 301 determines the number of contents to be used in the poster specified by the user. If the content is text, the determination unit 301 determines whether or not there is a plurality of pieces of character information linked to the title among the character information acquired from the text designation unit 202. If there is a plurality of pieces of character information linked to the title, the determination unit 301 outputs the acquired character information linked to the title to the content impression estimation unit 302. If the content is images, the determination unit 301 determines whether or not the number of images designated in the poster creation condition designation unit 201 is 0. If the number of images is not 0, the determination unit 301 further determines whether or not the number of image data acquired by the image acquisition unit 211 is greater than the number of images designated in the poster creation condition designation unit 201. If the number of image data acquired by the image acquisition unit 211 is greater, the determination unit 301 outputs the image data acquired by the image acquisition unit 211 to the content impression estimation unit 302.

[0037] The content impression estimation unit 302 estimates impressions of a plurality of pieces of content. The estimation of impressions of content will be described later.

[0038] The content evaluation unit 303 calculates the distance between the target impression specified by the user and the impression of the content estimated by the content impression estimation unit 302. In this embodiment, the Euclidean distance is used as the distance (hereinafter, the simple distance will be referred to as the Euclidean distance). The smaller the value indicated by the Euclidean distance, the closer the target impression and the impression of the content are.

[0039] The content selection unit 304 selects the top N pieces of content for each title and image that have the smallest distance values ​​calculated by the content evaluation unit 303. In this embodiment, the content selection unit 304 selects the top title and the number of image data pieces for the number of images specified by the poster creation condition specification unit 201 in order of smallest distance. Here, the number of selections N may be a fixed value or may be variable depending on the conditions specified by the poster creation condition specification unit 201. The number of selected images may be greater than the number of images specified by the poster creation condition specification unit 201. If the minimum value of the distance calculated by the content evaluation unit 303 is greater than a predetermined threshold, the content selection unit 304 may display a warning screen on the display 105 indicating that there are no content candidates that have an impression similar to the target impression.

[0040] Returning to the explanation of Fig. 2, the image analysis unit 212 performs an image data analysis process on one or more pieces of image data acquired from the content selection unit 220, and acquires information indicating image feature amounts. Specifically, the image analysis unit 212 performs an object recognition process, which will be described later, and acquires information indicating image feature amounts of the image data. The image analysis unit 212 also links the acquired information indicating the image feature amounts to the image data and outputs the information to the layout unit 217.

[0041] The skeleton acquisition unit 213 acquires one or more skeletons that meet the conditions specified by the poster creation condition specification unit 201 and the content selection unit 220 from the HDD 104. In this embodiment, a skeleton is information that indicates the layout of content (character strings and images) and graphics to be arranged on a poster.

[0042] FIG. 4 is a diagram illustrating an example of a skeleton. Three graphic objects 402, 403, and 404, one image object 405, and four text objects 406, 407, 408, and 409, in which text is arranged, are arranged on a skeleton 401 in FIG. 4(a). Each object is recorded with metadata necessary for generating a poster, in addition to the position, size, and angle indicating the placement location. FIG. 4(b) is a diagram illustrating an example of metadata. For example, text objects 406 to 409 hold, as metadata attributes, the type of text information to be arranged. Here, text object 406 indicates that a title will be arranged, text object 407 indicates that a subtitle will be arranged, and text objects 407 and 408 indicate that the main text will be arranged. Furthermore, graphic objects 402 to 404 hold, as metadata attributes, color scheme numbers (color scheme IDs) indicating the shape and color scheme pattern of the graphic. Here, the attributes of graphic objects 402 and 403 indicate that they are rectangular, and the attribute of graphic object 404 indicates that they are elliptical. Also, assume that color scheme number 1 is assigned to graphic object 402, and color scheme number 2 is assigned to graphic objects 403 and 404. Here, the color scheme number is information referenced when assigning colors, which will be described later, and indicates that different colors are assigned to different color scheme numbers. Note that the types of objects and metadata are not limited to these. For example, there may be a map object for placing a map, or a barcode object for placing a QR code (registered trademark) or barcode. Furthermore, metadata for character objects may include metadata indicating the line spacing and character spacing. The metadata may include the purpose of the skeleton and be used to control whether or not to use the skeleton depending on the purpose.

[0043] The skeletons may be stored in HDD 104 in, for example, CSV format, or in a DB format such as SQL. Skeleton acquisition unit 213 outputs one or more skeletons acquired from HDD 104 to skeleton selection unit 214.

[0044] The skeleton selection unit 214 selects one or more skeletons that match the desired impression specified by the desired impression specification unit 204 from the skeletons acquired from the skeleton acquisition unit 213, and outputs them to the layout unit 217. Since the layout of the entire poster is determined by the skeleton, preparing various types of skeletons in advance can increase the variety of posters that can be generated.

[0045] Color scheme selection unit 215 obtains from HDD 104 one or more color schemes that match the desired impression specified by desired impression specification unit 204, and outputs them to layout unit 217. A color scheme is a combination of colors to be used in a poster.

[0046] FIG. 5 is a diagram showing an example of a color scheme table. In this embodiment, a color scheme pattern is expressed as a combination of four colors. The color scheme ID column in FIG. 5 is an ID for uniquely identifying a color scheme pattern. The color 1 to color 4 columns represent colors in RGB order, with each RGB color value ranging from 0 to 255 ((R, G, B) = (0 to 255, 0 to 255, 0 to 255)). Note that, although this embodiment uses a color scheme pattern consisting of a combination of four colors, other numbers of colors may be used, or multiple numbers of colors may be mixed.

[0047] Font selection unit 216 selects one or more font patterns that match the desired impression specified by desired impression specification unit 204, obtains them from HDD 104, and outputs them to layout unit 217. A font pattern is a combination of at least one of a title font, a subtitle font, and a body font.

[0048] The layout unit 217 generates one or more poster data items equal to or greater than the specified number of posters to be created by laying out various contents for each of the one or more skeletons obtained from the skeleton selection unit 214. The layout unit 217 arranges, for each skeleton, the text (title) selected by the content selection unit 220, the text (other than the title) obtained from the text specification unit 202, and the image data obtained from the image analysis unit 212 or the content selection unit 220. The layout unit 217 also applies the color pattern obtained from the color pattern selection unit 215 and the font pattern obtained from the font selection unit. The layout unit 217 outputs the generated one or more poster data items to the poster impression estimation unit 218.

[0049] The poster impression estimation unit 218 estimates the impression of each piece of poster data among the multiple poster data acquired from the layout unit 217, and links the estimated impression (estimated impression) to each piece of poster data. Then, the poster impression estimation unit 218 outputs one or more pieces of poster data linked with the estimated impression to the poster selection unit 219.

[0050] The poster selection unit 219 compares the target impression specified by the target impression specification unit 204 with each estimated impression of the multiple poster data associated with an estimated impression obtained from the poster impression estimation unit 218, and selects poster data associated with an estimated impression that is close to the target impression. The poster selection unit 219 selects posters equal to or greater than the number of posters to be created specified by the poster creation condition specification unit 201. At this time, the poster selection unit 219 selects posters equal to or greater than the number of posters to be created in ascending order of the value (e.g., Euclidean distance) indicating the difference between the target impression and the estimated impression. The closeness between the target impression and the estimated impression is determined based on the difference in impression value for each impression factor. The selection result is saved in the HDD 104. The poster selection unit 219 outputs the selected poster data to the poster display unit 205.

[0051] The poster display unit 205 displays a poster image based on the poster data acquired from the poster selection unit 219 on the display 105. The poster image is, for example, bitmap data. Note that the poster generation unit 210 generates poster data for the number of posters specified by the poster creation condition specification unit 201 or more, and therefore, a list of previews of the poster images is displayed on the display 105. When the user clicks on a poster image with the pointing device 107, the poster image is selected.

[0052] The poster creation application may be provided with a function (not shown) that allows the user to edit the layout, color, shape, etc. of images, text, and graphics through additional operations after the generated result is displayed on the poster display unit 205, and further change the design to suit the user's needs. Also, if the application is provided with a function that prints the poster data stored in the HDD 104 using a printer according to the conditions specified in the poster creation condition specification unit 201, the user will be able to obtain a printed copy of the poster they have created.

[0053] The image impression estimation unit 221 and the UI change unit 222 are software blocks used in a content priority mode, which will be described later. The image impression estimation unit 221 estimates an impression of the image data acquired from the image acquisition unit 211, and outputs the estimated impression to the UI change unit 222.

[0054] The UI modification unit 222 determines information for changing the setting conditions for the desired impression and the UI for setting the desired impression, based on the impression of the image data acquired from the image impression estimation unit 221, and outputs the information to the desired impression designation unit 204. In the content priority mode, the desired impression designation unit 204 changes the operable range of the desired impression and the display form of the operation object (UI) on the generation condition setting screen 701, based on the setting conditions for the desired impression acquired from the UI modification unit 222 and the information for changing the UI.

[0055] <Example of display screen> 6 is a diagram showing an example of a mode selection screen 601 provided by the poster creation application. The mode selection screen 601 is displayed on the display 105. The user sets an operation mode, which will be described later, via the mode selection screen 601.

[0056] The content-priority mode button 602 on the mode selection screen 601 is a button for setting the operating mode of the poster creation application to content-priority mode. When the content-priority mode button 602 is pressed, a generation condition setting screen 701 shown in FIG. 7 is displayed on the display 105. Note that, on the generation condition setting screen 701 displayed in content-priority mode, when the user specifies content (e.g., image data) to be placed on the poster, the display state of the UI for setting a target impression based on the impression of that image data is changed and displayed. Details of the content-priority mode will be described later.

[0057] The impression-priority mode button 603 on the mode selection screen 601 is a button for setting the operating mode in the poster creation application to impression-priority mode. When the impression-priority mode button 603 is pressed, a creation condition setting screen 701 shown in FIG. 7 is displayed on the display 105.

[0058] 7 is a diagram showing an example of a generation condition setting screen 701 provided by the poster creation application. The generation condition setting screen 701 is displayed on the display 105. The user specifies the text and images, which are the content to be placed on the poster, the desired impression of the poster to be created, and the poster creation conditions (size, number to be created, number of images, and use category) via the generation condition setting screen 701. The poster creation condition specification unit 201, image specification unit 203, and text specification unit 202 acquire the contents specified by the user via this UI screen.

[0059] The generation condition setting screen 701 is provided with a content input area 724 and a condition setting area 725 .

[0060] The title box 702, subtitle box 703, and main text box 704 in the content input area 724 accept user specification of text information to be placed on the poster. In this embodiment, three types of text information are accepted, but this is not limited to this. For example, additional text information such as location, date and time may also be accepted. Also, text information does not need to be entered in all boxes, and some boxes may be left blank.

[0061] The add title button 718 is a button that is operated when displaying an additional title box 719. For example, this button is operated when there are multiple candidates for the character information of the title and the user wants to specify the character information of multiple titles. Note that in the content priority mode, the add title button 718 cannot be pressed.

[0062] Similar to the title box 702, the additional title box 719 accepts the specification of character information. The user can specify multiple pieces of character information as title candidates by specifying character information in the title box 702 and the title box 719. Note that in this embodiment, the specification of multiple pieces of character information is accepted only for the title, but this is not limited thereto. Add buttons and boxes for addition may also be provided for the subtitle and the main text, and the specification of multiple pieces of character information may also be accepted. Note that the method for specifying multiple pieces of character information is not limited thereto. For example, the text specification unit 202 may divide the character information specified by the user in the title box 702 by line feed characters, and the divided pieces of character information may be specified as multiple title candidates.

[0063] The image designation area 705 in the content input area 724 is an area for designating an image to be placed on the poster. Image 706 represents an image designated by the user or a thumbnail selected by the content selection unit 220. The add image button 707 is a button for adding an image to be placed on the poster. When the add image button 707 is pressed by the user, the image designation unit 203 displays an image designation screen 801 for selecting an image file and accepts the user's designation of a group of images. When a group of images is designated, the content selection unit 220 selects one or more images from the group of images. A thumbnail of the image selected by the content selection unit 220 is displayed in the image designation area 705.

[0064] Here, the image designation screen 801 will be described with reference to Fig. 8. The image designation screen 801 is displayed on the display 105. The user can use the image designation screen 801 to designate an image to be placed on a poster, or to designate an acquisition destination for an image group including multiple images. The image designation unit 203 acquires settings from the user through this UI screen.

[0065] Radio buttons 802 to 807 are buttons for setting the method for specifying candidate image data or image groups. The user can set the method for specifying image data by pressing radio buttons 802 to 807 to set on / off. Although multiple radio buttons are displayed, only one radio button can be set to on. In other words, when an off radio button is set to on, that radio button is set to on, but any radio button that was on before the setting was made is automatically set to off.

[0066] Radio button 802 is a button for setting a method for specifying one or more image data as a method for specifying an image. Specification box 808 accepts the specification of one or more image data. The user can specify candidate image data by specifying the file path of the image data in specification box 808. Browse button 809 is a button for specifying one or more image data. When the user presses Browse button 809, image specification unit 203 displays a dialog screen for selecting a file saved in HDD 104 and accepts the user's selection of an image file.

[0067] Radio button 803 is a button for setting a method for specifying an image by specifying a folder containing one or more image data as the acquisition destination for the image group. Specification box 810 accepts the specification of a folder containing one or more image data. By specifying a folder path in specification box 810, the user can specify all image data contained in that folder as an image group. Browse button 811 is a button for specifying the acquisition destination folder. When the user presses Browse button 811, image specification unit 203 displays a dialog screen for selecting a folder saved in HDD 104 and accepts the folder selection by the user.

[0068] The radio button 804 is a button for setting a method of specifying all application material images as the image specifying method. When the radio button 804 is on, all application material images are specified as candidate images.

[0069] Radio button 805 is a button for setting a method for specifying some application material images as a method for specifying images. Designation box 812 displays the name of the application material image specified via browse button 813. Browse button 813 is a button for specifying one or more application material images. When the user presses browse button 813, image designation unit 203 displays a dialog screen for selecting application material images and accepts the image selection by the user. Note that if tag information is assigned to the application material images, it may be possible to select all application material images with the tag by specifying the tag.

[0070] The radio button 806 is a button for setting a method for specifying a linked material image as a method for specifying an image. The specification box 814 displays the name of the linked material image specified through the reference button 815. The reference button 815 is a button for specifying one or more linked material images. When the reference button 815 is pressed by the user, the image specification unit 203 displays a dialog screen for selecting a linked material image and accepts the image selection by the user. Note that if tag information is assigned to the linked material image, it may be possible to select all linked material images with the tag by specifying the tag.

[0071] Radio button 807 is a button for setting a method for generating an image using an image generation AI as a method for specifying an image. Prompt box 816 accepts specification of a prompt to be used as input for the image generation AI. Then, image specification unit 203 generates an image using the specified prompt and image generation AI, and saves the generated image in HDD 104. In this embodiment, image data is generated from the prompt using stable diffusion. Then, image specification unit 203 specifies the saved AI-generated image.

[0072] If the user has clearly decided which image they want to use for the poster, they turn on one of radio buttons 802, 805, 806, or 807 to specify the source of one or more images to use for the poster. On the other hand, if the user has not clearly decided which image they want to use for the poster, they can turn on radio button 803 or radio button 804 to specify an entire image folder or all in-app materials as the image group. They can also specify one of radio buttons 802, 805, 806, or 807 to specify multiple image candidates as the image group. In any case, the user does not need to decide on a specific image to use for the poster.

[0073] Cancel button 817 is a button for canceling the designation of an image. When cancel button 817 is pressed, each setting information on image designation screen 801 is ignored, and the screen displayed on display 105 transitions to generation condition setting screen 701. When OK button 818 is pressed by the user, the screen displayed on display 105 transitions to generation condition setting screen 701. At this time, a thumbnail of an image selected from the group of images designated on image designation screen 801 by content selection processing, which will be described later, is added to image designation area 705 of generation condition setting screen 701.

[0074] Returning to FIG. 7, impression sliders 708-711 on generation condition setting screen 701 are operation objects (UI) that allow the user to set values ​​indicating the degree of each factor (hereinafter referred to as impression factor) of the target impression of the poster to be created. For example, impression slider 708 is a slider that sets a value indicating the degree of the target impression related to the impression factor "luxury." The target impression is set so that the more to the right the slider is slid, the more luxurious the poster will be, and the more to the left the poster will be slid, the less luxurious (cheaper) it will be. Furthermore, by combining the target impression factors set by each slider, an overall target impression is set that reflects not only the impression factor set by one slider, but also the impression factors set by other sliders.

[0075] For example, if the impression slider 708 corresponding to the impression factor "luxury" is set to the right of the center, and the impression slider 711 corresponding to the impression factor "profoundness" is set to the left of the center, a poster with a high sense of luxury but low sense of profoundness, and an elegant impression, is generated. Alternatively, if the impression slider 708 corresponding to the impression factor "luxury" is set to the right of the center, and the impression slider 711 corresponding to the impression factor "profoundness" is set to the right of the center, a poster with a high sense of luxury and profoundness, and an elegant impression, is generated. In this way, by combining target impression factors indicated by multiple impression sliders, it is possible to set target impressions with different directionalities, such as a "refined" target impression and a "gorgeous" target impression, even if the common target impression factor of "luxury" is set.

[0076] In other words, the target impression is composed of and determined by multiple factors that indicate the impression, but it may also be determined by a single factor that indicates the impression. In this embodiment, the state in which the slider is set to the far left is set to -2, and the state in which it is set to the far right is set to +2, and the value indicating the impression is corrected to a value between -2 and +2. These numerical values ​​indicate that the impression is low (-2), slightly low (-1), neutral (0), slightly high (+1), and high (+2). Note that the purpose of correcting to -2 to +2 is to align the scale with the estimated impression to facilitate distance calculation, which will be described later, but this is not limiting and normalization using values ​​between 0 and 1 may also be used.

[0077] Radio button 712 controls whether to enable or disable the setting of each impression factor. The user can enable or disable the setting of each impression factor by pressing radio button 712 to set it to on or off. For example, by selecting off with radio button 712, that impression factor is excluded from impression control. For example, if the user wants to create a subdued poster with low dynamism but has no other impressions specified, the user can generate a poster that focuses on low dynamism by turning off the radio buttons 712 other than dynamism. Note that FIG. 7 shows a state in which luxury and familiarity are set to on, and dynamism and profoundness are set to off. This allows for highly flexible control, such as whether to use all impression factors in poster generation or only some of them. Note that if the state in which each slider is set to the leftmost position is equivalent to a state in which the corresponding impression factor is not set, radio button 712 may be omitted. In this case, the user can disable the setting of each target impression by setting each slider to the leftmost position.

[0078] The size list box 713 is a list box for setting the size of the poster to be created. By clicking with the pointing device 107, the user can display a list of poster sizes that can be created and select one. The number to create box 714 can be used to set the number of posters to be created. The number of images box 720 can be used to set the number of images to be used in the poster. The category list box 715 can be used to set the purpose category of the poster to be created.

[0079] The reset button 716 is a button for resetting each setting information on the generation condition setting screen 701 .

[0080] When the user presses the OK button 717, the poster creation condition specification unit 201 and the target impression specification unit 204 output the information set on the generation condition setting screen 701 to the poster creation unit 210. At that time, the poster creation condition specification unit 201 obtains the size of the poster to be created from the size list box 713 and the number of posters to be created from the number of creations box 714. It also obtains the number of images to be used from the number of images box 720 and the purpose category of the poster to be created from the category list box 715. The text specification unit 202 obtains the text information entered in the title boxes 702 and 719, the subtitle box 703, and the main text box 704. The image specification unit 203 obtains the file path of the image to be placed on the poster from the image specification area 705. The target impression specification unit 204 obtains the target impression of the poster to be created from the impression sliders 708 to 711 and the radio button 712. The poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 may process the values ​​set on the creation condition setting screen 701. For example, the text specification unit 202 may remove unnecessary blank characters at the beginning or end of the input character information. Furthermore, the target impression specification unit 204 may correct the target impression values ​​specified by the impression sliders 708 to 711.

[0081] Furthermore, if the number of image data items specified in image specification area 705 is smaller than the number specified in number of images box 720, poster creation condition specification unit 201 displays a dialog screen to prompt the user to add images. Alternatively, poster creation condition specification unit 201 may change the number that can be specified in number of images box 720 depending on the number of image data items specified in image specification area 705. For example, if the number of image data items specified in image specification area 705 is 2, poster creation condition specification unit 201 may limit the number that can be specified in number of images box 720 to 0, 1, or 2.

[0082] 9 is a diagram showing an example of a poster preview screen 901 on which a poster image 902 generated by the poster generation unit 210 is displayed on the display 105. When the OK button 717 on the generation condition setting screen 701 is pressed and the poster generation is completed, the screen displayed on the display 105 transitions to the poster preview screen 901.

[0083] Poster image 902 is a poster image output by poster display unit 205. Poster generation unit 210 generates poster data in the number of pieces specified by poster creation condition specification unit 201 or more, and therefore displays a list of poster images 902 in the number corresponding to the generated poster data. When the user clicks on any of the poster images 902 with pointing device 107, the poster data corresponding to that poster image 902 is selected.

[0084] An edit button 903 is a button for transitioning to a function for editing the selected poster data. With the edit function, the poster data can be edited through a UI (not shown).

[0085] A print button 904 is a button for transitioning to a function for printing the selected poster data. With the print function, the poster data can be printed via a control UI of a printer (not shown).

[0086] <Quantifying the impression of posters> Here, we will explain the process of quantifying the impression of a poster (hereinafter referred to as the poster impression quantification process). The poster impression quantification process is a pre-processing required for executing the poster impression estimation process (S1512 in FIG. 15) described later. The poster impression estimation process is executed in the poster generation process (FIGS. 15 and 23) described later.

[0087] The poster impression quantification process is performed by a vendor or the like who develops the poster creation application during the development stage of the poster creation application. The poster impression quantification process may be executed by the poster generation device 100, or may be executed by an information processing device different from the poster generation device 100. When executed by an information processing device different from the poster generation device 100, the process is executed by the CPU of that information processing device.

[0088] The poster impression quantification process quantifies the impressions people have of various posters. At the same time, a correspondence between the poster image and the poster impression is derived. This makes it possible to estimate the impression of the poster from the generated poster image. If the impression can be estimated, it becomes possible to control the impression of the poster by modifying the poster image, or to search for a poster image that has a certain target impression. The poster impression quantification process is performed, for example, by running an impression learning application in a poster generation device to learn the impression of the poster image in advance of the poster generation process.

[0089] Fig. 10 is a flowchart showing the poster impression quantification process. The flowchart shown in Fig. 10 is realized, for example, by CPU 101 reading a program stored in HDD 104 into RAM 103 and executing it. The poster impression quantification process will be described with reference to Fig. 10. Note that the symbol "S" in the description of each process indicates a step in the flowchart (the same applies hereinafter in this specification).

[0090] In S1001, CPU 101 acquires subjective evaluations of impressions of a poster. FIG. 11 illustrates an example of a method for subjectively evaluating impressions of a poster. CPU 101 presents a poster to a subject and acquires subjective evaluations of the impressions of the poster from the subject. Measurement methods such as the Semantic Differential (SD) method and the Likert scale can be used for this. FIG. 11 illustrates an example of a questionnaire using the SD method, in which adjective pairs expressing impressions are presented to multiple evaluators and scores are assigned to the adjective pairs evoked by the target poster. After acquiring subjective evaluation results for multiple posters from multiple subjects, CPU 101 calculates the average of the responses to each adjective pair and sets the average as a representative score for the corresponding adjective pair. Note that a subjective evaluation method other than the SD method may be used as long as words expressing impressions and corresponding scores are determined.

[0091] In S1002, CPU 101 performs a factor analysis of the subjective evaluation results acquired in S1001. If the subjective evaluation results are left as they are, the number of adjective pairs will increase the number of dimensions, making control complex. Therefore, it is desirable to reduce the number of dimensions to an efficient level using an analytical method such as principal component analysis or factor analysis. In this embodiment, the description will be given assuming that the dimensions have been reduced to four factors using factor analysis. Naturally, this number varies depending on the adjective pairs selected for the subjective evaluation and the factor analysis method. Furthermore, the output of the factor analysis is assumed to be standardized. That is, each factor is scaled so that the mean and variance of the poster used in the analysis are 0 and 1, respectively. This allows the impressions of -2, -1, 0, +1, and +2 specified by target impression specification unit 204 to directly correspond to -2σ, -1σ, the mean value, +1σ, and +2σ for each impression, facilitating the calculation of the distance between the target impression and the estimated impression, as described below. In this embodiment, the four factors are luxury, familiarity, dynamism, and solidity as shown in Figure 7, but these are names given for convenience in order to convey impressions to the user through the user interface, and each factor is composed of the interaction of multiple adjective pairs. Also, CPU 101 stores in HDD 104 a conversion formula (hereinafter referred to as "impression conversion formula") from the subjective evaluation results of each adjective pair to each impression value obtained by factor analysis.

[0092] In S1003, the CPU 101 associates poster images with impressions. While it is possible to quantify posters that have been subjectively evaluated using the method described above, it is also necessary to estimate impressions for posters to be created in the future without subjective evaluation. The association between poster images and impressions can be achieved by learning a model that estimates impressions from poster images. Specifically, for example, deep learning methods such as Convolution Neural Network (CNN) or Visual Transformer (ViT), or machine learning methods using decision trees can be used. In this embodiment, the CPU 101 inputs a poster image and outputs four factors to perform supervised deep learning using CNN. That is, a deep learning model is created by learning the subjectively evaluated poster images and the corresponding impressions as correct answers, and an impression is estimated by inputting an unknown poster image into the learning model.

[0093] In S1004, the CPU 101 stores in the HDD 104 the model configuration and learned parameters of the deep learning model for impression estimation created in S1003.

[0094] The poster impression estimation unit 218 loads the deep learning model stored in the HDD 104 into the RAM 103 and executes it. The poster impression estimation unit 218 converts the poster data acquired from the layout unit 217 into an image, and estimates the impression of the poster by running the deep learning model loaded in the RAM 103 on the CPU 101 or the GPU 109. Note that, although a deep learning method is used in this embodiment, this is not limiting. For example, when using a machine learning method such as a decision tree, it is possible to extract feature amounts such as the average brightness value and edge amount of the poster image by image analysis, and create a machine learning model that estimates the impression based on the feature amounts.

[0095] <Quantifying content impressions> Next, with reference to FIG. 12, a process for quantifying the impression of content (hereinafter referred to as content impression quantification process) will be described. The content impression quantification process is a pre-processing for executing the text impression estimation process (S1602 in FIG. 16) or the image impression estimation process (S1607 in FIG. 16). The content impression quantification process is performed by a vendor or the like who develops a poster creation application at the development stage of the poster creation application. Note that the content impression quantification process may be executed by the poster generation device 100, or may be executed by an information processing device different from the poster generation device 100. Note that when executed by an information processing device different from the poster generation device 100, it is executed by the CPU of that information processing device.

[0096] The content impression quantification process derives a correspondence between the content itself and the impression of the content in a space where the impression of the poster is quantified. This makes it possible to find content that suits the impression of the poster you want to generate. The content impression quantification process is executed, for example, in a poster generation device by running an impression learning application to learn the impression of the content in advance of the poster generation process. Furthermore, the content impression quantification process must be executed after the poster impression quantification process because it uses the impression transformation formula acquired in the poster impression quantification process shown in Figure 10.

[0097] Fig. 12 is a flowchart showing the content impression quantification process. The flowchart shown in Fig. 12 is implemented, for example, by the CPU 101 reading a program stored in the HDD 104 into the RAM 103 and executing it. First, the image impression quantification process will be described with reference to Fig. 12(a).

[0098] In S1201, CPU 101 executes acquisition of subjective evaluations of impressions of images. The subjective evaluation may be performed using the same method as the subjective evaluation performed in quantifying impressions of posters. After acquiring subjective evaluation results from multiple subjects for multiple images, CPU 101 calculates the average value of the responses to each adjective pair and sets this average value as the representative score value for the corresponding adjective pair. Note that the subjective evaluation method for impressions may be other than the SD method, as long as words expressing impressions and corresponding scores are determined.

[0099] In S1202, the CPU 101 acquires from the HDD 104 the impression conversion formula obtained by the factor analysis performed in the poster impression quantification process, and applies it to the subjective evaluation results acquired in S1201 to acquire an impression value for each image. By applying the impression conversion formula obtained in the poster impression quantification, the impression of the image can be quantified on a dimension that has the same meaning as the impression of the poster.

[0100] In S1203, the CPU 101 associates images with impressions. While the above-described method can quantify images that have been subjectively evaluated, the poster generation process of this embodiment requires that impressions be estimated for unknown images without subjective evaluation. The association between images and impressions can be achieved by learning a model that estimates impressions from images. Specifically, for example, deep learning methods using Convolution Neural Network (CNN) or Visual Transformer (ViT), or machine learning methods using decision trees, can be used. In this embodiment, the CPU 101 performs supervised deep learning using CNN, taking images as input and four factors as output. That is, a deep learning model is created by learning the subjectively evaluated images and the corresponding impressions as correct answers, and an impression is estimated by inputting an unknown image into the learning model.

[0101] In S1204, the CPU 101 stores in the HDD 104 the model configuration and learned parameters of the deep learning model for impression estimation created in S1203.

[0102] Next, the text impression quantification process will be described with reference to FIG.

[0103] In S1211, CPU 101 acquires subjective evaluations of impressions of the text. The subjective evaluations may be performed using the same method as the subjective evaluations performed in quantifying impressions of the posters. After acquiring subjective evaluation results from multiple subjects for multiple texts, CPU 101 calculates the average of the responses to each adjective pair and sets the average as the representative score value for the corresponding adjective pair. Note that the subjective evaluation method for impressions may be other than the SD method, as long as words expressing impressions and their corresponding scores are determined.

[0104] In S1212, the CPU 101 obtains from the HDD 104 the impression conversion formula obtained by the factor analysis performed in the poster impression quantification, and applies it to the subjective evaluation results obtained in S1211 to obtain an impression value for each piece of text. By applying the impression conversion formula obtained in the poster impression quantification, the impression of the text can be quantified on a dimension that has the same meaning as the impression of the poster.

[0105] In S1213, CPU 101 associates text with impressions. While it is possible to quantify images that have been subjectively evaluated using the method described above, it is also necessary to estimate impressions for unknown text without subjective evaluation. Association between text and impressions can be achieved by training a model that estimates impressions from text, using, for example, a deep learning method using Transformer or a machine learning method using a decision tree. In this embodiment, CPU 101 inputs an image and performs supervised deep learning using Transformer, with four factors as output. That is, a deep learning model is created by learning the subjectively evaluated image and the corresponding impression as the correct answer, and an impression is estimated by inputting unknown text into the learning model.

[0106] In S1214, the CPU 101 stores in the HDD 104 the model configuration and learned parameters of the deep learning model for impression estimation created in S1213.

[0107] <Operation mode explanation> The differences in the process flow and the posters output in the content-first mode and impression-first mode, which are operating modes of the poster creation application, will be outlined with reference to Fig. 13. Note that Fig. 13 illustrates an example in which the content used in the poster is an image, but in this embodiment, text information such as a title can also be treated as content.

[0108] The impression-priority mode is a mode in which content appropriate for a user-specified target impression is automatically selected. In the processing flow, the poster creation application first accepts the user's specification of a target impression 1311. Next, the poster creation application accepts the user's specification of content candidates 1312 to be used in the poster. When specifying content candidates, the user inputs title candidates. The user may specify specific images as image candidates, or may specify a group of images associated with a directory, tag, or app stock image itself on the image specification screen 801 shown in FIG. 8 . Alternatively, multiple images generated by AI may be specified by specifying a prompt. The poster creation application selects content 1313 appropriate for the specified target impression 1311 from the content candidates 1312. The poster creation application uses the selected content 1313 to generate posters 1314 and 1315 that have impressions similar to the target impression. In the impression-priority mode, when multiple patterns of posters are generated, different patterns of content may be used for each poster.

[0109] The content-first mode is a mode in which the UI for setting a target impression is changed so that a target impression appropriate for the content used in the poster is prioritized. As a processing flow, first, the poster creation application accepts designation of content 1301 to be used in the poster from the user. Next, the poster creation application determines a target impression setting range 1302 appropriate for the designated content 1301. After that, the poster creation application accepts a designation of a target impression by the user within the determined target impression setting range 1302. The designated target impression is indicated by reference numeral 1303 in the figure. The poster creation application uses the designated content 1301 to generate posters 1304 and 1305 having impressions close to the designated target impression 1303. In the content-first mode, even when multiple patterns of posters are generated, the content 1301 designated by the user is used for all posters.

[0110] As described above, in impression-priority mode, the process starts when the user specifies a target impression, whereas in content-priority mode, the process starts when the user specifies content. Also, in impression-priority mode, different content may be used in the poster generated depending on the target impression specified by the user, whereas in content-priority mode, the content specified by the user is reliably used in the poster generated. By selecting one of these operating modes, the user can execute the poster generation process in a production flow that matches their intentions.

[0111] <Operation mode switching process> 14 is a flowchart showing the operation mode switching process of the poster creation application, which starts when the poster creation application is started by a user operation.

[0112] The flowchart shown in Fig. 14 is realized, for example, by CPU 101 reading a program stored in HDD 104 into RAM 103 and executing it. In this embodiment, the components shown in Fig. 2 that function when CPU 101 executes the poster creation application will be described as executing the processes. The operation mode switching process will be described with reference to Fig. 14. Note that the symbol "S" in the description of each process indicates a step in the flowchart (the same applies hereinafter in this specification).

[0113] In S1401, the poster creation application displays the mode selection screen 601 on the display 105. The user uses the pointing device 107 to specify one of the operation modes displayed on the mode selection screen 601.

[0114] In S1402, the operation mode designation unit 230 acquires the operation mode designated on the mode selection screen 601.

[0115] In S1403, the operation mode designation unit 230 determines whether the operation mode acquired in S1402 is the content-priority mode or the impression-priority mode. If the operation mode is the content-priority mode, the process proceeds to S1404, and if the operation mode is the impression-priority mode, the process proceeds to S1405.

[0116] In S1404, the poster generation unit 210 executes the poster generation process in the content priority mode.

[0117] In S1405, the poster generation unit 210 executes the poster generation process in the impression-priority mode.

[0118] The above is the processing flow of the poster creation application. The poster generation process in the content-first mode executed in S1404 and the poster generation process in the impression-first mode executed in S1405 will be described in detail below.

[0119] <Poster generation process in impression-priority mode> FIG. 15 is a flowchart showing the poster generation process in the impression-priority mode. The flowchart shown in FIG. 15 is started when the impression-priority mode is selected in the above-described operation mode switching process. The flowchart shown in FIG. 15 is realized, for example, by the CPU 101 reading a program stored in the HDD 104 into the RAM 103 and executing it. In this embodiment, the components shown in FIG. 2 that function when the CPU 101 executes the poster creation application will be described as executing the processes. Note that in the impression-priority mode, the image impression estimation unit 221 and the UI change unit 222 in FIG. 2 are not used. The poster generation process in the impression-priority mode will be described with reference to FIG. 15.

[0120] In S1501, the poster creation application displays the creation condition setting screen 701 shown in FIG.

[0121] In S1502, the poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 accept settings for each setting item displayed on the creation condition setting screen 701 and acquire the set data. The user inputs setting values ​​for each setting item using the keyboard 106 or pointing device 107. The image specification unit 203 acquires information on the acquisition destination of the image file or image group. The text specification unit 202 acquires character information entered in the title box 702, subtitle box 703, and body box 704. The target impression specification unit 204 accepts designation of the target impression by operating the impression sliders 708 to 711 or radio button 712, and acquires information on the designated target impression. The poster creation condition specification unit 201 accepts designation of the poster size, number of creations, number of images, and poster usage category, and acquires the designated information.

[0122] 8, the user sets radio button 802 on and specifies the file path of the image data in designation box 808. If the image to be used in the poster is an in-app material image or an externally linked material image, the user sets radio button 804 or radio button 805 on and specifies the file path of the material image data in designation box 812 or designation box 814.

[0123] If the user has not yet decided on a specific image to use for the poster, the user turns on radio button 803 on image specification screen 801 shown in FIG. 8 and specifies the folder path of the image folder in specification box 810. This allows multiple image data stored in the image folder to be specified as a group of images at once. Alternatively, the user can specify all application material images as a group of images by turning on radio button 804 for application material images (all). Alternatively, the user can specify multiple images as candidate images (group of images) by turning on radio buttons 802, 805, and 806 and specifying the file paths of multiple image data in specification boxes 808, 812, and 814. Alternatively, the user can specify images generated by AI as candidate images (group of images). In this case, the user turns on radio button 807 and specifies a prompt in prompt box 816 to have the image generation AI generate images.

[0124] In S1503, the image acquisition unit 211 acquires image data. Specifically, the image acquisition unit 211 reads the image data or image group designated by the user in the image designation unit 203 from a designated acquisition destination (for example, the HDD 104) into the RAM 103.

[0125] In S1504, the content selection unit 220 executes content selection processing. That is, the content selection unit 220 selects image data to be used for a poster from the image data or image group acquired in S1503. Furthermore, the content selection unit 220 acquires character information for a title from the character information specified in the text specification unit 202 in S1502, and selects character information for a title to be used for a poster from that character information. The image selected by the content selection unit 220 may be displayed in the image specification area 705 of the generation condition setting screen 701. Furthermore, the title selected by the content selection unit 220 may be displayed in the title box 702 or the additional title box 719 of the generation condition setting screen 701.

[0126] <Content selection process> The content selection process executed in S1504 will now be described in detail with reference to Fig. 16. Fig. 16 is a flowchart for explaining S1504 in detail. The process of this flowchart is executed by the determination unit 301, content impression estimation unit 302, content evaluation unit 303, and content selection unit 304 of the content selection unit 220 shown in Fig. 3.

[0127] In S1601, the determination unit 301 determines whether there is a plurality of pieces of title character information acquired from the text designation unit 202. If there is a plurality of pieces of title character information, the process proceeds to S1602. If there is a single piece of title character information, the title is uniquely determined, so the content selection unit 220 selects that title and stores it in RAM 103, and the process proceeds to S1605.

[0128] In S1602, the content impression estimation unit 302 estimates the impressions of each of a plurality of titles (text information) using the trained model generated by the content impression quantification process shown in FIG. 12(b).

[0129] In S1603, the content evaluation unit 303 determines a distance, which is information indicating the difference between the target impression acquired in S1502 and the impression of each title (text information) estimated in S1602. In this embodiment, the Euclidean distance is used as the distance. The smaller the distance value, the closer the target impression and the impression of the title are.

[0130] In S1604, the content selection unit 304 selects the top N titles with the smallest distance values ​​determined in S1603. In this embodiment, the content selection unit 304 selects the top one title. The content selection unit 304 stores the selected title in RAM 103.

[0131] The number N of titles to be selected may be set as a fixed value, or may be variable depending on the conditions specified in the poster creation condition specification unit 201. For example, if the number to be created is specified as 6 in the number to be created box 714 on the generation condition setting screen 701, the poster generation unit 210 will generate six posters. The layout unit 217, which will be described later, generates a poster by combining skeletons, color patterns, and fonts selected in the skeleton selection unit 214, color pattern selection unit 215, and font selection unit 216. Therefore, for example, by selecting two skeletons, two color patterns, and two fonts, 2 × 2 × 2 = 8 posters can be generated, thereby satisfying the condition that the number to be created is 6. In this way, the number N of titles to be selected may be determined depending on the conditions specified in the poster creation condition specification unit 201.

[0132] Furthermore, if the minimum value of the distance determined in S1603 is greater than a predetermined threshold, the content selection unit 304 may display a warning screen on the display 105 indicating that there is no title having an impression close to the target impression.

[0133] In S1605, the determination unit 301 determines whether the number of images specified in the poster creation condition specification unit 201 is 0. If the number of images is not 0 (S1605; YES), the process proceeds to S1606. If the number of images is 0 (S1605; NO), there is no need to select an image, so this flowchart ends and the process proceeds to S1505 in FIG.

[0134] In S1606, the determination unit 301 determines whether the number of image data (number of candidate images) acquired by the image acquisition unit 211 in S1503 is greater than the number of images specified by the poster creation condition designation unit 201. If the number of image data (number of candidate images) acquired by the image acquisition unit 211 is greater (S1606; YES), the process proceeds to S1607. If the number of image data (number of candidate images) acquired by the image acquisition unit 211 matches the number of images specified by the poster creation condition designation unit 201 (S1606; NO), the image to be used is uniquely determined. Therefore, the image is selected and stored in RAM 103, the present flowchart is terminated, and the process proceeds to S1505 in FIG. 15. Note that if the number of image data (number of candidate images) acquired by the image acquisition unit 211 is less than the number of images specified by the poster creation condition designation unit 201, a process prompting addition of an image is performed when the OK button 717 on the generation condition setting screen 701 is pressed, as described above.

[0135] In S1607, the content impression estimation unit 302 estimates the impressions of each of the multiple pieces of image data using the trained model generated by the content impression quantification process shown in FIG. 12(a).

[0136] In S1608, the content evaluation unit 303 determines the distance between the target impression acquired in S1502 and the impression of each image data estimated in S1607. The smaller the distance value, the closer the target impression and the impression estimated from the image data are.

[0137] In S1609, the content selection unit 304 selects the top N image data having the smallest distance values ​​determined in S1608, and stores them in RAM 103. In this embodiment, the content selection unit 304 selects image data for the number of images specified by the poster creation condition specification unit 201. As in the case of text, the method for setting the selection number N is not limited to this, and it may be variable depending on the conditions specified by the poster creation condition specification unit 201. Furthermore, if the minimum distance value calculated in S1607 is greater than a predetermined threshold, the content selection unit 304 may display a warning screen on the display 105 indicating that there are no candidate images having an impression similar to the target impression.

[0138] When the process of S1609 ends, this flowchart ends and the process proceeds to S1505 in Fig. 15. The content selection process of this flowchart results in the selection of one title and the specified number of image data.

[0139] Returning to FIG. 15(a), in S1505, the number of selections is determined so that posters can be generated according to the number of creations specified in the poster creation condition specification unit 201. That is, the skeleton selection unit 214 determines the number of skeletons to select, the color scheme pattern selection unit 215 determines the number of color scheme patterns to select, and the font selection unit 216 determines the number of fonts to select. In this embodiment, the layout unit 217 generates poster data equal to the number of skeletons x the number of color scheme patterns x the number of fonts using a method described below. The skeleton selection unit 214, the color scheme pattern selection unit 215, and the font selection unit 216 determine the number of skeletons, the number of color scheme patterns, and the number of fonts to select so that the number of posters to be generated is equal to or greater than the number of creations specified in the poster creation condition specification unit 201 (the number specified in the number of creations box 714 in FIG. 7). For example, the number of skeletons, the number of color scheme patterns, and the number of fonts can each be determined according to the following equation 1.

[0140]

number

[0141] For example, if the number of creations is 6, the number of selections is 3, and the layout unit 217 generates 27 pieces of poster data, of which the poster selection unit 219 selects 6 pieces of poster data. This allows the poster selection unit 219 to select posters whose overall impression more closely matches the target impression from among the poster data generated in excess of the number of creations. Note that the method for determining the number of selections is not limited to this, and other methods may also be used. The number of selections may also be a fixed value.

[0142] In S1506, the image analysis unit 212 performs an analysis process on the image data selected in the content selection process (FIG. 16) in S1504, and acquires information indicating feature amounts related to the image. Examples of information indicating feature amounts include meta information stored in the image and information indicating image feature amounts that can be acquired by analyzing the image. This information is used in object recognition processing, which is an analysis process. Note that in this embodiment, object recognition processing is executed as the analysis process, but this is not limited to this, and other analysis processes may also be executed. Furthermore, the process of S1506 may be omitted. Below, details of the process performed by the image analysis unit 212 in S1506 will be described.

[0143] The image analysis unit 212 executes object (material) recognition processing on the image selected in S1504. Here, a known method can be used for the object recognition processing. In this embodiment, objects are recognized by a classifier created by Deep Learning. The classifier outputs the likelihood of a pixel constituting the image being a pixel constituting each object as a value between 0 and 1, and recognizes that an object exceeding a certain threshold is present in the image. By recognizing the object image, the image analysis unit 212 can acquire the type and position of objects such as faces, pets such as dogs or cats, flowers, food, buildings, ornaments, and landmarks.

[0144] In S1507, the skeleton acquisition unit 213 acquires skeletons that meet the various set conditions. In this embodiment, it is assumed that each skeleton is described in one file and stored in the HDD 104. The skeleton acquisition unit 213 sequentially reads skeleton files from the HDD 104 to the RAM 103, leaves skeletons that meet the set conditions on the RAM 103, and deletes skeletons that do not meet the conditions from the RAM 103. Figure 15(b) is a flowchart of the condition determination process performed by the skeleton acquisition unit 213 in S1507. The condition determination process performed by the skeleton acquisition unit 213 will be described with reference to Figure 15(b).

[0145] In S1521, skeleton acquisition unit 213 determines whether the skeleton size matches the poster size specified by poster creation condition specification unit 201 for the skeleton read into RAM 103. Note that although it is confirmed that the sizes match here, it is also sufficient that the aspect ratios match. In this case, skeleton acquisition unit 213 acquires a skeleton that matches the poster size specified by poster creation condition specification unit 201 by enlarging or reducing the coordinate system of the read skeleton.

[0146] In S1522, the skeleton acquisition unit 213 determines whether the use category specified in the poster creation condition specification unit 201 matches the skeleton category. For skeletons that are used only for specific purposes, the use category is written in the skeleton file, and the skeleton is not acquired unless the corresponding use category is selected. This prevents a skeleton that is designed specifically for a specific purpose, such as one that uses graphics to evoke a school or a sporting goods pattern, from being used in other use categories. Note that if no use category is set on the generation condition setting screen 701, S1522 is skipped.

[0147] In S1523, skeleton acquisition unit 213 determines whether the number of image objects of the loaded skeleton matches the number of images specified by poster creation condition specification unit 201. While this embodiment illustrates an example of determining whether the number of image objects of the skeleton matches the number of images specified by poster creation condition specification unit 201, the determination is not limited thereto and may instead be made as to whether the number of image objects of the skeleton is equal to or less than the specified number of images. In this case, some of the posters generated by poster generation unit 210 will not satisfy the number of images specified by the user. That is, when the number of images specified by poster creation condition specification unit 201 is three, posters with three images, posters with two images, and posters with one image will also be generated. Therefore, compared to posters generated when the number of image objects of the skeleton matches the number of images specified by poster creation condition specification unit 201, more image selection patterns of posters can be generated.

[0148] In S1524, the skeleton acquisition unit 213 determines whether the character object of the loaded skeleton matches the character information specified in the text designation unit 202. More specifically, it determines whether the type of character information specified in the text designation unit 202 is present in the skeleton. For example, assume that character strings are specified in the title box 702 and the body box 704 on the generation condition setting screen 701, and that the subtitle box 703 is left blank. In this case, all character objects in the skeleton are searched, and if both a character object with "title" set as the type of character information in the metadata and a character object with "body" specified are found, the skeleton is deemed suitable; otherwise, it is deemed unsuitable. If the character object of the loaded skeleton matches the character information specified in the text designation unit 202, the skeleton acquisition unit 213 leaves the skeleton in RAM 103; if they do not match, the skeleton is deleted from RAM 103.

[0149] As described above, the skeleton acquisition unit 213 stores in RAM 103 skeletons whose skeleton size, usage category, number of image objects, and type of text object all match the conditions set on the generation condition setting screen 701. In this embodiment, the skeleton acquisition unit 213 examines all skeleton files on HDD 104, but this is not limiting. For example, the poster creation application may store in advance in HDD 104 a database that associates the file paths of skeleton files with search conditions (skeleton size, number of image objects, and type of text object). In this case, the skeleton acquisition unit 213 can quickly acquire skeleton files by reading only those skeleton files that match the results of a search in the database from HDD 104 to RAM 103. Return to FIG. 15(a).

[0150] In S1508, the skeleton selection unit 214 selects a skeleton from the skeletons acquired in S1507 that matches the target impression specified by the target impression designation unit 204. FIG. 17 is a diagram illustrating a method for the skeleton selection unit 214 to select a skeleton. FIG. 17(a) is a diagram illustrating an example of a table linking skeletons to impressions. The skeleton name column in FIG. 17(a) lists the skeleton file names, and the luxury, familiarity, dynamism, and profoundness columns represent numbers (numeric values) indicating the degree to which each skeleton influences each impression factor. These numerical values ​​indicate that the impression is low (-2), slightly low (-1), neutral (0), slightly high (+1), and high (+2), respectively. First, the skeleton selection unit 214 calculates the distance between the target impression acquired from the target impression designation unit 204 and the impression of each skeleton shown in the skeleton impression table in FIG. 17(a). For example, if the target impression is "luxury +1, familiarity -1, dynamism -2, and profoundness +2," the distance calculated by the skeleton selection unit 214 will be as shown in FIG. 17(b). In this embodiment, Euclidean distance is used as the distance (hereinafter, simple distance will be referred to as Euclidean distance). The smaller the value indicated by the Euclidean distance, the closer the target impression and the impression of the skeleton are. Next, the skeleton selection unit 214 selects the top N selection number of skeletons with the smallest distance values ​​in FIG. 17(b). In this embodiment, the skeleton selection unit 214 selects the top two skeletons. That is, the skeleton selection unit 214 selects skeleton 1 and skeleton 4.

[0151] As described above, the number N of skeletons to be selected may be a fixed value, or may be a variable value depending on the conditions specified in the poster creation condition specification unit 201. When the number N is a variable value, it may be determined using the above-described formula 1, or may be determined by other methods. For example, if the number of skeletons to be created is set to 6 in the creation number box 714 on the creation condition setting screen 701, the poster generation unit 210 generates six posters. The layout unit 217, described later, generates a poster by combining the skeletons, color patterns, and fonts selected in the skeleton selection unit 214, color pattern selection unit 215, and font selection unit 216. Therefore, for example, by selecting two skeletons, two color patterns, and two fonts, 2 × 2 × 2 = 8 posters can be generated, thereby satisfying the condition of 6 posters to be created. In this way, the number N of skeletons to be selected may be determined depending on the conditions specified in the poster creation condition specification unit 201.

[0152] Furthermore, the value range of each impression in the skeleton impression table of FIG. 17(a) does not need to be the same as the value range of the impression specified by the target impression specifying unit 204. In this embodiment, the value range of the impression specified by the target impression specifying unit 204 is -2 to +2, but the value range of the impression in the skeleton impression table may be different from this. In this case, the value range of the skeleton impression table is scaled to match the value range of the target impression before the distance calculation is performed. Furthermore, the distance calculated by the skeleton selecting unit 214 is not limited to Euclidean distance, and any method that can calculate the distance between vectors, such as Manhattan distance or cosine similarity, may be used. Furthermore, impression factors set to off using the radio button 712 are excluded from the distance calculation.

[0153] The skeleton impression table is created in advance by estimating the impression of a poster image generated based on each skeleton, for example, by fixing the color scheme, font, and image and text data to be placed on the skeleton, and then saving it on HDD 104. That is, by estimating the impression of each poster image that uses the same text color and image, but has a different text and image arrangement, the relative characteristics compared to other skeletons are tabulated. In this process, it is desirable to cancel the impression due to the color scheme, image, etc. used, by standardizing the overall estimated impression or averaging the impressions of multiple poster images generated from a single skeleton using multiple color schemes and images. This allows the table to show, for example, how the impression of a skeleton with a small image is determined by elements such as graphics and text, regardless of the image, or how tilted image or text arrangement creates a stronger sense of dynamism.

[0154] FIG. 17(c) shows examples of skeletons corresponding to Skeletons 1 to 4 in FIG. 17(a). For example, Skeleton 1 has image objects and text objects arranged in a regular pattern and a small image area, resulting in a low sense of dynamism. Skeleton 2 has circular graphic and image objects, resulting in a high sense of familiarity and a low sense of solidity. Skeleton 3 has large image objects and tilted graphic objects placed on top of the image objects, resulting in a high sense of dynamism. Skeleton 4 has images placed across the entire skeleton and minimizes text objects, resulting in a high sense of solidity and a low sense of dynamism. In this way, when a poster image includes text or images, poster images with different target impressions are generated depending on how the text or images are arranged. Note that the method of creating the skeleton impression table is not limited to this; it may be estimated from the characteristics of the arrangement information itself, such as the area and coordinates of the image and title string, or it may be adjusted manually. The skeleton impression table is stored in the HDD 104, and the skeleton selection unit 214 reads the skeleton impression table from the HDD 104 into the RAM 103 and refers to it.

[0155] In S1509, color scheme selection unit 215 selects a color scheme that matches the target impression specified by target impression specification unit 204. Using a method similar to S1507, color scheme selection unit 215 references the impression table corresponding to the color scheme and selects a color scheme according to the target impression. FIG. 18(a) shows an example of a color scheme impression table that links color scheme patterns with impressions. Color scheme selection unit 215 calculates the distance between the impressions indicated by the columns for luxury to dignity in FIG. 18(a) and the target impression, and selects the top N color scheme patterns with the smallest distance values. In this embodiment, the top two color scheme patterns are selected. Note that, like the skeleton impression table, the color scheme impression table can tabulate the impression trends of color scheme patterns by creating and estimating impressions of posters with different color scheme patterns while fixing the skeleton, font, and images other than the color scheme pattern.

[0156] In S1510, the font selection unit 216 selects a font combination that matches the target impression specified by the target impression specification unit 204. The font selection unit 216 references the impression table corresponding to the font and selects a font according to the target impression in the same manner as in S1507. FIG. 18(b) shows an example of a font impression table that links fonts with impressions. The font selection unit 216 calculates the distance value between the impression indicated by the columns for luxury to dignity in FIG. 18(b) and the target impression, and selects the top N fonts with the smallest distance values. Note that, like the skeleton impression table, the font impression table can tabulate the tendency of font impressions by fixing the skeleton, color pattern, and image other than the font, and then creating and estimating the impression of posters with different fonts.

[0157] In S1511, the layout unit 217 sets character information, images, color schemes, and fonts for the skeleton selected by the skeleton selection unit 214, and generates a poster.

[0158] Next, the processing of S1511 and the software configuration of the layout unit 217 will be described in detail with reference to FIGS.

[0159] 19 is an example of a software block diagram illustrating in detail the layout unit 217. The layout unit 217 includes a color allocation unit 1901, an image layout unit 1902, an image correction unit 1903, a font setting unit 1904, a text layout unit 1905, and a text decoration unit 1906.

[0160] Fig. 20 is a flowchart for explaining the layout processing of S1511 in detail. Fig. 21 is a diagram for explaining the information input to the layout unit 217. Fig. 21(a) is a table summarizing the character information specified in the text specification unit 202 and the image data 2101 selected in the content selection unit 220. Fig. 21(b) is an example of a table showing color patterns acquired from the color pattern selection unit 215, and Fig. 21(c) is an example of a table showing fonts acquired from the font selection unit 216. Fig. 22 is a diagram for explaining the processing steps of the layout unit 217.

[0161] First, the layout process in S1511 will be described in detail with reference to FIG.

[0162] In S2001, the layout unit 217 lists all combinations of skeletons obtained from the skeleton selection unit 214, color patterns obtained from the color pattern selection unit 215, and fonts obtained from the font selection unit 216. The layout unit 217 generates poster data for each combination in turn by performing layout processing from S2002 onwards. For example, if the number of skeletons obtained from the skeleton selection unit 214 is three, the number of color patterns obtained from the color pattern selection unit 215 is two, and the number of fonts obtained from the font selection unit 216 is two, the layout unit 217 generates 3 x 2 x 2 = 12 poster data. Next, in S2001, the layout unit 217 selects one of the listed combinations and executes the processes of S2002 to S2007. Note that the content selection unit 304 may select more than one title in S1604 or may select more images than the number of images specified by the poster creation condition specification unit 201 in S1609. In these cases, the layout unit 217 lists combinations that take into consideration the title and image.

[0163] In S2002, the color scheme assignment unit 1901 assigns the color scheme pattern acquired from the color scheme pattern selection unit 215 to the skeleton acquired from the skeleton selection unit 214. FIG. 22(a) is a diagram showing an example of a skeleton. In this embodiment, an example will be described in which a color scheme pattern is assigned to the skeleton 2201 in FIG. 22(a) when the color scheme ID in FIG. 21(b) is 1. The skeleton 2201 in FIG. 22(a) is made up of two graphic objects 2202 and 2203, one image object 2204, and three text objects 2205, 2206, and 2207. First, the color scheme assignment unit 1901 assigns colors to the graphic objects 2202 and 2203. Specifically, the color scheme assignment unit 1901 assigns a corresponding color from the color scheme pattern based on the color scheme number, which is metadata described in the graphic object. Next, the color scheme assignment unit 1901 selects a character object (Text) from among the character objects whose metadata is type and whose attribute is "title".<type=Title> ), for example, the last color in the color scheme pattern is assigned to the character placed in character object 2205. That is, in this embodiment, color 4 is assigned to the character placed in character object 2205. Next, for the character placed in character objects 2206 and 2207 whose metadata is type and whose attribute is other than "title", the character color is set based on the brightness of the background of the character object. In this embodiment, if the brightness of the background of the character object is equal to or lower than a threshold, the character color is set to white; otherwise, the character color is set to black. Figure 22(b) is a diagram showing the state of skeleton 2208 after the above color scheme assignment process has been performed. The color scheme assignment unit 1901 outputs the colored skeleton data 2208 to the image arrangement unit 1902.

[0164] In S2003, the image placement unit 1902 places the image data acquired from the image analysis unit 212 in the skeleton data 2208 acquired from the color scheme assignment unit 1901 based on the accompanying analysis information. If image analysis by the image analysis unit 212 has not been performed, the image placement unit 1902 places the image data selected by the content selection unit 220 in the skeleton data 2208. In this embodiment, the image placement unit 1902 assigns the image data 2101 to the image object 2204 in the skeleton. Furthermore, if the aspect ratios of the image object 2204 and the image data 2101 differ, the image placement unit 1902 performs trimming so that the aspect ratio of the image data 2101 matches the aspect ratio of the image object 2204. More specifically, based on the position of the object obtained by the image analysis unit 212 analyzing the image data 2101, trimming is performed so that the object area reduced by trimming is minimized. The trimming method is not limited to this, and other trimming methods may be used, such as trimming the center of the image, or devising a composition so that the face position forms a triangle. The image arrangement unit 1902 outputs the skeleton data to which images have been assigned to the image correction unit 1903.

[0165] In S2004, the image correction unit 1903 obtains skeleton data with images assigned from the image placement unit 1902 and corrects the images placed on the skeleton. In this embodiment, if the image resolution is insufficient, upsampling processing is performed using super-resolution processing. First, the image correction unit 1903 determines whether the image placed on the skeleton meets a certain resolution. For example, assume that an image of 1600px x 1200px is assigned to a 200mm x 150mm area on the skeleton. In this case, the print resolution of the image can be calculated using Equation 2.

[0166]

number

[0167] Next, if the print resolution of the image is determined to be less than the threshold, the image correction unit 1903 increases the resolution through super-resolution processing. On the other hand, if the print resolution of the image is determined to be equal to or greater than the threshold and has sufficient resolution, no image correction is performed. In this embodiment, super-resolution processing is performed when the print resolution of the image is less than 300 dpi.

[0168] In S2005, the font setting unit 1904 sets the font acquired from the font selection unit 216 to the image-corrected skeleton data acquired from the image correction unit 1903. FIG. 21C shows an example of a font combination selected by the font selection unit 216. In this embodiment, an example of font assignment will be described in which the font ID "2" in FIG. 21C is assigned to the image-corrected skeleton data. In this embodiment, fonts are set for the character objects 2205, 2206, and 2207 of the skeleton 2208. Note that in posters, a font that is easy to see is often set for the title from the perspective of eye-catchingness, and a font that is easy to read from the perspective of visibility is set for the other characters. Therefore, in this embodiment, the font selection unit 216 selects two types of fonts: a title font and a body font. The font setting unit 1904 sets the title font to the character object 2205 whose attribute is "title," and sets the body font to the other character objects 2206 and 2207. The font setting unit 1904 outputs the skeleton data with the font set to the text layout unit 1905. In this embodiment, the font selection unit 216 selects two types of fonts, but this is not limiting. For example, only the title font may be selected. In this case, the font setting unit 1904 uses a font corresponding to the title font as the body font. That is, if the title is a Gothic font, a typical Gothic font with high readability should be selected for the other text objects, and if the title is a Mincho font, a typical Mincho font should be selected for the other text objects. Naturally, the title font and the body font may be the same. Alternatively, different fonts may be used depending on the degree of prominence desired, such as using a title font for the title and subtitle character objects and a body font for other character objects, or using a title font for a certain font size or larger.

[0169] In S2006, the text layout unit 1905 lays out the text specified by the text designation unit 202 and the text selected by the content selection unit 220 in the skeleton data with font settings obtained from the font setting unit 1904. In this embodiment, each piece of text shown in FIG. 21(a) is assigned by referring to the metadata attributes of the skeleton character objects. That is, "Summer Big Thank You Sale," whose attribute is the title, is assigned to character object 2205, and "Blow Away the Midsummer Heat," whose attribute is the subtitle, is assigned to character object 2206. Because no text has been set, nothing is assigned to character object 2207. FIG. 22(c) shows skeleton 2209, which is an example of skeleton data after processing by the text layout unit 1905. The text layout unit 1905 outputs the skeleton data 2209 with text layout to the text decoration unit 1906.

[0170] In S2007, the text decoration unit 1906 decorates the character objects in the skeleton with text already placed, obtained from the text placement unit 1905. In this embodiment, if the color difference between the title character and its background area is equal to or less than a threshold, a process of bordering the title character is performed. This improves the readability of the title. The text decoration unit 1906 outputs the decorated skeleton data, i.e., the poster data for which the layout has been completed, to the poster impression estimation unit 218.

[0171] In S2008, the layout unit 217 determines whether poster data has been generated for all combinations. If the layout unit 217 determines that poster data has been generated for all combinations of skeletons, color patterns, and fonts, it ends the layout process and proceeds to S1512. If it determines that not all poster data has been generated, it returns to S2001 and generates poster data for combinations that have not yet been generated.

[0172] The layout process in step S1511 has been described above. Returning to the description of FIG.

[0173] In S1512, the poster impression estimation unit 218 performs rendering processing on each poster data acquired from the layout unit 217, estimates the impression of the rendered poster image, and associates the estimated impression with the poster data. The rendering processing is a process of converting poster data into image data. For example, even if the color scheme is the same, the layout will change if the skeleton is different, and the actual area in which each color is used will differ. Therefore, this processing is performed at this timing because it is necessary to evaluate the impression of the final poster, not just the individual impression trends of the color scheme and skeleton. This makes it possible to evaluate the impression of the final poster, including images and text, in addition to the impression of individual elements of the poster, such as the color scheme and layout.

[0174] In S1513, the poster selection unit 219 selects a poster to be output to the display 105 (to be presented to the user) from the poster data and the estimated impression linked to the poster data acquired from the poster impression estimation unit 218. In this embodiment, the poster selection unit 219 selects a poster for which the distance between the target impression and the estimated impression of the poster is equal to or less than a predetermined threshold.

[0175] In this embodiment, Euclidean distance is used as the distance. The smaller the Euclidean distance, the closer the target impression and the estimated impression are. Furthermore, the distance calculated by the poster selection unit 219 is not limited to Euclidean distance; it can be calculated using any method that can calculate the distance between vectors, such as Manhattan distance or cosine similarity.

[0176] Furthermore, if the number of selected posters is less than the number of posters to be created specified by the poster creation condition specification unit 201, the poster selection unit 219 selects the missing posters in ascending order of the distance between the target impression and the estimated impression of the poster. In this embodiment, the poster selection unit 219 also selects the missing posters, but this is not limited to this. For example, if the number of posters selected by the poster selection unit 219 is less than the number of posters to be created, the poster preview screen 901 (FIG. 9) may display a message indicating that there are not enough posters. Alternatively, the poster selection unit 219 may select the missing posters and then display them on the poster preview screen 901 so that posters whose distance between the target impression and the estimated impression is equal to or less than a threshold value can be distinguished from posters whose distance is equal to or greater than the threshold value. For example, if there are not enough selected posters, the process may return to S1505 and increase the number of skeletons, color patterns, and fonts to be selected.

[0177] In S1514, the poster display unit 205 renders the poster data selected by the poster selection unit 219 and outputs the poster image to the display 105. That is, the poster preview screen 901 in FIG.

[0178] The above is a description of the poster generation process (S1405) in the impression-priority mode. As described above, in the poster generation process in the impression-priority mode, the poster creation application accepts a target impression designated by the user and selects content such as images and text to be placed on the poster from a content group based on the accepted target impression. At this time, the poster creation application selects content from the content group that is suitable for the target impression designated by the user. Specifically, the poster creation application selects content that is estimated to have an impression close to the target impression as content that is suitable for the target impression designated by the user. The poster creation application also generates a poster based on the target impression designated by the user using the selected content. As a result, the first content selected when accepting a designation of a first target impression is different from the second content selected when accepting a designation of a second target impression different from the first target impression. As a result, a poster is generated in which different content is arranged according to the target impression.

[0179] Therefore, when impression-priority mode is enabled, it becomes easier to generate a poster with the intended design even if the desired content is not clearly determined. For example, a user may want to create a poster with a calm impression, but may not be able to prepare an appropriate image. In such a case, the user specifies a "calm impression" as the target impression. The poster creation application then automatically finds an image suitable for creating a calm impression from a group of images containing images with various impressions, and automatically generates a poster close to the target impression. This reduces the number of attempts required to obtain a poster with content appropriate for the target impression, making it easier to obtain a poster with the intended design. Note that this can be done not only for images, but also for text information such as titles. If the user cannot determine an appropriate title themselves, the user can specify the target impression, and the poster creation application automatically finds an appropriate title from a group of text containing characters with various impressions, and automatically generates a poster close to the target impression.

[0180] In the above-described embodiment, the text designation unit 202 is configured to accept user input of a character string in the title box 702 or the additional title box 719 of the generation condition setting screen 701, but this is not limited thereto. As shown in the image designation screen 801 in FIG. 8 , text candidates may be acquired from the user by specifying a source of a text group containing multiple pieces of text. In this case, it may be possible to specify source text (application source text) created in advance for this application, source text (linked source text) provided by a material providing service, and text generated by the generation AI. Furthermore, the source of such text may be specified in the text designation unit 202, thereby acquiring a text group containing multiple pieces of text. Furthermore, the designation of a text group may be accepted not only for titles but also for subtitles and main texts.

[0181] <Poster generation process in content-first mode> Next, the poster generation process in the content priority mode will be described.

[0182] FIG. 23 is a flowchart for explaining the poster generation processing in the content-priority mode in detail. This flowchart is executed in S1404, which is entered when the content-priority mode is set in the operation mode switching processing shown in FIG. 14. Note that steps in FIG. 23 that are given the same reference numerals as those in FIG. 15 (poster generation processing in the impression-priority mode) execute the same processing, and therefore their explanations are omitted. In the poster generation processing in the content-priority mode, S1502 (setting acquisition) and S1503 (image acquisition) in the poster generation processing in the impression-priority mode are deleted. Instead, S2301 (image acquisition), S2302 (image impression estimation), S2303 (setting UI change for target impression), and S2304 (setting acquisition) are added. Note that in the content-priority mode, the image impression estimation unit 221 and UI change unit 222 in FIG. 2 are used. The following description will focus on the differences from the impression-priority mode.

[0183] In S1501 of FIG. 23(a), it is assumed that the Add Image button 707 is pressed on the generation condition setting screen 701, and the OK button 717 is pressed after the user has specified one or more images on the image specification screen 801.

[0184] In S2301, the image acquisition unit 211 acquires image data designated by the user. Specifically, the image acquisition unit 211 reads one or more image data from the acquisition source designated by the image designation unit 203 into the RAM 103.

[0185] In S2302, the image impression estimation unit 221 estimates the impression of one or more pieces of image data acquired in S2301 and associates the estimated impression value with each piece of image data. As in S1607 described above, the impression of the image data can be estimated using the trained model generated by the impression quantification process for the content (image) shown in Figure 9(a).

[0186] In S2303, the UI change unit 222 changes the state of the UI for specifying a target impression on the generation condition setting screen 701, based on the estimated impressions of one or more images acquired in S2302. For example, the UI change unit 222 changes the on / off setting of the radio button 712 corresponding to each impression factor according to the impression estimated from the image. In this embodiment, if the standard deviation of the estimated impressions of images for a certain impression factor is equal to or greater than a predetermined threshold, that is, if the distribution of estimated impressions for that impression factor is wide, the radio button 712 for that impression factor is set to off. The radio buttons 712 for other impression factors are set to on. This is because, when an image with a wide distribution of estimated impressions is specified, there will be suitable images no matter what target impression the user specifies, and this does not contribute to narrowing down the images.

[0187] In S2304 , the poster creation condition specification unit 201 , the text specification unit 202 , and the desired impression specification unit 204 acquire the corresponding settings from the creation condition setting screen 701 .

[0188] After S2304, the process proceeds to the content selection process of S1504. The process from S1504 onwards is the same as the process described in the impression-priority mode. Also, in this flowchart, an example has been described in which the UI change unit 222 changes the UI for setting the desired impression based on the estimated impression of an image, but the same can be applied to text. That is, the UI change unit 222 may change the UI for setting the desired impression based on the estimated impression of text specified by the user.

[0189] As described above, in the poster generation process in content-first mode, the UI for setting a target impression changes according to the impression of one or more pieces of content specified by the user, and the target impressions that the user can specify are changed. This makes it easier to specify a target impression that is close to the impression of the content. Furthermore, when creation conditions such as text specification, poster size, and usage category are specified, content that is close to the target impression is selected by the content selection unit 220, and a poster that includes the selected content and meets the creation conditions is generated.

[0190] <<Modification 1 of the First Embodiment>> In the first embodiment, the impression sliders 708 to 711 on the generation condition setting screen 701 are used as objects to be operated to set the desired impression, but the method for setting the desired impression is not limited to this.

[0191] FIG. 24 shows an example of a UI for setting a target impression. FIG. 24(a) shows an example of setting a target impression using a UI on a radar chart 2400. By operating a handle 2401 on the radar chart 2400 in FIG. 24(a), a target impression value can be set for each impression factor set on each axis. The target impression specification unit 204 acquires a target impression value for each axis, for example, such that when the handle 2401 is at the center of the UI, the value is -2, and when it is at the outermost position, the value is +2. In FIG. 24(a), the target impressions are +0.8 for luxury, +1.1 for familiarity, -0.1 for dynamism, and -0.7 for profoundness. As such, the target impressions may be decimals. Also, the radar chart 2403 in FIG. 24(b) shows a state in which some impression factors have been turned off by the user or the UI change unit 222. For example, the user can double-click a handle with the pointing device 107 to turn off the target impression for the axis corresponding to that handle, thereby hiding it. The user can turn on the target impression and redisplay the radar chart 2403 by clicking the axis 2402 on the radar chart 2403 again with the pointing device 107. In Figure 24(b), the target impression is the same as in Figure 24(a) except for the dynamism, but the dynamism is set to off.

[0192] In the content-priority mode, when a target impression is set using a UI on a radar chart, the UI change unit 222 changes the operable impression factors (axes) in the same way as the UI on a slider described in the first embodiment. Specifically, as shown in the radar chart 2403, if the standard deviation of the estimated impressions of the image for the impression factor is equal to or greater than a predetermined threshold, that is, if the distribution of the estimated impressions for the impression factor is wide, the handle of the axis corresponding to the impression factor is set to off. In this case, the non-operable axes may be displayed in a different display format from the operable axes, for example, by being displayed in gray.

[0193] Furthermore, the target impression specifying unit 204 may accept the specification of the target impression by specifying information expressing the impression (hereinafter referred to as expression information). FIG. 24(c) shows an example of a UI for setting the target impression from a poster image sample (hereinafter referred to as sample poster image), which is expression information, rather than words such as "luxury" or "familiarity." A sample poster display area 2409 displays an array of sample poster images 2404 to 2407 that enhance one of the impressions. A check box 2408 is also displayed on each sample poster image. The user can select a sample poster image that the user thinks is closest to the impression of the poster they want to create by clicking the pointing device 107 to turn on the check box 2408. The target impression specifying unit 204 determines the target impression by referring to the impression value corresponding to the selected sample poster image.

[0194] FIG. 24(d) is a table showing the impression values ​​corresponding to the sample poster images 2404 to 2407 in FIG. 24(c) and the final target impression value. The columns for luxury, familiarity, dynamism, and stateliness represent numbers indicating the degree to which each sample poster image influences each impression factor. For example, assume that sample poster images 2404 and 2407 are selected as shown in FIG. 24(c). In this case, the target impression designation unit 204 determines the final target impression value as a composite impression value of the sample poster images 2404 and 2407. In this example, the value with the largest absolute value among the values ​​of the impression factors corresponding to the selected sample poster images 2404 and 2407 is set as the value of each impression factor for the final target impression. Note that while an example in which a poster image with the highest impression is presented has been shown, this is not limiting. Sample poster images with high impression values ​​for multiple impression factors may be used, or sample poster images with more than the number of impression factors may be presented. This allows the user to intuitively designate the target impression from an image of the actual product rather than using words.

[0195] FIG. 25 is a diagram showing an example of a target impression setting screen 2504 for specifying a target impression using media files such as image files, video files, and music files as expression information. By pressing a target impression setting button (not shown) provided on the generation condition setting screen 701, the target impression setting screen 2504 is displayed on the display 105. Radio buttons 2510 to 2512 are buttons for selecting the type of media file to be used in setting the target impression. The user can set the type of media file by pressing one of the radio buttons 2510 to 2512 to set it on / off. Note that, of the multiple radio buttons 2510 to 2512, only one radio button can be set to on. In other words, when an off radio button is set to on, that radio button is turned on, but any radio button that was on before the setting is automatically set to off.

[0196] Radio button 2510 is a button for setting an image file as the type of media file. Designation box 2513 accepts the designation of an image file. The user can designate an image that represents a target impression by designating the file path of the image in designation box 2513. Browse button 2516 is a button for designating the acquisition location of the image file. When the user presses browse button 2516, the poster creation application displays a dialog screen for selecting a file saved in HDD 104 and accepts the user's selection of an image file.

[0197] Radio button 2511 is a button for setting a video file as the type of media file. Designation box 2514 accepts the designation of a video file. The user can designate a video that expresses a target impression by designating the file path of the video in designation box 2514. Browse button 2517 is a button for designating the acquisition destination of the video file. When the user presses browse button 2517, the poster creation application displays a dialog screen for selecting a file saved in HDD 104 and accepts the user's selection of a video file.

[0198] Radio button 2512 is a button for setting a music file as the type of media file. Designation box 2515 accepts the designation of a music file. The user can designate music that expresses a target impression by designating the music file path in designation box 2515. Browse button 2518 is a button for designating the source from which the music file is obtained. When the user presses browse button 2518, the poster creation application displays a dialog screen for selecting a file saved in HDD 104 and accepts the user's selection of a music file.

[0199] The cancel button 2519 is a button for canceling the setting of the desired impression on the desired impression setting screen 2504 and returning to the generation condition setting screen 701. When the OK button 2520 is pressed, the media file linked to the radio button that is turned on is sent to a content impression estimation unit (not shown). The content impression estimation unit performs impression estimation on the content (images, videos, music) of the media file using, for example, a machine learning model.

[0200] Here, we will explain the impression estimation method for each media file. For images, the content impression quantification process described in Figure 9(a) can be used to quantify impressions and build an estimation model. For videos, the content impression quantification process described in Figure 9(a) is performed by replacing images with videos and then running CNN or ViT. Other deep learning methods such as recurrent neural networks (RNNs), long short-term memory (LSTMs), and 3D CNNs can also be used to quantify impressions and build estimation models. For music, the process described in Figure 9(a) is performed by replacing images with music and then using deep learning methods such as RNNs and LSTMs to quantify impressions and build estimation models.

[0201] In the impression-priority mode, the target impression designation unit 204 acquires an impression estimated from an image, video, and music as the target impression. The target impression is then used for selecting content in the content selection unit 220, selecting a skeleton in the skeleton selection unit 214, selecting a color pattern in the color scheme selection unit 215, and selecting a font in the font selection unit 216. In the content-priority mode, the impression estimated from the image, video, and music is acquired as the target impression, and the estimated impression is further transmitted to the UI modification unit 222. The UI modification unit 222 reflects the impression estimated from the media file in changes to the UI for setting the target impression (for example, impression sliders 708 to 711 on the generation condition setting screen 701).

[0202] In the content priority mode, the UI changing unit 222 may also change the sample poster images that can be specified for the UI (FIG. 24(c)) for setting a target impression from a sample poster image. Specifically, the UI changing unit 222 selects a sample poster image in which the impression value of a specific impression factor is 0 or close to 0 from among sample poster images having various impressions, and displays the selected sample poster image in the sample poster display area 2409. Alternatively, the UI changing unit 222 limits the sample poster images that can be specified to sample poster images having a specific impression value.

[0203] <<Modification 2 of First Embodiment>> In the first embodiment, the content selection unit 220 selects the top N (N is an integer equal to or greater than 1) pieces of content with the smallest distance between the target impression specified by the user and the estimated impression of the content. However, the content selection method is not limited to this. For example, content may be selected based on the similarity to the expression information specified by the user in addition to the distance from the target impression. In this example, the expression information is text or an image expressing the content the user wants to use. The expression information can be specified by the user, for example, in an expression information input area (not shown) on the generation condition setting screen 701. The content selection unit 220 may calculate the similarity between the expression information specified by the user and an image or each image included in an image group specified by the user, and select content based on the similarity in addition to the distance from the target impression of the image. The similarity between the expression information and the image can be calculated using an image classification model such as CLIP. Among the pieces of content with a small distance between the target impression specified by the user and the estimated impression, the content selection unit 220 may select, as content to be used for the poster, the content with the greatest similarity to the expression information.

[0204] <<Second embodiment>> In the first embodiment, an example was described in which, in impression-priority mode, content suitable for a target impression specified by the user is selected from a content group, and a poster is generated based on the target impression specified by the user using the selected content. A poster creation application in a second embodiment generates multiple poster data in which content included in a content group is arranged, and selects from these posters a poster whose overall impression of the poster image is closest to the target impression. In the second embodiment, a combination generation unit 2601 is provided to generate multiple posters. The combination generation unit 2601 generates combinations of poster components (skeleton, font, color scheme) and content that will result in an overall impression of the poster that is close to the target impression, based on a genetic algorithm. This allows a poster to be generated using content from the content group that will result in an overall impression of the poster that is close to the target impression. In other words, content whose overall impression is suitable for the target impression is selected from the content group.

[0205] <Software block diagram> Fig. 26 is a software block diagram when the impression-priority mode is set in the poster creation application of the second embodiment. As shown in Fig. 26, the poster creation application has a poster creation condition specification unit 201, a text specification unit 202, an image specification unit 203, a target impression specification unit 204, a poster display unit 205, and a poster generation unit 2600. The poster generation unit 2600 has an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a layout unit 217, a poster impression estimation unit 218, a poster selection unit 2602, and a combination generation unit 2601. The operation mode specification unit 230 is omitted from Fig. 2.

[0206] The difference from the poster generation unit 210 of the first embodiment is that a combination generation unit 2601 is provided instead of the skeleton selection unit 214, color scheme pattern selection unit 215, font selection unit 216, and content selection unit 220 in Fig. 2. Also, the poster selection unit 2602 performs processing different from that of the poster selection unit 219 of the first embodiment. In Fig. 26, the components denoted by the same reference numerals as in Fig. 2 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0207] The combination generation unit 2601 acquires one or more skeletons from the skeleton acquisition unit 213, poster data and an estimated poster impression from the poster impression estimation unit 218, and a desired impression from the desired impression designation unit 204. The combination generation unit 2601 also acquires a list of color scheme patterns and fonts from the HDD 104. The combination generation unit 2601 also acquires the number of images from the poster creation condition designation unit 201, image data from the image analysis unit 212, and character information of the title from the text designation unit 202. The combination generation unit 2601 generates a combination of poster components (skeleton, color scheme pattern, font, image, title) to be used for poster generation. The combination generation unit 2601 outputs the generated combination of poster components to the layout unit 217. The generation of combinations will be described later.

[0208] The poster selection unit 2602 selects posters from the multiple posters generated from the multiple combinations generated by the combination generation unit 2601, where the distance between the estimated impression of the poster and the target impression specified by the target impression specification unit 204 is equal to or less than a threshold, and saves these in RAM 103. The poster selection unit 2602 also determines whether the number of selected and saved posters has reached the number specified in the number of posters to be created box 714 in Figure 7, and repeats generating combinations and selecting posters until the number of posters to be created is reached. The poster selection unit 219 outputs the selected poster data to the poster display unit 205.

[0209] <Poster generation process in impression-priority mode> FIG. 27 is a flowchart showing the poster generation process executed when the impression-priority mode is set in the poster creation application of the second embodiment. The process shown in this flowchart is started when the impression-priority mode is selected (S1405; YES) in the operation mode switching process (FIG. 14). In FIG. 27, the processes denoted with the same reference numerals as in FIG. 15 (poster generation process in impression-priority mode of the first embodiment) are the same as those in the first embodiment, and therefore will not be described. In addition, in the poster generation process in impression-priority mode of the second embodiment shown in this flowchart, S1504 (content selection process) and S1505 (determination of number of selections) shown in FIG. 15 are omitted. In addition, after the skeleton is acquired in S1507, S1508 to S1510 (skeleton selection, color scheme selection, font selection) are omitted, and S2701 (content table generation) and S2702 (combination generation) are added. Also, instead of S1513 (select poster), S2703 (select poster) and S2704 (determine whether generation is complete) are added. The following mainly describes the differences from the first embodiment.

[0210] In S1501 to S1507, the target impression, text, and images are specified on the generation condition setting screen 701, and the poster creation conditions (size, number of creations, number of images, and use category) are specified. The image acquisition unit 211 acquires the specified image data, and the image analysis unit 212 analyzes the image data to obtain image features. Furthermore, the skeleton acquisition unit 213 acquires a skeleton that meets the various setting conditions from the HDD 104.

[0211] In S2701, the combination generation unit 2601 generates each table related to the content (images and titles) to be used in generating a poster. Fig. 28(a) shows a list of skeletons that the combination generation unit 2601 has acquired from the skeleton acquisition unit 213. Figs. 28(b) and 28(c) show a list of fonts and a list of color schemes, respectively, that the combination generation unit 2601 has acquired from the HDD 104. Fig. 28(d) shows a list of image data that the combination generation unit 2601 has acquired from the image analysis unit 212. The combination generation unit 2601 generates a list of image data acquired from the image analysis unit 212. Fig. 28(e) shows a list of character information of titles that the combination generation unit 2601 has acquired from the text designation unit 202. The combination generation unit 2601 generates a list of character information of titles acquired from the text designation unit 202.

[0212] <Combination generation process> The processes of S2702, S1511, S1512, S2703 and S2704 are repeated. The processes of S2702 to S2704 will be explained separately for the first execution and the second and subsequent loops.

[0213] When S2702 is executed for the first time, the combination generation unit 2601 obtains the skeleton, color scheme, and font tables shown in Figs. 28(a) to 28(c) used to generate a poster. The combination generation unit 2601 generates random combinations from five tables, including the above three tables, the image table (Fig. 28(d)) generated in S2701, and the title table (Fig. 28(e)). In this embodiment, 100 combinations are generated. Fig. 28(f) shows the combination table generated in this embodiment.

[0214] The combination generation unit 2601 executes layout (S1511), poster impression estimation (S1512), and poster selection (S2703) processes for all the generated combinations. As in the first embodiment, poster impression estimation can be performed by running a deep learning model for poster impression estimation, which is loaded in RAM 103, on the CPU 101 or GPU 109. Note that, instead of deep learning, when using a machine learning method such as a decision tree, feature amounts such as the average brightness or edge amount of the poster image may be extracted by image analysis, and a machine learning model that estimates an impression based on the feature amounts may be created.

[0215] In the second and subsequent loops of S2702, the combination generation unit 2601 calculates the distance between the estimated poster impression obtained from the poster impression estimation unit 218 and the target impression, and associates the calculated distance with each combination in the combination table. FIG. 29 is a diagram explaining the operation of S2702 in the second and subsequent loops. FIG. 29(a) is a table in which the distance value between the estimated poster impression and the target impression is associated with each combination in the combination table shown in FIG. 28(f). More specifically, the layout unit 217 generates a poster based on each combination in the combination table shown in FIG. 28(f), and the poster impression estimation unit 218 estimates an impression for each generated poster. The value shown in the "distance" column in FIG. 29(a) indicates the distance between the estimated impression and the target impression of the poster generated using the combination in the corresponding row.

[0216] The combination generation unit 2601 generates a new combination table from the results shown in FIG. 29(a). FIG. 29(b) shows the newly generated combination table. In the second embodiment, new combinations are generated using tournament selection and uniform crossover in a genetic algorithm. The combination generation unit 2601 first randomly selects N combinations from the table in FIG. 29(a). Here, for example, N=3. Next, from the selected combinations, the top two combinations with the smallest distance (i.e., closest to the target impression) are selected. Finally, new combinations are generated by randomly swapping each element of the combination (skeleton ID, color scheme ID, font ID, image ID, title ID) of the two selected combinations. For example, combinations IDs 1 and 2 in FIG. 29(b) show the results generated from combinations IDs 1 and 3 in FIG. 29(a), and in the example shown, the color scheme ID numbers have been swapped. By repeating the above procedure, 100 new combinations are generated, as shown in FIG. 29(b).

[0217] This allows efficient combination search based on the distance between the target impression and the estimated poster impression. While 100 combinations were generated in this embodiment, the number of combinations to be generated is not limited to this. While tournament selection and uniform crossover were used, other methods, such as ranking selection, roulette selection, and one-point crossover, may also be used. Mutation may be incorporated to prevent the search from reaching a local optimum. While the skeleton (layout), color scheme, font, image, and title were used as the components of the poster to be searched, other components may also be used. For example, multiple patterns may be prepared to be inserted into the poster background, and which patterns to use or not to use may be determined through search. Increasing the number of components to be searched allows for the generation of posters with greater variety, thereby broadening the range of impression expression.

[0218] In S2703, the poster selection unit 2602 calculates the distance value between the poster estimated impression and the target impression for the new combination table in the same way as in S2702. Also, as in Fig. 29(a), a table is created in which the distance values ​​between the poster estimated impression and the target impression are linked to the new combination table. The poster selection unit 2602 saves in RAM 103 poster images whose distance values ​​from the target impression are equal to or less than a threshold.

[0219] In S2704, the poster selection unit 2602 determines whether the number of poster images saved in RAM 103 in S2703 has reached the number of creations specified in the number of creations box 714 in Figure 7. If it is determined that the number of creations has been reached, the process proceeds to S1514. If it is determined that the number of creations has not been reached, the process returns to S2702. In other words, the process of S2702 in the second loop described above is executed, and the processes of S2702, S1511, S1512, and S2703 are repeatedly executed until the number of poster images for which the value of the distance between the poster estimated impression and the target impression is equal to or less than the threshold reaches the specified number of creations.

[0220] If there are poster images with a distance value between the estimated poster impression and the target impression that is equal to or less than a threshold value, equal to or greater than the specified number of poster images, the poster selection unit 2602 compares the distance values ​​of the respective poster images. The poster image with the smaller distance value, i.e., the poster image closest to the target impression, may be stored in RAM 103. Furthermore, if a poster image once stored in RAM 103 has a larger distance value than a poster image selected thereafter, the poster selection unit 2602 may delete the poster image stored in RAM 103 from RAM 103. The poster image selected thereafter may then be stored in RAM 103 in place of the deleted poster image.

[0221] In this embodiment, a search for combinations of poster components is performed using a genetic algorithm, but the search method is not limited to this, and other search methods such as a neighborhood search method or a Tabu search method may also be used.

[0222] As described above, the poster creation application of the second embodiment accepts a user's designation of a target impression. The poster creation application also automatically generates a combination of content (images and titles), skeletons, fonts, and color schemes to be used in a poster. That is, the application generates multiple poster data by combining content included in a content group with the components of the poster. The poster selection unit 2602 then selects a combination that creates an overall poster impression similar to the target impression. This allows a poster to be generated using content that creates an overall poster impression similar to the target impression designated by the user. In other words, a poster generated by the poster creation application when a first target impression designation is accepted differs from a poster generated by the poster creation application when a second target impression designation different from the first target impression is accepted. Furthermore, the content used in these posters is not necessarily the same; different content may be used. Therefore, the content included in the poster selected based on the first target impression may differ from the content included in the poster selected based on the second target impression. The user can select and output a poster with a desired design from the posters selected by the poster creation application. Therefore, according to the poster creation application of the second embodiment, even a user who is not clear about the content he or she wants to use can easily obtain a product with the design he or she wants.

[0223] <<Third Embodiment>> In the first embodiment, an example was described in which, in impression-priority mode, the poster creation application selects content suitable for the target impression from a content group based on the target impression specified by the user, and generates a poster. In the second embodiment, the poster creation application creates a large number of different combinations of candidate content, and selects from among them a poster whose overall impression is closest to the target impression. This generates a poster including content suitable for the target impression specified by the user. In these examples, the poster creation application automatically selects content suitable for the target impression from a content group including multiple candidate content. Therefore, there is a possibility that the content the user wants to use will not ultimately be used in the poster.

[0224] Therefore, in the third embodiment, as in the first and second embodiments, the poster creation application first selects multiple pieces of content suitable for the target impression from a content group based on the target impression specified by the user, and sets these as content candidates.The poster creation application then presents the selected content candidates to the user and accepts the user's specification of the content to actually use.As a result, the poster creation application presents content candidates suitable for the target impression narrowed down from the content group, and the user can reliably select the desired content from among them to generate a poster.

[0225] <Software block diagram> FIG. 30 is a software block diagram of the poster creation application of the third embodiment when the impression-priority mode is set. As shown in FIG. 30, the poster creation application includes a poster creation condition specification unit 201, a text specification unit 3003, an image specification unit 3002, a target impression specification unit 204, a poster display unit 205, and a poster generation unit 3000. The poster generation unit 3000 of the third embodiment includes an image acquisition unit 3004, an image analysis unit 212, a skeleton acquisition unit 3005, a skeleton selection unit 214, a color scheme pattern selection unit 215, a font selection unit 216, a layout unit 217, a poster impression estimation unit 218, a poster selection unit 219, and a content selection unit 3001. In FIG. 30, components designated by the same reference numerals as those in FIG. 2 are the same as those in the first embodiment, and therefore will not be described again. The operation mode specification unit 230 is also omitted.

[0226] The content selection unit 3001 acquires the number of images from the poster creation condition designation unit 201. It also acquires one or more candidate character information items from the text designation unit 3003. It also acquires one or more candidate image data items (image group) from the image acquisition unit 3004. It also acquires the designated target impression from the target impression designation unit 204. The content selection unit 3001 then selects candidate image data items to be used for the poster from the image group based on the target impression and the number of images. It also selects candidate character information items to be used for the poster from the candidate character information items based on the target impression. The content selection unit 3001 outputs the selected candidate image data items to the image designation unit 3002 and the selected candidate character information items to the text designation unit 3003.

[0227] In addition to the processing described in the image designation unit 203 of the first embodiment, the image designation unit 3002 accepts designation by the user of image data to be used for a poster from the image data candidates selected by the content selection unit 3001. The image designation unit 3002 outputs the file path of the designated image to the image acquisition unit 3004.

[0228] In addition to the processing described in the text specification unit 202 of the first embodiment, the text specification unit 3003 accepts specification by the user of character information to be used for the poster from the character information candidates selected by the content selection unit 3001. The text specification unit 3003 associates each piece of character information so that it can be determined as to what type of information it is, such as a title, date and time, or location, and then outputs the information to the skeleton acquisition unit 3005 and the layout unit 217.

[0229] In addition to the processing described in the image acquisition unit 211 of the first embodiment, the image acquisition unit 3004 outputs to the image analysis unit 212 the image data to be used for the poster specified by the user in the image designation unit 3002 from the selected image data candidates.

[0230] The skeleton acquisition unit 3005 acquires, from the HDD 104, one or more skeletons that meet the conditions specified in the poster creation condition specification unit 201 and the text specification unit 3003. The skeleton acquisition unit 3005 outputs the one or more skeletons acquired from the HDD 104 to the skeleton selection unit 214.

[0231] <Example of display screen> 31 is a diagram showing an example of a content specification screen 3101 provided by a poster creation application. The content specification screen 3101 is displayed on the display 105 after the OK button 717 on the generation condition setting screen 701 shown in FIG. 7 is pressed and processing by the content selection unit 3001, which will be described later, is performed. The user specifies an image and title to be used in the poster via the content specification screen 3101, and the image specification unit 3002 and text specification unit 3003 acquire the settings from the user via this UI screen.

[0232] Image designation area 3102 is an area where candidate images to be used for the poster are displayed. Image 3103 represents a thumbnail of an image selected by content selection unit 3001. Image 3103 can be selected by the user by clicking on it with pointing device 107. In FIG. 31, an image surrounded by a selection frame 3104 indicates that the image has been selected by the user. Selection frame 3104 is displayed around the thumbnail selected by the user, and if the user selects the same thumbnail again in this state, selection frame 3104 is hidden and the selection is cancelled. Note that multiple images 3103 to be used for the poster can be selected.

[0233] Title designation area 3105 is an area where candidate titles to be used for the poster are displayed. Title box 3106 displays the title selected by content selection unit 3001. Title box 3106 can be selected by the user clicking on it with pointing device 107. In FIG. 31, a title box surrounded by selection frame 3107 indicates that the title box has been selected by the user. Selection frame 3107 is displayed around the title box selected by the user, and if the same title box is selected again in this state, selection frame 3107 is hidden and the selected state is cancelled. In this way, content designation screen 3101 displays candidate content (multiple images and multiple pieces of character information) selected by content selection unit 3001 in a selectable manner.

[0234] Back button 3108 is a button for canceling the details specified on content specification screen 3101 and returning to generation condition setting screen 701. When back button 3108 is pressed, each setting information on content specification screen 3101 is ignored, and the screen displayed on display 105 transitions to generation condition setting screen 701.

[0235] When the user presses OK button 3109, image designation unit 3002 and text designation unit 3003 output the content designated on content designation screen 3101 to poster generation unit 3000. Image designation unit 3002 acquires the file path of the image selected in image designation area 3102. Text designation unit 3003 acquires text information of the title selected in title designation area 3105. If the number of images designated in image designation area 3102 does not match the number designated in image count box 720 on generation condition setting screen 701, image designation unit 3002 displays a dialog screen informing the user of the mismatch. Then, image designation unit 3002 displays content designation screen 3101 again, prompting the user to reselect an image. If the number of titles designated in title designation area 3105 is greater than one, text designation unit 3003 displays a dialog screen informing the user that the number is large. Then, content designation screen 3101 is displayed again, prompting the user to reselect a title.

[0236] <Poster generation process> Fig. 32 is a flowchart showing the poster generation process executed when the impression-priority mode is set in the poster creation application of the third embodiment. The process shown in this flowchart is started when the impression-priority mode is selected (S1405; YES) in the operation mode switching process (Fig. 14). In Fig. 32, the processes denoted with the same reference numerals as in Fig. 15 (poster generation process in impression-priority mode of the first embodiment) are the same as those in the first embodiment, and therefore description thereof will be omitted. In addition, in the poster generation process in impression-priority mode of the third embodiment shown in this flowchart, S3201 to S3203 are executed instead of S1504 shown in Fig. 15. The following description will focus on the differences from the first embodiment.

[0237] In S1501 to S1507, the target impression, text candidates, and image group are specified on the generation condition setting screen 701, and the poster creation conditions (size, number of images to be created, number of images, and use category) are specified. The image acquisition unit 3004 acquires image data of the specified image group, and the image analysis unit 212 analyzes the image data to obtain image features. The skeleton acquisition unit 213 acquires a skeleton that meets the various setting conditions from the HDD 104. It is assumed that an image group containing multiple images, such as an image folder or all of the material images, is selected as the image, and a text group containing multiple pieces of text is specified as the text. In this embodiment, the character information of a title will be described as an example of text.

[0238] In S3201, the content selection unit 3001 selects multiple image data items that are candidates for images to be used for a poster from the group of images acquired in S1503. The content selection unit 3001 also selects multiple pieces of text information that are candidates for titles to be used for a poster from the text information of the multiple titles acquired in S1502.

[0239] <Content selection process> The content selection process executed in S3201 will now be described in detail with reference to Fig. 33. Fig. 33 is a flowchart for explaining S3201 in detail, and is executed by the content selection unit 3001 of the third embodiment. Note that in the flowchart of Fig. 33, the processes denoted with the same reference numerals as in Fig. 16 are similar to the content selection process of the first embodiment, and therefore description thereof will be omitted.

[0240] In S3301, the content selection unit 3001 selects the top N titles determined in S1603 based on the smallest distance between the estimated impression and the target impression for each candidate title. Here, the selection number N must be a value equal to or greater than the number of titles to be used in the poster. In this embodiment, the number of titles to be used in the poster is 1, so the selection number N of titles may be 1 or greater. As an example, the content selection unit 3001 selects the top four titles. Note that if the number of titles specified on the generation condition setting screen 701 in S1501 is less than four, the selection number may be less than four. The third embodiment differs from the content selection unit 220 in the first embodiment in that, whereas in the first embodiment, the number of titles selected was one, the same as the number of titles to be used in the poster, in the third embodiment, a number of titles equal to or greater than the number of titles to be used in the poster are selected as candidates.

[0241] The selection number N may be set to a fixed value or may be specified by the user. For example, a box (not shown) for specifying the number of titles to be candidates (number of candidates) may be provided on the generation condition setting screen 701, and the specified number of candidates may be used as the selection number N of titles in S3301.

[0242] In S3302, the content selection unit 3001 selects the top N image data having the smallest distance between the estimated impression and the target impression for each candidate image data calculated in S1608. Here, the selection number N needs to be a value equal to or greater than the number of images to be used in the poster. In the present embodiment, as an example, the content selection unit 3001 selects image data that is twice the number of images specified in the poster creation condition specification unit 201. Note that the method for setting the selection number N of images is not limited to this, and any number greater than or equal to the number of images to be used in the poster may be used. The difference from the content selection unit 220 in the first embodiment is that, whereas in the first embodiment the number of image data selected was the same as the number of images specified in the poster creation condition specification unit 201, in the third embodiment, image data equal to or greater than the specified number of images is selected as candidates.

[0243] 33 results in a state where a plurality of titles close to the target impression are selected from a group of titles designated by the user on the generation condition setting screen 701. Also, a plurality of image data close to the target impression are selected from a group of images designated by the user on the generation condition setting screen 701 (image designation screen 801).

[0244] Returning to the explanation of Fig. 32, in S3202, the poster creation application acquires the content (plurality of titles, pluralities of image data) selected in S3201. Then, a content selection screen 3101 showing these plural titles and pluralities of image data as candidates is displayed on the display 105. The user inputs each setting via the UI screen of the content selection screen 3101 using the keyboard 106 or pointing device 107.

[0245] In S3203, the image designation unit 3002 acquires image data designated by the user on the content designation screen 3101. Furthermore, the text designation unit 3003 acquires character information of the title designated by the user on the content designation screen 3101. Thereafter, similarly to the first embodiment, the processes from S1505 onwards are executed.

[0246] As described above, the poster creation application of the third embodiment acquires a content group designated by the user, selects from the group a plurality of content candidates suitable for the target impression, and displays the content candidates on the content designation screen 3101. The content designation screen 3101 also accepts designation of content to be used in the poster from the user. This allows the user to designate desired content from among several content candidates suitable for the target impression. Furthermore, a poster can be obtained in which the content designated by the user is reliably used.

[0247] Therefore, even if a user is not sure what content they want to use, the poster creation application will select several content candidates from which the user can choose the content they want to use, making it easier to create a finished product with the design they want.

[0248] <<Fourth Embodiment>> In the first and second embodiments, an example was described in which a poster creation application selects content to be used in a poster from a content group based on a target impression specified by a user, and generates a poster. In the third embodiment, an example was described in which a poster creation application selects and presents multiple content candidates to be used in a poster from a content group specified by the user, based on a target impression specified by the user. An example was also described in which a poster is generated after the user selects the content to actually use from the presented content candidates.

[0249] In the fourth embodiment, in the impression-priority mode, when acquiring image data from an image group based on a target impression specified by a user, the poster creation application extracts images suitable for the target impression in advance and limits the images that the user can specify according to the target impression. Also, an example will be described in which the poster creation application accepts a title (text information) input by the user and displays whether the title entered by the user is suitable for the target impression based on the target impression specified by the user. This allows the user to select the image they want to use from among candidate images suitable for the target impression simply by specifying the target impression. Furthermore, when entering a title, they can know whether the title is suitable for the target impression. This makes it possible to generate a poster that is suitable for the target impression more efficiently.

[0250] <Software block diagram> FIG. 34 is a software block diagram of the poster creation application of the fourth embodiment when the impression-priority mode is set. As shown in FIG. 34, the poster creation application includes a poster creation condition specification unit 201, a text specification unit 3403, an image specification unit 3402, a target impression specification unit 204, a poster display unit 205, and a poster generation unit 3400. The poster generation unit 3400 includes an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a skeleton selection unit 214, a color scheme pattern selection unit 215, a font selection unit 216, a layout unit 217, a poster impression estimation unit 218, a poster selection unit 219, and a content extraction unit 3401. In FIG. 34, components designated by the same reference numerals as those in FIG. 2 are the same as those in the first embodiment, and therefore will not be described again. The operation mode specification unit 230 is also omitted.

[0251] When acquiring a group of images designated by the user in the image designation unit 3402, the content extraction unit 3401 extracts images from the group of images based on the target impression acquired from the target impression designation unit 204 and then acquires the images. As a result, the images that the user can designate in the image designation unit 3402 are narrowed down based on the target impression. Furthermore, based on the target impression acquired from the target impression designation unit 204, the content extraction unit 3401 indicates to the user whether the title entered by the user is suitable for the target impression in the title designation process executed by the text designation unit 3403. The image determination by the content extraction unit 3401 and the title designation process will be described later.

[0252] The image designation unit 3402 presents the images determined by the content extraction unit 3401 to the user as candidates for images to be used for the poster, and accepts the user's designation of an image from among the candidate images. The image designation unit 3402 outputs the file path of the image designated by the user to the image acquisition unit 211.

[0253] The text specification unit 3403 receives from the user specification of character information to be used for the poster. At that time, the text specification unit 3403 indicates to the user whether the title entered by the user is suitable for the target impression. The text specification unit 3403 outputs the character information specified by the user to the skeleton acquisition unit 213 and the layout unit 217.

[0254] <Example of display screen> FIG. 35(a) shows an example of a generation condition setting screen 3505 in the fourth embodiment, and FIG. 35(b) shows an example of a content setting screen 3501 in the fourth embodiment. The generation condition setting screen 3505 and the content setting screen 3501 are displayed on the display 105. The user specifies the target impression and creation conditions of the poster (described later) via the generation condition setting screen 3505. The user also accepts designation of text and images to be used in the poster via the content setting screen 3501. The poster creation condition designation unit 201, image designation unit 3402, and text designation unit 3403 acquire settings from the user via these UI screens. In FIG. 35, components designated with the same reference numerals as those in FIG. 7 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0255] Reset button 3506 on generation condition setting screen 3505 is a button for resetting the information set on generation condition setting screen 3505. When the user presses Next button 3507, poster creation condition specification unit 201 and target impression specification unit 204 output the settings set on generation condition setting screen 3505 to poster generation unit 3400.

[0256] Back button 3502 on content setting screen 3501 is a button for returning to generation condition setting screen 3505. Reset button 3503 is a button for resetting each setting information on content setting screen 3501. When the user presses OK button 3504, text designation unit 3403 and image designation unit 3402 output the contents set on content setting screen 3501 to poster generation unit 3400.

[0257] <Poster generation process> Fig. 36 is a flowchart showing the poster generation process executed when the impression-priority mode is set in the poster creation application of the fourth embodiment. The process shown in this flowchart is started when the impression-priority mode is selected (S1405; YES) in the operation mode switching process (Fig. 14). In Fig. 36, the processes denoted with the same reference numerals as in Fig. 15 (poster generation process in impression-priority mode of the first embodiment) are the same as those in the first embodiment, and therefore description thereof will be omitted. In the poster generation process in impression-priority mode of the fourth embodiment shown in this flowchart, S3601 to S3605 are executed instead of S1501 to S1505 shown in Fig. 15. The following description will focus on the differences from the first embodiment.

[0258] In S3601, the poster creation application first displays a generation condition setting screen 3505 shown in Fig. 35(a) on the display 105. The user inputs each setting through the UI screen of the generation condition setting screen 3505 using the keyboard 106 or pointing device 107. When the user presses the Next button 3507, the process proceeds to S3602.

[0259] In S3602, the poster creation condition specification unit 201 and the target impression specification unit 204 acquire corresponding settings from the creation condition setting screen 3505. Specifically, the poster creation condition specification unit 201 acquires the user's specifications of poster size, number of posters to be created, number of images, and usage category. The target impression specification unit 204 acquires the user's specification of the target impression.

[0260] In S3603, the poster creation application displays a content setting screen 3501 on the display 105. The user inputs each setting via the UI screen of the content setting screen 3501 using the keyboard 106 or pointing device 107. When the user presses the add image button 707 on the content setting screen 3501, the process proceeds to S3604, and the processing shown in Fig. 38(a) is executed. Also, when a title is input in the title box 702 on the content setting screen 3501, the processing shown in Fig. 38(b) is executed in S3604.

[0261] In S3604, the content extraction unit 3401 determines an image that can be designated by the image designation unit 3402 based on the target impression acquired by the target impression designation unit 204 in S3602 (processing in FIG. 38(a)). Also, the content extraction unit 3401 performs processing (processing in FIG. 38(b)) to be described later regarding the designation of a title based on the target impression acquired by the target impression designation unit 204 in S3602.

[0262] <Content extraction process> The content extraction process of S3604 and content extraction unit 3401 will be described in detail with reference to Figures 37 and 38. Figure 37 is an example of a software block diagram illustrating content extraction unit 3401 in detail. Content extraction unit 3401 has content acquisition unit 3701, content impression estimation unit 3702, content evaluation unit 3703, and content adoption unit 3704. Figure 38 is a flowchart for describing the processing of S3604 in detail.

[0263] The processing flow shown in Fig. 38(a) is executed when the user specifies an image on the image specification screen 801 (Fig. 8), which is transitioned to when the Add Image button 707 on the content setting screen 3501 displayed in S3603 is pressed. Specifically, this processing flow is executed when the Browse Image File button 809, the Browse Image Folder button 811, the Browse In-App Material (Part) button 813, or the Browse External Linkage Material button 815 is pressed. Alternatively, this processing flow is executed when the OK button 818 is pressed with the In-App Material (All) radio button 804 and the AI ​​Image Generation radio button 807 specified.

[0264] In S3801, the content acquisition unit 3701 acquires image data associated with a reference button or a specified radio button on the image designation screen 801, based on the reference button pressed or the specified radio button. Specifically, when the reference buttons 809 and 811 are pressed, the content acquisition unit 3701 acquires an image owned by the user stored on the HDD 104. When the reference button 813 is pressed or when the OK button 818 is pressed with the radio button 804 specified, the content acquisition unit 3701 acquires an application material image stored on the HDD 104. When the reference button 815 is pressed, the content acquisition unit 3701 acquires a linked material image via a network or a server. When the OK button 818 is pressed with the radio button 807 specified, the content acquisition unit 3701 acquires an AI-generated image generated by an image generation AI. The image generation AI may be provided in the poster generation device, or may be provided in another information processing device accessible from the poster generation device, and acquire the AI-generated image via a network.

[0265] In S3802, the content impression estimation unit 3702 estimates the impressions of each of the multiple image data acquired in S3801 using the trained model generated by the content impression quantification process shown in FIG. 9(a).

[0266] In S3803, the content evaluation unit 3703 determines the distance between the target impression acquired in S3602 and the estimated impression of each image estimated in S3802.

[0267] In S3804, the process starts repeating S3805 and S3806 for each of all images for which the distance has been determined in S3803.

[0268] In S3805, the content evaluation unit 3703 determines whether the distance determined in S3803 for the target image is equal to or less than a predetermined threshold. If the content evaluation unit 3703 determines that the distance determined in S3803 is equal to or less than the predetermined threshold, the process proceeds to S3806. If the content evaluation unit 3703 determines that the distance determined in S3803 is greater than the predetermined threshold, the process skips S3806 and loops the processes of S3805 and S3806 for the next image.

[0269] In S3806, the content adoption unit 3704 adopts the target image as a candidate image. If the processing of this flowchart is started by pressing any of the browse buttons 809, 811, 813, and 815, the adopted candidate image is displayed on a dialog screen for the user to specify. Note that in this embodiment, images not adopted are not displayed on the dialog screen for the user to specify, but this is not limited to this. For example, the unadopted image may be displayed on the dialog screen in a state where it cannot be selected. Alternatively, the unadopted image may be displayed on the dialog screen in a state where it can be selected, and if selected by the user, a warning screen may be displayed indicating that the image is not suitable for the specified target impression. On the other hand, if the processing of this flowchart is started by pressing the OK button 818 while the radio button 804 or 807 is selected, the adopted candidate image is treated as the image specified by the user, and processing from S3605 onwards in FIG. 36 is carried out. When the processing of S3805 and S3806 is completed for all images whose distances have been determined in S3803, the processing of this flowchart ends.

[0270] Next, the processing flow shown in Fig. 38(b) will be described. This processing flow is executed when a title is entered on content setting screen 3501 displayed in S3603. Specifically, after the user enters a title in title box 702 on content setting screen 3501, the processing is executed with the areas other than title box 702 active.

[0271] In S3810, the content acquisition unit 3701 acquires the text information entered in the title box 702.

[0272] In S3811, the content impression estimation unit 3702 estimates the impression of the character information acquired in S3810 using the trained model generated by the content impression quantification process shown in FIG. 9(b).

[0273] In S3812, the content evaluation unit 3703 determines the distance between the target impression acquired in S3602 and the impression of the text information estimated in S3811.

[0274] In S3813, the content evaluation unit 3703 determines whether the distance value determined in S3812 is greater than a predetermined threshold. If the content evaluation unit 3703 determines that the distance value determined in S3812 is greater than the predetermined threshold, it transitions to S3814. If not, it ends this flowchart.

[0275] In S3814, the content adoption unit 3704 displays a warning screen indicating that the character information entered in the title box 702 is not suitable for the target impression. The above is the content extraction process executed in S3604 in Fig. 36. Return to Fig. 36.

[0276] In S3605, the image designation unit 3402 and the text designation unit 3403 acquire the corresponding content from the content setting screen 3501. Specifically, the image designation unit 3402 accepts an image designation by the user on a dialog screen that displays the images adopted by the processing of FIG. 38(a) as image candidates. The image acquisition unit 211 acquires the designated image data. The text designation unit 3403 acquires character information that was input by the user by the processing of FIG. 38(b) and for which no warning screen was displayed. The subsequent processing (S1505 to S1514) is the same as in the first embodiment.

[0277] As described above, when acquiring a group of images specified by a user, the poster creation application of the fourth embodiment pre-extracts images suitable for the target impression from the group of images based on the target impression specified by the user. This limits the images that the user can specify. Furthermore, the poster creation application of the fourth embodiment determines whether the title entered by the user is suitable for the target impression specified by the user and notifies the user. This allows the user to efficiently specify content suitable for the target impression. This improves usability when the user specifies content to be placed on a poster. Furthermore, it becomes possible to obtain a poster with the intended design with fewer attempts, allowing posters suitable for the target impression to be generated efficiently. This makes it easier for users who are not clear about the content they want to use to obtain a product with the intended design.

[0278] Although preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. For example, in the second embodiment, content may be selected or extracted as in the third or fourth embodiment, and then a poster may be generated by combining content candidates with components such as a skeleton, color scheme, and font. Furthermore, in the second to fourth embodiments, a radar chart or expression information such as a media file or sample image may be used as an object for specifying a target impression. Furthermore, in the second to fourth embodiments, content or a poster may be selected based on the similarity to the expression information specified by the user in addition to the distance from the target impression. It is clear that a person skilled in the art could conceive of various modifications or alterations within the scope of the disclosed technical ideas, and it is understood that such modifications also fall within the technical scope of the present disclosure.

[0279] <<Other embodiments>> The above-described embodiments can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and a computer (CPU, MPU, etc.) of the system or device reads and executes the programs. The programs may be executed by a single computer or by multiple computers in cooperation with each other. Furthermore, all of the above-described processes do not need to be implemented by software; some or all of the processes may be implemented by hardware such as an ASIC. Furthermore, the CPU is not limited to a single CPU that performs all processes; multiple CPUs may perform processes in cooperation with each other as appropriate. The functions of the above-described embodiments are not necessarily realized by a computer reading and executing program code. This also includes cases where an operating system or the like running on a computer performs some or all of the actual processes based on the instructions of the program code, thereby realizing the functions of the above-described embodiments.

[0280] The disclosure of the above-described embodiment includes the following configurations.

[0281] (Configuration 1) An information processing device that generates data of a production, a receiving means for receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection means for selecting content to be placed in the production from a content group based on the desired impression received by the reception means, An information processing device characterized in that the first content selected by the selection means when the receiving means receives a designation of a first target impression is different from the second content selected by the selection means when the receiving means receives a designation of a second target impression different from the first target impression.

[0282] (Configuration 2) The information processing device according to configuration 1, wherein the selection means selects content to be placed in the production based on the target impression received by the reception means and an impression estimated from each piece of content included in the content group.

[0283] (Configuration 3) 3. The information processing device according to configuration 2, wherein a value indicating a difference between the target impression received by the receiving means and an impression estimated from the content selected by the selecting means is smaller than a predetermined threshold value.

[0284] (Configuration 4) The information processing device described in configuration 3 is characterized in that the selection means selects, as the first content, content from the content group in ascending order of the value indicating the difference between the first target impression and the impression estimated from the first content, the value indicating the difference being smaller than the threshold, and selects, as the second content, content from the content group in ascending order of the value indicating the difference, the value indicating the difference being smaller than the threshold.

[0285] (Configuration 5) 5. The information processing device according to any one of configurations 1 to 4, further comprising a generating unit that generates data of the production using the content selected by the selecting unit.

[0286] (Configuration 6) The information processing device according to configuration 5, characterized in that a value indicating a difference between the impression estimated from the data of the work generated by the generating means and the target impression received by the receiving means is smaller than a predetermined threshold value.

[0287] (Configuration 7) a generating unit for generating data of a plurality of said productions in which said content included in said content group is arranged, The information processing device according to any one of configurations 1 to 4, characterized in that the selection means selects the content to be placed in the work from the data of the plurality of works generated by the generation means based on the target impression accepted by the acceptance means.

[0288] (Configuration 8) 8. The information processing device according to configuration 7, wherein the generating means generates data of a plurality of said products by combining components of said products with said content included in said content group.

[0289] (Configuration 9) the receiving means receives a user's designation of expression information that expresses the content; The information processing device according to any one of configurations 1 to 8, wherein the selection means selects the content to be placed in the production from the content group based on the target impression and the expression information received by the reception means.

[0290] (Configuration 10) 10. The information processing device according to configuration 9, wherein the selection means selects the content from the content group based on a similarity between the content included in the content group and the expression information.

[0291] (Configuration 11) The accepting means further accepts a designation by the user of the number of the contents to be arranged in the production; 11. The information processing device according to any one of configurations 1 to 10, wherein the selection means selects from the content group a number of contents equal to or greater than the number accepted by the acceptance means.

[0292] (Configuration 12) 12. The information processing apparatus according to configuration 11, further comprising display means for displaying on a screen the plurality of contents selected by the selection means as candidates that can be designated by the user.

[0293] (Configuration 13) 13. The information processing apparatus according to configuration 12, wherein the selection means further accepts a selection by the user of the content to be used in the production from the candidates displayed on the screen.

[0294] (Configuration 14) The receiving means further receives designation by the user of information for acquiring the content group; 14. The information processing device according to any one of configurations 1 to 13, further comprising an acquisition unit that acquires the content group based on the information for acquiring the content group accepted by the acceptance unit.

[0295] (Configuration 15) 15. The information processing apparatus according to configuration 14, wherein the accepting means displays a screen for accepting designation by the user of information for acquiring the content group.

[0296] (Configuration 16) 15. The information processing apparatus according to configuration 14, wherein the acquiring means extracts and acquires the content accepted by the accepting means when acquiring the content group.

[0297] (Configuration 17) 15. The information processing apparatus according to configuration 14, wherein the content group is acquired by the acquisition means from a storage area designated by the user in the acceptance means.

[0298] (Configuration 18) The content included in the content group has attribute information, 15. The information processing apparatus according to configuration 14, wherein the content group is acquired by the acquisition means based on the attribute information designated by the user at the reception means.

[0299] (Configuration 19) 15. The information processing device according to configuration 14, wherein the content group includes material content created in advance for the production.

[0300] (Configuration 20) 15. The information processing device according to configuration 14, wherein the content group includes content generated by a generation AI based on a prompt specified by the user in the reception means.

[0301] (Configuration 21) 21. The information processing device according to any one of configurations 1 to 20, wherein the content to be arranged in the production includes at least one of an image and text information.

[0302] (Configuration 22) 22. The information processing device according to any one of configurations 1 to 21, wherein the accepting means accepts the user's designation of the target impression as information expressing an impression.

[0303] (Configuration 23) 23. The information processing device according to claim 22, wherein the information expressing the impression is a media file including at least one of an image, a video, and music.

[0304] (Configuration 24) 24. The information processing device according to any one of configurations 1 to 23, wherein the product is a poster.

[0305] (Configuration 25) An information processing method for generating data of a production, comprising: a receiving step of receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection step of selecting content to be placed in the production from a content group based on the target impression received in the reception step, An information processing method characterized in that the first content selected in the selection step when a designation of a first target impression is accepted in the acceptance step is different from the second content selected in the selection step when a designation of a second target impression different from the first target impression is accepted in the acceptance step.

[0306] (Configuration 26) A program for causing a computer to execute an information processing method for generating data of a production, The information processing method includes: a receiving step of receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection step of selecting content to be placed in the production from a content group based on the target impression received in the reception step, The program is characterized in that the first content selected in the selection step when a designation of a first target impression is accepted in the acceptance step is different from the second content selected in the selection step when a designation of a second target impression different from the first target impression is accepted in the acceptance step.

Claims

1. An information processing device that generates data of a production, a receiving means for receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection means for selecting content to be placed in the production from a content group based on the desired impression received by the reception means, An information processing device characterized in that the first content selected by the selection means when the receiving means receives a designation of a first target impression is different from the second content selected by the selection means when the receiving means receives a designation of a second target impression different from the first target impression.

2. The information processing device according to claim 1 , wherein the selection means selects content to be placed in the production based on the target impression received by the reception means and an impression estimated from each content included in the content group.

3. 3. The information processing apparatus according to claim 2, wherein a value indicating a difference between the target impression received by the receiving means and the impression estimated from the content selected by the selecting means is smaller than a predetermined threshold value.

4. 4. The information processing device according to claim 3, wherein the selection means selects, as the first content, from the content group in ascending order of the value indicating the difference between the first target impression and the impression estimated from the first content, the content for which a value indicating the difference between the first target impression and the impression estimated from the first content is smaller than the threshold, and selects, as the second content, from the content group in ascending order of the value indicating the difference, the content for which a value indicating the difference between the second target impression and the impression estimated from the second content is smaller than the threshold.

5. 2. The information processing apparatus according to claim 1, further comprising: a generating unit that generates data of the work using the content selected by the selecting unit.

6. 6. The information processing apparatus according to claim 5, wherein a value indicating a difference between the impression estimated from the data of the product generated by the generating means and the target impression received by the receiving means is smaller than a predetermined threshold value.

7. a generating unit for generating data of a plurality of said productions in which said content included in said content group is arranged, The information processing device according to claim 1, characterized in that the selection means selects the content to be placed in the work by selecting data of the work from the data of the plurality of works generated by the generation means based on the target impression accepted by the acceptance means.

8. 8. The information processing apparatus according to claim 7, wherein said generating means generates data of a plurality of said products by combining components of said products with said content included in said content group.

9. the receiving means receives a user's designation of expression information that expresses the content; 2. The information processing apparatus according to claim 1, wherein the selection means selects the content to be arranged in the product from the content group based on the target impression and the expression information received by the reception means.

10. 10. The information processing apparatus according to claim 9, wherein the selection means selects the content from the content group based on a similarity between the content included in the content group and the expression information.

11. The accepting means further accepts a designation by the user of the number of contents to be arranged in the production; 2. The information processing apparatus according to claim 1, wherein the selection means selects from the content group a number of contents equal to or greater than the number accepted by the acceptance means.

12. 12. The information processing apparatus according to claim 11, further comprising a display unit that displays the plurality of contents selected by the selection unit on a screen as candidates that can be designated by the user.

13. 13. The information processing apparatus according to claim 12, wherein the selection means further accepts a selection by the user of the content to be used in the production from the candidates displayed on the screen.

14. The receiving means further receives designation by the user of information for acquiring the content group; 2. The information processing apparatus according to claim 1, further comprising: an acquisition unit that acquires the content group based on the information for acquiring the content group received by the reception unit.

15. 15. The information processing apparatus according to claim 14, wherein the accepting unit displays a screen for accepting designation by the user of information for acquiring the content group.

16. 15. The information processing apparatus according to claim 14, wherein the acquiring means extracts and acquires the content accepted by the accepting means when acquiring the content group.

17. 15. The information processing apparatus according to claim 14, wherein the content group is acquired by the acquisition means from a storage area designated by the user via the acceptance means.

18. The content included in the content group has attribute information, 15. The information processing apparatus according to claim 14, wherein the content group is acquired by the acquisition means based on the attribute information designated by the user via the reception means.

19. 15. The information processing apparatus according to claim 14, wherein the content group includes material content created in advance for the production.

20. 15. The information processing apparatus according to claim 14, wherein the content group includes content generated by a generation AI based on a prompt specified by the user at the reception means.

21. The information processing apparatus according to claim 1 , wherein the content arranged in the production includes at least one of an image and text information.

22. 2. The information processing apparatus according to claim 1, wherein the accepting unit accepts the user's designation of the desired impression as information expressing an impression.

23. 23. The information processing apparatus according to claim 22, wherein the information expressing the impression is a media file including at least one of an image, a video, and music.

24. 2. The information processing apparatus according to claim 1, wherein the work is a poster.

25. An information processing method for generating data of a production, comprising: a receiving step of receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection step of selecting content to be placed in the production from a content group based on the target impression received in the reception step, An information processing method characterized in that the first content selected in the selection step when a designation of a first target impression is accepted in the acceptance step is different from the second content selected in the selection step when a designation of a second target impression different from the first target impression is accepted in the acceptance step.

26. A program for causing a computer to execute an information processing method for generating data of a production, The information processing method includes: a receiving step of receiving from a user a designation of a target impression, which is an impression that the production is ultimately required to maintain; a selection step of selecting content to be placed in the production from a content group based on the target impression received in the reception step, A program characterized in that a first content selected in the selection step when a designation of a first target impression is accepted in the acceptance step is different from a second content selected in the selection step when a designation of a second target impression different from the first target impression is accepted in the acceptance step.

Images