Information processing device and its control method

Abstract

To allow for appropriately generating a poster expressing a user-intended impression even when performing character decoration.SOLUTION: An information processing device disclosed herein is configured to acquire characters, acknowledge a target impression, determine a character decoration method according to the target impression, and generate a poster based on the determined character decoration method and the characters.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This invention relates to a technology for generating posters. [Background technology]

[0002] Traditionally, a method has been proposed to generate posters by preparing a template that stores information such as the shape and arrangement of images, text, and graphics that make up the poster, and then automatically arranging the images, text, and graphics according to the template.

[0003] Patent Document 1 discloses a technique for ensuring readability while maintaining the impression of text image data by changing the text decoration (hereinafter referred to as text decoration) without changing the text color, in cases where changing the color of the text would alter the impression of the text image data. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-46994 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, Patent Document 1 does not consider how to appropriately generate a poster that expresses the user's intended impression when the text decoration is changed.

[0006] The present invention aims to appropriately generate posters that express the user's intended impression, even when text decoration is applied. [Means for solving the problem]

[0007] To solve the above problems, the information processing apparatus of the present invention comprises a character acquisition means for acquiring characters, a reception means for receiving a target impression, and a poster generation means for generating a poster based on the characters and the target impression, wherein the poster generation means receives the target impression and the readability of the text The method for decorating the characters is determined based on the above, and a poster is generated based on the determined method for decorating the characters and the characters themselves. [Effects of the Invention]

[0008] According to the present invention, even when applying text decoration, it is possible to appropriately generate a poster that expresses the impression intended by the user. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the hardware configuration of an information processing device. [Figure 2] This is a software block diagram for a poster creation application. [Figure 3] This is a diagram explaining the skeleton. [Figure 4] This is a diagram illustrating color scheme patterns. [Figure 5] This figure shows the display screen provided by a poster creation application. [Figure 6] This figure shows the display screen provided by a poster creation application. [Figure 7] This flowchart shows the poster generation process, which involves quantifying the impression given by a poster. [Figure 8] This is a diagram illustrating the subjective evaluation of posters. [Figure 9] This is a flowchart showing the poster generation process. [Figure 10] This is a diagram explaining how to select a skeleton. [Figure 11] This diagram illustrates how to select color schemes and fonts. [Figure 12] This is a software block diagram that explains the layout section in detail. [Figure 13] It is a flowchart showing layout processing. [Figure 14] It is a diagram explaining the input of the layout section. [Figure 15] It is a diagram explaining the operation of the layout section. [Figure 16] It is a diagram showing an example of the target impression specification section. [Figure 17] It is a diagram explaining character decoration. [Figure 18] It is a software block diagram explaining the text decoration section in detail. [Figure 19] It is a flowchart showing text decoration processing. [Figure 20] It is a diagram explaining the operation of the tile division section. [Figure 21] It is a diagram explaining the readability of characters. [Figure 22] It is a diagram explaining the method of selecting a character decoration method. [Figure 23] It is a flowchart showing decoration color determination processing. [Figure 24] It is a flowchart showing decoration width determination processing. [Figure 25] It is a diagram explaining the method of determining the decoration width. [Figure 26] It is a diagram explaining the boldness of characters by font. [Figure 27] It is a diagram explaining the influence of the decoration width on the impression. [Figure 28] It is a diagram explaining the effects of the second embodiment. [Figure 29] It is a software block diagram explaining the color scheme assignment section in detail. [Figure 30] It is a flowchart showing character color determination processing. [Figure 31] It is a flowchart showing layout processing.

Mode for Carrying Out the Invention

[0010] Embodiments of the present invention will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the scope of the present invention as defined in the claims, and not all combinations of features described in these embodiments are necessarily essential to the solution of the present invention. The same reference numeral is used for identical components, and their descriptions are omitted.

[0011] <<First Embodiment>> In this embodiment, a method for automatically generating a poster by running a poster creation application (hereinafter also referred to as "the app") in an information processing device will be described as an example. In the following description, unless otherwise specified, "image" includes still images and frame images extracted from videos.

[0012] Figure 1 is a block diagram showing the hardware configuration of an information processing device. The information processing device 100 can be, for example, a personal computer (hereinafter referred to as a PC) or a smartphone. In this embodiment, the information processing device will be described as a PC. The information processing device 100 includes a CPU 101, ROM 102, RAM 103, HDD 104, display 105, keyboard 106, pointing device 107, and data communication unit 108.

[0013] The CPU (Central Processing Unit / Processor) 101 comprehensively controls the information processing device 100 and, for example, reads a program stored in the ROM 102 into the RAM 103 and executes it, thereby realizing the operation of this embodiment. In Figure 1, there is one CPU, but it may be composed of multiple CPUs. The ROM 102 is a general-purpose ROM and stores, for example, programs executed by the CPU 101. The RAM 103 is a general-purpose RAM and is used, for example, as working memory to temporarily store various information when the CPU 101 executes a program. The HDD (Hard Disk Drive) 104 is a storage medium (storage unit) for storing image files, a database that holds processing results such as image analysis, and skeletons used by poster creation applications.

[0014] The display 105 is a display unit that displays the user interface (UI) and the resulting layout of image data (hereinafter also referred to as "image") to the user. Although not shown, it also has a display control unit for displaying on the display unit. The keyboard 106 and pointing device 107 receive instructions from the user. The display 105 may also have a touch sensor function. The keyboard 106 is used, for example, when the user inputs the number of double-page spreads of the poster they want to create on the UI displayed on the display 105. The pointing device 107 is used, for example, when the user clicks a button on the UI displayed on the display 105.

[0015] The data communication unit 108 communicates with external devices via a wired or wireless network. For example, the data communication unit 108 transmits data laid out by the automatic layout function to a printer or server that can communicate with the information processing device 100. The data bus 109 connects each block in Figure 1 so that they can communicate with each other.

[0016] The configuration shown in Figure 1 is merely an example and is not limited to it. For example, the information processing device 100 does not have a display 105, and the UI may be displayed on an external display.

[0017] In this embodiment, the poster creation application is stored on the HDD 104. As will be described later, the application is launched by the user selecting the application icon displayed on the display 105 using the pointing device 107 and clicking or double-clicking it.

[0018] Figure 2 is a software block diagram of a poster creation application. The poster creation application includes a poster creation condition specification unit 201, an image specification unit 202, a text specification unit (character acquisition unit) 203, a target impression specification unit 204, a poster display unit 205, and a poster generation unit 210. The poster generation unit 210 includes an image acquisition unit 211, an image analysis unit 212, a skeleton acquisition unit 213, a skeleton selection unit 214, a color scheme pattern selection unit 215, a font selection unit 216, a layout unit 217, an impression estimation unit 218, and a poster selection unit 219.

[0019] Each program module corresponding to each component shown in Figure 2 is included in the poster creation application described above. The CPU 101 then executes each program module, causing the CPU 101 to function as each component shown in Figure 2. Hereafter, each component shown in Figure 2 will be described as performing various processes. Figure 2 also specifically shows a software block diagram related to the poster generation unit 210, which performs the automatic poster creation function.

[0020] The poster creation condition specification unit 201 specifies poster creation conditions to the poster generation unit 210 in response to UI operations performed by the pointing device 107. In this embodiment, the poster creation conditions specify the poster size, the number of posters to be created, and the usage category. The poster size may be specified as the actual width and height, or as a paper size such as A1 or A2. The usage category indicates what purpose the poster will be used for, such as restaurants, school events, or sales.

[0021] The text specification unit 202 specifies the text information to be placed on the poster through UI operations using the keyboard 106. The text information to be placed on the poster refers to strings of characters such as titles, dates and times, and locations. The text specification unit 203 identifies whether each piece of text information is of a specific type, such as title, date and time, or location, and then outputs it to the skeleton acquisition unit 213 and the layout unit 217.

[0022] The image selection unit 203 specifies a group of images stored on the HDD 104 to be placed on the poster. The image group may be specified based on the structure of the file system containing the images, such as the device and directory, or it may be specified by accompanying information of individual images, such as the date and time of shooting, or by attribute information. The image selection unit 203 outputs the file paths of the specified images to the image acquisition unit 211.

[0023] The target impression specification unit 204 specifies the target impression of the poster to be created. The target impression is the impression that the poster should ultimately convey. In this embodiment, the degree to which a word representing an impression should be emphasized is specified by UI operation using the pointing device 107. Details about impressions will be described later.

[0024] The image acquisition unit 211 acquires the image group specified by the image selection unit 203 from the HDD 104. The image acquisition unit 211 outputs the acquired image data to the image analysis unit 212. It also outputs the number of acquired images to the skeleton acquisition unit 213. Images stored in the HDD 104 include still images and frame images extracted from videos. Still images and frame images are acquired from imaging devices such as digital cameras and smart devices. The imaging device may be provided by the information processing device 100 or by an external device. If the imaging device is an external device, the images are acquired via the data communication unit 108. As another example, still images may be illustrations created with image editing software or CG images created with CG production software. Still images and extracted images may be images acquired from a network or server via the data communication unit 108. Examples of images acquired from a network or server include social networking service images (hereinafter referred to as "SNS images"). Furthermore, the program executed by CPU 101 analyzes the data associated with each image to determine its source. SNS images may be obtained from SNS via the application, and the source may be managed within the application. The images are not limited to those described above; other types of images may also be used.

[0025] The image analysis unit 212 analyzes the image data acquired from the image acquisition unit 211 using a method described later to obtain image features. The analysis process involves performing object recognition and obtaining these image features. The image analysis unit 212 associates the acquired image features with the image data and outputs them to the layout unit 217.

[0026] The skeleton acquisition unit 213 acquires a group of skeletons from the HDD 104 that match the conditions specified by the poster creation condition specification unit 201, the text specification unit 202, and the image acquisition unit 211. In this embodiment, a skeleton is information that represents the arrangement of text strings, images, figures, etc. to be placed on a poster.

[0027] Figure 3 shows an example of a skeleton. Three graphic objects 302, 303, and 304, one image object 305, and four text objects 306, 307, 308, and 309 are placed on the skeleton 301 in Figure 3(a). Each object records not only its position, size, and angle, which indicate where it will be placed, but also metadata necessary to generate the poster. Figure 3(b) shows an example of metadata. For example, text objects 306 to 309 hold metadata indicating what kind of text information will be placed. Here, text object 306 indicates the title, text object 307 indicates the subtitle, and text objects 307 and 308 indicate the body text. Text objects 306 to 309 also hold metadata indicating the font size and number of lines. In this embodiment, all text objects are processed as being on one line. Graphic objects 302 to 304 hold metadata indicating the shape and color scheme number of the graphic. Here, graphic objects 302 and 303 indicate rectangles, and graphic object 304 indicates an ellipse. Furthermore, graphic object 302 is assigned color scheme number 1, and graphic objects 303 and 304 are assigned color scheme number 2. Here, the color scheme number is information referenced when applying color schemes, as described later, and indicates that different color scheme numbers are assigned to different colors. Note that the types of objects and metadata are not limited to these. For example, there may be map objects for placing maps, or barcode objects for placing QR codes (registered trademarks) or barcodes. Also, metadata for text objects may include metadata that indicates the line spacing and character spacing. The metadata may include the purpose of the skeleton and be used to control whether or not the skeleton can be used depending on the purpose.

[0028] The skeleton may be saved on HDD104 in CSV format, for example, or in a database format such as SQL. The skeleton acquisition unit 213 outputs the skeletons acquired from HDD104 to the skeleton selection unit 214. Note that the target impression changes depending on the arrangement of images, text, graphics, fonts, color schemes, and graphics of the skeleton.

[0029] The skeleton selection unit 214 selects a group of skeletons from those acquired from the skeleton acquisition unit 213 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 217. Since the overall layout of the poster is determined by the skeleton, the variations of the generated poster can be increased by preparing various types of skeletons in advance.

[0030] The color scheme pattern selection unit 215 retrieves a group of color scheme patterns from the HDD 104 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 217. A color scheme pattern is a combination of colors used for a poster.

[0031] Figure 4 shows an example of a color scheme pattern table. In this embodiment, the color scheme pattern is represented as a combination of four colors. The Color ID column in Figure 4 is an ID for uniquely identifying the color scheme. Columns Color 1 to Color 4 represent the colors in RGB order, with each color value ranging from 0 to 255. Although a four-color scheme pattern is used in this embodiment, other numbers of colors may be used, or multiple numbers of colors may be mixed.

[0032] The font selection unit 216 retrieves a set of fonts from the HDD 104 that match the target impression specified by the target impression specification unit 204, and outputs them to the layout unit 217.

[0033] The layout unit 217 generates poster data by laying out various data for each skeleton acquired from the skeleton selection unit 214. The layout unit 217 places text acquired from the text specification unit 202 and image data acquired from the image analysis unit 212 for each skeleton and applies a color scheme acquired from the color scheme pattern selection unit 216. The layout unit 217 also controls the application of text decoration based on the target impression acquired from the target impression specification unit 204. The layout unit 217 outputs the generated poster data to the impression estimation unit 218.

[0034] The impression estimation unit 218 estimates the impression of each poster data obtained from the layout unit 217 and associates it with each poster data. The impression estimation unit 218 outputs the estimated impression associated with each poster data to the poster selection unit 219.

[0035] The poster selection unit 219 compares the target impression specified by the target impression specification unit 204 with the estimated impression obtained from the impression estimation unit 218, and selects the poster data with the estimated impression that is closest to the target impression. The selection result is saved to the HDD 104. The poster selection unit 217 outputs the selected poster data to the poster display unit 205.

[0036] The poster display unit 205 outputs a poster image for display on the display 105 according to the poster data acquired from the poster selection unit 219. The poster image is, for example, bitmap data. The poster display unit 205 displays the poster image on the display 105.

[0037] When the poster creation application is installed on the information processing device 100, a launch icon appears on the top screen (desktop) of the OS (operating system) running on the information processing device 100. When the user double-clicks the launch icon displayed on the display 105 with the pointing device 107, the application program stored on the HDD 104 is loaded into the RAM 103 and launched by being executed by the CPU 101.

[0038] Although not shown in the illustration, the poster creation application may also include a function that allows the user to edit the arrangement, color, and shape of images, text, and graphics after the generated result is displayed on the poster display unit 205, thereby further modifying the design to the user's specifications.

[0039] Furthermore, if the system includes a function to print poster data stored on the HDD104 using a printer according to the conditions specified in the poster creation condition specification unit 201, the user will be able to obtain a printed copy of the poster they created.

[0040] <Example of display screen> Figure 5 shows an example of an application screen 501 provided by the poster creation application. The application screen 501 (first screen) is displayed on the display 105 when the application is launched and is a screen for receiving the target impression from the user. The user sets the poster creation conditions, text, and images, which will be described later, via the application screen 501, and the poster creation condition specification unit 201, image specification unit 202, and text specification unit 203 obtain the settings from the user through this UI screen.

[0041] The title box 502, subtitle box 503, and body box 504 accept the specification of text information to be placed on the poster. In this embodiment, three types of text information are accepted, but this is not limited to these. For example, additional text information such as location and date and time may be accepted. Also, it is not necessary for all specifications to be completed, and blank fields may be left.

[0042] The image selection area 505 is the area that displays the image to be placed on the poster. Image 506 represents a thumbnail of the selected image. The image add button 507 is a button for adding an image to be placed on the poster. When the user presses the image add button 507, the image selection unit 202 displays a dialog screen for selecting a file stored on the HDD 104 and accepts the user's selection of an image file. The thumbnail of the selected image is added to the image selection area 507.

[0043] The impression sliders 508-511 are used to set the target impression of the poster you are creating. For example, slider 508 sets the target impression of luxury, with the right side indicating a higher level of luxury and the left side indicating a lower level of luxury (cheaper-looking poster). For example, setting impression slider 508 to the right and impression slider 511 to the left, meaning a poster with a high level of luxury and low level of weightiness, will have an elegant impression. On the other hand, keeping impression slider 508 on the right and setting impression slider 511 to the right as well, meaning a poster with a high level of both luxury and weightiness, will have a gorgeous impression. In this way, by combining multiple impression sliders, it is possible to set different directions of impression, even with the same level of luxury.

[0044] Radio buttons 512 can be used to control the on / off state of each target impression. Figure 5 shows the state where luxury and approachability are on, and dynamism and gravitas are off. Selecting off with radio button 512 excludes that impression from impression control. For example, a user who wants to create a calm poster with low dynamism but has no specific requirements for other impressions can turn off all radio buttons 512 except for dynamism to generate a poster that specializes in low dynamism. This allows for highly flexible control, such as using all target impressions or using only some target impressions in poster generation.

[0045] The size list box 513 is a list box for setting the size of the poster to be created. A user can click on it using the pointing device 107 to display a list of available poster sizes, from which they can select one.

[0046] The "Number of creations" box (514) allows you to set the number of candidate posters to create.

[0047] Category list box 515 allows you to set the intended use category for the poster you are creating.

[0048] The reset button 516 is used to reset the various settings information on the app screen 501.

[0049] The text decoration radio button 518 is a radio button that sets whether or not to perform text decoration automatically. When the text decoration radio button 518 is ON, the poster generation unit 206 is allowed to perform text decoration automatically. Even if it is ON, if the poster generation unit 206 determines that text decoration is unnecessary, text decoration will not be performed. When the text decoration radio button 518 is OFF, the poster generation unit 206 skips the text decoration process.

[0050] When the user presses the OK button 517, the poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 output the content set on the application screen 501 to the poster generation unit 210. At this time, the poster creation condition specification unit 201 obtains the size of the poster to be created from the size list box 513, the number of posters to be created from the number box 514, and the purpose category of the poster to be created from the category list box 515. The text specification unit 202 obtains the text information to be placed on the poster from the title box 503, subtitle box 503, and body box 504. The image specification unit 203 obtains the image file path to be placed on the poster from the image specification area 505. The target impression specification unit 204 obtains the target impression of the poster to be created from the impression sliders 508~511 and radio buttons 512. The poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 may process the values ​​set on the application screen 501. For example, the text specification unit 202 may remove unnecessary whitespace characters at the beginning or end of the input text information. The target impression specification unit 204 may also format the values ​​specified by the impression sliders 508 to 511. In this embodiment, the leftmost setting of the slider is set to -2, and the rightmost setting is set to +2, and the values ​​are formatted to integer values ​​from -2 to +2. This corresponds to the impression being low (-2), slightly low (-1), neutral (0), slightly high (+1), and high (+2). The reason for formatting to -2 to +2 is to match the scale with the estimated impression described later and facilitate distance calculation, but it is not limited to this and may be normalized to 0 to 1, for example.

[0051] Figure 6 shows an example of a poster preview screen where the poster image generated by the poster display unit 205 is displayed on the display 105. When the OK button 309 on the application screen 501 is pressed and poster generation is complete, the screen displayed on the display 105 transitions to the poster preview screen 601.

[0052] Poster image 602 is a poster image output by the poster display unit 205. The poster generation unit 210 generates the number of posters specified by the poster creation condition specification unit 201, so the created posters are also displayed as a list in poster image 602. The user can select a poster by clicking on it with the pointing device 107.

[0053] The edit button 603 allows you to edit the selected poster through a UI that provides editing functions not shown in the image.

[0054] The print button 604 allows you to print the selected poster via the printer's control UI (not shown).

[0055] <Quantifying the impression of a poster> This section explains a method for quantifying the impression of a poster, which is necessary for the poster generation process described later. This poster impression quantification involves quantifying the impressions people have of various posters. Simultaneously, it derives a correspondence between poster images and the impressions they convey. This allows us to estimate the impression of a poster from the poster image we will create. Once the impression is estimated, it becomes possible to control the poster's impression by modifying it, or to search for a poster that possesses a specific target impression. Note that the poster impression quantification process is performed, for example, by running an impression learning application in an information processing device before the poster generation process to learn the poster's impression.

[0056] Figure 7 is a flowchart showing the process of quantifying the impression of a poster. The flowchart shown in Figure 7 is implemented, for example, by the CPU 101 reading a program stored in the HDD 104 into the RAM 103 and executing it. The process of quantifying the impression of a poster will be explained with reference to Figure 7. In the description of each process, the symbol "S" indicates a step in the flowchart (the same applies hereafter in this specification).

[0057] In S701, the subjective evaluation acquisition method performs subjective evaluations of impressions of posters. Figure 8 illustrates an example of a method for subjective evaluation of impressions of posters. The subjective evaluation acquisition method presents posters to subjects and obtains subjective evaluations of the impressions they receive from the posters. Measurement methods such as the Semantic Differential (SD) method and the Likert scale can be used. Figure 8 shows an example of a questionnaire using the SD method, where multiple evaluators are presented with pairs of adjectives that express impressions, and scores are assigned to the pairs of adjectives that are recalled from the target poster. After obtaining subjective evaluation results from multiple subjects for multiple posters, the subjective evaluation acquisition method calculates the average value of the responses for each pair of adjectives to obtain a representative score for that pair. Note that the subjective evaluation method for impressions does not have to be the SD method; it is sufficient to have words that express impressions and corresponding scores.

[0058] In S702, the factor analysis means performs factor analysis on the subjective evaluation results obtained by the subjective evaluation acquisition means. If the subjective evaluation results are left as they are, the number of adjective pairs becomes a dimensionality, making control complex. Therefore, it is desirable to reduce the dimensionality to an efficient level using analytical methods such as principal component analysis or factor analysis. In this embodiment, we will explain the process assuming that the dimensionality has been reduced to four factors by factor analysis. Naturally, this number varies depending on the selection of adjective pairs in the subjective evaluation and the factor analysis method used. Furthermore, the output of the factor analysis is assumed to be standardized. That is, each factor is scaled so that the mean is 0 and the variance is 1 in the poster used for the analysis. This allows the -2, -1, 0, +1, and +2 of the impressions specified in the target impression specification unit 204 to directly correspond to -2σ, -1σ, mean, +1σ, and +2σ in each impression, facilitating the calculation of the distance between the target impression and the estimated impression, as described later. In this embodiment, the four factors are referred to as a sense of luxury, familiarity, dynamism, and gravitas, as shown in Figure 5. However, these are names conveniently assigned to convey an impression to the user through the user interface, and each factor is composed of multiple adjective pairs influencing each other.

[0059] In S703, the impression learning method associates poster images with impressions. While it is possible to quantify the impressions of posters that have been subjectively evaluated using the method described above, it is necessary to estimate impressions of posters to be created without subjective evaluation. Associating poster images with impressions can be achieved by training a model that estimates impressions from poster images using deep learning methods such as Convolutional Neural Networks (CNNs) or machine learning methods using decision trees. In this embodiment, the impression learning method takes poster images as input and four factors as outputs, and performs supervised deep learning using a CNN. That is, it learns subjectively evaluated poster images and their corresponding impressions as ground truth to create a deep learning model, and then estimates impressions by inputting unknown poster images into that trained model.

[0060] The deep learning model created above is stored, for example, on HDD104, and the impression estimation unit 218 loads the deep learning model stored on HDD104 into RAM103 and executes it. The impression estimation unit 218 converts the poster data acquired from the layout unit 217 into an image and estimates the impression of the poster by running the deep learning model loaded into RAM103 on CPU101 or GPU109. In this embodiment, a deep learning method is used, but it is not limited to this. For example, if a machine learning method such as a decision tree is used, feature quantities such as the average brightness and edge quantity of the poster image may be extracted by image analysis, and a machine learning model that estimates the impression based on those feature quantities may be created.

[0061] <Processing flow> Figure 9 is a flowchart of the poster generation process of a poster creation application. The flowchart shown in Figure 9 is realized, for example, when the CPU 101 reads a program stored in the HDD 104 into the RAM 103 and executes it. In the explanation of Figure 9, each component shown in Figure 2 is described as performing a process, which is activated when the CPU 101 executes the poster creation application. The poster generation process will be explained with reference to Figure 9. In the explanation of each process, the symbol "S" indicates a step in the flowchart (the same applies hereafter in this specification).

[0062] In the S901, the poster creation application displays the application screen 501 on the display 105. The user inputs various settings through the UI screen of the application screen 501 using the keyboard 106 or pointing device 107.

[0063] In S902, the poster creation condition specification unit 201, text specification unit 202, image specification unit 203, and target impression specification unit 204 obtain settings from the application screen 501.

[0064] In S904, the image acquisition unit 211 acquires image data. Specifically, the image acquisition unit 211 reads the image file in the HDD 104 specified by the image selection unit 203 into the RAM 103.

[0065] In S903, the skeleton selection unit 214, the color scheme pattern selection unit 215, and the font selection unit 216 determine the number of skeletons, color scheme patterns, and fonts to select according to the number of posters to be created specified in the poster creation condition specification unit 201. In this embodiment, the layout unit 217 generates poster data with a number of skeletons × number of color scheme patterns × number of fonts by a method described later. At this time, the number of skeletons, color scheme patterns, and fonts to select is determined so that the number of posters generated exceeds the number to be created. In this embodiment, the number of skeletons, color scheme patterns, and fonts are determined according to Equation 1, respectively.

[0066]

number

[0067] For example, if the number of posters to be created is 6, the number of posters to be selected is 3, resulting in 27 poster data generated by the layout unit 217, from which the poster selection unit 219 selects 6 poster data. This allows the poster selection unit 219 to select posters from among the generated poster data that better match the target impression in terms of overall impression.

[0068] In S905, the image analysis unit 212 performs analysis on the image data acquired in S904 and obtains feature quantities. Examples of feature quantities include metadata stored in the image and image features that can be obtained by analyzing the image. In this embodiment, object recognition is performed as the analysis process, but it is not limited to this, and other analysis processes may be performed. The details of the processing performed by the image analysis unit 212 in S905 will be described below.

[0069] The image analysis unit 212 performs object recognition on the image acquired in S904. Here, known methods can be used for the object recognition process. In this embodiment, objects are recognized by a discriminator created by Deep Learning. The discriminator outputs a likelihood of 0 to 1 that a certain pixel in the image is a pixel that constitutes each object, and recognizes that there is an object in the image if the likelihood exceeds a certain threshold. By recognizing the object image, the image analysis unit 212 can obtain the type and location of objects such as faces, pets such as dogs or cats, flowers, food, buildings, ornaments, and landmarks.

[0070] In S906, the skeleton acquisition unit 213 acquires skeletons that meet various setting conditions. In this embodiment, it is assumed that each skeleton is written to a separate file and stored in the HDD 104. The skeleton acquisition unit 213 sequentially reads the skeleton files from the HDD 104 into the RAM 103, keeping the skeletons that meet the setting conditions in the RAM 103 and deleting the skeletons that do not meet the conditions from the RAM 103. The skeleton acquisition unit 213 first determines whether the size of the skeleton matches the poster size specified in the poster creation condition specification unit 201. Note that while the size matches here, it is also acceptable if only the aspect ratio matches. In that case, the skeleton acquisition unit 213 expands or shrinks the coordinate system of the read skeleton to acquire a skeleton that matches the poster size specified in the poster creation condition specification unit 201. Next, it determines whether the category of the skeleton matches the usage category specified in the poster creation condition specification unit 201. Skeletons used only for specific purposes have their purpose category recorded in the skeleton file, and are not retrieved unless the corresponding purpose category is selected. This prevents the skeleton from being used in other purpose categories if it is designed specifically for a particular purpose, such as if it depicts a school image using graphics or a sports equipment pattern. Next, the skeleton retrieval unit 213 determines whether the number of image objects in the loaded skeleton matches the number of images retrieved by the image retrieval unit 211. Finally, the skeleton retrieval unit 213 determines whether the character objects in the loaded skeleton match the character information specified in the text specification unit 202. More specifically, it determines whether the type of character information specified in the text specification unit 202 exists within the skeleton. For example, suppose a string is specified in the title box 502 and body box 504 on the app screen 501, and a blank is specified in the subtitle box 503.In this case, the skeleton is searched for all character objects within the skeleton, and is considered a match if both a character object with "Title" set as the type of character information in the metadata and a character object with "Body" specified are found; otherwise, it is considered a failure. As described above, the skeleton acquisition unit 213 stores on RAM 103 skeletons in which the skeleton size, the number of image objects, and the type of character objects all match the set conditions. In this embodiment, the skeleton acquisition unit 213 checked all skeleton files on HDD 104, but is not limited to this. For example, a poster creation application may store a database on HDD 104 that associates the file paths of skeleton files with search conditions (skeleton size, number of image objects, and type of character objects) in advance. In that case, the skeleton acquisition unit 213 can acquire skeleton files at high speed by reading only the skeleton files that match the search results from the database from HDD 104 to RAM 103.

[0071] In S907, the skeleton selection unit 214 selects a skeleton from among the skeletons acquired in S906 that matches the target impression specified in the target impression specification unit 204. Here, Figure 10 is a diagram illustrating how the skeleton selection unit 214 selects a skeleton. Figure 10(a) is a diagram showing an example of a table that links skeletons to impressions. The skeleton name column in Figure 10(a) contains the file name of the skeleton, and the columns for luxury, familiarity, dynamism, and solidity indicate how much the skeleton contributes to each impression. First, the skeleton selection unit 214 calculates the distance between the target impression acquired from the target impression specification unit 204 and the skeleton impression table in Figure 10(a). For example, if the target impression is "luxury +1, familiarity -1, dynamism -2, solidity +2", the distance calculated by the skeleton selection unit 214 will be as shown in Figure 10(b). In this embodiment, Euclidean distance is used as the distance. Next, the skeleton selection unit 214 selects the top N skeletons with the smallest distance in Figure 8(b). In this embodiment, the skeleton selection unit 214 selects the top two skeletons, namely skeleton 1 and skeleton 4.

[0072] Here, the setting method for N may be a fixed value, or it may be variable depending on the conditions specified in the poster creation condition specification unit 201. For example, if the number of posters to be created is specified as 6 in the number of posters box 514 on the app screen 501, the poster generation unit 210 will generate 6 posters. In the layout unit 217, which will be described later, the poster is generated by combining the skeleton, color scheme, and font selected in the skeleton selection unit 214, the color scheme selection unit 215, and the font selection unit 216. For example, by selecting 2 skeletons, 2 color schemes, and 2 fonts, 2 x 2 x 2 = 8 posters can be generated, thus satisfying the condition of 6 posters to be created. In this way, the number of skeletons to be selected may be determined according to the conditions specified in the poster creation condition specification unit 201.

[0073] Furthermore, the range of values ​​for each impression in the skeleton impression table in Figure 10(a) does not need to be the same as the range of values ​​for the impression specified by the target impression specification unit 204. In this embodiment, the range of values ​​for the impression specified by the target impression specification unit 204 is -2 to +2, but the range of values ​​for the impressions in the skeleton impression table may be different. In that case, the distance calculation is performed after scaling the range of values ​​in the skeleton impression table to match the range of values ​​for the target impression. Also, the distance calculated by the skeleton selection unit 214 is not limited to Euclidean distance; it is acceptable as long as it can calculate the distance between vectors, such as Manhattan distance or cosine similarity. In addition, if the target impression is set to off using radio button 517, it is excluded from the distance calculation.

[0074] A skeleton impression table can be created by fixing elements such as color schemes, fonts, and images and text information placed on the skeleton, generating posters based on each skeleton, and estimating their impressions. In other words, by estimating the impressions of posters that use the same colors and images but have different arrangements, the relative characteristics of each skeleton can be compiled into a table. At that time, it is desirable to perform a process to cancel out the impression caused by the color schemes and images used, such as standardizing the overall estimated impression or averaging the impressions of multiple posters generated from a single skeleton using multiple color schemes and images. This allows for the creation of a table showing the influence of arrangement on the impression, such as how a skeleton with a small image will have its impression determined by elements such as graphics and text rather than the image, or how a tilted arrangement of images and text will create a strong sense of dynamism. Figure 10(c) is an example of skeletons corresponding to skeletons 1 to 4 in Figure 10(a). For example, in skeleton 1, image objects and text objects are arranged regularly, and the image area is small, resulting in a low sense of dynamism. Skeleton 2 has a high sense of familiarity and a low sense of weight due to the circular shape and image objects. Skeleton 3 has a high sense of dynamism due to the large placement of image objects and the placement of tilted shape objects overlapping the image objects. Skeleton 4 has a high sense of weight and a low sense of dynamism because images are placed throughout the entire skeleton and text objects are kept to a minimum. In this way, the impression changes depending on the placement of images, shapes, and text. Note that the method of creating the skeleton impression table is not limited to these methods; it may also be estimated from the characteristics of the placement information itself, such as the area and coordinates of images and title strings, or it may be adjusted manually. The skeleton impression table is stored in HDD 104, and the skeleton selection unit 214 reads the skeleton impression table from HDD 104 into RAM 103 and references it.

[0075] In S908, the color scheme pattern selection unit 215 selects a color scheme pattern that matches the target impression specified in the target impression specification unit 204. Similar to S906, the color scheme pattern selection unit 215 refers to the impression table corresponding to the color scheme pattern and selects a color scheme pattern according to the target impression. Figure 11(a) shows an example of a color scheme pattern impression table that links color scheme patterns to impressions. The color scheme pattern selection unit 215 calculates the distance between the "luxury" column to the "heavy" column in Figure 11 and the target impression, and selects the top N color scheme patterns with the smallest distances. In this embodiment, the top two color scheme patterns are selected. The color scheme pattern impression table, similar to the skeleton impression table, is created by fixing the skeleton other than the color scheme pattern, as well as fonts and images, and then creating posters with different color scheme patterns and estimating the impressions, thereby creating a table of impression trends for color scheme patterns.

[0076] In S909, the font selection unit 216 selects a font that matches the target impression specified in the target impression specification unit 204. Similar to S906, the font selection unit 216 refers to the impression table corresponding to the font and selects a font according to the target impression. Figure 11(b) shows an example of a font impression table that links fonts to impressions. Note that, similar to the skeleton impression table, the font impression table can be created by fixing the skeleton other than the font, the color scheme, and the image, and then creating posters with different fonts and estimating the impressions, thereby creating a table of font impression trends.

[0077] In S910, the layout unit 217 sets text information, images, color schemes, and fonts for the skeleton selected by the skeleton selection unit 214 to generate a poster.

[0078] S910 and the layout unit 217 will be explained in detail using Figures 12, 13, 14, and 15. Figure 12 is an example of a software block diagram that explains the layout unit 217 in detail. Figure 13 is a flowchart that explains S910 in detail. Figure 14 is a diagram that explains the information input to the layout unit 217. Figure 14(a) is a table summarizing the character information specified by the text specification unit 202 and the image specified by the image specification unit 203. Figure 14(b) is an example of a color scheme pattern obtained from the color scheme pattern selection unit 215, and Figure 14(c) is an example of a font obtained from the font selection unit 216. Figure 15 is a diagram that explains the processing process of the layout unit 217.

[0079] In S1301, the layout unit 217 enumerates all possible combinations of skeletons obtained from the skeleton selection unit 214, color patterns obtained from the color pattern selection unit 215, and fonts obtained from the font selection unit 216. The layout unit 217 then generates poster data for each combination in order through subsequent layout processing. For example, if the number of skeletons obtained from the skeleton selection unit 214 is 3, the number of color patterns obtained from the color pattern selection unit 215 is 2, and the number of fonts obtained from the font selection unit 216 is 2, the layout unit 217 generates 3 × 2 × 2 = 12 poster data.

[0080] In S1302, the color assignment unit 1201 assigns each color pattern obtained from the color pattern selection unit 215 to each skeleton obtained from the skeleton selection unit 214. Figure 15(a) shows an example of a skeleton. In this embodiment, an example of assigning the color pattern of color ID 1 in Figure 14(b) to the skeleton 1501 in Figure 15(a) will be described. The skeleton 1501 in Figure 15(a) consists of two graphic objects 1502 and 1503, one image object 1504, and three text objects 1505, 1506, and 1507. First, the color assignment unit 207 assigns colors to the graphic objects 1502 and 1503. Specifically, it assigns the corresponding color from the color pattern based on the color number, which is metadata described in the graphic object. Next, it assigns the last color from the color pattern to the text object whose metadata type is "Title". In other words, in this embodiment, color 4 is assigned to the character object 1505. Next, for character objects whose metadata type is anything other than "Title", the character color is set based on the brightness of the background of the character object. In this embodiment, the L* value obtained when the input data is converted to L*a*b* using the L*a*b* color space, which is a typical color space, is used as the brightness. Note that the method of calculating brightness is not limited to this. For example, the value of L may be used as brightness using the known HunterLab conversion formula, or the value of V may be used as brightness using the known HSV conversion formula. Any method that can calculate a value related to brightness is acceptable. In this embodiment, if the brightness of the background of the character object is below a threshold, the character color is set to white, and otherwise the character color is set to black. Figure 15(b) is a diagram showing the state of the skeleton 1508 after the above color assignment process has been performed. The color assignment unit 1201 outputs the colored skeleton data to the image placement unit 1202.

[0081] In S1303, the image placement unit 1202 places the image data obtained from the image analysis unit 212 onto the skeleton data obtained from the color assignment unit 1201, based on the accompanying analysis information. In this embodiment, the image placement unit 1202 assigns the image data 1401 to the image object 1504 within the skeleton. Furthermore, if the aspect ratio of the image object 1504 and the image data 1401 are different, the image placement unit 1202 performs trimming to match the aspect ratio of the image data 1401 to that of the image object 1504. More specifically, based on the face position and object position obtained by the image analysis unit 202 analyzing the image data 1401, the image placement unit 1202 performs trimming to minimize the reduction in the face area and object area due to trimming. Note that the trimming method is not limited to this, and other trimming methods may be used, such as trimming the center of the image or devising a composition so that the face position forms a triangular composition. The image placement unit 1202 outputs the skeleton data with image assignments to the image correction unit 1203.

[0082] In S1304, the image correction unit 1203 obtains the image-assigned skeleton data from the image placement unit 1202 and corrects the images placed on the skeleton. In this embodiment, if the image resolution is insufficient, upsampling processing is performed using super-resolution processing. First, the image correction unit 1203 checks whether the images placed on the skeleton meet a certain resolution. For example, suppose a 1600px × 1200px image is assigned to a 200mm × 150mm area on the skeleton. In this case, the print resolution of the image can be calculated using Equation 2.

[0083]

number

[0084] Next, the image correction unit 1203 compensates for the perceived resolution by performing super-resolution processing if the print resolution of the image is below a threshold, and conversely, does not perform any image correction if the resolution is sufficient. In this embodiment, super-resolution processing is performed when the resolution is less than 300 dpi.

[0085] In S1305, the font setting unit 1204 sets the font obtained from the font selection unit 216 to the image-corrected skeleton data obtained from the image correction unit 1203. Figure 10(c) shows an example of the font selected by the font selection unit 216. In this embodiment, the font is set for the character objects 1505, 1506, and 1507 of the skeleton 1508. In posters, from the perspective of attracting attention, it is common to set a font that stands out for the title and a font that is easy to read for the other characters. Therefore, in this embodiment, the font selection unit 216 selects two types of fonts: a title font and a body font. The font setting unit 1204 sets the title font for the character object 1505, which is the title, and the body font for the other character objects 1506 and 1507. The font setting unit 1204 outputs the skeleton data with the font set to the text placement unit 1205. In this embodiment, the font selection unit 216 selected two types of fonts, but this is not limited to this; for example, only the title font may be selected. In that case, the font setting unit 1204 uses the font corresponding to the title font as the body font. That is, if the title is a Gothic font, then a representative Gothic font with high readability should be chosen for the other text as well; if the title is a Mincho font, then a representative Mincho font should be chosen for the other text as well, and so on. The body font should be set to match the type of title font. Of course, the title font and body font can also be the same. Alternatively, the font can be used differently depending on the degree to which you want to make something stand out, such as using the title font for the title and subtitle text objects and the body font for other text objects, or using the title font for fonts above a certain size.

[0086] In S1306, the text placement unit 1205 places the text specified by the text specification unit 202 onto the font-configured skeleton data obtained from the font-configured unit 1204. In this embodiment, each text shown in 14(a) is assigned by referring to the metadata of the skeleton's character objects. That is, the title "Summer Grand Sale" is assigned to character object 1505, and the subtitle "Beat the Summer Heat" is assigned to character object 1506. Since no body text is set, nothing is assigned to character object 1507. Figure 15(c) shows skeleton 1509, which is an example of skeleton data after processing by the text placement unit 1205. The text placement unit 1205 outputs the skeleton data with the placed text to the text decoration unit 1206.

[0087] In S1307, the text decoration unit 1206 evaluates the readability of the character objects in the text-placed skeleton obtained from the text placement unit 1205, and adds decoration based on the target impression obtained from the target impression specification unit 204.

[0088] Here, text decoration will be explained using Figure 17. Compared to text 1701 without decoration, text 1702 with outline decoration and text 1703 with glow decoration have improved readability. However, the impression the user receives differs between text 1702 with outline decoration and text 1703 with glow decoration. Therefore, in this embodiment, a process for appropriately generating a poster that expresses the user's intended impression, even when text decoration is applied, will be described.

[0089] The processing performed by the text decoration unit 1206 will be explained in detail using Figures 18 and 19. Figure 18 is an example of a software block diagram that explains the text decoration unit 1206 in detail. The text decoration unit 1206 consists of a tile division unit 1801, an evaluation value calculation unit 1802, a determination unit 1803, and a decoration execution unit 1804.

[0090] The tile division unit 1801 performs tile division after imaging the areas where character objects are placed in the skeleton data with text already placed, obtained from the text placement unit 1205. The evaluation value calculation unit 1802 evaluates the readability of the characters. The determination unit 1803 determines whether the readability of the characters is above a threshold. The decoration execution unit 1804 determines a character decoration method and related parameters that match the target impression specified in the target impression specification unit 204 and adds decoration to the characters.

[0091] Figure 19 is a flowchart that explains in detail the processing performed by the text decoration unit 1206.

[0092] In S1901, the tile division unit 1801 renders the area where a text object is placed, including other objects placed in the background, for text objects whose metadata type is "Title". The tile division unit 1801 then divides the rendered image into tiles. For example, if the skeleton shown in Figure 3 is used, the tile division unit 1801 renders the area indicated by the rectangle of the text object 306, including its background, and then divides it into tiles.

[0093] Here, Figure 20 is a diagram illustrating the tile division process. In this embodiment, as shown in Figure 20(a), the text object area is divided into squares (shown by dotted lines in the figure) with sides equal to the length of the shorter side. Note that although the tile size is set to the shorter side of the text object area, the method of tile division is not limited to this. For example, as shown in Figure 20(b), a square of a fixed size may be used. In this case, the fixed size is kept as a value for a certain poster size and converted to the poster size specified in the poster creation condition specification unit 201. For example, if the fixed size is kept as 10 mm for A4 size, and the poster size specified in the poster creation condition specification unit 201 is A2 size, then the text can be divided into squares with sides of 20 mm. Also, in this embodiment, the entire area of ​​the text object is processed, but a rectangular area 1.5 times the size of the text may be processed based on the center of each text actually entered. The method is not limited to these methods as long as an appropriate processing area can be set to evaluate readability. The processing from S1902 to S1904 is performed for each divided tile.

[0094] In S1902, the evaluation value calculation unit 1802 evaluates the readability of the characters. Here, Figure 21 is a diagram illustrating the readability of the characters. In Figure 21, to simply represent readability, rectangles 2101 to 2103 are described as characters. Rectangles 2101 and 2102 represent characters without decoration, while rectangle 2103 represents a character with contour decoration. The line profile 2105 shows the brightness of pixels on line 2106 in the character image 2104. The brightness of rectangle 2101 is 0, the brightness of rectangles 2102 and 2103 is 60, the brightness of the contour of rectangle 2103 is 0, and the brightness of the background of the character image 2104 is 80. As can be seen from Figure 21, rectangle 2102 has lower readability compared to rectangle 2101. Also, the brightness difference between rectangle 2101 and the background is 80, and the brightness difference between rectangle 2102 and the background is 20. Thus, the smaller the difference in brightness between the characters and the surrounding area, the more difficult it becomes to distinguish between the characters and the surrounding area, resulting in lower readability. Furthermore, rectangle 2103 has high readability despite having a brightness difference of 20 between the characters themselves and the background, similar to rectangle 2102. This is because the brightness difference between the outline of rectangle 2103 and the background, or between the outline of rectangle 21003 and the characters, is large, making it easy to distinguish between the characters and the surrounding area. In this embodiment, such a brightness difference between the characters and the surrounding area is also called the readability evaluation value. The brightness difference between the characters and the background is called the readability evaluation value for characters and background, and the brightness difference between the characters and decorative parts is called the readability evaluation value for characters and decorations.

[0095] In this embodiment, readability evaluation values ​​are calculated for the text and background. In this embodiment, the brightness of the text is calculated based on the text color assigned to the text object. In this embodiment, the brightness of the background is calculated as the brightness of the average color of the background area excluding the text. Note that the method for calculating the brightness of the background is not limited to this. For example, a 3D color histogram of the background pixels may be calculated, the color with the most pixels may be extracted, and the brightness of that color may be calculated. Any method that can calculate a value related to the brightness of the background area is acceptable.

[0096] In S1903, the determination unit 1803 determines whether the readability evaluation value calculated in S1902 is greater than a threshold. A smaller threshold means that the evaluation of readability is stricter. The threshold is set within the range of brightness values ​​calculated in S1902. In this embodiment, the threshold is set to 30. If it is determined that the readability evaluation value is greater than the threshold, the process proceeds to S1904. If it is determined that the readability evaluation value is not greater than the threshold, the process proceeds to S1905.

[0097] In S1904, the determination unit 1803 determines whether the processes from S1902 to S1903 were executed for all tiles divided in S1901. If it is determined that the processes were executed for all tiles (S1903 is YES), the text decoration unit 1206 decides not to apply text decoration. If it is not determined that the processes were executed for all tiles, the process from S1902 is repeated for different tiles.

[0098] In the described processing flow, decoration is performed if the readability evaluation value of one tile is greater than a threshold. However, decoration may also be performed if, for example, a predetermined percentage of tiles are greater than the threshold (for example, if 10% or more of the tiles in a text block are greater than the threshold). Furthermore, the threshold may be variable depending on the size of the text block. Alternatively, decoration may be performed if the maximum or average readability evaluation value of all tiles is greater than the threshold.

[0099] In S1905, the decoration execution unit 1804 determines a character decoration method for character objects whose readability evaluation value was determined to be below a threshold in S1903, based on the target impression specified by the target impression specification unit 204. Here, Figure 22 is a diagram illustrating how the text decoration unit 1206 selects a decoration method. Figure 22(a) is a diagram showing an example of a table that links character decoration methods and impressions (hereinafter referred to as the character decoration impression table). The character decoration method column in Figure 22(a) describes the types of character decoration methods. In this embodiment, examples of decoration methods include outline, glow, shadow, and bold. Other decoration methods may include chamfering and underlining. Any character decoration method that can improve the readability of the characters is acceptable. The columns for luxury, familiarity, dynamism, and gravitas in Figure 22(a) show how much each character decoration method contributes to each impression. Figure 22(b) is an example of when the decoration methods described in Figure 22(a) are applied to characters. It can be seen that characters with various decorations have improved readability compared to characters without decorations. The text decoration unit 1206, like S804, refers to an impression table corresponding to the decoration method and selects a character decoration method according to the target impression. In addition, the character decoration impression table can be created by fixing the skeleton other than the character decoration, the color scheme pattern, and the font, and then creating posters with different character decoration methods and estimating the impressions, thereby creating a table showing the impression trends of different character decoration methods.

[0100] In this embodiment, a text decoration impression table was used, but the method for determining the text decoration method is not limited to this. For example, fonts and text decoration methods may be linked in advance, and the text decoration method may be switched according to the font of the text object being processed. As for how to link fonts and text decoration methods, the text decoration method impression table and the font impression table used in S806 can be used to link the text decoration method that gives the impression closest to the degree of contribution of each font to the impression. This method allows the text decoration method to be determined without performing distance calculations, thus reducing the computational load.

[0101] In S1906, the decoration execution unit 1804 determines the character decoration parameters related to the character decoration method determined in S1905. In this embodiment, it determines the parameters related to the color and width of the decoration.

[0102] Here, the color of the decoration can be set for text decoration methods such as outlines, glows, and shadows. Figure 23 will be used to explain in detail how to determine the decoration color. Figure 23 is a flowchart that explains in detail how to determine the decoration color in the decoration parameter determination of S1906 in Figure 19.

[0103] In S2301, the decoration execution unit 1804 obtains the representative color of the background of the text object. In this embodiment, the average color of the background of the text object is used as the representative color. However, the method of obtaining the background representative color is not limited to this. For example, a 3D color histogram of the background pixels may be calculated, and the color with the most pixels may be obtained as the background representative color.

[0104] In S2302, the decoration execution unit 1804 calculates the decorative color. In this embodiment, a similar color obtained by changing the brightness of the background representative color acquired in S2301 is calculated as the decorative color. By using a similar color for decoration, a natural decoration can be applied, and thus, changes in impression due to decoration can be suppressed. Specifically, the background representative color is converted to the Lab color space using the conversion formula used when calculating the brightness in S1902, and then only the brightness of the background representative color is changed. In this embodiment, using formula 3, a constant is added to or subtracted from the brightness of the background representative color based on the relationship between the brightness of the background representative color and the brightness of the text color.

[0105]

number

[0106] Here, L back L' represents the brightness of the background's representative color before the change. back The brightness of the representative background color after the change is L textrepresents the brightness of the character, and C represents a constant. The constant is set within the range of values ​​of the brightness to be calculated. In this embodiment, the constant is set to 15. If the brightness calculated by Equation 3 exceeds the range of values ​​of the brightness, it is clipped within the range of values ​​of the brightness. Then, the decorative color is calculated using Equation 4.

[0107]

number

[0108] Here, DecoColor Lab In the Lab color space, decorative colors are a back and b back This indicates the chromaticity, which represents the hue and saturation of the background representative color converted to the Lab color space.

[0109] In S2303, the decoration execution unit 1804 adjusts the decorative color calculated in S2302 so that the readability evaluation value for the text color and the decorative color calculated in S2302 is equal to or greater than a threshold. In this embodiment, if the readability evaluation value for the text color and decorative color is below a threshold, the brightness of the decorative color is adjusted so that the readability evaluation value becomes equal to or greater than the threshold. In this embodiment, using equations 5 and 6, the brightness of the decorative color is added to or subtracted from the brightness of the decorative color by an amount obtained by subtracting the readability evaluation value for the text color and decorative color from the threshold, based on the relationship between the brightness of the decorative color and the brightness of the text color.

[0110]

number

[0111] Here, Thre sets the threshold, L decо L' is the brightness of the decorative color before adjustment. decо This indicates the brightness of the adjusted decorative color. The threshold is set within the range of values ​​of the brightness to be calculated. Furthermore, the threshold used in S2303 is set to a value greater than or equal to the threshold used in S1903. In this embodiment, the threshold is set to 40. If the brightness calculated by Equation 6 exceeds the range of values ​​of the brightness, it is clipped within the range of values ​​of the brightness. Then, the decorative color is calculated using Equation 7.

[0112] [Numerical]

[0113] When the readability evaluation values for the character color and the decorative color do not reach the threshold even with the above processing, adjust to create a brightness difference in the reverse direction. Specifically, the brightness of the decorative color is calculated using Equation 8.

[0114] [Numerical]

[0115] Here, L′′ decо indicates the calculated brightness of the decorative color. When the brightness calculated by Equation 8 exceeds the value range of brightness, clip it within the value range of brightness. Then, calculate the decorative color using Equation 9.

[0116] [Numerical]

[0117] [[ID= thirty-seven ]] By converting the color space of the decorative color calculated as above into the RGB color space, the decorative color is determined.

[0118] [[ID= forty ]] [[ID= forty-one ]] In this embodiment, in the calculation of the decorative color in S2302, analogous colors with different brightnesses were calculated based on the background representative color, but the method of calculating analogous colors is not limited to this. For example, analogous colors with different brightnesses may be calculated based on the character color. In this case, since a difference in brightness occurs between the character and the periphery of the character, the readability of the character can be improved similarly.

[0119] In this embodiment, in the calculation of decorative colors in S2302, analogous colors with different brightness levels were calculated based on the background representative color, but the method for calculating analogous colors is not limited to this. For example, analogous colors with different hues may be calculated based on the background representative color, or analogous colors with different 3D color vectors may be calculated without being limited to a single element. In this case, a difference is created between the color of the text and the color of the text, and similarly, the readability of the text can be improved.

[0120] Furthermore, the range of decorative elements such as outlines and glows can be set.

[0121] Returning to Figure 19, we will now explain how to set the decoration width in the decoration parameter determination process of S1906.

[0122] Figure 27 illustrates the effect of decorative width on the overall impression. Text object 2701 has a font size of 36pt with a 1.0pt wide outline. Text object 2702 has half the font size and decorative width compared to text object 2701. Text object 2703 has the same font size as text object 2702 and the same decorative width as text object 2701. Comparing text object 2701 and text object 2703, it can be seen that even with the same decorative width, the appearance differs depending on the font size, changing the overall impression. Similarly, comparing text object 2702 and text object 2703, it can be seen that even with the same font size, the appearance differs depending on the decorative width, changing the overall impression. This change in impression is due to a significant change in the ratio between the thickness of the drawn text and the thickness of the decorative width. Therefore, by determining the decorative width according to the font size, decoration can be applied without changing the overall impression.

[0123] Figures 24 and 25 will be used to explain in detail how to determine the decorative width. Figure 24 is a flowchart of the decorative width determination process in the determination of decorative parameters in S1906.

[0124] In S2401, the decoration execution unit 1804 obtains the font size that the character object has as metadata.

[0125] In S2402, the decoration execution unit 1804 determines the decoration width based on the font size of the character object to be decorated. Here, Figure 25 is a diagram illustrating the relationship between font size and decoration width (hereinafter referred to as the decoration width calculation function). In Figure 25, decow min and decow max These represent the lower and upper limits of the decorative width, respectively, and fontsize min and fontsize max These indicate the font size thresholds for setting the lower and upper limits of the decorative width, respectively. min is decow max Set the following value, fontsize min is fontsize max Set the values ​​as follows. Note that both font size and decoration width are in points (pt).

[0126] In this embodiment, the decorative width is determined based on the decorative width calculation function shown in Figure 25. Specifically, the decorative width is calculated using Equation 10.

[0127]

number

[0128] Here, FontSize represents the font size of the character object obtained in S2401, and DecoWidth represents the decoration width. By determining the decoration width based on Equation 10, the smaller the font size, the smaller the decoration width can be set.

[0129] Here, the function for calculating the decorative width is not limited to the form of Equation 10. For example, it could be a simple linear function or a multidimensional function. It could also be stored as a LUT. Anything is acceptable as long as the relationship between font size and decorative width can be defined.

[0130] Furthermore, the method for setting the decorative width may be defined according to the font type. Figure 26 illustrates the thickness of characters drawn with various fonts and the same font size. Gothic font 2601 and sans-serif font 2603 are fonts with uniform character thickness. In contrast, Mincho font 2602 and serif font 2604 are fonts with uneven character thickness, and are relatively thinner than Gothic font 2601 and sans-serif font 2603. Therefore, it is desirable to use a gentler gradient relationship for Mincho font 2602 and serif font 2604 compared to Gothic font 2601 and sans-serif font 2603. For example, this can be achieved by multiplying the DecoWidth calculated by equation 10 for Gothic fonts by 0.5. In this way, by defining different decorative width calculation functions according to the font, it becomes possible to set an appropriate decorative width regardless of the font type.

[0131] Furthermore, the method for setting the decorative width may be defined according to the decorative color. Colors can be categorized into expanding and contracting colors, with some colors appearing to expand and others appearing to contract. Therefore, when the decorative color is an expanding color, it is desirable to use a relationship with a gentler gradient compared to when the decorative color is a contracting color. For example, this can be achieved by multiplying the DecoWidth calculated using Equation 10 (when the decorative color is a contracting color) by 0.5. In this way, by defining different decorative width calculation functions according to the decorative color, it becomes possible to set an appropriate decorative width regardless of the decorative color.

[0132] This completes the process for determining the decorative width shown in Figure 24.

[0133] Returning to the explanation of Figure 19, in this embodiment, the processing of S1901 to S1906 is performed on all text objects whose metadata type is "Title" within the text object, thereby completing the processing of the text decoration unit 1206. The text decoration unit 1206 outputs the decorated skeleton data, that is, the poster data with the layout completed, to the impression estimation unit 218.

[0134] In this embodiment, the process was performed on a text object whose metadata type is "Title," but it is not limited to this. For example, it may target multiple types of text objects, including "Subtitle," or it may target all text objects. This allows for control over each type for which readability is important.

[0135] Furthermore, in this embodiment, the brightness difference between the character and the surrounding area was used as the readability evaluation value, but the color difference between the character and the surrounding area may also be used. For example, the difference in hue may be used, or the difference in three-dimensional vectors in the L*a*b* color space may be used. In this case, it becomes possible to switch whether or not to perform character decoration based on whether or not the color difference can be distinguished.

[0136] Furthermore, in this embodiment, the character object is tiled in S1901, but tiling is not required. In this case, for example, the readability evaluation value is calculated using the brightness of the background of the entire character object. This reduces the processing load compared to when tiling is performed. Alternatively, the area in which the characters are thickened may be calculated, and the readability evaluation value may be calculated using the brightness of the background of that area. This eliminates the need to view the entire character object, thus reducing the processing load.

[0137] Furthermore, although the processing was performed on the text object in this embodiment, the processing may also be performed line by line or character by character within the text object. This reduces the number of characters to which text decoration is applied compared to when the processing is performed on the entire text object.

[0138] In S1308, the layout unit 217 determines whether all poster data has been generated. If poster data has been generated for all combinations of skeletons, color schemes, and fonts, the layout process ends and the program proceeds to S911. If not all poster data has been generated, the program returns to S1301 and generates poster data for the remaining combinations.

[0139] This concludes the explanation of S910. Return to Figure 9.

[0140] In S911, the impression estimation unit 218 renders each poster data acquired from the layout unit 217, estimates the impression of the rendered poster image, and links the estimated impression to the poster data. This allows for the evaluation of the impression of the final poster, including images and text, as well as the impression of individual elements of the poster, such as color scheme and arrangement. For example, even with the same color scheme, the arrangement will change if the skeleton is different, so the actual area occupied by each color will differ. Therefore, it is necessary to evaluate not only the impression trends of individual color schemes and skeletons, but also the impression of the final poster.

[0141] In S912, the poster selection unit 219 selects posters to present to the user from the poster data and estimated impression obtained from the impression estimation unit 218. In this embodiment, the poster selection unit 219 selects posters in which the distance between the target impression and the estimated impression of the poster is less than or equal to a predetermined threshold. If the number of selected posters is less than the number of posters to be created specified in the poster creation condition specification unit 201, the poster selection unit 219 selects the remaining posters in ascending order of the distance between the target impression and the estimated impression of the poster. In this embodiment, the poster selection unit 219 also selects the remaining posters, but is not limited to this. For example, if the number of posters selected by the poster selection unit 219 is less than the number to be created, a message indicating the shortage may be displayed on the preview screen 601. Alternatively, after selecting the remaining posters, the poster selection unit 219 may display on the preview screen 601 that posters with a distance between the target impression and the estimated impression below a threshold are distinguishable from posters with a distance above the threshold. As another example, if there are not enough selected posters, the process returns to S903, and the number of skeletons, color patterns, and fonts to be selected may be increased.

[0142] In S913, the poster display unit 205 renders the poster data selected by the poster selection unit 219 and outputs the poster image to the display 105. That is, it displays the preview screen 601 shown in Figure 6.

[0143] The above is a description of the poster generation process flow, which generates a poster based on the user's specified impression.

[0144] As explained above, this embodiment makes it possible to generate posters that express the impression desired by the user. More specifically, in this embodiment, by selecting and combining each element that constitutes a poster, such as the skeleton, color scheme, font, and text decoration, based on the target impression, multiple variations of poster candidates that conform to the target impression can be generated. Furthermore, by estimating the overall impression of the poster and selecting a poster that is close to the target impression from the group of poster candidates, it is possible to generate a poster that not only has individual elements but also an overall impression that aligns with the user's intention. More specifically, for example, in this embodiment, suppose the target impression on the app screen 501 is set to luxury -1, familiarity +1, and dynamism and gravitas are both set to off. In that case, for example, the poster image 602 on the preview screen 601 is generated with an estimated impression close to the target impression, such as luxury -1.2, familiarity +0.9, dynamism +0.2, and gravitas -1.3. Note that dynamism and gravitas, which are set to off for the target impression, are not controlled and can have any value. Furthermore, according to this embodiment, the readability of the text can also be improved by applying text decoration.

[0145] <<Modification of the First Embodiment>> In the first embodiment, the target impression was set using the impression setting slider bars 508-511 on the app screen 501, but the method of setting the target impression is not limited to this.

[0146] Figure 16 shows an example of a UI for setting target impressions. Figure 16(a) shows an example of setting target impressions using a UI on a radar chart. By manipulating the handle 1601 on the radar chart in Figure 16(a), the target impression for each axis can be set. The target impression specification unit 204 acquires the target impression in the UI such that it is -2 when the handle 1601 is in the center and +2 when it is outside. In Figure 19(a), the target impressions are luxury +0.8, familiarity +1.1, dynamism -0.1, and solidity -0.7. Thus, target impressions can also be decimal numbers. Furthermore, the radar chart in Figure 16(b) shows an example of turning off some target impressions. For example, the user can turn off the target impression for the axis corresponding to the handle by double-clicking the handle 1601 with the pointing device 107. The user can then turn on the target impression again by clicking the axis 1602 on the radar chart with the pointing device 107. Figure 16(b) has the same target impression as Figure 16(a), except for the sense of dynamism, but the sense of dynamism is turned off.

[0147] Figure 16(c) shows an example of a UI where the target impression is set from an image rather than a word. The sample poster display area 1603 displays poster images 1604 to 1607, each representing a larger or smaller impression. A checkbox 1608 is displayed next to each poster image. The user can select a poster by clicking the checkbox 1608 with the pointing device 107, which indicates the poster closest to the desired poster. The target impression specification unit 204 determines the target impression by referring to the impressions corresponding to the selected poster images. Figure 16(d) is a table showing the impressions corresponding to the poster images 1604 to 1607 in Figure 16(c) and the final target impression. For example, suppose poster images 1604 and 1607 are selected as shown in Figure 16(c). In this case, the target impression specification unit 204 determines the combined impression 1613, which is an example of impressions 1609 and 1612 respectively, as the target impression. In this example, the target impression is determined by taking the value with the maximum absolute value among the impressions corresponding to the selected poster images. While the example shown presents the poster image that maximizes each impression, this is not the only approach. Multiple poster images that enhance different impressions may be used, or more poster images than the number of impressions may be presented. This allows users to intuitively specify their target impression based on the actual poster rather than words.

[0148] <<Second Embodiment>> In the first embodiment, an example was described in which the skeleton, color scheme, and font, which are components of a poster, are selected based on the target impression, and the text decoration is controlled to generate a poster in which readability is ensured. In the second embodiment, an example was described in which a color scheme is selected based on the target impression, and a poster in which readability is ensured by the method of assigning text colors is described. More specifically, candidate colors to be assigned to text objects are defined, and the color that provides the best readability among the candidate colors is assigned to the text object. This makes it possible to generate a poster that expresses the impression desired by the user while also being highly readable.

[0149] Here, Figure 28 illustrates the effects of the second embodiment. Figure 28(a) shows candidate colors to be assigned to text objects. In Figure 28(a), three colors, Color 1 to Color 3, are candidate colors. Colors 1 to 3 are assumed to be acquired based on the target impression. Figure 28(b) shows an example where each candidate color is assigned to a text object. Color 1 is assigned to text object 2801, Color 2 to text object 2802, and Color 3 to text object 2803. In this embodiment, when there are multiple candidate colors to be assigned to a text object, the color that provides the best readability is assigned to the text object. In the case of Figure 28(b), since text object 2801, to which Color 1 is assigned, has the highest readability, Color 1 is assigned to the text object. According to this embodiment, the color that provides the best readability among the colors acquired based on the target impression is assigned to the text object. Therefore, it is possible to generate a poster that expresses the impression desired by the user while also having high readability.

[0150] The software block diagram in this embodiment is the same as that in Figure 2, so its explanation is omitted.

[0151] <Processing flow> Figure 31 is a flowchart showing the processing of the poster generation unit 210 of the poster creation application in this embodiment. Here, steps other than S3101 in Figure 31 are the same as the steps in Figure 13, so their explanation is omitted.

[0152] In S3101, the color assignment unit 1201 assigns a color pattern obtained from the color pattern selection unit 215 to the skeleton obtained from the skeleton selection unit 214. More specifically, for text objects whose metadata type is "Title", the unit determines the color to be assigned to the text object while evaluating the readability of the text. Note that the method for assigning color patterns to text objects whose metadata type is other than "Title", and to objects other than text objects, is the same as in the first embodiment, so a description is omitted in this embodiment.

[0153] The processes performed by the color assignment unit 1201 will be explained in detail using Figures 29 and 30. Figure 29 is an example of a software block diagram that explains the color assignment unit 1201 in detail. Figure 30 is a flowchart that explains the processes performed by the color assignment unit 1201 in detail.

[0154] In S3001, the character color assignment unit 2901 obtains the color to be assigned to the character object. In this embodiment, the colors included in the color scheme obtained from the color scheme pattern selection unit 215 are defined as candidate character colors, and one color is selected from among the candidate character colors. Note that the method of defining candidate character colors is not limited to this. For example, multiple colors, including white and black, added to the colors included in the color scheme, may be defined as candidate character colors. This makes it possible to set colors that are basically used as character colors.

[0155] In S3002, the character color assignment unit 2901 assigns the color obtained in S3001 to the character object.

[0156] In S3003, the evaluation value calculation unit 2902 renders the area where the character object is placed, including other objects placed in the background, into an image.

[0157] In S3004, the evaluation value calculation unit 2902 evaluates the readability of the characters. In this embodiment, readability evaluation values ​​are calculated for the characters and background. The method for calculating the readability evaluation values ​​is the same as in the first embodiment, so the explanation is omitted in this embodiment. The calculated readability evaluation values ​​are stored in association with the colors obtained in S3001.

[0158] In S3005, the determination unit 2903 determines whether or not readability evaluation values ​​have been calculated for all colors included in the candidate character colors defined in S1601. If it is determined that the calculation of readability evaluation values ​​has been completed for all colors, the process proceeds to S3006. If it is determined that the calculation of readability evaluation values ​​has not been completed for all colors, the process from S3001 is repeated.

[0159] In S3006, the character color determination unit 2904 determines the color associated with the highest readability evaluation value among the readability evaluation values ​​calculated in S3004 as the character color of the character object.

[0160] Furthermore, in this embodiment, the process was performed on a text object whose metadata type is "Title," but it is not limited to this. For example, multiple types of text objects, including "Subtitle," may be targeted, or all text objects may be targeted. This allows for control over each type for which readability is important.

[0161] Furthermore, if the metadata within a text object includes text objects other than "Title," the system may be controlled to assign different colors to each type. For example, if "Title" and "Subtitle" are targeted, and the text color determination process 30 assigns color 1 to "Title," then the "Subtitle" may be assigned the color that provides the highest readability from a color scheme other than color 1. This allows for differentiation between types.

[0162] As described above, according to this embodiment, the color scheme used for the poster is selected to match the target impression, and the color with the highest readability among the selected color schemes is assigned to the text object. This makes it possible to generate a poster that closely matches the target impression while maintaining the readability of the text.

[0163] <<Other Embodiments>> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0164] Some features of the embodiment can be summarized as follows: [1] A means of obtaining characters, A means of receiving target impressions, The system includes a poster generation means that generates a poster based on the aforementioned characters and the aforementioned target impression, The poster generation means is characterized by determining a method for decorating the characters based on the target impression, and generating a poster based on the determined method for decorating the characters and the characters themselves.

[0165] [2] The information processing apparatus according to [1], characterized in that the difference between the impression of the poster generated by the poster generation means and the target impression is less than or equal to a predetermined threshold.

[0166] [3] The poster generation means determines a method for decorating the characters based on the target impression and the readability of the characters, and generates a poster based on the determined method for decorating the characters and the characters, as described in [1] or [2].

[0167] [4] The poster includes a readability evaluation means for evaluating the readability of the characters contained in the poster, The poster generation means determines a method for decorating characters such that the readability is above a predetermined threshold, and generates a poster based on the determined method for decorating characters and the characters, as described in [3].

[0168] [5] The information processing apparatus according to [4], characterized in that the readability is the difference between the brightness of the characters and the brightness of the background area.

[0169] [6] The information processing apparatus according to [4], characterized in that the readability is the difference between the color of the characters and the color of the background area.

[0170] [7] It has an image acquisition means for acquiring images, The poster generation means is characterized in that it generates a poster based on the image, the characters, and the target impression, as described in any one of [1] to [6].

[0171] [8] The information processing apparatus according to any one of [1] to [7], characterized in that the method of decorating characters includes at least one of outlines, glows, and shadows.

[0172] [9] It has a display control means that displays a screen for receiving target impressions, The information processing apparatus according to any one of [1] to [8], characterized in that the receiving means receives a target impression via the screen.

[0173]

[10] A method for controlling an information processing device, The character acquisition process for obtaining characters, The reception process for receiving target impressions, The process includes a poster generation step that generates a poster based on the aforementioned characters and the aforementioned target impression, The poster generation step is characterized by determining a method for decorating the characters based on the target impression, and generating a poster based on the determined method for decorating the characters and the characters themselves.

[0174]

[11] A program readable by a computer for causing the computer to function as an information processing device as described in any one of paragraphs [1] to [9].

Claims

1. A means of obtaining characters, A means of receiving target impressions, The system includes a poster generation means that generates a poster based on the aforementioned characters and the aforementioned target impression, The poster generation means is characterized by determining a method for decorating the characters based on the target impression and the readability of the characters, and generating a poster based on the determined method for decorating the characters and the characters.

2. The information processing apparatus according to claim 1, characterized in that the difference between the impression of the poster generated by the poster generation means and the target impression is less than or equal to a predetermined threshold.

3. The poster includes a readability evaluation means for evaluating the readability of the characters contained in the poster, The information processing apparatus according to claim 1, wherein the poster generation means determines a method for decorating characters such that the readability is above a predetermined threshold, and generates a poster based on the determined method for decorating characters and the characters.

4. The information processing apparatus according to claim 3, characterized in that the readability is the difference between the brightness of the characters and the brightness of the background area.

5. The information processing apparatus according to claim 3, characterized in that the readability is the difference between the color of the characters and the color of the background area.

6. It has an image acquisition means for acquiring images, The information processing apparatus according to claim 1, characterized in that the poster generation means generates a poster based on the image, the text, and the target impression.

7. The information processing apparatus according to claim 1, characterized in that the method of decorating the characters includes at least one of outline, glow, and shadow.

8. It has a display control means that displays a screen for receiving target impressions, The information processing apparatus according to claim 1, characterized in that the receiving means receives a target impression via the screen.

9. A method for controlling an information processing device, The character acquisition process for obtaining characters, The reception process for receiving target impressions, The poster generation process includes generating a poster based on the aforementioned characters, the target impression, and the readability of the characters, The poster generation step is characterized by determining a method for decorating the characters based on the target impression, and generating a poster based on the determined method for decorating the characters and the characters themselves.

10. A program readable by a computer for causing the computer to function as an information processing device according to claim 1.

Images