Program, image creation support system, image creation support method, and image creation support device

Abstract

To enable further efficient creation of an image according to a customer request.SOLUTION: A program is for causing a computer to execute first reception step (step S1) for receiving first text data, first transmission step (step S3) for transmitting the first text data to an image generation AI device, first image acquisition step (step S6) for acquiring first image data generated by the image generation AI device corresponding to the first text data from the image generation AI device, and external device transmission step (step S13) for transmitting the first text data and the first image data in association with each other to an external device.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a program, an image creation support system, an image creation support method, and an image creation support device. [Background technology]

[0002] Traditionally, when salespeople at printing companies discuss images of printed materials with customers, the customers often do not have a clear image of what they want to achieve. Therefore, salespeople have to draw out an image from something that does not exist. Therefore, the salesperson creates images several times and has the customer check them. This allows the salesperson to determine the direction of the design, and then the salesperson passes the images to the designer to complete the official images.

[0003] On the other hand, image generation AI (Artificial Intelligence) such as Stable Diffusion, which generates images by inputting text, is known. Furthermore, Patent Document 1 describes a design support learning model that receives an input image of a natural object and outputs an output image in which the input image is multiplied with an artificial object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-168078 Summary of the Invention [Problem to be solved by the invention]

[0005] In the current situation described above, when a salesperson shows a customer an image created based on the fine-tuning process, the customer may feel that the image is different from what they had in mind and the work must be redone. Therefore, in order to find a design direction that would satisfy the customer, the salesperson had to create images many times, which resulted in a burden on both the customer and the salesperson in terms of man-hours and costs. Therefore, by using image generation AI such as the one mentioned above to generate images, it is possible to efficiently determine the direction of a design draft that the customer can agree on.

[0006] However, if only the image generated in this way is provided to the designer, the image will contain a mixture of parts that the customer is particular about and parts that the customer does not mind being modified. This makes it difficult for the designer to understand which parts are the client's preference and which parts can be modified. In the end, images may have to be sent back and forth between the client and the designer multiple times.

[0007] SUMMARY OF THE PRESENT EMBODIMENT It is therefore an object of the present invention to more efficiently create images that meet customer needs. [Means for solving the problem]

[0008] In order to solve the above problem, the program according to the present invention comprises: a first receiving step of receiving first text data; a first transmission step of transmitting the first text data to an image generating AI device; a first image acquisition step of acquiring, from the image generation AI device, first image data generated by the image generation AI device in response to the first text data; an external device transmission step of transmitting the first text data and the first image data to an external device in association with each other; Have the computer run the following.

[0009] In addition, the image creation support system according to the present invention comprises: a first reception unit that receives first text data; a first transmission unit that transmits the first text data to an image generation AI device; a first image acquisition unit that acquires, from the image generation AI device, first image data generated by the image generation AI device in response to the first text data; an external device transmission unit that transmits the first text data and the first image data to an external device in association with each other; Equipped with.

[0010] Further, the image creation support method according to the present invention comprises: An image creation support method performed by an image creation support system, comprising: a first receiving step of receiving first text data; a first transmission step of transmitting the first text data to an image generating AI device; a first image acquisition step of acquiring, from the image generation AI device, first image data generated by the image generation AI device in response to the first text data; an external device transmission step of transmitting the first text data and the first image data to an external device in association with each other; has.

[0011] Further, the image creation support device according to the present invention comprises: a first reception unit that receives first text data; a first image acquisition unit that acquires first image data generated in response to the first text data using an image generation AI; an external device transmission unit that transmits the first text data and the first image data to an external device in association with each other; Equipped with. Effect of the Invention

[0012] According to the present invention, images that meet customer needs can be created more efficiently. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram showing a configuration of an image creation support system. [Diagram 2]FIG. 11 is a flow diagram of an image creation support process. [Diagram 3] FIG. 13 is an image diagram of an image creation support screen. [Figure 4] FIG. 13 is an image diagram of an image creation support screen. [Diagram 5] FIG. 13 is an image diagram of a history screen. [Figure 6] FIG. 13 is an image diagram of an image creation support screen. [Figure 7] FIG. 13 is an image diagram of an image creation support screen. [Figure 8] FIG. 11 is a flow diagram of an image creation support process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the scope of the present invention is not limited to the following embodiments and drawings.

[0015] <Image Creation Support System 100> First, the configuration of an image creation support system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the image creation support system 100 includes an image creation support device 1, a cloud server 2, and an external device 3. The image creation support device 1, the cloud server 2, and the external device 3 transmit and receive information via a communication network N.

[0016] The image creation support device 1 is a device that supports image creation by users such as salespeople at printing companies and their customers. Specifically, the image creation support device 1 displays an image generated by the cloud server 2 based on the concept of text data input by the user, and accepts various operations on the image by the user. The cloud server 2 is an image generation AI device that creates images based on the concepts of the text data. The cloud server 2 is also a device that analyzes the saliency of the created images, which will be described later. The cloud server 2 may be an on-premise device. The external device 3 is a device on which the designer displays images, concepts, and the like that he or she has created. The communication network N is a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like.

[0017] <Image creation support device 1> Next, the configuration of the image creation support device 1 will be described with reference to Fig. 1. As shown in Fig. 1, the image creation support device 1 includes a control unit 11, a display unit 12, an operation unit 13, a communication unit 14, and a storage unit 15.

[0018] The control unit 11 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The CPU of the control unit 11 reads out various programs stored in the storage unit 15, expands them in the RAM, executes various processes according to the expanded programs, and controls the operations of each part of the image creation support device 1.

[0019] The display unit 12 is configured with a monitor such as an LCD (Liquid Crystal Display), and displays various screens and the like according to instructions of a display signal input from the control unit 11.

[0020] The operation unit 13 is a keyboard equipped with cursor keys, numeric input keys, various function keys, etc., a pointing device such as a mouse, a touch panel laminated on the surface of the display unit 12, etc. The operation unit 13 is configured to be operable by an operator. The operation unit 13 also outputs various signals to the control unit 11 based on operations performed by the operator.

[0021] The communication unit 14 is capable of transmitting and receiving various signals and various data to and from other devices connected via the communication network N.

[0022] The storage unit 15 is configured with a non-volatile semiconductor memory, a hard disk, or the like, and stores various programs executed by the control unit 11, parameters required for executing the programs, various data, and the like.

[0023] <Cloud Server 2> Next, the configuration of the cloud server 2 will be described with reference to Fig. 1. As shown in Fig. 1, the cloud server 2 includes a control unit 21, a communication unit 22, and a storage unit . Cloud server 2 functions as an image generation AI and as a saliency analysis module.

[0024] The control unit 21 is composed of a CPU, a RAM, etc. The CPU of the control unit 21 reads out various programs stored in the storage unit 23, loads them in the RAM, executes various processes according to the loaded programs, and controls the operation of each part of the cloud server 2. The control unit 21 can extract design requirements from the concept of the text data. Design requirements are subdivisions of elements included in a concept. For example, if the concept is "an easy miso sauce that makes young people living alone want to make delicious boiled fish," the control unit 21 can extract design requirements such as "fish dishes," "stews," "miso sauce," "beginners," and "for young people." The control unit 21 can create a design image using an image generation AI based on the concept, design requirements, and images. Examples of image generation AI include Stable Diffusion, DALL·E2, Midjourney, starryai, and Dream by WOMBO. The control unit 21 can analyze the saliency of the created design image. That is, the control unit 21 can display the analysis result regarding saliency in the following cases. Specifically, this is the case when an evaluator (user, etc.) wants to know whether or not a part of an image that he or she wants to make especially visually prominent has been made prominent. When a part that he or she wants to make especially visually prominent has been made prominent, it is said that there is saliency. Also, this is the case when an evaluator wants to know how to make that part of the image to be evaluated more prominent.

[0025] <Saliency analysis method> Here, the saliency analysis method will be described. First, the functions of the control unit 21 will be described in detail. The control unit 21 functions as a feature extraction unit and a generation unit.

[0026] The control unit 21, which functions as a feature extraction unit, extracts low-order image feature amounts and high-order image feature amounts from the acquired evaluation target image. A specific method in which the control unit 21 acting as a feature extracting unit extracts low-order image feature amounts and high-order image feature amounts from the evaluation target image will be described later.

[0027] Here, the low-order image feature is a physical image feature including, for example, color, brightness, direction (edge ​​direction, shape), etc., and is a component that exogenously or passively guides a person's gaze to gaze. In this embodiment, the low-order image feature is a concept that broadly includes at least one of color, brightness distribution contrast, and movement. The impact an image has on the viewer and the degree to which it attracts attention (prominence, salience) varies depending on factors such as the color contrast used in each part of the image (for example, color differences in the red-green direction and yellow-blue direction), the distribution of brightness contrast (luminance difference) between parts, and directional (orientation) contrast.

[0028] For example, a portion (such as a boundary portion) with a large color difference along the red-green or yellow-blue direction is likely to attract the gaze and become highly salient. Furthermore, for example, when the entirety is arranged in a fixed direction, if there is something arranged in a different direction (edge ​​direction), the line of sight tends to be drawn to that part.

[0029] Furthermore, the image to be evaluated is not limited to a still image, but may be a moving image. When the image to be evaluated is a moving image, various actions (movements, movements) in the image also affect the degree of the viewer's gaze. For example, when an image in which the whole image moves in a constant direction at a nearly constant speed has one part moving at a different speed or in a different direction, the gaze tends to be drawn to that part.

[0030] The high-order image features are physiological and mental image features that reflect people's memories, experiences, knowledge, etc., and are components that intrinsically and actively guide people's gaze to fixate. More specifically, they are components derived from people's mental and psychological tendencies, gaze movement tendencies, etc. that are said to affect the impact on the person viewing the image and the degree of fixation (prominence, salience). In this embodiment, the high-order image features include at least one degree of position bias, processing fluency, and face components.

[0031] For example, position bias includes the following tendencies in eye movement. Specifically, it is a center bias, where the gaze tends to be drawn to things in the center of an image. For example, in magazines and web pages, the gaze tends to move from the upper left to the lower right of the image, and the gaze tends to be drawn to the upper left. When viewing a document written vertically, the gaze tends to move from the upper right to the lower left, and the gaze tends to be drawn to the upper right. For example, when considering going to a store such as a supermarket, the gaze tends to be drawn to parts of the store layout that are close to eye level. As described above, the positional bias affects the degree of gaze (prominence, salience) of a person viewing an image or the like.

[0032] Moreover, processing fluency indicates that, in general, people find it easy to process simple things that are easy to recognize, and difficult to process complex things that are difficult to understand. In this embodiment, people tend to focus their gaze on and gaze at parts of an image that are easy to recognize and have high processing fluency, and tend not to focus on parts that are difficult to recognize and have low processing fluency. As described above, processing fluency affects the degree of attention (prominence, salience) given by the viewer to an image.

[0033] In this embodiment, the degree of processing fluency includes that determined by at least one of the complexity, density of the depicted object, and spatial frequency of the luminance distribution. In other words, the parts that are difficult to recognize are the messy and complex parts, i.e., the parts drawn in the image. In areas where objects are crowded together in a cluttered manner, there are sudden changes in the image such as edges, and the spatial frequency of the luminance distribution is high in these areas. Processing fluency is low in areas where the complexity, density of the depicted objects, and spatial frequency of the luminance distribution are too high. On the other hand, areas where the complexity, density of depicted objects, and spatial frequency of the luminance distribution are too low, in other words, areas that do not contain information, are difficult to read, are hard for the human brain to process, and tend not to be focused on.

[0034] Furthermore, when there is a part in an image that is recognized as a face, there is a general tendency for people to focus on that part. In other words, parts that are recognized as a face tend to be highly salient.

[0035] Furthermore, the high-level image features may include characters and fonts. When the elements that make up an image are legible characters, the degree to which the viewer focuses on them also varies depending on the type and size of the font. Fonts contain characters in a specific style, and come in a variety of styles, including print, block, and cursive. The degree to which the viewer focuses on something can vary depending on the font in which it is expressed. Larger characters also tend to attract more attention than smaller characters, even within the same style.

[0036] In addition, the control unit 21, as a generation unit, generates a feature saliency map for each type of image feature that indicates the saliency in the image to be evaluated based on the image feature, and generates a saliency map that integrates all the feature saliency maps. A specific method in which the control unit 21 as a generating unit generates the saliency map will be described later. The control unit 21 then performs blurring processing on the evaluation target image using a Gaussian filter. The blurring processing is processing to reduce the resolution of the image. Specifically, a group of images (multi-resolution representation of the image, Gaussian pyramid) in which multiple Gaussian filters with different blurring levels are applied to the evaluation target image in a stepwise manner is generated for each low-order image feature.

[0037] When the control unit 21 generates an image group (Gaussian pyramid) for each component of the image feature, the control unit 21 uses this multi-resolution representation to obtain (calculate) inter-image differences with different scales for each element of the image feature. In the process of calculating the inter-image difference, the control unit 21 converts at least one of the color difference and the luminance difference from the RGB data to L * a * b * Calculated based on color space. * a * b * The color space is more suited to human perception of color difference than the RGB color space. Therefore, at least one of the color difference and luminance difference is expressed as L * a * b *Calculation in color space has the following effects. Specifically, the luminance contrast and chromaticity contrast values ​​extracted from the image to be evaluated can be expressed as lightness difference and color difference that match human sense. Therefore, the saliency indicated by the finally obtained saliency map can be made to match human sense more closely.

[0038] When the difference image is acquired, the control unit 21 normalizes it and combines feature amount maps of all scales for each component of the image feature amount. Next, the control unit 21 generates a feature amount saliency map according to the feature amount map. When viewed for each low-level image feature, the feature saliency map is a saliency map that indicates that areas with large color contrasts (red-green or yellow-blue color differences) are highly salient for color components. For luminance components, the map indicates that the boundary between the black and white parts of a laptop computer screen is highly salient. For directional components, the edges of the notebook or laptop computer are indicated as areas of high salience.

[0039] In this embodiment, processing fluency (complexity, etc.), position bias, and face components are extracted as high-order image features by the control unit 21 as a feature extraction unit. Although processing fluency, position bias, and face components are exemplified as high-level image features here, as described above, the high-level image features are not limited to these. The high-level image features may also include various elements (components) other than these.

[0040] As mentioned above, processing fluency can be measured by the degree of complexity, and can be analyzed and quantified, for example, using a technique called fractal dimension. That is, the image to be evaluated is divided into multiple meshes, and an analysis is performed to determine which parts have a dense structure represented by dots, and which parts have a sparse structure. As a result, parts with a high fractal dimension are evaluated as complex and messy. Parts with a low fractal dimension are evaluated as simple and low in information.

[0041] As mentioned above, the background area where almost no information exists has a low fractal dimension, but is not very noticeable and has low saliency. Therefore, as a feature saliency map for processing fluency, the background area where almost no information exists or areas that are too complex are evaluated as having low saliency, and areas with moderate complexity are evaluated as having the highest saliency.

[0042] Furthermore, the control unit 21 generates a feature saliency map of position bias according to the location and direction to which the gaze is likely to be guided when considering the psychological characteristics of a person, according to the characteristics and type of the evaluation target image. The characteristics and type of the image include, for example, whether it is an image intended to be published in a book or a web page, an image to be inserted into a vertically written document, etc. For example, if the image to be evaluated is one that will be posted on a web page, the map will show high salience in the upper left corner of the screen and low salience in the lower right corner.

[0043] Furthermore, the control unit 21 extracts areas that can be recognized as faces from the evaluation target image using a face area detection AI or the like, and generates a face component feature saliency map. In the face component feature saliency map, areas that can be recognized as faces have high saliency.

[0044] Then, when the feature amount saliency maps are generated for the low-order and high-order image feature amounts, the control unit 21 integrates the feature amount saliency maps. Then, the control unit 21 calculates where the gaze will go as a whole when a person looks at the evaluation target image, and which part will attract high attention and gaze.

[0045] Furthermore, in the process of integrating all feature amount saliency maps, the control unit 21 generates the saliency map so as to maximize the sum of the similarities between the saliency map and all the feature amount saliency maps. Specifically, the control unit 21 generates the saliency map so as to satisfy the following formula (1).

number

[0046] By generating a saliency map so as to satisfy the above formula (1), the following saliency map can be generated. Specifically, when multiple feature amount saliency maps emphasize the same parts in the image to be evaluated, the saliency map emphasizes those same parts to the same degree as the feature amount saliency map. In addition, a case will be described in which one feature saliency map overemphasizes a portion of the image to be evaluated, while another feature saliency map does not overemphasize that portion. In this case, the saliency map does not overemphasize that portion.

[0047] Let's return to explaining the configuration of cloud server 2. The communication unit 22 is capable of transmitting and receiving various signals and various data to and from other devices connected via the communication network N.

[0048] The storage unit 23 is configured with a non-volatile semiconductor memory, a hard disk, or the like, and stores various programs executed by the control unit 11, parameters required for executing the programs, various data, and the like.

[0049] <External device 3> Next, the configuration of the external device 3 will be described with reference to Fig. 1. As shown in Fig. 1, the external device 3 includes a control unit 31, a display unit 32, an operation unit 33, a communication unit , and a storage unit .

[0050] The control unit 31 is composed of a CPU, a RAM, etc. The CPU of the control unit 31 reads out various programs stored in the storage unit 35, loads them in the RAM, executes various processes according to the loaded programs, and controls the operations of each part of the external device 3.

[0051] The display unit 32 is configured with a monitor such as an LCD, and displays various screens and the like according to instructions of a display signal input from the control unit 31.

[0052] The operation unit 33 is a keyboard equipped with cursor keys, numeric input keys, various function keys, etc., a pointing device such as a mouse, a touch panel laminated on the surface of the display unit 32, etc. The operation unit 33 is configured to be operable by an operator. In addition, the operation unit 33 outputs various signals to the control unit 31 based on operations performed by the operator.

[0053] The communication unit 34 is capable of transmitting and receiving various signals and various data to and from other devices connected via the communication network N.

[0054] The storage unit 35 is configured with a non-volatile semiconductor memory, a hard disk, or the like, and stores various programs executed by the control unit 11, parameters required for executing the programs, various data, and the like.

[0055] <Image creation support processing> Next, the image creation support process will be described with reference to FIG. The image creation support process is a process that supports the user in creating an image based on the concept of the text data entered by the user, such as a salesperson of a printing company or a customer of the printing company. Finally, the created image data, concept, and design requirements are sent to the designer's terminal (external device 3). It is assumed that the image creation support screen 12A shown in FIG.

[0056] Before describing the image creation support process, the screens displayed on the display unit 12 will be described. The image creation support screen 12A shown in FIG. 3 will be described. Area A1 is where the user enters text for a concept. Area A2 is an area for displaying an image generated by cloud server 2. The image generated by cloud server 2 is an image that is the basis of a final design image, and may be generated with at least one of the resolution, number of gradations, and number of colors lower than the final design image. Area A3 is an area that displays design requirements extracted from the concept by cloud server 2. Area A4 is an area where the user can input any comment. Area A5 is where the user enters text for additional concepts. Button B1 is a button for selecting the style of the image generated by cloud server 2. For example, the style may be pictogram style, illustration style, etc. Button B2 is a button for selecting a feeling to be felt from the image generated by the cloud server 2. For example, the feeling may be energetic, casual, elegant, etc. Button B3 is a button for selecting the style of the image generated by the cloud server 2. The style is image characteristics that can be obtained by learning a group of past images of the same customer. Specifically, if a customer has previously requested the creation of a design image for a product label, the image generation AI of the cloud server 2 is trained using an image from which only the text portion of that design image has been deleted, and the image characteristics are acquired. Button B4 is a button for inputting characters into the image displayed in area A2. Button B5 is a button for adjusting the image displayed in area A2. For example, adjusting the image refers to adjusting the size, position, transparency, etc. of components of the image generated by cloud server 2. Button B6 is a button for performing a saliency analysis of the image displayed in area A2. The button B7 is a button for drawing an image in the area A2 using various information inputted into the image creation support screen 12A. Button B8 is a button for deciding (confirming) the image displayed in area A2. When any of the buttons B1 to B6 is pressed, a pop-up screen opens, allowing various selections, inputs, and analysis results to be displayed.

[0057] As shown in FIG. 3, in the initial display of image creation support screen 12A, area A1 and buttons B1, B2, B3, and B7 are active, and the user can use operation unit 13 to input text or press a button. FIG. 4 shows the image creation support screen 12A after button B7 is pressed, with an image generated by the cloud server 2 displayed in area A2. A thumbnail image is displayed in the upper left of area A2, and an image corresponding to the selected thumbnail image is displayed on the right. Area A3 displays a list of design requirements extracted by the cloud server 2 from the text data of the concept entered in area A1. By unchecking the boxes displayed in area A3, it is also possible to remove the image configuration from the images in area A2 that corresponds to the removed design requirement.

[0058] FIG. 5 shows the history screen 12B, and a history is recorded each time the button B7 is pressed. Area A6 is an area for displaying a history number. Area A7 is an area for displaying the image displayed in area A2. Area A8 is an area for displaying the design requirements displayed in area A3. Area A9 is an area for displaying the comments entered in area A4. Area A10 is a check box for selecting the history to be displayed on display unit 12. Button B9 is a button for displaying the history of items checked in area A10 on display unit 12. When button B9 is pressed, image creation support screen 12A is displayed on display unit 12.

[0059] The image creation support screen 12A and the history screen 12B are switched by switching the selection of tabs T1 and T2.

[0060] Returning to the description of the image creation support process. First, the user inputs a concept as text in the area A1 using the operation unit 13. The control unit 11 accepts the input text data (step S1, first reception step). Note that the present invention is not limited to the example of accepting text data entered as text in the area A1, and text data may be accepted from an external device (e.g., a user terminal). Next, the user uses the operation unit 13 to press the button B3 to display a pop-up screen on the display unit 12, and selects an image showing the style displayed on the pop-up screen. The control unit 11 accepts the selected style (step S2, style information acceptance step). Note that although the style is shown as an example here, style or sensibility may also be used. Next, the user presses button B7 using the operation unit 13. The control unit 11 transmits the data accepted in steps S1 and S2 to the cloud server 2 via the network N (step S3, first transmission step).

[0061] Next, the control unit 21 extracts design requirements using the data received via the network N (step S4). Next, the control unit 21 generates an image by the image generation AI using the data and design requirements received via the network N (step S5). Next, the control unit 21 transmits the design requirement data extracted in step S4 and the image data generated in step S5 to the image creation support device 1 via the network N (step S6, first image acquisition step).

[0062] Next, the control unit 11 uses the data received via the network N to cause the display unit 12 to display the image creation support screen 12A shown in FIG. 4 described above (step S7, display step). Furthermore, the control unit 11 stores data received via the network N in the storage unit 15 as history.

[0063] Next, the user presses button B6 using the operation unit 13. The control unit 11 instructs the cloud server 2 to perform saliency analysis of the image via the network N (step S8). In this embodiment, since the cloud server 2 has both a function as an image generation AI and a function as a saliency analysis module, the control unit 11 instructs the cloud server 2 to perform saliency analysis of the image generated by the cloud server 2. If the saliency analysis module is a device separate from the cloud server 2, the control unit 11 transmits the image data to the saliency analysis module (step S8, third transmission step).

[0064] Next, the control unit 21 performs a saliency analysis of the image generated in step S5 (step S9). Next, the control unit 21 transmits the saliency analysis data to the image production support device 1 via the network N (step S10, saliency analysis result acquisition step). The control unit 11 uses the data received via the network N to display the saliency analysis result in a pop-up.

[0065] Next, the user selects an image in area A2 and selects (deletes) a design requirement in area A3 on image creation support screen 12A shown in Fig. 4. Control unit 11 accepts the selection of the image and the design requirement (step S11, image selection step). Each design requirement corresponds to a component of the image displayed in area A2, and when the user selects or deletes a design requirement, control unit 11 deletes or returns the component corresponding to the design requirement from the image displayed in area A2.

[0066] Next, if there is an additional concept on the image creation support screen 12A shown in Fig. 4 (step S12; YES), the user inputs the additional concept in text form into the area A5. The control unit 11 advances the image creation support process to step S1. Note that, from the second loop of the image creation support process onwards, the control unit 11 may also transmit data on the image displayed in the area A2, design requirements, etc. (associated with each other) in step S3. From the second loop of the image creation support process onwards, the above-mentioned first reception step, first transmission step, and first image acquisition step may be read as "second" and "third". If there is no additional concept (step S12; NO), the user presses the button B8. When the control unit 11 accepts the pressing of the button B8, it advances the image creation support process to step S13. Next, the control unit 11 transmits various data input or displayed on the image creation support screen 12A to the external device 3 via the network N (step S13, external device transmission step).

[0067] Next, the control unit 31 receives various data via the network N and causes the display unit 32 to display the data (step S14). The various data includes not only images, but also concepts, design requirements (including not only required requirements but also prohibited requirements), saliency, style, sensibility, artistic style, history, and comments from customers on their impressions of the generated images, allowing designers to understand the direction of the design.

[0068] <Other> The image creation support screen may be a screen such as an image creation support screen 12C shown in FIG. 6 or an image creation support screen 12D shown in FIG. In image creation support screen 12C shown in FIG. 6, when the user directly inputs design requirements in area A11 and presses the confirm button on the right, control unit 11 adds the design requirements to the list in area A13. Moreover, when the user presses a button B10 on the image creation support screen 12C shown in FIG. 6, the displayed list of design requirements is transmitted to the cloud server 2, and the control unit 31 performs image generation. In addition, the image creation support screen 12C shown in FIG. 6 displays a history like that in area A15 (the changes in design requirements and images in the order of (1)(2)(3)(4)) on the same screen, making it easy for the user to trace the changes in design requirements and images. In the image creation support screen 12D shown in Fig. 7, the user directly inputs design requirements in area A16, and when he presses the add button to the right, it is added to the list of design requirements in area A18. Area A18 is provided with check boxes and a trash can, and the user can select design requirements or delete the design requirements themselves. The user can also input any comments in area A20.

[0069] In the flow of the image creation support process shown in FIG. 2, when the button B8 (OK) is pressed, the control unit 11 transmits data to the external device 3. When button B8 (decision) is pressed, control unit 11 may be configured to transmit the data as decision data to cloud server 2. Then, based on an acquisition instruction from external device 3 to cloud server 2, control unit 21 may be configured to transmit the data to external device 3.

[0070] The text data may be based on at least one of free text and selection from prepared text.

[0071] Cloud server 2 has the functionality of an image generation AI, so in addition to generating images from text, it may also have the following functions: The cloud server 2 may obtain user information of the image creation support device 1, obtain learning data that can be used to identify the user from the storage unit 23, and generate an image based on the learning data. The cloud server 2 may reflect the determined image result the next time an image is generated. The cloud server 2 may avoid imitating images that already exist by deliberately creating learning data with low purity, or by determining the degree of similarity and excluding images that are above a certain level.

[0072] In the above, the image creation support process has been described with the image creation support device 1 and the cloud server 2 as separate devices, but the image creation support device 1 may be given various functions of the cloud server 2, and the image creation support process may be executed in the image creation support device 1. The flow of the image creation support process in this case is shown in Fig. 8. The processing content of each step is similar to that of the image creation support process shown in Fig. 2, but data transmission and reception between the image creation support device 1 and the cloud server 2 is not required.

[0073] <Effects> As described above, the program causes the computer of the image creation support device 1 to execute a first reception step (step S1) of receiving first text data, a first transmission step (step S3) of transmitting the first text data to the image generation AI device, a first image acquisition step (step S6) of acquiring from the image generation AI device first image data generated by the image generation AI device in response to the first text data, and an external device transmission step (step S13) of associating the first text data with the first image data and transmitting them to an external device. This allows images that meet customer needs to be created more efficiently. In addition, even if the customer's needs are vague and they do not have a specific image, an image that draws out the customer's needs can be created efficiently. In other words, the customer can solidify their image by looking at the created image while talking to the salesperson. In addition, the salesperson can ultimately reach a common understanding and reach a consensus with the customer based on that image.

[0074] The program also has a second receiving step (step S1) of receiving second text data as a comment on the first image data, and an external device sending step of associating the first text data, the first image data, and the second text data and sending them to the external device. This allows images to be polished more efficiently to meet customer needs.

[0075] The program also has a second transmission step (step S3) of transmitting the first image data and the second text data to the image generation AI device, and a second image acquisition step (step S6) of acquiring second image data from the image generation AI device corresponding to the first image data and the second text data, and the external device transmission step associates the first text data, the first image data, the second text data, and the second image data and transmits them to the external device. This allows images to be polished more efficiently to meet customer needs.

[0076] The program also has a third receiving step (step S1) of receiving third text data as a comment on the second image data, and an external device sending step of associating the first text data, the first image data, the second text data, the second image data, and the third text data and sending them to the external device. This allows images to be polished more efficiently to meet customer needs.

[0077] Moreover, the external device transmitting step transmits based on an acquisition instruction from the external device. This allows the designer to receive data at any time.

[0078] The text data is based on at least one of free description and selection from prepared text. This improves the usability of the screen.

[0079] The program also has a display step (step S7) of acquiring design requirements extracted from the text data by the image generation AI device and displaying the extracted design requirements on the display unit. This allows clients to easily see their design requirements from concept to key points.

[0080] Additionally, for each design requirement displayed, the option to delete it is available. This allows customers to easily select the design requirements they wish to use.

[0081] The program also has a display step (step S7) in which the first image data acquired in the first image acquisition step includes multiple images, and an image selection step (step S11) in which the user selects a desired image from the multiple displayed images, and transmits the selected desired image as the first image in an external device transmission step. This allows images that meet customer needs to be created more efficiently.

[0082] The program also has a style information receiving step (step S2) capable of receiving input of style information. This allows images to be created more efficiently in accordance with the style desired by the customer.

[0083] Moreover, the image style information is image feature data obtained by learning a group of past images of the same customer. This allows images to be created more efficiently in accordance with the style desired by the customer.

[0084] Moreover, the first image is an image that is the basis of the final design image, and is lower in at least one of the resolution, number of gradations, and number of colors than the final design image. This makes it possible to prevent copying of images that already exist.

[0085] The program also has a third transmission step (step S8) of transmitting the first image data to the saliency analysis device, and a saliency analysis result acquisition step (step S10) of acquiring a saliency analysis result of the first image data or the second image data from the saliency analysis device, and has a display step (step S10) of displaying the acquired saliency analysis result on a display unit. This makes it easier to objectively evaluate the saliency of the generated image.

[0086] In addition, the image creation support system 100 includes a first reception unit (control unit 11) that receives first text data, a first transmission unit (communication unit 14) that transmits the first text data to the image generation AI device, a first image acquisition unit (control unit 11) that acquires first image data generated by the image generation AI device in response to the first text data from the image generation AI device, and an external device transmission unit (communication unit 14) that associates the first text data and the first image data and transmits them to an external device. This allows images that meet customer needs to be created more efficiently.

[0087] The image creation support method is an image creation support method performed by an image creation support system, The method includes a first receiving step (step S1) of receiving first text data, a first transmitting step (step S3) of transmitting the first text data to an image generation AI device, a first image acquiring step (step S6) of acquiring first image data generated by the image generation AI device in response to the first text data from the image generation AI device, and an external device transmitting step (step S13) of associating the first text data and the first image data with each other and transmitting them to an external device. This allows images that meet customer needs to be created more efficiently.

[0088] In addition, the image creation support device 1 includes: a first reception unit (control unit 11) that receives first text data; A first image acquisition unit (control unit 11) that acquires first image data generated in response to the first text data using an image generation AI; an external device transmission unit (communication unit 14) that associates the first text data and the first image data and transmits them to an external device; Equipped with. This allows images that meet customer needs to be created more efficiently.

[0089] The present invention has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified as appropriate without departing from the spirit of the present invention. For example, in the above description, a hard disk or a non-volatile semiconductor memory is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As another computer-readable medium, a portable recording medium such as a CD-ROM can be used.

[0090] In addition, the detailed configuration and operation of each device may be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0091] 100 Image Creation Support System 1 Image creation support device 11 control unit (first reception unit, first image acquisition unit) 12 Display section 13 Control section 14 Communication unit (first transmission unit, external device transmission unit) 15 Storage section 2. Cloud Server 21 Control section 22 Communications Department 23 storage section 3 External device 31 Control Unit 32 Display section 33 Operation section 34 Communications Department 35 Storage section N Communication Network

Claims

1. a first receiving step of receiving first text data; a first transmission step of transmitting the first text data to an image generating AI device; a first image acquisition step of acquiring first image data generated by the image generation AI device in response to the first text data from the image generation AI device; an external device transmission step of transmitting the first text data and the first image data to an external device in association with each other; A program for causing a computer to execute the following.

2. a second receiving step of receiving second text data as a comment on the first image data; 2. The program according to claim 1, wherein the external device transmission step transmits the first text data, the first image data, and the second text data to the external device in association with each other.

3. a second transmitting step of transmitting the first image data and the second text data to the image generating AI device; A second image acquisition step of acquiring second image data corresponding to the first image data and the second text data from the image generation AI device, 3. The program according to claim 2, wherein the external device transmission step transmits the first text data, the first image data, the second text data, and the second image data to the external device in association with each other.

4. a third receiving step of receiving third text data as a comment on the second image data; 4. The program according to claim 3, wherein the external device transmission step transmits the first text data, the first image data, the second text data, the second image data, and the third text data to the external device in association with each other.

5. 5. The program according to claim 1, wherein the external device transmission step transmits the information based on an acquisition instruction from the external device.

6. 2. The program according to claim 1, wherein the text data is based on at least one of free description and selection from prepared text.

7. The program according to claim 1 , further comprising a display step of acquiring design requirements extracted from the text data by the image generation AI device and displaying the extracted design requirements on a display unit.

8. 8. The program according to claim 7, wherein a deletion option is available for each displayed design requirement.

9. the first image data acquired in the first image acquisition step includes a plurality of images; a display step of displaying the plurality of images on a display unit; an image selection step of allowing a user to select a desired image from the displayed plurality of images; 2. The program according to claim 1, wherein the selected desired image is transmitted as the first image in the external device transmitting step.

10. 2. The program according to claim 1, further comprising a style information receiving step capable of receiving input of style information.

11. The program according to claim 10, wherein the image style information is image feature data obtained by learning a group of past images of the same customer.

12. 2. The program according to claim 1, wherein the first image data is image data that is a basis for a final design image, and is image data that is lower in at least one of resolution, number of gradations, and number of colors than the final design image.

13. a third transmitting step of transmitting the first image data to a saliency analysis device; a saliency analysis result acquisition step of acquiring a saliency analysis result of the first image data from the saliency analysis device; The program according to claim 1 , further comprising a display step of displaying the acquired saliency analysis result on a display unit.

14. a first receiving unit that receives first text data; a first transmission unit that transmits the first text data to an image generation AI device; a first image acquisition unit that acquires from the image generation AI device first image data generated by the image generation AI device in response to the first text data; an external device transmission unit that transmits the first text data and the first image data to an external device in association with each other; An image creation support system comprising:

15. An image creation support method performed by an image creation support system, comprising: a first receiving step of receiving first text data; a first transmission step of transmitting the first text data to an image generating AI device; a first image acquisition step of acquiring first image data generated by the image generation AI device in response to the first text data from the image generation AI device; an external device transmission step of transmitting the first text data and the first image data to an external device in association with each other; The image creation assistance method includes the steps of:

16. a first receiving unit that receives first text data; a first image acquisition unit that acquires first image data generated in response to the first text data using an image generation AI; an external device transmission unit that transmits the first text data and the first image data to an external device in association with each other; An image creation support device comprising:

Images