Computer program, information processing apparatus, information processing method, and image generation model generation method

Abstract

To provide a computer program, an information processor, an information processing method, and an image generation model generation method capable of generating a pattern derived from a natural material.SOLUTION: Acquiring at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquiring a pattern image derived from the natural material generated by inputting the acquired at least one of the attribute information and the reference image to an image generation model, and outputting the acquired pattern image derived from the natural material.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing device, an information processing method, and an image generation model generation method. [Background technology]

[0002] Conventionally, methods for creating wood grain pattern designs for interior and exterior building sheets or decorative sheets for furniture and interiors have involved, for example, (1) cutting out valuable logs with high design value to serve as the original, (2) processing the cut logs into wooden boards, (3) polishing the wooden boards or applying powder or the like to create the desired finish, (4) photographing the finished wooden boards, (5) editing the photographed images to add texture such as color tone and growth rings, vessels, and knots, and (6) performing endless (repeat) processing to match the printing press.

[0003] However, conventional methods not only require costs (in terms of money and time) for a series of processes, from cutting the logs to photographing them and processing the images, but also pose problems in terms of protecting the natural environment.Patent Document 1 discloses a method in which, without using natural materials, conditions related to tree growth are given to a computer, a tree model that grows based on the given conditions is generated, and a wood grain pattern is generated from the generated tree model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-230754 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method of Patent Document 1 involves growing a tree model in a computer, and then using cut surface creation software to create a cut surface of the tree model based on the grown tree model, thereby creating a wood grain pattern. This requires trying out a variety of different ways to create the cut surface, making it difficult to determine the optimal cut surface, and therefore requiring trial and error when generating a new wood grain pattern. Furthermore, there is also a demand for the creation of patterns derived from natural materials such as stone and bamboo, in addition to wood grain patterns.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a computer program, an information processing device, an information processing method, and an image generation model generation method that are capable of generating patterns derived from natural materials. [Means for solving the problem]

[0007] The present application includes multiple means for solving the above-mentioned problems, and as one example, a computer program causes a computer to execute a process of acquiring at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquiring an image of the pattern derived from the natural material generated by inputting at least one of the acquired attribute information and reference image into an image generation model, and outputting the acquired image of the pattern derived from the natural material. [Effects of the Invention]

[0008] According to the present invention, patterns derived from natural materials can be created. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a first example of the configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of attribute information. [Figure 3] FIG. 10 is a diagram illustrating an example of a method for generating an image generation model. [Figure 4] FIG. 10 is a diagram illustrating an example of a training dataset. [Figure 5]FIG. 10 is a diagram illustrating an example of a method for generating a pattern image derived from natural materials using an image generation model. [Figure 6] FIG. 10 is a diagram illustrating an example of a prompt when generating a layout. [Figure 7] FIG. 10 is a diagram showing an example of a prompt when generating material. [Figure 8] FIG. 1 is a diagram showing an example of a technical expression and a non-technical expression corresponding to the technical expression. [Figure 9] FIG. 2 is a diagram illustrating a second example of the configuration of the information processing system according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example of application to a 3D presentation system. [Figure 11] FIG. 10 is a diagram illustrating an example of application to the proposed system. [Figure 12] FIG. 10 is a diagram showing an example of an ordering transaction for a component having a pattern image derived from natural materials generated by an information processing device. [Figure 13] FIG. 10 is a diagram illustrating an example of a processing procedure for generating an image generation model. [Figure 14] FIG. 10 is a diagram illustrating an example of a processing procedure for generating a pattern image by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing a first example of the configuration of an information processing system according to this embodiment. The information processing system includes an information processing device 50. The information processing device 50 is connected to a training database 100 and a generation AI 200 via a communication network 1.

[0011] The generation AI 200 is an AI that can generate various content, such as sentences (text), images, audio, music, and videos. The generation AI 200 can learn from an unstructured data set and generate new content. In this embodiment, the generation AI 200 will be described using an image generation AI as an example.

[0012] Image generation AI consists of two main components: a text encoder and an image generator. The text encoder converts input text into feature vectors, and the image generator converts the converted feature vectors into images. A feature vector is a string of numbers that represents the meaning of the input text or the content of an image. Feature vectors with similar text meanings are located close to each other in vector space. The role of the text encoder is to understand the natural language input to the image generation AI. Pre-training for natural language understanding is performed using language modeling. Language modeling divides the text data into small chunks (which can be tokens) and performs an embedding process on each chunk to generate a vector representation (embedding) of each chunk. The embedding process includes quantifying the type of chunk and its relative positions. This allows each chunk to be represented as a point in a high-dimensional space, where chunks with similar meanings are located close to each other and chunks with different meanings are located far apart. Text encoding can numerically capture the semantic relevance and similarity between chunks and understand the semantics, or meaning, of text data. The text encoder can be, for example, but is not limited to, a Transformer.

[0013] The image generator uses, for example, a diffusion model. In the diffusion process, noise following a normal distribution is gradually added to the data to generate an image with complete noise following a normal distribution added, and in the de-diffusion process, the noise added to the image is gradually removed to generate an image. The noise to be removed at each step is estimated by a neural network (e.g., U-Net, Transformer, etc.). The feature vector converted by the text encoder is input to the neural network.

[0014] The image generation model 210 is generated and registered using training data including training images 110 recorded in the training database 100 and attribute information 120 corresponding to the training images 110. In other words, the image generation model 210 is a model capable of generating unique images by learning from independently collected and created images and attribute information corresponding to the images. The image generation model 210 can generate pattern images derived from natural materials. Natural material patterns include, for example, wood grain patterns, stone patterns, and bamboo patterns. In this embodiment, a wood grain pattern will be used as an example of a natural material pattern. The image generation model 210 can be, but is not limited to, models such as DALL·E, StyleGAN, TransGAN, and Stable Diffusion. While the example in FIG. 1 illustrates the image generation model 210 as being included in the generation AI 200, this is not limiting, and the information processing device 50 may also include the image generation model 210.

[0015] The training database 100 is a database that records training data for learning (generation) of the image generation model 210. The training database 100 records training images 110 and attribute information 120.

[0016] The training images 110 include, for example, pattern data derived from natural materials previously created by experts (e.g., designers) on patterns derived from natural materials, scanned image data of natural materials such as raw wood boards, and pattern data of various natural materials created and accumulated through open sources or within a company (in-house). The training images 110 may also include image data of pattern images derived from natural materials generated by the image generation model 210.

[0017] The attribute information 120 is information that represents a pattern derived from a natural material, and specifically includes various information for expressing a pattern derived from a natural material. The attribute information 120 will be described in detail later.

[0018] The information processing device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a memory 53, a display unit 54, an operation unit 55, a learning processing unit 56, a generation instruction unit 57, a storage unit 59, and an export unit 61. Note that the information processing device 50 may be configured with multiple information processing devices, for example, with functions distributed.

[0019] The control unit 51 may be configured by incorporating a required number of central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), etc. The control unit 51 may also be configured by combining digital signal processors (DSPs), field-programmable gate arrays (FPGAs), etc.

[0020] The communication unit 52 includes a communication module and has the function of communicating with the training database 100 and the generation AI 200 via the communication network 1.

[0021] The display unit 54 is configured with a liquid crystal display, an organic EL display, or the like, and provides a UI (user interface) to the user by displaying required information. The display unit 54 is equipped with a microphone and a speaker, and can input and output audio. Note that an external display device may be used instead of the display unit 54.

[0022] The operation unit 55 is configured, for example, by a touch panel, and can operate icons displayed on the display unit 54, move and operate a cursor, input characters, etc. The operation unit 55 may be configured by buttons, switches, etc., or may be configured by a keyboard, mouse, etc. The operation unit 55 provides a UI to the user by accepting user operations. Note that an external terminal device for operation may be provided instead of the operation unit 55.

[0023] The learning processing unit 56 performs learning (generation) processing of the image generation model 210 under the control of the control unit 51 .

[0024] The generation instruction unit 57 includes a prompt receiving unit 58, and outputs an instruction to the image generation model 210 to generate a pattern image derived from natural materials.

[0025] The prompt receiving unit 58 receives prompts, which are input to the image generation model 210. That is, the prompt receiving unit 58 receives prompts input by a user. The prompts include elements such as a task (command) to be executed by the image generation model 210 and attribute information that enables the image generation model 210 to understand the task and make an appropriate decision. The prompts also include positive prompts that specify elements that the image generation model 210 should include in a pattern image derived from natural materials generated by the image generation model 210, and negative prompts that specify elements that the image generation model 210 should not include in a pattern image derived from natural materials generated by the image generation model 210. The prompt receiving unit 58 can input the received prompts to the image generation model 210. In this case, the text data included in the prompt is divided into chunks using, for example, LangChain, which is one of the libraries, and each divided chunk is converted into a vector representation.

[0026] The storage unit 59 can be configured with a semiconductor memory or a hard disk, and stores a computer program (program product) 60 and required information.

[0027] The computer program 60 can be stored in the storage unit 59 by reading the computer program 60 recorded on a recording medium (for example, an optically readable disk storage medium such as a CD-ROM) M using a recording medium reading unit (not shown). The computer program 60 may also be read from a recording medium such as a storage device (semiconductor memory such as a solid state drive (SSD)) connected according to a standard for connecting to a computer (for example, USB (Universal Serial Bus) or other standard) and stored in the storage unit 59. The computer program 60 may also be downloaded from an external device via the communication unit 52 and stored in the storage unit 59.

[0028] The memory 53 can be configured with semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc. A computer program 60 can be loaded into the memory 53, and the control unit 51 can execute the computer program 60. The control unit 51 can execute the processing defined by the computer program 60. That is, the processing by the control unit 51 is also processing by the computer program 60. Furthermore, each processing by the learning processing unit 56 and the generation instruction unit 57 can be realized by the processing by the computer program 60. That is, the control unit 51 can execute each processing by the learning processing unit 56 and the generation instruction unit 57.

[0029] The export unit 61 converts the image data of the pattern image derived from natural materials generated by the image generation model 210 into a format that can be read by an external application (such as a system or program), and outputs the converted image data to the external application.

[0030] Figure 2 shows an example of attribute information. The attribute information includes various information for describing patterns derived from natural materials. Users who interact with components using patterns derived from natural materials can be classified into various layers, including ordinary consumers who purchase products or parts using the components, customer representatives or internal sales representatives who handle the products or parts, and designers responsible for the design of the components. Ordinary consumers, customer representatives or internal sales representatives, and designers may describe the same pattern derived from natural materials in different ways. For example, professionals such as designers often use technical terms commonly used in the design industry. On the other hand, while customer representatives and internal sales representatives may use industry terminology for products or parts, like ordinary consumers, they rarely use technical terms commonly used in the design industry. Rather, they often use non-technical terms such as sensory expressions or color expressions that are influenced by patterns derived from natural materials. It is also possible to objectively describe patterns derived from natural materials regardless of user differences.

[0031] Therefore, in this embodiment, the expression methods of attribute information are classified into three types: (1) professional expressions, (2) non-professional expressions, and (3) objective expressions. Professional expressions are expressions used by experts such as designers, and are relatively frequently used in professional industries for mutual understanding between experts. Non-professional expressions are expressions that are not professional and are used by ordinary consumers, customers, and internal sales personnel. Objective expressions are literally objective expressions, and therefore can be commonly understood by experts such as designers, ordinary consumers, customers, and internal sales personnel. Even if the pattern derived from natural materials is the same, the pattern derived from natural materials can be expressed in three ways: professional expressions, non-professional expressions, and objective expressions.

[0032] As shown in FIG. 2, the technical expressions include interior information, wood grain expressions, and pattern information. The interior information includes, for example, expressions such as "natural," "modern," "industrial," and "Scandinavian." The wood grain expressions include, for example, expressions such as "straight grain," "cross grain," "luster," "knot," "coarse / fine," and "mottled." The pattern information includes, for example, expressions such as "pattern name (e.g., pine pattern, oak pattern, etc.)" and "design name."

[0033] The non-technical expressions include color information, emotional expressions, etc. The color information includes, for example, expressions such as "reddish," "bluish," "bright," "subdued," "vivid," and "dull." The emotional expressions include, for example, expressions such as "new / old," "luxury," "unique," "universal," "stimulating," and "gentle."

[0034] The objective expressions include tree species, product information, and types of components. Tree species refers to the type of natural material (the type of pattern derived from natural materials), and examples of such expressions include walnut, oak, and ash. Product information includes, for example, expressions such as brand name, product name, and product number. Types of components include, for example, expressions about the parts of products and parts, such as wallpaper and flooring. Note that the attribute information is an example and is not limited to the example in FIG. 2.

[0035] Next, a method for generating (learning) the image generation model 210 will be described.

[0036] 3 is a diagram showing an example of a method for generating the image generation model 210. The image generation model 210 can be generated (trained) by inputting training images and attribute information associated with the training images into the image generation model 210. The training images are, for example, images of patterns derived from natural materials, and the attribute information represents the patterns derived from the natural materials. A large number of combinations of these training images and attribute information are prepared and registered as a training dataset in the image generation model 210 for training.

[0037] 4 is a diagram showing an example of a training dataset. The training dataset includes, for each training data ID, a set of training images and attribute information associated with the training images. For example, a dataset with a training data ID of ID=0001 includes a training image G0001 and attribute information associated with the training image G0001, which includes a technical expression A0001, a non-technical expression B0001, and an objective expression C0001.

[0038] Furthermore, the data set with a learning data ID of ID=0002 includes learning images G0002 and G0003, and non-technical expression B0002 as attribute information associated with the learning images G0002 and G0003.

[0039] Furthermore, the data set with a learning data ID of ID=0003 includes a learning image G0004, and, as attribute information associated with the learning image G0004, a technical expression A0002 and a non-technical expression B0003.

[0040] Furthermore, the data set with the training data ID of ID=0004 includes training images G0005 and G0006, and a technical expression A0003 as attribute information associated with the training images G0005 and G0006. The same applies to the other training data IDs.

[0041] As described above, each piece of training data identified by a training data ID may include one or more training images, and the attribute information corresponding to the training images may include only technical expressions, only non-technical expressions, or only objective expressions, or may include at least two or more expressions of technical expressions, non-technical expressions, and objective expressions. Note that the symbols G0001, A0001, B0001, and C0001 are assigned for convenience, and the technical expressions, non-technical expressions, and objective expressions of the attribute information are expressed as shown in FIG. 2.

[0042] The image generation model 210 can be trained by inputting into the image generation model 210 each piece of training data identified by the corresponding training data ID.

[0043] As described above, the control unit 51 acquires training data including image data of patterns derived from natural materials (image data of learning images) and attribute information representing the patterns derived from natural materials, and based on the acquired training data, inputs the image data of patterns derived from natural materials and the attribute information representing the patterns derived from natural materials into the image generation model 210 that generates the pattern images derived from natural materials, thereby generating the image generation model 210.

[0044] By preparing a large number of combinations of training images and attribute information corresponding to the training images, the image generation model 210 is able to generate unprecedented, original and varied patterns derived from natural materials without being limited to existing patterns.

[0045] As shown in FIG. 4, the attribute information may include technical terms relating to the pattern derived from natural materials and non-technical terms relating to the pattern derived from natural materials.

[0046] Designers and other professionals use design terminology and specialized expressions specific to wood grain patterns during the design process, while ordinary consumers, client representatives, and internal sales representatives tend to use non-technical expressions. The differences in expression between the two groups make it difficult to communicate with each other. As a result, designs have traditionally tended to depend on the designer's sensibilities, and the designer's subjective evaluation has been the primary focus. The pattern expression cannot be evaluated qualitatively or quantitatively, making it difficult to evaluate the pattern itself. In particular, ordinary consumers, client representatives, and internal sales representatives often have difficulty accurately grasping what the pattern design will be before it is even designed. For the same reasons, it has also been difficult for senior designers to transfer knowledge to junior designers.

[0047] However, in this embodiment, the learning dataset used to generate (train) the image generation model 210 includes technical expressions, non-technical expressions, and objective expressions as attribute information corresponding to each learning image, so the image generation model 210 can learn that the technical expressions, non-technical expressions, and objective expressions, which are different in expression, are each expressions related to the corresponding learning images.

[0048] For example, as will be described later, when generating a pattern image derived from a natural material using the image generation model 210, if a technical expression is input as a prompt as attribute information to the image generation model 210, the image generation model 210 generates a pattern image derived from a natural material based on the input technical expression. If the pattern derived from a natural material generated by the image generation model 210 can be expressed using a non-technical expression, inputting the non-technical expression as a prompt as attribute information to the image generation model 210 allows the image generation model 210 to generate a pattern image derived from a natural material similar to that generated when a technical expression is input.

[0049] As described above, according to this embodiment, the image generation model 210 is generated (trained) using attribute information including at least technical and non-technical expressions, and therefore the image generation model 210 learns the relationship between the design terminology and technical expressions specific to wood grain patterns used by experts such as designers in the design process and the non-technical expressions used by ordinary consumers, customer representatives, and in-house sales representatives for patterns derived from natural materials. This enables communication between the two parties even when their expressions differ. This makes it possible to evaluate pattern expressions qualitatively or quantitatively.

[0050] For example, when in-house designers create a new pattern derived from natural materials, they input technical expressions as prompts into the image generation model 210 to generate a pattern image derived from natural materials. The generated pattern image derived from natural materials is highly likely to be a pattern image that is intended by ordinary consumers, customer representatives, and in-house sales representatives, since the image generation model 210 also takes into account non-technical expressions that correspond to the input technical expressions.

[0051] Furthermore, when an in-house sales representative shares an image of a pattern derived from natural materials with a building materials manufacturer, which is the ordering party, in a BtoB scenario, technical expressions or expressions using in-house jargon cannot be used, so the sales representative advances the dialogue by generating an image or picture of a pattern derived from natural materials using general emotional expressions and color information that the building materials manufacturer's representative can understand, and then inputs non-technical expressions as prompts into the image generation model 210 to determine the image of the pattern to be requested from the designer. Even in such a case, the image generation model 210 also takes into consideration technical expressions that correspond to the input non-technical expressions, so there is a high possibility that the pattern image will be understandable to the designer.

[0052] The attribute information also includes objective expressions, specifically, at least one of product information regarding the pattern derived from natural materials, the type of pattern derived from natural materials (such as tree species), and the type of component. By using objective expressions, a common understanding can be achieved between experts such as designers, ordinary consumers, and customers and internal sales personnel.

[0053] Image data of pattern images derived from natural materials generated by the image generation model 210 can be further added to the training data, and the control unit 51 can further acquire the training data and input further image data of the acquired pattern images derived from natural materials to the image generation model 210. This enables the image generation model 210 to generate even more pattern images derived from natural materials, making it possible to generate a wide variety of novel patterns.

[0054] Furthermore, image data selected by the user from the image data of pattern images derived from natural materials generated by the image generation model 210 is further added to the training data. The image data selected by the user includes, for example, image data of pattern images that match the needs of ordinary consumers and the intentions of customers. The control unit 51 can further acquire the training data and further input the acquired image data of pattern images derived from natural materials to the image generation model 210. This makes it possible to generate pattern images derived from natural materials that match the needs of ordinary consumers and the intentions of customers.

[0055] Furthermore, when image data selected by a user from the image data of pattern images derived from natural materials generated by the image generation model 210 is further added to the training data, the training data may include, as attribute information, a set of prompts input to the image generation model 210 when the image generation model 210 generated the pattern image derived from natural materials. This enables the image generation model 210 to generate even more diverse pattern images derived from natural materials, making it possible to generate a wide variety of novel patterns.

[0056] Next, a method for generating a pattern image derived from natural materials using the image generation model 210 will be described.

[0057] FIG. 5 is a diagram showing an example of a method for generating a pattern image derived from a natural material using the image generation model 210. The image generation model 210 shown in FIG. 5 is an image generation model 210 that has been trained using the method described with reference to FIGS. 3 and 4, etc. As shown in FIG. 5, the control unit 51 inputs a prompt (attribute information) and a reference image of a pattern derived from a natural material to the image generation model 210. The image generation model 210 generates and outputs a pattern image (image data) derived from a natural material based on the input prompt and reference image. The control unit 51 can acquire the pattern image derived from a natural material generated and output by the image generation model 210. Note that the image generation model 210 can not only generate a pattern image derived from one natural material, but also simultaneously generate one or more pattern images derived from natural materials.

[0058] The reference image of a pattern derived from a natural material is an image that represents the pattern that you want the image generation model 210 to generate, and in the case of a wood grain pattern, for example, it can include images that define (control) the pattern derived from natural materials, such as the grain pattern (shading due to annual rings, expression of straight grain or flat grain, etc.), photographs, paintings, abstract paintings, geometric patterns (e.g., herringbone), etc. This makes it possible to generate new, artificial pattern images derived from natural materials that have never been seen before.

[0059] At least one of attribute information and a reference image can be input to the image generation model 210, but it is preferable to input attribute information to the image generation model 210. In other words, a reference image is not necessarily required, but attribute information is preferably required.

[0060] As described above, the control unit 51 can acquire at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquire a pattern image derived from a natural material generated by inputting at least one of the acquired attribute information and reference image into the image generation model 210, and output the acquired pattern image derived from a natural material. The pattern image may be output and displayed on the display unit 54, or may be output to an external application via the export unit 61.

[0061] This makes it possible to create unprecedented, innovative patterns derived from natural materials with a wide variety of variations, without being limited by existing patterns.

[0062] As shown in Figure 4, the attribute information may include both technical terms related to the natural material pattern and non-technical terms related to the natural material pattern. This allows for communication between experts such as designers and ordinary consumers, client representatives, and internal sales representatives. This also allows for qualitative and quantitative analysis of the pattern expression.

[0063] The attribute information also includes objective expressions, specifically, at least one of product information regarding the pattern derived from natural materials, the type of pattern derived from natural materials (such as tree species), and the type of component. By using objective expressions, a common understanding can be achieved between experts such as designers, ordinary consumers, and customers and internal sales personnel.

[0064] Next, an example of a prompt will be described. The prompt may be automatically generated by the generation instruction unit 57, or may be manually input by the user. The prompt input by the user can be received by the prompt receiving unit 58.

[0065] Fig. 6 is a diagram showing an example of prompts when generating a layout. Fig. 6 shows examples of prompts by a designer, by a customer or an internal salesperson, and by an ordinary consumer.

[0066] Designers and other experts understand the characteristics and names of materials, such as tree species and wood character, and also have their own unique material layout analysis criteria. For example, a prompt such as "Please create an image of a pattern derived from natural materials using the following attribute information as a reference" can be generated. The attribute information can be, for example, "Include knots and troughs, oak blast processing, dyed and whitened texture, and a textured look."

[0067] At the level of customers and internal salespeople, they have a general understanding of tree species and wood character, and can associate them with images such as "Nordic" or "vintage." For example, a prompt such as "Please create an image of a pattern derived from natural materials using the following attribute information as a reference" can be generated. The attribute information can be, for example, "trendy furniture pattern, lots of character, vintage feel."

[0068] The average consumer may not know much about tree species or characters, but they can vaguely understand images such as "Nordic style," "chic feel," and "natural texture." For example, a prompt such as "Please create an image of a pattern derived from natural materials using the following attribute information as a reference" can be generated. The attribute information can be, for example, "Wood grain pattern of overseas furniture tabletops."

[0069] Even if the image generation instructions are expressed differently by people at different levels, such as designers, clients, internal sales staff, or ordinary consumers, the resulting pattern image will have a similar layout.

[0070] Since the image generation model 210 has learned that the technical expressions included in the attribute information of the prompt by the designer and the non-technical expressions included in the attribute information of the prompt by the client, in-house salesperson, etc., or the general public correspond to each other, the image generation model 210 can generate a similar pattern image derived from natural materials regardless of whether the prompt is from a designer, a client, in-house salesperson, etc., or a general public. As a result, for example, when a general public inputs the prompt exemplified in Fig. 6 into the image generation model 210 and instructs image generation, a pattern image derived from natural materials similar to that generated when a designer inputs the prompt exemplified in Fig. 6 into the image generation model 210 and instructs image generation can be generated.

[0071] The prompt illustrated in FIG. 6 is an example, and is not limited to the example in FIG.

[0072] Figure 7 shows an example of prompts for generating materials. Figure 7 shows examples of prompts from designers and from ordinary people.

[0073] Designers and other experts (who use specialized expressions for patterns derived from natural materials) can identify tree species and wood character names, and can link these characteristics to image names related to interior and exterior design, such as Nordic style or vintage style. For example, a prompt such as "Please create an image of oak with fine grain using the following attribute information as a reference." The attribute information can be, for example, "Material type is white ash with white paint."

[0074] Ordinary people (who use non-technical terms for patterns derived from natural materials) may not be familiar with the names of tree species or wood character, but they do know image names for interior and exterior designs such as "Nordic style" and "vintage style." For example, a prompt could be generated with the instruction "Please refer to the following attribute information to create an image of wood grain that suits a calming Japanese space." The attribute information could be, for example, "Material type is wood grain white."

[0075] Even if the image generation instructions are expressed differently by people at different levels, such as designers or ordinary people, the resulting pattern image will be of the same material.

[0076] Since the image generation model 210 has learned that the technical expressions included in the attribute information of the prompt by the designer and the non-technical expressions included in the attribute information of the prompt by the general public correspond to each other, the image generation model 210 can generate similar pattern images derived from natural materials regardless of whether a prompt from a designer or a general public is input. As a result, for example, when a general public inputs the prompt exemplified in Fig. 7 into the image generation model 210 and instructs image generation, it is possible to generate a pattern image derived from natural materials similar to that generated when a designer inputs the prompt exemplified in Fig. 7 into the image generation model 210 and instructs image generation.

[0077] The prompt illustrated in FIG. 7 is merely an example, and is not limited to the example in FIG.

[0078] Figure 8 is a diagram showing examples of technical expressions and corresponding non-technical expressions. For convenience, Figure 8 shows examples of technical expressions and corresponding non-technical expressions, divided into material type, layout type, style, prompt, etc. For example, in the material type, the technical expression is "white ash white paint" and the corresponding non-technical expression is "wood grain white color" (illustrated in Figure 7).

[0079] In the layout type category, the professional expression is "pine pattern with a seam feel," and the corresponding non-professional expression is "overseas wood grain furniture pattern." In the style category, the professional expression is "knotty blasted oak pattern," and the corresponding non-professional expression is "vintage-style wood grain trend pattern." In the style category, the professional expression is "knotty dry chestnut pattern," and the corresponding non-professional expression is "organic-style wood grain pattern." In the prompt category, the professional expression is "knotless glossy walnut pattern," and the corresponding non-professional expression is "luxurious dark brown floor pattern."

[0080] The image generation model 210 has already learned that the technical expressions illustrated in FIG. 8 and the corresponding non-technical expressions are mutually corresponding expressions, and therefore the image generation model 210 can generate similar pattern images derived from natural materials regardless of whether the technical expressions illustrated in FIG. 8 or the non-technical expressions are input as a prompt.

[0081] Next, a description will be given of a usage scenario of the pattern image derived from natural materials generated by the information processing device 50 (image generation model 210) of this embodiment. The pattern image derived from natural materials generated by the information processing device 50 of this embodiment can be used, for example, as texture data for building interior and exterior decoration materials, car interiors, etc. Furthermore, the pattern image derived from natural materials generated by the information processing device 50 of this embodiment can be used for license sales, etc., intended for use in various applications by construction companies, building material and furniture manufacturers, automobile manufacturers, etc. In this case, construction companies, building material and furniture manufacturers, automobile manufacturers, etc. can be made to order building materials having the pattern image derived from natural materials generated by the information processing device 50 of this embodiment online. A specific example will be described below.

[0082] Figure 9 is a diagram showing a second example of the configuration of the information processing system of this embodiment. The difference from the first example shown in Figure 1 is that an order server 300 is provided. The information processing device 50 is connected to the training database 100, the generation AI 200, and the order server 300 via the communication network 1.

[0083] The ordering server 300 is a server for ordering components (for example, products and parts of construction companies, building material and furniture manufacturers, automobile manufacturers, etc.) using pattern images derived from natural materials generated by the image generation model 210. The information processing device 50 can output an ordering transaction to the ordering server 300. The ordering server 300 can place an order for the components to the supplier based on the ordering transaction.

[0084] Fig. 10 is a diagram showing an example of application to a 3D presentation system. The 3D presentation system may be any display system that displays components using patterns derived from natural materials, and includes, for example, an application (system, program, etc.) that displays the interior layout of a building in 3D, as shown in Fig. 10. The export unit 61 of the information processing device 50 can convert image data of the pattern image derived from natural materials generated by the image generation model 210 into image data in a data format used by the 3D system, and output the converted image data to the 3D system.

[0085] The 3D system can acquire image data output by the information processing device 50, and automatically apply and display pattern images derived from natural materials to components corresponding to the acquired image data (for example, flooring, wall materials, door materials, decorative paneling, ceiling materials, etc.). The 3D system also displays multiple selection candidates for pattern images derived from natural materials for each component corresponding to the acquired image data, and can apply and display, for example, a pattern image derived from natural materials selected by an ordinary consumer to the corresponding component. In the example of FIG. 10, the pattern images derived from natural materials "P0050," "P0060," "P0065," and "P0070" output by the information processing device 50 are displayed as selectable images, and when the user operates the "select" icon, the selected pattern image (in the example of FIG. 10, "P0050") is applied to the corresponding component (flooring) and displayed.

[0086] Fig. 11 is a diagram showing an example of application to a proposal system. The proposal system may be any display system that displays components using patterns derived from natural materials, and may include, for example, an application (system, program, etc.) that proposes a layout inside a building, as shown in Fig. 11. The export unit 61 of the information processing device 50 can convert image data of the pattern image derived from natural materials generated by the image generation model 210 into image data in a data format used by the proposal system, and output the converted image data to the proposal system.

[0087] The proposed system can acquire image data output by the information processing device 50 and automatically apply and display pattern images derived from natural materials to components (e.g., flooring, wall, door, decorative paneling, ceiling, etc.) corresponding to the acquired image data. The proposed system also displays multiple selection options for pattern images derived from natural materials for each component corresponding to the acquired image data. For example, the proposed system can apply and display a pattern image derived from natural materials selected by a general consumer to the corresponding component. In the example of FIG. 11 , the pattern images "P0031," "P0032," and "P0033" derived from natural materials output by the information processing device 50 are displayed as selectable images for the floor of a living, dining, and kitchen (LDK) room in a home. "P0032" is selected, and a 3D layout of the LDK room with the selected pattern image applied is displayed. Furthermore, the pattern images "P0071," "P0072," and "P0073" derived from natural materials output by the information processing device 50 are displayed as selectable images for the accent wall of a Western-style room 1 in the home. "P0073" is selected, and a 3D layout of the Western-style room 1 with the selected pattern image applied is displayed. Furthermore, pattern images "P0081," "P0082," and "P0083" derived from natural materials output by the information processing device 50 are displayed selectably on the floor of the toilet in the home, and when "P0081" is selected, a 3D layout of the toilet with the selected pattern image applied is displayed.

[0088] As described above, the control unit 51 can display a component of a display system that displays a component using a pattern derived from a natural material with a pattern image derived from a natural material. Furthermore, the control unit 51 can accept the selection of a desired pattern image derived from a natural material from a plurality of pattern images derived from a natural material, and can display the component of a display system that displays a component using a pattern derived from a natural material with the selected pattern image derived from a natural material.

[0089] FIG. 12 is a diagram showing an example of an ordering transaction for components having a pattern image derived from natural materials generated by the information processing device 50. The person ordering the components can place an order for the components by operating the operation unit 55 or an external terminal device. As shown in FIG. 12, the ordering transaction includes, for each order number, information such as the pattern image to be applied to the component (the pattern image derived from natural materials generated by the information processing device 50), the component to be ordered, the quantity of the component, and the delivery date. For example, order number "001" indicates an order for 100 units of component "AAA" to which pattern image "P0001" is applied.

[0090] The control unit 51 can output to the ordering server 300 an ordering transaction for materials using pattern images derived from natural materials, based on the order details input from the operation unit 55.

[0091] Fig. 13 is a diagram showing an example of a processing procedure for generating an image generation model 210. Fig. 13 shows a process in which an information processing device 50 generates an image generation model 210, but the image generation model 210 may be generated by a device other than the information processing device 50.

[0092] The control unit 51 acquires training data including learning images of patterns derived from natural materials and attribute information associated with the learning images (S11), and inputs the acquired learning images and attribute information into the image generation model 210 to train the image generation model 210 (S12).

[0093] The control unit 51 determines whether or not there is training data (S13), and if there is training data (YES in S13), continues the processing from step S11 onwards. If there is no training data (NO in S13), the control unit 51 registers the trained image generation model 210 (S14) and ends the processing. The registered image generation model 210 may be stored in the storage unit 59, for example.

[0094] 14 is a diagram showing an example of a processing procedure for generating a pattern image by the information processing device 50. The control unit 51 receives a prompt (S21) and selects an image generation model (S22). The image generation model 210 is trained (generated) using, for example, different learning data, and includes multiple image generation models for different uses, different designers, domestic or overseas use, etc. The control unit 51 determines whether or not there is a reference image (S23), and if there is a reference image (YES in S23), acquires the reference image (S24), inputs the prompt and reference image into the image generation model 210 (S25), and performs the processing of step S27 described below.

[0095] If there is no reference image (NO in S23), the control unit 51 inputs a prompt to the image generation model 210 (S26) and generates a pattern image derived from natural materials using the selected image generation model 210 (S27). The control unit 51 acquires the pattern image derived from natural materials generated by the image generation model 210 (S28). The control unit 51 outputs the acquired pattern image derived from natural materials (S29) and determines whether to redo the image generation (S30).

[0096] When image generation is to be redone (YES in S30), the control unit 51 determines whether or not to change the generation conditions (S31). There are four patterns for changing the generation conditions: (1) modifying both the prompt and the reference image, (2) modifying only the prompt, (3) modifying only the reference image, and (4) not modifying either the prompt or the reference image. The change of the generation conditions may also accept a change in the image generation model selection in step S22.

[0097] If the generation conditions are to be changed (YES in S31), the control unit 51 continues the processing from step S21 onwards. If the generation conditions are not to be changed (NO in S31), the control unit 51 continues the processing from step S27 onwards, and generates a new pattern image derived from natural materials. If the image generation is not to be redone (NO in S30), the control unit 51 ends the processing.

[0098] Although the above description has been given using a wood grain pattern as an example, this embodiment is not limited to wood grain patterns and can also be applied to the creation of patterns derived from natural materials such as stone and bamboo.

[0099] (Supplementary Note 1) The computer program causes a computer to execute a process of acquiring at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquiring an image of the pattern derived from the natural material generated by inputting at least one of the acquired attribute information and the reference image into an image generation model, and outputting the acquired image of the pattern derived from the natural material.

[0100] (Supplementary Note 2) In the computer program according to Supplementary Note 1, the attribute information includes technical expressions relating to the pattern derived from the natural material and non-technical expressions relating to the pattern derived from the natural material.

[0101] (Supplementary Note 3) In the computer program according to Supplementary Note 1 or Supplementary Note 2, the attribute information includes at least one of product information relating to the pattern derived from natural materials, the type of the pattern derived from natural materials, and the type of component.

[0102] (Appendix 4) A computer program in any one of Appendices 1 to 3 causes a computer to execute a process of displaying components of a display system that displays components using patterns derived from natural materials with an image of the pattern derived from the natural material.

[0103] (Appendix 5) The computer program in any one of Appendices 1 to 4 causes a computer to execute a process of accepting a selection of a desired pattern image derived from natural materials from among a plurality of pattern images derived from natural materials, and displaying the components of a display system that displays components using patterns derived from natural materials with the pattern image derived from the natural material that was selected.

[0104] (Supplementary Note 6) The computer program causes a computer to execute a process for outputting an order transaction for a component using the pattern image derived from natural materials in any one of Supplementary Note 1 to Supplementary Note 5.

[0105] (Appendix 7) The information processing device includes a control unit, which acquires at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquires an image of the pattern derived from the natural material generated by inputting at least one of the acquired attribute information and the reference image into an image generation model, and outputs the acquired image of the pattern derived from the natural material.

[0106] (Appendix 8) The information processing method acquires at least one of attribute information representing a pattern derived from a natural material and a reference image of the pattern derived from the natural material, acquires a pattern image derived from a natural material generated by inputting at least one of the acquired attribute information and the reference image into an image generation model, and outputs the acquired pattern image derived from a natural material.

[0107] (Appendix 9) The image generation model generation method acquires training data including image data of patterns derived from natural materials and attribute information representing the patterns derived from natural materials, and inputs the image data of patterns derived from natural materials and the attribute information representing the patterns derived from natural materials into an image generation model that generates an image of a pattern derived from natural materials based on the acquired training data.

[0108] (Supplementary Note 10) In the image generation model generation method of Supplementary Note 9, the attribute information includes technical expressions related to the pattern derived from the natural material and non-technical expressions related to the pattern derived from the natural material that are associated with the technical expressions.

[0109] (Supplementary Note 11) In the image generation model generation method according to Supplementary Note 9 or Supplementary Note 10, the attribute information includes at least one of product information relating to the pattern derived from the natural material, the type of the pattern derived from the natural material, and the type of component.

[0110] (Appendix 12) The image generation model generation method according to any one of Appendices 9 to 11 further acquires training data including image data of the pattern image derived from the natural material generated by the image generation model, and further inputs the acquired image data of the pattern image derived from the natural material into the image generation model.

[0111] (Appendix 13) The image generation model generation method according to any one of Appendices 9 to 12 further acquires training data including image data selected by a user from the image data of the pattern images derived from natural materials generated by the image generation model, and further inputs the acquired image data into the image generation model.

[0112] (Appendix 14) In the image generation model generation method according to any one of Appendices 9 to 13, the training data includes, as the attribute information, a prompt input to the image generation model when the image generation model generates the pattern image derived from the natural material.

[0113] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0114] 1. Communication Network 50 Information processing equipment 51 Control section 52 Communications Department 53 Memory 54 Display section 55 Operation section 56 Learning processing unit 57 Generation instruction section 58 Prompt Reception Department 59 Memory section 60 Computer Programs 61 Export Department 100 Learning Databases 110 training images 120 Attribute information 200 Generation AI 210 Image Generation Model 300 Order Server

Claims

1. acquiring at least one of attribute information representing a pattern derived from natural materials and a reference image of the pattern derived from natural materials; acquiring a pattern image derived from a natural material, which is generated by inputting at least one of the acquired attribute information and a reference image into an image generation model; Output the acquired pattern image derived from the natural material. A computer program that causes a computer to perform a process.

2. The attribute information is A technical expression relating to the pattern derived from the natural material and a non-technical expression relating to the pattern derived from the natural material, 2. The computer program of claim 1.

3. The attribute information is The information includes at least one of product information regarding the pattern derived from natural materials, the type of the pattern derived from natural materials, and the type of component.

2. The computer program of claim 1.

4. a display system for displaying a component using a pattern derived from a natural material, the component being displayed with an image of the pattern derived from the natural material; 4. A computer program according to claim 1, which causes a computer to execute a process.

5. Accepting a selection of a desired pattern image derived from natural materials from among the plurality of pattern images derived from natural materials; a display system for displaying components using patterns derived from natural materials, the components being displayed with a pattern image derived from the selected natural material; 4. A computer program according to claim 1, which causes a computer to execute a process.

6. outputting an order transaction for a component using the pattern image derived from the natural material; 4. A computer program according to claim 1, which causes a computer to execute a process.

7. A control unit is provided, The control unit acquiring at least one of attribute information representing a pattern derived from natural materials and a reference image of the pattern derived from natural materials; acquiring a pattern image derived from a natural material, which is generated by inputting at least one of the acquired attribute information and the reference image into an image generation model; Output the acquired pattern image derived from the natural material. Information processing device.

8. acquiring at least one of attribute information representing a pattern derived from natural materials and a reference image of the pattern derived from natural materials; acquiring a pattern image derived from a natural material, which is generated by inputting at least one of the acquired attribute information and a reference image into an image generation model; Output the acquired pattern image derived from the natural material. Information processing methods.

9. Acquire training data including image data of patterns derived from natural materials and attribute information representing the patterns derived from natural materials; inputting image data of patterns derived from natural materials and attribute information representing the patterns derived from natural materials into an image generation model that generates the pattern images derived from natural materials based on the acquired training data; Image generation model generation method.

10. The attribute information is a technical expression relating to the pattern derived from the natural material; and a non-technical expression relating to the pattern derived from the natural material, which is associated with the technical expression; The image generation model generation method according to claim 9.

11. The attribute information is The information includes at least one of product information regarding the pattern derived from natural materials, the type of the pattern derived from natural materials, and the type of component. The image generation model generation method according to claim 9.

12. Further acquiring training data including image data of the pattern image derived from natural materials generated by the image generation model; Further inputting image data of the acquired pattern image derived from natural materials into the image generation model.

12. The image generation model generating method according to claim 9.

13. Further acquiring training data including image data selected by a user from the image data of the pattern images derived from natural materials generated by the image generation model; Further inputting the acquired image data into the image generation model; 12. The image generation model generating method according to claim 9.

14. The training data is The attribute information includes a prompt input to the image generation model when the image generation model generates the pattern image derived from the natural material.

12. The image generation model generating method according to claim 9.

Images