Text and design matching method and device

Abstract

The present invention relates to a text and design matching method performed by a text and design matching device. According to an embodiment of the present invention, the method includes the steps of acquiring an image from information including an image and text, learning features of the acquired image, extracting text from the information and learning about a pair of the extracted text and the acquired image, extracting trend words extracted a predetermined number of times or more from the extracted text, learning about a pair of the trend words and the acquired image, and identifying design feature information that matches the trend words from the learned features of the image.

Description

[Technical field]

[0001] The present invention relates to a method and an apparatus for matching a text with a design, and more particularly to a method and an apparatus for matching a text with a design that reflects design trends. [Background technology]

[0002] In the fashion market, the latest trendy designs are produced and consumed in short cycles as the proportion of so-called fast fashion or SPA (Specialty retailer of Private label Apparel) brands increases. These fast fashion brands have the advantage of providing consumers with the opportunity to purchase trendy items at low cost by securing a mass production system and global distribution network for their products and employing many designers to reflect the latest trends in their designs.

[0003] On the other hand, the production and sales system of fast fashion brands intensifies competition in the fashion design industry and creates entry barriers for new entrants into the market.

[0004] In order to solve such problems, a text-based design image generation technology has been disclosed, which allows a large number of images to be automatically generated simply by inputting text. In particular, the technology disclosed in the present invention allows not only designers but also general users to generate design images simply by inputting text.

[0005] In order for the images generated by the disclosed technology to be used in products, a technology for generating high-resolution designs is also required, as well as a technology for generating designs that reflect rapidly changing fashion trends. Summary of the Invention [Problem to be solved by the invention]

[0006] The technical problem to be solved by the present invention is to provide a method and an apparatus for generating conditions for automatic generation of designs by learning acquired images and text matched thereto. Specifically, the technical problem to be solved by the present invention is to provide a method and an apparatus for automatically generating designs based on trend information reflected in text by learning characteristics of text matched to images.

[0007] Another technical problem to be solved by the present invention is to provide a method and an apparatus for automatically generating a design in response to input of text corresponding to generated conditions. Specifically, another technical problem to be solved by the present invention is to provide a method and an apparatus for recognizing a product category selection and text input by a user as conditions for design generation and generating a design under the recognized conditions. More specifically, the technical problem to be solved by the present invention is to provide a method and an apparatus for interpreting input text as detailed characteristics of a selected category and generating a design reflecting the characteristics for products in the category.

[0008] Another technical problem to be solved by the present invention is to provide a method and an apparatus for enhancing the resolution of a design image generated by inputting text. Specifically, another technical problem to be solved by the present invention is to provide a method and an apparatus for maximizing the resolution of a design image by considering feature-based loss, not pixel-based loss, on an original image and a generated design image.

[0009] The technical problems of the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned can be clearly understood by a person skilled in the art of the present invention from the following description. [Means for solving the problem]

[0010] The design generation method based on learned conditions for solving the technical problem is performed by a design generation device based on learned conditions, and the method includes the steps of acquiring an image from information including an image and text, learning features of the acquired image, extracting text from the information and matching the extracted text to the learned features of the image, learning the conditions of a design to be generated based on the image through the matching, a step of a user inputting user text for generating a design image, a step of identifying conditions corresponding to the user's text, and a step of generating a design image based on the identified conditions.

[0011] According to one embodiment, the method may include a step of performing resolution conversion on the generated design image, and a step of generating a final design image through the resolution conversion, wherein the step of performing resolution conversion may include a step of extracting features of the generated design image, a step of comparing the learned features of the image with the extracted features of the generated design image, and a step of performing resolution conversion of the generated design based on difference value information as a result of the comparison.

[0012] According to one embodiment, the step of learning conditions for a design to be generated based on the image may include a step of learning text that corresponds to the learned features of the image from among the text extracted through the matching as the conditions, and a step of determining the text learned based on the conditions as a category of a product that includes the image features.

[0013] In one embodiment, identifying conditions corresponding to the user's text may include vectorizing the user's text, generating a vector value of the user's text as a result of the vectorization, and determining whether the vector value matches the category; and if the vector value matches the category, identifying the category and the vector value with the condition.

[0014] According to one embodiment, if the vector value does not match the category, the method may include a step of extracting a transformed vector value that matches the category by reflecting a style variable in the category and the vector value, and a step of identifying the transformed vector value and the category by the condition, which includes a step of identifying the category by the condition. Effect of the Invention

[0015] According to an embodiment of the present invention, even if different users use different words for product design, it is possible to generate a design image that matches the user's intention based on the words input by the user.

[0016] Furthermore, according to another embodiment of the present invention, there is an advantage that a design image reflecting the latest design trend information is provided to the user. That is, according to the embodiment, there is an advantage that the latest trends that are difficult for the user to grasp are reflected in the design image generation model, the user's input text is interpreted as the latest trend, and a design image reflecting the latest trend can be generated.

[0017] In addition, according to another embodiment of the present invention, there is an advantage that the amount of calculation for generating a design image can be reduced by restricting the target of design generation based on the category and the input text.

[0018] Further, according to still another embodiment of the present invention, there is an advantage that the resolution of the generated design image is improved, thereby increasing the efficiency of commercializing the design.

[0019] The effects of the present invention are not limited to the effects described above, and other effects not mentioned will be clearly understood by those skilled in the art of the present invention from the following description. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 illustrates an exemplary system for generating designs based on learned conditions according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a conceptual diagram for explaining a design generation process based on learned conditions referenced in some embodiments of the present invention. [Diagram 3] FIG. 13 is a block diagram of a design generator based on learned conditions according to another embodiment of the present invention. [Figure 4] FIG. 2 is an exemplary diagram of the architecture of the design generation software referenced in some embodiments of the present invention. [Diagram 5] 13 is a flowchart of a method for generating designs based on learned conditions according to another embodiment of the present invention. [Figure 6] 4 is a flowchart of a matching method between a captured image and an extracted text according to yet another embodiment of the present invention. [Figure 7] FIG. 2 is an exemplary diagram illustrating a method for learning an image and a text pair to be matched therewith in some embodiments of the present invention. [Figure 8] FIG. 2 is an illustrative diagram for explaining the trend word selection process referenced in some embodiments of the present invention. [Figure 9] FIG. 2 is an illustrative diagram of an interface for generating designs in response to input text as referenced in some embodiments of the present invention. [Figure 10] FIG. 13 is an illustrative diagram of a high-resolution design image generated by a post-generator referenced in some embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. The advantages and features of the present invention, and the method of achieving them, will become apparent by referring to the following embodiments in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be realized in various forms. The present embodiments are provided merely to make the disclosure of the present invention complete and to fully convey the scope of the invention to those skilled in the art to which the present invention pertains, and the present invention is only defined by the claims. The same reference numerals refer to the same elements throughout the specification.

[0022] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in the sense that can be commonly understood by those having ordinary knowledge in the technical field to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless specifically defined. The terms used in this specification are intended to describe the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless otherwise specified in the text.

[0023] In this specification, the design generation system based on learned conditions, the design generation device based on learned conditions, and the design generation method based on learned conditions may be abbreviated as the design generation system, the design generation device, and the design generation method, respectively.

[0024] As used herein, images can include photographs, pictures, and video content, and text can include not only literal elements such as letters, morphemes, words, and sentences, but also elements capable of conveying semantics, such as numbers, symbols, and the like.

[0025] Fig. 1 is an example diagram of a design generation system based on learned conditions according to an embodiment of the present invention, and Fig. 2 is a conceptual diagram for explaining a design generation process based on learned conditions referred to in some embodiments of the present invention.

[0026] Referring to FIG. 1, a design generation system 10 can include a design generation device 100, a manufacturing terminal 300, and a number of external terminals that provide information 30 through a network.

[0027] The design generating device 100, the manufacturing terminal 300, and the multiple external terminals are computer devices that can exchange data with each other through a network. The design generating device 100 can access information 30 including images 50 and text 70 through the network.

[0028] In one embodiment, the design generator 100 can crawl, scrape, and / or parse numerous pages on the web through a network and classify, identify, extract, and process information 30 contained on the pages.

[0029] In particular, according to an embodiment of the present invention, the design generating device 100 can obtain an image 50 belonging to the information 30 and extract text 70. The page can include at least one of a blog, a social networking site, and an internet shopping mall page.

[0030] In one embodiment, the design generator 100 can learn the features of the captured images 50 through an artificial neural network. For example, if the images 50 are fashion design images, the design generator 100 can learn the features of the captured images from countless pages to create a category for each fashion design garment, a shape-based classification for at least a portion of the garment, a pattern-based classification for the garment, and a texture-based classification for the garment.

[0031] In one embodiment, the design generating device 100 can learn the extracted text 70. In particular, the design generating device 100 can analyze the extracted text 70 into morphemes, words, and sentence units and learn the relationship between each unit. For example, the design generating device 100 can extract the text 70, one-piece, dots, patterns, belts, wrap dresses, semi-suits, black / white, and spring / summer season. The design generating device 100 can classify the extracted text by unit, for example, by word or morpheme, and then embed the text.

[0032] In one embodiment, the design generator 100 can match pairs of the acquired images 50 and text 70, and can also learn a matching relationship between the images 50 and the text 70 based on the learning results on the features of the images 50 and the vector values ​​extracted by embedding the text images 70. For example, the design generator 100 can identify a shape design element of a dress among the garments based on learning on the feature points extracted from the garment image 50 and the interrelationships between the feature points, and match this with the learned "dress" among the text 70 extracted on the page.

[0033] In one embodiment, the design generation device 100 can learn the conditions for design generation. The conditions for design generation are conditions that restrict the range of designs to be generated when a design is generated by inputting text into the design generation device 100. The input text may be a keyword related to the design to be generated, for example, a type of clothing design such as a coat, a dress, or a jacket. That is, when the text "dress" is input, the designs to be generated are limited to the type of dress, and specifically, an image having the learned feature points can be generated by matching the text "dress".

[0034] According to an embodiment of the present invention, for responsiveness of real-time design generation and saving of computing resources, category information such as clothing type, color, pattern, etc. can be generated as conditions through condition learning.

[0035] In one embodiment, text for design generation is input to the design generation device 100, and category information is selected, so that a design that corresponds to the category and matches the text can be generated.

[0036] In FIG. 2, a case is illustrated in which the category 201 is a type of clothing, and the input text 202 is, for example, a sleeve length, a color, a specific design element, a zipper, a pocket, a pattern, and the like.

[0037] When the category 201 and the text 202 are input to the design generating device 100, a style generator 210 of the design generating device 100 can generate a conditional design. As an example, the style generator 210 may use a design generation model based on GAN, particularly cGAN (conditional GAN).

[0038] Here, GAN is a type of unsupervised learning in which a Generator (G), which generates images, and a Discriminator (D), which evaluates the images generated by the Generator, compete with each other to improve each other's performance, generating new images.

[0039] Specifically, D strives to determine that only the original data is true, while G generates fake data that D cannot distinguish from false, and the performance of both models improves until D is ultimately unable to distinguish between real and fake data. cGAN also differs from existing GANs in that it does not generate data randomly, but can generate what is desired.

[0040] 1 again, the manufacturing terminal 300 receives the design generated by the design generating device 100, and the design product is a computer device at the manufacturing site. In another example, the manufacturing terminal 300 may be a terminal of an influencer who receives the generated design and publicizes it. In this case, the influencer can use the received design in various ways by modifying and processing it through an SNS page, etc., for the content he or she is creating.

[0041] When the manufacturing terminal 300 receives the generated design and directly manufactures the design product, the design elements such as the shape and pattern of the design, each area and its connection relationship, etc. must be clear. That is, a high-resolution design must be generated and provided to the manufacturing terminal 300. For this reason, the design generation device 100 can manage the resolution of the design image through the final generation process for the primarily generated design.

[0042] Referring again to FIG. 2, if a problem occurs that the resolution of the design image generated through the style generator 210 is low, a high-resolution design image may be generated using the post generator 230. As an example, the post generator 230 may also use a cGAN. That is, according to an embodiment of the present invention, a final design image 250 may be generated using a neural network separate from the style generator 210. In particular, the final design image 250 may include an image set having a priority according to the intention of the user who inputs the text 202. The post generator 230 may be referred to as a high-resolution converter.

[0043] The design generation system 10 according to an embodiment of the present invention is not limited to FIG. 1 and the above description.

[0044] Next, the function and operation of the design generating apparatus 100 will be described in more detail with reference to Fig. 3. Fig. 3 is a block diagram of a design generating apparatus based on learned conditions according to another embodiment of the present invention.

[0045] Referring to FIG. 3, the design generation device 100 may include one or more processors 101, a network interface 102 for wired or wireless communication with external devices, a memory 103 for loading computer programs to be executed by the processor 101, storage 104 for storing the computer programs, and a display 107.

[0046] The processor 101 controls the overall operation of each component of the design generating device 100. The processor 101 may include a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), an application processor (AP), or any other type of processor known in the technical field of the present invention. Furthermore, the processor 101 may perform calculations for at least one application or program for executing a method according to an embodiment of the present invention. The design generating device 100 may include one or more processors.

[0047] The network interface 102 supports wired and wireless network communication of the design generation device 100. The network interface 102 can support various communication methods in addition to the Internet, which is a public communication network. To this end, the network interface 102 can be configured to include a communication module well known in the technical field of the present invention. The network interface 102 can provide a connection to a physical output means such as a printer device and support outputting content reflected by a user interface according to an embodiment of the present invention.

[0048] According to an embodiment of the present invention, the design generator 100 can receive information including images and text using a network interface 102 .

[0049] The memory 103 stores various data, instructions and / or information. The memory 103 can load one or more programs 105 from the storage 104 to perform the method according to the embodiment of the present invention. The memory 103 can be configured as a volatile memory that can be written and read and has a high read or write speed. As an example, the memory 103 can include at least one of a RAM, a DRAM, or an SRAM.

[0050] The storage 104 can non-temporarily store the one or more programs 105 and the generated design image database 106. In Fig. 2, as an example of the one or more programs 105, a design generation S / W (hereinafter, design generation software) 105 is illustrated.

[0051] Storage 104 may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or any other form of computer-readable recording medium known in the art to which the present invention pertains.

[0052] According to an embodiment of the present invention, the design image database 106 can store not only information on each of the text and the image, but also a matching relationship between the text and the image. The design image database 106 according to an embodiment of the present invention can store designs generated to be provided to the manufacturing terminal 300, etc. Therefore, the designs provided to the manufacturing terminal 300 include design images generated in real time in response to the text input from the design generation device 100, rather than images randomly generated and stored in advance.

[0053] Meanwhile, the design generation software 105 according to an embodiment of the present invention may include various modules for implementing various functions of the design generation device 100. Hereinafter, in this specification, a module may refer to a functional unit of software.

[0054] 4 is an exemplary diagram of the structure of the design generation software referred to in some embodiments of the present invention. Each module of the design generation software 105 will be described below with reference to FIG.

[0055] Referring to FIG. 4, the design generation software 105 may include an image acquisition module 115, a text recognition module 125, an image text matching module 135 for matching the acquired image with the extracted text, a design generation module 145 based on the above-mentioned cGAN, and a resolution conversion module 155 for performing high-resolution image generation for the generated image.

[0056] For example, the image acquisition module 115 may acquire the image 50 of FIG. 1 by at least one of crawling, scraping, and parsing, and learn features therefrom to identify design elements, such as type, color, shape, and the like, of the image acquired by the design generation device 100. To this end, the image acquisition module 115 may include a machine learning model.

[0057] In one example, feature learning can extract feature points on a large number of acquired images by driving a machine learning model by the processor 101 in Fig. 3. Here, the machine learning model can include an artificial neural network, and can be configured to include at least one of a convolution neural network (CNN), an autoencoder, a feedforward neural network, a radial basis function network, a Kohonen self-organizing network, and a recurrent neural network (RNN).

[0058] For example, when the machine learning model is a convolutional neural network, the processor 101 drives the machine learning model to filter the feature values ​​of the one-piece image 50 in Fig. 1, and only specific values ​​are newly obtained from a large number of image pixels by a convolution operation. By repeating this learning process, desired feature points can be extracted.

[0059] The feature points are parts of a garment that can be grasped by the processor 101 executing the design generation program 105, and may be points corresponding to, for example, the outline of a one-piece garment, a pattern on the outer surface of the one-piece garment, a belt portion, etc. Specifically, the feature points may be points located on a line or area constituting the shape and / or pattern of a one-piece garment.

[0060] According to an embodiment of the present invention, the design generating device 100 can associate each feature point by analyzing the direction and similarity based on the extracted feature points. That is, the design generating device 100 can generate lines by associating and connecting each feature point, and can identify the object of the design, for example, the type of clothing, based on the generated lines.

[0061] The text recognition module 125 enables the design generator 100 to extract text 70 on the information 30 of Figure 1. For example, the text 70 is text on a web page of a shopping mall, and may include product description text by the page provider and review text by product buyers.

[0062] The text recognition module 125 allows the design generating device 100 to extract text such as one piece, dot, pattern, belt, A-style, semi-suit, BLACK / WHITE, SS season, from the product description text and the review text. The extracted text may be text that commonly appears in the product description text and the review text. As another example, the extracted text may be text that belongs to the product description text and the review text, respectively, and has a mutual semantic relationship. Furthermore, the extracted text may include words that are frequently exposed in relation to the design, words tagged by the text inputter, evaluations regarding the design, and text of positive or negative reactions.

[0063] In one embodiment, the design generation device 100 extracts text from the information 30, learns on pairs of the extracted text and the acquired images, and classifies the text into a first group of text related to product descriptions and a second group of text corresponding to review information.

[0064] The design generating device 100 can extract text that is included simultaneously in the first group of text and the second group of text, and can perform learning on the pair of acquired images 50 based on the text that is included simultaneously. The design generating device 100 can include a step of generating category text for the acquired image based on the learning result. For example, the design generating device 100 can generate "one piece" as a category of the design.

[0065] In one embodiment, the text recognition module 125 may include a network for encoding the extracted text. The encoding network may be implemented with four layers, with the lower two layers being limited within the boundaries of sentences for sentence-based representation, and the upper two layers being document-based representation, not limited to sentence boundaries, and learning the entire document concatenated. For example, the encoding network may be a language model based on Bidirectional Encoder Representations from Transformers (BERT). The BERT is a language model trained on a corpus containing 3.3 billion words using a bidirectional transformer structure.

[0066] According to one embodiment, the text recognition module 125 can learn a document including images and texts such as the information 30 of FIG. 1, which reflects both texts and images, as a learning target such as the corpus. In this way, by learning the text 70 matched with the features of the image 50 in the information 30, the conditions of the design to be generated according to the embodiment of the present invention can be learned. For example, the learned conditions can be text related to a category matched to the image to be designed, or text corresponding to a shape or pattern that is likely to be reflected in the design target. That is, the text "One Piece" matched to the image 50 of FIG. 1 can be learned as the category of the image 50. Also, it can be learned that the "dot" pattern is a pattern that applies to the category "One Piece".

[0067] According to an embodiment, the text recognition module 125 may be configured to learn a function that considers the characteristics of the domain to which the design object belongs and determines which category the object belongs to. For example, if it is determined that the design object is clothing, the domain is the fashion industry, and the category is clothing types, then, among the text 70 in FIG. 1, the design generator 100 may determine that the text "wrap dress" belongs to the category "one piece dress."

[0068] The design generation device 100 determines whether the vector value obtained by vectorizing the text is matched to a category, and if so, can generate a design reflecting this. Meanwhile, if the vector value obtained by vectorizing the text is not matched to a category, for example, if the vector value of "back pocket of pants" is not matched to the category of a dress, the design generation device 100 can apply a conversion value thereto and generate a converted vector value. In this case, the design generation device 100 can apply a pocket design on the skirt part of the dress to the dress. The vector conversion value can be generated based on the vector value of the dress belonging to the category.

[0069] The image-text matching module 135 matches the features identified as a result of learning the captured image with the vectorization results of the extracted text using a language model.

[0070] For example, the image text matching module 135 allows the design generating device 100 to include a deep learning model that combines the learning results of the image features by the image acquisition module 115 with the analysis of text learned by the text recognition module 125. This allows the design generating device 100 to generate conditions for the design to be generated. The generated conditions can be called a condition set or conditional.

[0071] At this time, the generated conditions can form a category. If the generated condition is "dress", the image text matching module 135 reflects the text "dress" in the category, and the dress category can reflect various dress-related image features and / or text embedding information such as shirt dress, tea dress, sweater dress, wrap dress, maxi dress, smock dress, skater dress, empire dress, etc. as conditions in the design generation process.

[0072] Meanwhile, although the case where the category is a condition for design generation has been mainly described, the embodiment of the present invention is not limited thereto, and text can also be reflected as a condition in the generated design.

[0073] The design generation module 145 controls the design generation device 100 to generate a design that reflects the input text under the generated conditions.

[0074] GAN is a technology that generates images starting from a randomly given vector, and is a method of generating random images, but it has the limitation that it is difficult to control the resulting image.To improve this, styleGAN has been proposed, which specifies a style vector instead of a random image and controls image generation through this.

[0075] From this perspective, according to an embodiment of the present invention, the design generation device 100 can generate a design that satisfies a constraint condition in response to a conditional input. That is, the design generation device 100 according to an embodiment of the present invention controls the application of a style to a design to be generated, thereby enabling the generation of a design that fully reflects the user's intention.

[0076] For this purpose, the text recognition module 125 can connect a network so that a design image can be generated using the encoding result of the encoding network described above as a style vector. The style vector is the vectorization result of the text by the language model described above, and is an element for applying a style to the image to be generated.

[0077] According to one embodiment of the present invention, in the image text matching module 135, a 512-dimension vector is used as the style vector, 8 fully-connected layers (FC layers) are used as the mapping network, and 18 CNN layers are used as the final design image generation network.

[0078] As a result, when the category is determined to be "one piece dress", which is a type of clothing, the design generation device 100 recognizes the input text as a keyword related to one piece dress, and the style vector value based on the input text is reflected in the one piece dress design image, thereby generating a design. For example, when the "one piece dress" category is selected in the design generation device 100, and the text "black", which is a color-related keyword, and "dot", which is a pattern-related keyword, are input, the style vector value of the input text is reflected, and a one piece dress design image printed with a black dot pattern can be generated.

[0079] Meanwhile, according to an embodiment of the present invention, the text recognition module 125 may generate a design in which a text image is not of one color but of a mixture of multiple colors in the design generation device 100. In the above example, when the text inputs "black", "dots", and "white" are input, the text recognition module 125 generates a style vector taking into consideration the relationship between "black" and "dots" and the input order of the text "white", the image text matching module 135 reflects the generated style vector in the design category as a condition, and the design generation module 145 generates a design image of a white dress with a black dot pattern.

[0080] A resolution conversion module 155 can then perform high resolution image generation on the generated image. The resolution conversion module 155 can support the functionality of the post generator 230 of FIG.

[0081] According to an embodiment of the present invention, the resolution conversion module 155 may include an image upscaling network that generates a result of the cGAN as a high-resolution image. The image upscaling network may be, for example, a Super-Resolution Generative Adversarial Network (SRGAN).

[0082] Generally, image generation requires a huge amount of computing power, so when using a deep learning network, the image resolution is limited to 256 x 256 pixels or 512 x 512 pixels to reduce the amount of calculation. However, when generating a result at a low resolution, it is difficult to use GAN in the fashion field where details are to be confirmed. To solve this problem, according to an embodiment of the present invention, the design generation device 100 can convert the result of the cGAN generation network into a high-resolution image using the SRGAN of the resolution conversion module 155.

[0083] Specifically, the design generating device 100 can generate a design image by reflecting similarity based on extracted features, rather than comparing pixels on the original image and the generated design image, by utilizing a network for increasing the resolution of an image, such as the SRGAN included in the resolution conversion module 155. For example, the resolution conversion module 155 can generate a high-resolution design image that is very similar even when viewed with the naked eye of a human being, by using the similarity of the encoding result using VGGNET as learning performance, rather than based on pixel-by-pixel similarity.

[0084] 3 again, the design generating device 100 may include a display 107, which may be a display 107 that is widely known in the technical field to which the present invention belongs. The display 107 may output a design image generated by the design generating device 100 and / or an image converted to a high resolution. If the display 107 is configured as a touch screen, it may function not only as an output unit but also as an input unit. In this case, the design generating device 100 may provide an interface for selecting the above-mentioned category and inputting text through the display 107.

[0085] Fig. 5 is a flowchart of a method for generating designs based on learned conditions according to another embodiment of the present invention, and Fig. 6 is a flowchart of a method for matching between an acquired image and an extracted text according to another embodiment of the present invention.

[0086] Each step referred to in FIGS. 5 and 6 below is performed by the design generating device 100, and specifically, can be executed by the processor 101 computing the design generating program 105.

[0087] Referring to Fig. 5, the design generating device 100 can acquire an image (S10). The acquired image may be an image included in the information 30 as referred to in Fig. 1. For example, it may be an image of a fashion design product, particularly a dress.

[0088] The design generating device 100 can extract text through the text recognition module 125 on the information 30 of FIG. 1, and match the extracted text with the acquired image using the image text matching module 135 (S20).

[0089] 6, in the matching step (S20), the design generating device 100 can perform learning on a text-image pair (S610). The text-image pair can be, for example, a pair of text 70 and image 50 on the information 30 in FIG. 1.

[0090] In the learning process of step (S610), text that is extracted with a high frequency can be extracted for design features that constitute at least a part of the dress analyzed by learning image 50 in Fig. 1. For example, if a one-piece image and the text "dot" matched therewith are extracted with a high frequency in various learned documents, the design generating device 100 can interpret that the design element on the one-piece image is "dot." This allows the text "dot" to be matched in relation to the one-piece in image 50.

[0091] An embodiment of the present invention can learn from pairs of images and extracted text obtained from documents that are subject to crawling, scraping, and / or parsing, and identify words that are extracted more frequently than a predetermined reference number of times, and determine these as trend words.

[0092] The design generating device 100 determines whether trend words are extracted during the learning process of step (S610) (S620). If trend words exist and are extracted, the design generating device 100 can perform learning on pairs of trend words and images (S630).

[0093] For example, if a page similar to information 30 in FIG. 1 is crawled a lot, or if multiple images 50 on the page are obtained as a result of parsing, this may be because there is a lot of information on polka dot dresses on the web page. This may be a reflection of trends such as an increase in the production and sales of polka dot dress designs among dress designs, and an increase in consumer interest. In this case, the text "dot" appears frequently in the product descriptions or user reviews on the page, and may be ranked high in the number of extractions when text is extracted.

[0094] In step S630, the design generating device 100 can learn the pair of the acquired one-piece image and the trend word "dot", and in the process, can identify the feature of the dot design on the one-piece image (S640). That is, since the one-piece image for the text "dot" has a dot pattern designed, as a result of learning, the design generating device 100 can identify the dot pattern, which is a design element of the one-piece, as feature information.

[0095] Furthermore, the design generating device 100 can also identify features of the dot pattern, such as the arrangement of dots, the number of dots, the area (size) of the dots, and the color of the dots, through the learning in step S630. This makes it possible to identify design feature information that corresponds to the latest trends among dot-patterned dresses.

[0096] In yet another embodiment, the design generating device 100 can determine as trend words not only texts with high ranking extraction counts but also texts with high extraction increase rates.

[0097] On the other hand, in step S620, a case may occur where trend words are not extracted. This may be the case where a market-leading design trend has not been formed or a term that represents the design of a particular product has not been established. In such a case, the design generating device 100 may adjust the criteria for extracting trend words (S650).

[0098] As an example, the design generating device 100 can change a preset reference number of times for extracting text.

[0099] As another example, the design generating device 100 can adjust the criteria to extract trend words according to the relationship between the extracted texts. For example, when there are multiple texts that are extracted alone infrequently but relatively frequently, the design generating device 100 can determine and extract each of the multiple texts as a trend word. In the above example, when "dot" is extracted infrequently in the pattern of a dress, but "dot" and "water droplets" are extracted relatively frequently, the design generating device 100 can determine that dots and water droplets represent the same design feature and perform learning in step (S630).

[0100] Specifically, if there is no trend word that is extracted more than a preset reference number of times, the design generation device 100 can identify a first design feature corresponding to a first text from the extracted text based on the learning results regarding image features and the learning results (S610) regarding the pair of the extracted text and the obtained image.

[0101] In a similar manner, the design generator 100 may also identify a second design feature from the extracted text that corresponds to a second text.

[0102] In one embodiment, when the first design feature and the second design feature have a similarity equal to or greater than a preset similarity, the design generating device 100 may determine each of the first text and the second text as trending words, i.e., determine both "water drop" and "dot" as trending words.

[0103] In one embodiment, the design generation device 100 may determine the first text and the second text as trend words when the first design feature and the second design feature have a similarity equal to or greater than a predetermined similarity, and when the sum of the number of times the first text is extracted and the number of times the second text is extracted is equal to or greater than a predetermined reference number.

[0104] As another example, the design generation device 100 may adjust the trend word extraction criteria by adjusting the trend word extraction schedule so as to increase the amount of learning related to the text-image pairs in step (S610).

[0105] Through the identification of the design feature information in step S640, the design generation device 100 can learn the text that means the design feature information as a condition for design generation.

[0106] 3, the design generation device 100 can learn conditions related to the design to be generated (S30). By learning in step (S30), the design generation device 100 can generate categories of designs. For example, the design generation device 100 can identify text that means a type of clothing, such as a dress, a jacket, pants, etc., and generate the text as a category.

[0107] In addition, the design generating device 100 can learn that the design classification of the details belonging to the type of clothing is subordinate to the category. For example, the design generating device 100 can learn that the polka dot dress in the image 50 of FIG. 1 belongs to the category "dress." Thus, when the dress category is selected and the text "dot" is input, the design generating device 100 can generate a design like the polka dot dress in the image 50.

[0108] The design generating device 100 can identify conditions corresponding to the input text (S40). In the above example, when the text "dot" is input, the design generating device 100 can identify design conditions corresponding to the dot. That is, the design generating device 100 can identify the category to which the text "dot" belongs and vector information related to the text.

[0109] The design generating device 100 may generate a design set based on the identified conditions (S50). At this time, the generated design set may be output through the display 107 of the design generating device 100, and the order in which the generated designs are displayed may be determined by reflecting the trend learned in step (S630) of Fig. 6. The design set is a group of design images including the generated designs, and the number of design images displayed on the screen may be determined by the display area of ​​the display 107 or a preset user interface.

[0110] The design generation device 100 can convert the generated design into a high resolution image (S60). To this end, the resolution conversion module 155 of Figure 5 enables the design generation device 100 to convert the resolution of the generated design based on the similarity of the design feature information, rather than on the similarity based on pixels.

[0111] Fig. 7 is an exemplary diagram for explaining a method for learning an image and a text pair to be matched therewith, which is referred to in some embodiments of the present invention. Fig. 8 is a diagram for explaining a trend word selection process, which is referred to in some embodiments of the present invention.

[0112] 7, the design generating device 100 can obtain a large number of image sets 700. The design generating device 100 can obtain a large number of images on a page such as the information 30 in FIG. 1, construct the image set 700 from the images, and determine the image set 700 as an image learning target for design feature extraction.

[0113] The design generator 100 can identify design features of a one-piece dress by studying the images 701, 702, 703, 704 on the image set 700.

[0114] The design generating device 100 can extract a large amount of text on a page such as the information 30. If the page is a page such as a blog or a shopping mall and has description information on the product to be designed, the design generating device 100 can extract text from the description information to generate a text set 710, and if review information 720 of the shopping mall exists, the design generating device 100 can also extract text from the review information 720.

[0115] The design generating device 100 can identify extracted text 730 that occurs more than a predetermined reference number of times from the text extracted from the text set 710 and the review information 720 .

[0116] FIG. 7 shows an example where words such as ONE PIECE, DRESS, ONE PIECE DRESS, Sleeve, Sleeveless, Black, Dark Gray, Cotton, Linen, AA Style, and AA Line are extracted.

[0117] The design generating device 100 can learn the text set 710 extracted through the description information together with the review information 720. As a result of learning, it is confirmed whether the text in the information for the product description is also used in the review, or whether the text described in the product description has a high similarity to the text reflected in the review, and the design generating device 100 can learn the text by product category.

[0118] 7, for the category of one-piece dresses, text such as AA, Double A, and A1 may be determined to be frequently extracted or to have an increasing amount of extraction, and may be determined as trend words 730. For example, AA, Double A, and A1 may be design terms related to design features of one-piece dresses, such as "dot."

[0119] 8, a graph 801 illustrates an example in which Black, DRESS, Sleeveless, AA, A1, and Double A are extracted a preset reference number of times x or more for the one-piece dress category at time T1 for an image 701. In particular, AA, A1, and Double A are determined as trend words at time T1.

[0120] Then, at time T2, the graph 802 illustrates a case where the design generation device 100 extracts Green, DRESS Sleeveless, AA, DA, and G.Pattern from the image 701 a preset reference number of times x or more. The design generation device 100, and in particular, at time T2, determined Green, DA, and G.Pattern as trend words.

[0121] Due to the change from time T1 to time T2, the text Black has been removed from the top extracted texts, and Green has been added. The amount of extraction of AA has decreased, and A1 has been removed. This may mean that the terms AA and A1 are not being used continuously. Furthermore, Double A has been changed to DA, which may mean that the design terms used by the user have been changed. The design generation device 100 can learn such changes in extracted text and identify trend changes for the category one piece of the image 701.

[0122] Furthermore, the design generation device 100 may detect trend changes for the dress category of the image 701 through new extraction of the newly added Green and G.Pattern. For example, the change from time T1 to time T2 may reflect fashionable colors due to seasonal changes, and the design generation device 100 may learn such trend changes based on changes in the extracted text.

[0123] FIG. 9 is an example diagram of an interface for generating designs in response to input text as referenced in some embodiments of the present invention.

[0124] 9, the design generation device 100 can generate an interface 900 and output it through the display 107 of FIG 3. In FIG 9, the interface 900 can include a text input window 903 and a design image output section 905.

[0125] The design generating device 100 can generate a design image 905 that reflects the text in the text input window 903 as a condition.

[0126] FIG. 10 is an exemplary diagram of a high resolution design image generated through a post generator as referred to in some embodiments of the present invention.

[0127] In FIG. 10, an original image 1000, a generated design image 1100, and a resolution-converted design image 1110 are shown.

[0128] 10, an original image 1000 can include background 1 (1003), background 2 (1005), design element 1 (1001), and design element 2 (1007) as components. A generated design image 1100 can include background 1 (1103), background (1105), design element 1 (1101), and design element 2 (1107).

[0129] Resolution-converted design image 1110 can include background 1 (1113), background (1115), design element 1 (1111), and design element 2 (1117).

[0130] The design generation device 100 can learn the original image 1000 and generate a design image 1100. At this time, as can be seen from the image 1100, it can be confirmed that background 1 (1103) and background 2 (1105) are distorted (smoothed) due to a resolution problem, and that design element 1 (1101) and design element 2 (1107) also differ from the original.

[0131] The generated design image 1100 is generated in comparison to the original image 1000, and although there may be differences in design, resolution issues cause the design image 1100 to be perceived as not being an actual existing original image, and to the user, it is identified as having been generated through image processing.

[0132] Referring to the resolution-converted image 1110, although there is a resolution problem like background 2 (1115), it shows a resolution similar to that of the actual original image, even though it is an image generated based on the original image 1000. The design generation device 100 can generate the resolution-converted image 1110 by using the image conversion module 155 to learn through comparison of design features, rather than pixel-by-pixel comparison of the original image 1000.

[0133] The methods according to the embodiments of the present invention described above with reference to the accompanying drawings may be performed by execution of a computer program embodied in computer readable code. The computer program may be transmitted from a first computer device to a second computer device over a network such as the Internet, and installed in the second computer device, thereby being used by the second computer device. The first computer device and the second computer device may include any of a server device, a fixed computer device such as a desktop PC, and a mobile computer device such as a laptop, a smartphone, and a tablet PC.

[0134] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art will understand that the present invention can be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting.

Claims

1. 1. A method for matching text and design performed by a text and design matching device, comprising: obtaining said image from information including images and text; learning features of the captured images; extracting text from the information and performing learning on pairs of the extracted text and the captured image; extracting trend words that have been extracted a predetermined number of times or more from the extracted text; learning about pairs of the trend words and the acquired images; Identifying design feature information that matches the trend word among the learned features of the image; Including, The step of extracting trend words extracted a predetermined number of times or more from the extracted text includes: If there is no trend word that has been extracted a predetermined number of times or more, identifying a first design feature corresponding to a first text from among the extracted texts based on a learning result regarding the image features and a learning result regarding the pair of the extracted text and the obtained image; and if there is no trend word extracted the predetermined reference number of times or more, identifying a second design feature corresponding to a second text from among the extracted texts based on a learning result regarding the image features and a learning result regarding the pair of the extracted text and the obtained image. How to match text with design.

2. 2. The method for matching text and design according to claim 1, further comprising a step of determining each of the first text and the second text as a trend word when the first design feature and the second design feature have a similarity equal to or greater than a predetermined similarity.

3. 2. The method for matching text and design according to claim 1, further comprising: determining the first text and the second text as trend words when the first design feature and the second design feature have a similarity equal to or greater than a predetermined similarity and when a total value of a number of times the first text is extracted and a number of times the second text is extracted is equal to or greater than a predetermined reference number.

4. The step of extracting text from the information and learning on pairs of the extracted text and the captured image includes: classifying said text into a first group of text and a second group of text; extracting text that is simultaneously included in the first group of text and the second group of text; learning the captured image pairs based on the text contained therein; and generating a category text for the acquired image based on the learning result.

Images