Non-abandoned image generation method according to user requirements

Abstract

The invention discloses a non-abandoned image generation method according to user requirements. The method comprises the following steps: training a generative adversarial network; establishing a demand mapping table; obtaining a user demand; querying feature coding information in a demand mapping table according to the keyword; generating feature coding information of the original image; merging the feature coding information of the original image and the queried feature coding information to obtain merged feature coding information; and inputting the combined feature coding information into a trained generative adversarial network, and outputting a non-missing image based on user requirements by the trained generative adversarial network. According to the method, a double-discriminator mechanism is adopted, the double discriminators evaluate the generated non-abandoned pattern from the two directions of non-abandoned authenticity and the user demand matching degree, it is ensured that the generated image is highly consistent with real data in the two aspects, the design demand of a user can be accurately understood and effectively met through application of the demand mapping table, and the user experience is improved. And the individuation and the use effect of the generated image are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of image generation, and in particular relates to a method for generating intangible cultural heritage images according to user requirements. Background Art

[0002] Intangible cultural heritage refers to various traditional cultural expressions that have been passed down from generation to generation by people of all ethnic groups and regarded as part of their cultural heritage, as well as physical objects and places related to traditional cultural expressions. It emphasizes people-centered skills, experience, and spirit. It is a living and changing cultural heritage that does not rely on material form to exist. It is more reflected in people's behavior, performance, traditional crafts, social practice, etc. Intangible cultural heritage is a "living" manifestation of national personality and national aesthetic habits, and is of great significance to the inheritance of national culture and the maintenance of cultural diversity.

[0003] In recent years, deep learning technology, especially GAN generative adversarial networks and CNN convolutional neural networks, has made breakthrough progress in the field of image generation. GAN is good at generating high-quality images from noisy data or under specific conditions, while CNN convolutional neural networks can extract useful visual features from large amounts of data for pattern analysis and generation. These technologies can be used to extract information such as geometric structure and color matching from traditional patterns, thereby achieving efficient automatic generation and optimization.

[0004] The design and production process of traditional intangible cultural heritage patterns usually relies on manual labor, which is time-consuming, difficult to mass-produce, and difficult to flexibly adjust. In addition, traditional handicrafts cannot meet the growing personalized needs in contemporary design. Automatically generating and optimizing the design of intangible cultural heritage patterns through deep learning technology can improve the efficiency of pattern design, ensure the inheritance of cultural elements, and meet the personalized and innovative needs in modern design. This automatic generation process can not only enhance the inheritance of intangible cultural heritage patterns, but also combine them with modern art design to form more works with contemporary characteristics. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for generating intangible cultural heritage images according to user needs in view of the deficiencies in the above-mentioned prior art. The method has a simple structure and a reasonable design. A dual discriminator mechanism is adopted. The dual discriminator evaluates the generated intangible cultural heritage patterns from two aspects: the authenticity of the intangible cultural heritage and the matching degree of user needs, respectively, to ensure that the generated images are highly consistent with the real data in these two aspects. Through the application of the demand mapping table, the user's design needs can be accurately understood and effectively realized, ensuring the personalization and use effect of the generated images.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for generating intangible cultural heritage images according to user needs, characterized in that it includes the following steps:

[0007] Step 1: Train the Generative Adversarial Network:

[0008] S11. Construct a generative adversarial network, which includes a generator G, a discriminator D1 and a discriminator D2, and set initial training parameters of the generative adversarial network;

[0009] S12. Obtaining a sample set: The sample set includes an intangible cultural heritage image set F and a user demand set X;

[0010] S13. Merge the first image in the non-legacy image set F and the second image in the user demand set X to obtain feature encoding information of the third image;

[0011] S14. Inputting the feature encoding information of the third image into the generator, and the generator outputs a new intangible heritage pattern;

[0012] S15. Input the new intangible cultural heritage pattern into the discriminator D1 and the discriminator D2 respectively, the discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to the first image, and the discriminator D2 outputs the probability p2 that the new intangible cultural heritage pattern belongs to the second image. If p1>p2, update the training parameters of the generator G and proceed to step S16; otherwise, update the training parameters of the generator G and return to step S13;

[0013] S16. Calculate the total loss D plus , update the training parameters of discriminator D1 and discriminator D2;

[0014] S17. Repeat steps S13 to S16 until the loss of the generative adversarial network converges, and then end the iteration to obtain a trained generative adversarial network.

[0015] Step 2: Establish a demand mapping table, which contains multiple key-value pairs, and the value of each key-value pair stores feature coding information corresponding to the keyword carrying its corresponding key;

[0016] Step 3: Generate intangible cultural heritage images based on user needs:

[0017] S31. Obtaining user requirements, including design requirements in the form of original images and natural language;

[0018] S32. Obtain keywords corresponding to design requirements, and query feature coding information in the requirements mapping table according to the keywords;

[0019] S33. Generate feature encoding information of the original image;

[0020] S34. Merge the feature coding information of step S32 and step S33 to obtain the merged feature coding information;

[0021] S35. Input the merged feature encoding information into the trained generative adversarial network, and the trained generative adversarial network outputs the intangible cultural heritage image based on user needs.

[0022] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized by further comprising step 4:

[0023] S41. Aesthetically classify the intangible cultural heritage images based on user needs. If the intangible cultural heritage images based on user needs are of low aesthetic quality, proceed to step S42, otherwise proceed to step S43;

[0024] S42. Merge the feature coding information of step S32, the feature coding information of step S33 and the random noise coding information to obtain the merged feature coding information; merge the feature coding information and input it into the trained generative adversarial network to regenerate the non-legacy image based on user needs;

[0025] S43. Output intangible cultural heritage images based on user needs.

[0026] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that the aesthetic classification is performed based on a binary classification model, and the sample set of the binary classification model is a binary classification sample set based on intangible cultural heritage patterns.

[0027] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that: the specific method of S13 is:

[0028] S131. The intangible cultural heritage image set F includes k types of images with labels; the user demand set X includes r types of images corresponding to preset user demands, where k and r are both positive integers greater than 1;

[0029] S132. randomly select a class h in the intangible cultural heritage image set F, and randomly select an image in the class h as the first image; randomly select a class t in the user demand set X, and randomly select an image in the class t as the second image;

[0030] S133. Combine the first image and the second image to obtain feature coding information of the third image, 1≤h≤k, 1≤t≤r.

[0031] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that: the specific method in S15 is: inputting the new intangible cultural heritage pattern into the discriminator D1 and the discriminator D2 respectively, the discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to class h, and the discriminator D1 outputs the probability p2 that the new intangible cultural heritage pattern belongs to class t. If p1>p2, update the training parameters of the generator G and enter step S16; otherwise update the training parameters of the generator G and return to step S13.

[0032] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that the intangible cultural heritage image set F includes at least 6 categories: patterns, characters, fabrics, New Year pictures, colors, eras, themes, and shapes.

[0033] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized by: D plus =α·D1+(1-α)·D2, where α represents the weight, D1 represents the loss of discriminator D1, D2 represents the loss of discriminator D2, and α>0.5.

[0034] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that: the demand mapping table includes mapping sub-tables of multiple categories, and the categories of the mapping sub-tables correspond to the categories of the user demand table X.

[0035] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that: if multiple value fields are obtained by querying the demand mapping table according to the key field of the design requirements, the merged images corresponding to the multiple value fields are first weighted merged, the weights are randomly assigned, and then merged with the feature coding information of the original image.

[0036] The above-mentioned method for generating intangible cultural heritage images according to user needs is characterized in that the third image is converted into feature coding information through the image encoder of the CLIP model.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. The invention has a simple structure, reasonable design, and is easy to implement and operate.

[0039] 2. The generative adversarial network of the present invention has two discriminators. Discriminator D1 and discriminator D2 respectively perform bidirectional evaluation on the generated new intangible cultural heritage pattern to ensure that the generated new intangible cultural heritage pattern can match the real data in both directions, thereby improving the accuracy of the generated new intangible cultural heritage pattern.

[0040] 3. The generator G of the generative adversarial network of the present invention does not generate data from scratch, but converts real data into generated data. Therefore, the input of the generator G is feature encoding information based on the intangible cultural heritage image and the corresponding image of the user's needs.

[0041] 4. The present invention establishes a demand mapping table for associating different keywords with feature coding information, thereby realizing data conversion and transmission. By clearly listing the mapping relationship between each demand and feature coding information, accurate understanding and effective implementation of the demand can be ensured.

[0042] 5. The present invention performs aesthetic classification on intangible cultural heritage images based on user needs, which can effectively screen images and provide users with intangible cultural heritage images based on user needs that are more aesthetically pleasing.

[0043] In summary, the present invention has a simple structure and a reasonable design. It adopts a dual discriminator mechanism. The dual discriminator evaluates the generated intangible cultural heritage patterns from two aspects: the authenticity of the intangible cultural heritage and the matching degree of user needs, respectively, to ensure that the generated image is highly consistent with the real data in these two aspects. Through the application of the demand mapping table, it can accurately understand and effectively realize the design needs of users, ensuring the personalization and use effect of the generated image.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The figure is a flow chart of the method of the present invention.

[0046] Figure 2 This is a partial image collection of the intangible cultural heritage image set F of the present invention.

[0047] Figure 3 It is a partial image collection of the user requirement set X of the present invention.

[0048] Figure 4 This is an example of the demand mapping table of the present invention.

[0049] Figure 5 It is the effect diagram of the present invention.

[0050] Figure 6 This is the effect diagram of the aesthetic classification of the present invention. DETAILED DESCRIPTION

[0051] The method of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0052] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0055] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0056] like Figure 1 As shown, a method for generating an intangible cultural heritage image according to user needs of the present invention comprises the following steps:

[0057] Step 1: Train the Generative Adversarial Network:

[0058] S11. Construct a generative adversarial network, which includes a generator G, a discriminator D1 and a discriminator D2, and set the initial training parameters of the generative adversarial network.

[0059] It should be noted that the generative adversarial network consists of two parts: the generator and the discriminator. Its main function is to generate new data with real data distribution. Through the adversarial process between the generator and the discriminator, the ability of the generator is continuously optimized, enabling it to generate more and more real data.

[0060] In one possible embodiment, the generative adversarial network uses a bidirectional generative adversarial network BGAN. The bidirectional generative adversarial network BGAN has two discriminators, which respectively perform bidirectional evaluations on the generated data to ensure that the generated data can match the real data in both directions. This setting helps to improve the diversity and accuracy of the generated data.

[0061] In one possible embodiment, the bidirectional generative adversarial network BGAN uses a cross entropy loss function to train the generator G, the discriminator D1 and the discriminator D2. Each discriminator is a multi-layer neural network, including a convolution layer, a pooling layer, a fully connected layer, etc., which is used to extract the features of the input data and output a probability value, indicating the possibility that the input data is real data. The generator G is a multi-layer neural network, including a convolution layer, an upsampling layer, etc. It should be noted that the generator G does not generate data from scratch, but converts real data into generated data. Therefore, the input of the generator G is the feature encoding information of the third image.

[0062] S12. Obtain a sample set: The sample set includes an intangible cultural heritage image set F and a user demand set X.

[0063] In this embodiment, the intangible cultural heritage image set F comes from the Chinese intangible cultural heritage gene database, and the intangible cultural heritage image set F includes at least 9 categories: jewelry, figures, fabrics, New Year paintings, colors, eras, themes, and shapes. Figure 2 It is a collection of some images of the intangible cultural heritage image set F. Figure 2 The first row contains four fabric images, namely animal patterns, curling grass patterns, plum and bamboo patterns, and tortoise shell patterns. Figure 2 The second row contains images of four characters, namely Fugui Wawa, Sun Wukong, Zhong Kui and Lin Chong. Figure 2 The third row contains images from four different eras, namely patterns from the Han Dynasty, Qing Dynasty, Tang Dynasty, and modern paper-cut flower and bird patterns. It should be noted that the same label in each category has multiple images.

[0064] The user requirement set X is generated based on the collected user requirements. The user requirement set X includes at least three categories, namely color, style and graphics. Figure 3 is a collection of partial images of user demand set X, Figure 3 The first row is color images, including kaleidoscope color matching, pure dark green color matching, rainbow color matching, and Dunhuang color matching. Figure 3 The second row is the style images, namely Van Gogh starry sky, watercolor style, sketch style and abstract style. Figure 3 The third row is a graphic image, which can also be understood as a geometric texture image, including bamboo leaves, cranes, dragons, and phoenixes. It should be noted that the same label in each category has multiple images.

[0065] S13. Merge the first image in the non-legacy image set F and the second image in the user requirement set X to obtain feature coding information of the third image.

[0066] S131. The intangible cultural heritage image set F includes k types of images with labels; the user demand set X includes r types of images corresponding to preset user demands, and k and r are both positive integers greater than 1.

[0067] In a possible embodiment, the intangible cultural heritage image set F includes at least 9 categories, namely jewelry, figures, fabrics, New Year paintings, colors, eras, themes, and forms. The user demand set X includes at least 3 categories, namely color, style, and graphics. k=9, r=3.

[0068] S132. Randomly select a class h in the intangible cultural heritage image set F, and randomly select an image in the class h as the first image; randomly select a class t in the user demand set X, and randomly select an image in the class t as the second image.

[0069] like Figure 6 As shown, in a possible embodiment, an image labeled as a curly grass pattern in the fabric image category is selected as the first image in the intangible cultural heritage image set F, and an image labeled as a watercolor style in the style category is selected as the second image in the user demand set X.

[0070] S133. Combine the first image and the second image to obtain feature coding information of the third image, 1≤h≤k, 1≤t≤r.

[0071] The third image is converted into feature encoding information through the image encoder of the CLIP model.

[0072] The image encoder of the CLIP model generally adopts the ResNet architecture or the Vision Transformer architecture. In a possible embodiment, the image encoder of the CLIP model adopts the ResNet architecture. The image encoder of the CLIP model includes an input layer, a feature extraction layer, an embedding layer and an output layer.

[0073] Input layer: The image encoder receives the raw image data as input. These image data are preprocessed and input into the feature extraction layer. The preprocessing includes resizing or normalization to meet the input requirements of the CLIP model. Feature extraction layer: The feature extraction layer is mainly composed of multiple convolutional layers and pooling layers. These layers extract features from the image through convolution and pooling operations. Embedding layer: The image encoder usually contains an embedding layer after the feature extraction layer, which is used to convert the extracted features into vector representations. Output layer: The output layer of the image encoder outputs a high-dimensional vector representation, which is a digital feature representation of the image and contains the key features and information in the image, so that the model can use these features in subsequent calculations for various tasks.

[0074] S14. Input the feature encoding information of the third image into the generator G, and the generator G outputs the new intangible cultural heritage pattern. In one possible embodiment, the generator G is a multi-layer neural network, including a convolution layer, an upsampling layer, etc. It should be noted that the generator G does not generate data from scratch, but is used to convert real data into generated data. Therefore, the input of the generator G is the feature encoding information of the third image.

[0075] S15. Input the new intangible cultural heritage pattern into discriminator D1 and discriminator D2 respectively. Discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to the first image, and discriminator D2 outputs the probability p2 that the new intangible cultural heritage pattern belongs to the second image. If p1>p2, update the training parameters of generator G and go to step S16; otherwise, update the training parameters of generator G and return to step S13.

[0076] The goal of the generator G is to generate data that is as realistic as possible, so that the discriminator D1 and the discriminator D2 cannot distinguish whether the generated data is real or generated. Therefore, the loss function usually used by the generator G is to make the probability of the discriminator's judgment on the generated data as close to 1 as possible. In a possible embodiment, the generator G uses a cross entropy loss function.

[0077] The specific method in S15 is: input the new intangible cultural heritage pattern into discriminator D1 and discriminator D2 respectively, discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to class h, and discriminator D2 outputs the probability p2 that the new intangible cultural heritage pattern belongs to class t. If p1>p2, update the training parameters of generator G and enter step S16; otherwise, update the training parameters of generator G and return to step S13.

[0078] Discriminator D1 and discriminator D2 respectively perform bidirectional evaluation on the generated new intangible cultural heritage pattern to ensure that the generated new intangible cultural heritage pattern can match the real data in both directions, thereby improving the accuracy of the generated new intangible cultural heritage pattern.

[0079] S16. Calculate the total loss D plus, update the training parameters of discriminator D1 and discriminator D2.

[0080] D plus =α·D1+(1-α)·D2, where α represents the weight, D1 represents the loss of discriminator D1, D2 represents the loss of discriminator D2, and α>0.5.

[0081] It should be noted that the total loss D plus is the weighted value of the loss of discriminator D1 and discriminator D2, among which the loss of discriminator D1 accounts for a larger proportion, making the total loss D plus The loss of the discriminator D1 is biased towards this, thereby ensuring that the output of the generative adversarial network is generated based on the non-heritage images, and the probability of belonging to the non-heritage images is higher.

[0082] The task of discriminator D1 is to distinguish whether the input data is real data of the intangible cultural heritage image set F or generated by generator G. Discriminator D1 hopes to maximize the accuracy of judging the real data while minimizing the accuracy of judging the generated data. Discriminator D2 is similar. The task of discriminator D2 is to distinguish whether the input data is real data of the user demand set X or generated by generator G. Discriminator D2 hopes to maximize the accuracy of judging the real data while minimizing the accuracy of judging the generated data. Therefore, the loss functions of both discriminator D1 and discriminator D2 use the binary cross entropy loss function.

[0083] S17. Repeat steps S13 to S16 until the loss of the generative adversarial network converges, end the iteration, and obtain a trained generative adversarial network.

[0084] Since it is necessary to generate intangible cultural heritage images based on user needs, two discriminators are set in the generative adversarial network. Discriminator D1 is used to output the probability of discriminating that the new intangible cultural heritage pattern belongs to a certain category in the intangible cultural heritage pattern set F, and discriminator D2 is used to output the probability of discriminating that the new intangible cultural heritage pattern belongs to a certain category in the user demand set X, thereby ensuring that the new intangible cultural heritage pattern retains both the characteristics of the intangible cultural heritage pattern and the characteristics of the user needs.

[0085] At the same time, during the training process, it is necessary to ensure that p1>p2, and to have the characteristics required by users while retaining the characteristics of the intangible cultural heritage patterns.

[0086] Step 2: Establish a demand mapping table, which contains multiple key-value pairs. The value of each key-value pair stores feature coding information corresponding to the keyword carrying its corresponding key.

[0087] The requirement mapping table is used to associate different keywords with feature coding information to achieve data conversion and transmission. By clearly listing the mapping relationship between each requirement and feature coding information, it can ensure accurate understanding and effective implementation of the requirements, and facilitate verification and confirmation of the requirements in subsequent stages.

[0088] In a possible embodiment, the demand mapping table includes multiple categories of mapping sub-tables, and the categories of the mapping sub-tables correspond to the categories of the user demand table X. The demand mapping table includes at least three mapping sub-tables, and the key fields of the three mapping sub-tables include color demand keywords, style demand keywords, and pattern demand keywords. Figure 4 As shown, the key field corresponds to the keyword and the value field corresponds to the image.

[0089] If multiple value fields are obtained from the requirement mapping table according to the key field of the design requirement, the merged images corresponding to the multiple value fields are first weighted and merged, with the weights randomly assigned, and then merged with the feature encoding information of the original image. Figure 4 As shown in the first and third rows of , there are 2 keywords each, which corresponds to images in the 2 user requirement sets X. The two images are merged to generate a merged image.

[0090] Step 3: Generate intangible cultural heritage images based on user needs:

[0091] S31. Obtain user requirements, which include design requirements in the form of original images and natural language.

[0092] It should be noted that the original image refers to the intangible cultural heritage image, and the design requirements in the form of natural language are as follows: Figure 4 As shown in the first column of . Natural language is the main way of daily communication for humans. Design requirements in natural language can express the requirements intuitively and clearly. Using natural language to describe design requirements facilitates the interaction between users and models.

[0093] S32. Obtain keywords corresponding to the design requirements, and query feature encoding information in the requirement mapping table according to the keywords. Extract keywords and synonyms of keywords from the design requirements in natural language form, query the corresponding merged image according to the keywords or synonyms, input the merged image into the feature encoder, and obtain feature encoding information of the merged image.

[0094] To obtain the keywords corresponding to the design requirements, a keyword extraction model is needed. The keyword extraction model is an important technology in the field of natural language processing. It can automatically extract key words or phrases from the text. These words or phrases can usually summarize the theme or core content of the text. This belongs to the prior art and will not be described in detail in this application.

[0095] S33. Generate feature encoding information of the original image;

[0096] S34. Merge the feature coding information of step S32 and step S33 to obtain the merged feature coding information;

[0097] S35. Input the merged feature encoding information into the trained generative adversarial network, and the trained generative adversarial network outputs the intangible cultural heritage image based on user needs.

[0098] like Figure 5 As shown in the figure, for the original image a1, the design requirements are: green ink painting style, delicate brushstrokes, exquisite details, clear layers and resolution. The keywords extracted based on the design requirements include the first keyword "ink" and the second keyword "green". The first keyword "ink" corresponds to Figure 4 The first image in the first row and the second keyword "green" correspond to Figure 4 The second image in the first row, the first image and the second image are merged to form a merged image a2, and the original image a1 and the merged image a2 are output through the trained generative adversarial network to form an image a3.

[0099] For the original image b1, the design requirements are: kaleidoscope color matching, geometric fractal shape, and gradient background. The keywords extracted based on the design requirements include the first keyword "kaleidoscope". The first keyword "kaleidoscope" corresponds to Figure 4 The first image in the second row is the merged image a2, the original image b1 and the merged image b2 based on design requirements, and the trained generative adversarial network output image b3.

[0100] Similarly, the original image c1 and the merged image c2 based on design requirements are trained to generate an output image c3 through the generative adversarial network. The original image d1 and the merged image d2 based on design requirements are trained to generate an output image d3 through the generative adversarial network. The original image e1 and the merged image e2 based on design requirements are trained to generate an output image e3 through the generative adversarial network.

[0101] Step 4:

[0102] S41. Aesthetically classify the intangible cultural heritage images based on user needs. If the intangible cultural heritage images based on user needs are of low aesthetic quality, proceed to step S42, otherwise proceed to step S43;

[0103] S42. Merge the feature coding information of step S32, the feature coding information of step S33 and the random noise coding information to obtain the merged feature coding information; merge the feature coding information and input it into the trained generative adversarial network to regenerate the non-legacy image based on user needs;

[0104] S43. Output intangible cultural heritage images based on user needs.

[0105] In this embodiment, aesthetic classification is performed based on a binary classification model. This method belongs to the prior art. This embodiment provides a method for achieving aesthetic classification: the binary classification model is composed of a neural network based on an attention mechanism. The attention mechanism can help the binary classification model pay more attention to the important parts of the image, thereby improving recognition accuracy and efficiency. The input of the binary classification model is an RGB original image with an input image size of 227×227×3. The neural network based on the attention mechanism extracts local detail features related to aesthetic quality prediction in the image, and then performs binary classification on the local detail features. The results of the binary classification are low aesthetic quality and high aesthetic quality. Through aesthetic classification, images can be effectively screened to provide users with image content that is more in line with aesthetic preferences.

[0106] In a possible embodiment, the sample set of the binary classification model is the CUHK dataset, which contains 12,000 images, half of which are marked as high-quality images with a binary classification label of 1, and the other half are marked as low-quality images with a binary classification label of 0.

[0107] In a possible embodiment, the sample set of the binary classification model is a binary classification sample set based on intangible cultural heritage patterns. Multiple intangible cultural heritage patterns are displayed to the user through a pop-up window, and the customer marks them as beautiful or ugly, thereby forming a binary classification sample set based on intangible cultural heritage patterns.

[0108] like Figure 6 As shown, Figure 5 The intangible heritage image e1 and the merged image e2 based on design requirements are trained on the generative adversarial network. The first output is Figure 6 a, Figure 6 a Aesthetic classification is performed through a binary classification model based on a convolutional neural network (CNN). Figure 6 a is of low aesthetic quality, so return to step S32 and regenerate Figure 6 b, Binary classification model based on convolutional neural network CNN Figure 6 b is of high aesthetic quality, so it is retained Figure 6 b, and output Figure 6 b.

[0109] The above description is only an embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for generating intangible cultural heritage images according to user needs, characterized in that: The following steps are involved: Step 1: Train the Generative Adversarial Network: S11. Construct a generative adversarial network, which includes a generator G, a discriminator D1 and a discriminator D2, and set initial training parameters of the generative adversarial network; S12. Obtaining a sample set: The sample set includes an intangible cultural heritage image set F and a user demand set X; S13. Merge the first image in the non-legacy image set F and the second image in the user demand set X to obtain feature encoding information of the third image; S14. Inputting the feature encoding information of the third image into the generator G, the generator G outputs a new intangible heritage pattern; S15. Input the new intangible cultural heritage pattern into the discriminator D1 and the discriminator D2 respectively, the discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to the first image, and the discriminator D2 outputs the probability p2 that the new intangible cultural heritage pattern belongs to the second image. If p1>p2, update the training parameters of the generator G and proceed to step S16; otherwise, update the training parameters of the generator G and return to step S13; S16. Calculate the total loss D plus , update the training parameters of discriminator D1 and discriminator D2; S17. Repeat steps S13 to S16 until the loss of the generative adversarial network converges, and then end the iteration to obtain a trained generative adversarial network. Step 2: Establish a demand mapping table, which contains multiple key-value pairs, and the value of each key-value pair stores feature coding information corresponding to the keyword carrying its corresponding key; Step 3: Generate intangible cultural heritage images based on user needs: S31. Obtaining user requirements, including design requirements in the form of original images and natural language; S32. Obtain keywords corresponding to design requirements, and query feature coding information in the requirements mapping table according to the keywords; S33. Generate feature encoding information of the original image; S34. Merge the feature coding information of step S32 and step S33 to obtain the merged feature coding information; S35. Input the merged feature encoding information into the trained generative adversarial network, and the trained generative adversarial network outputs the intangible cultural heritage image based on user needs.

2. A method for generating intangible cultural heritage images according to user needs according to claim 1, characterized in that: Also includes step 4: S41. Aesthetically classify the intangible cultural heritage images based on user needs. If the intangible cultural heritage images based on user needs are of low aesthetic quality, proceed to step S42, otherwise proceed to step S43; S42. Merge the feature coding information of step S32, the feature coding information of step S33 and the random noise coding information to obtain the merged feature coding information; Merge the feature encoding information into the trained generative adversarial network to regenerate the intangible cultural heritage images based on user needs; S43. Output intangible cultural heritage images based on user needs.

3. A method for generating intangible cultural heritage images according to user needs according to claim 2, characterized in that: Aesthetic classification is carried out based on a binary classification model, and the sample set of the binary classification model is a binary classification sample set based on intangible cultural heritage patterns.

4. A method for generating intangible cultural heritage images according to user needs according to claim 1, characterized in that: The specific method of S13 is: S131. The intangible cultural heritage image set F includes k types of images with labels; the user demand set X includes r types of images corresponding to preset user demands, where k and r are both positive integers greater than 1; S132. randomly select a class h in the intangible cultural heritage image set F, and randomly select an image in the class h as the first image; randomly select a class t in the user demand set X, and randomly select an image in the class t as the second image; S133. Combine the first image and the second image to obtain feature coding information of the third image, 1≤h≤k, 1≤t≤r.

5. A method for generating intangible cultural heritage images according to user needs according to claim 4, characterized in that: The specific method in S15 is: input the new intangible cultural heritage pattern into discriminator D1 and discriminator D2 respectively, discriminator D1 outputs the probability p1 that the new intangible cultural heritage pattern belongs to class h, and discriminator D2 outputs the probability p2 that the new intangible cultural heritage pattern belongs to class t. If p1>p2, update the training parameters of generator G and enter step S16; otherwise, update the training parameters of generator G and return to step S13.

6. A method for generating intangible cultural heritage images according to user needs according to claim 1 or 5, characterized in that: The intangible cultural heritage image set F includes at least six categories: patterns, figures, fabrics, New Year pictures, colors, eras, themes, and forms.

7. The method for generating intangible cultural heritage images according to user needs according to claim 1, characterized in that: D plus =α·D1+(1-α)·D2, where α represents the weight, D1 represents the loss of discriminator D1, D2 represents the loss of discriminator D2, and α>0.

5.

8. The method for generating intangible cultural heritage images according to user needs according to claim 1, characterized in that: The demand mapping table includes mapping sub-tables of multiple categories, and the categories of the mapping sub-tables correspond to the categories of the user demand table X.

9. A method for generating intangible cultural heritage images according to user needs according to claim 1 or 8, characterized in that: If multiple value fields are obtained by querying the requirement mapping table according to the key field of the design requirement, the merged images corresponding to the multiple value fields are first weighted merged, the weights are randomly assigned, and then merged with the feature coding information of the original image.

10. The method for generating intangible cultural heritage images according to user needs according to claim 1, characterized in that: The third image is converted into feature encoding information through the image encoder of the CLIP model.

Images