A method for extracting ethnic costume sketches based on cycle-consistent generative adversarial networks

Abstract

The present invention relates to a method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network, belonging to the field of image translation technology. The present invention utilizes a cycle-consistent generative adversarial network to automatically generate sketches of ethnic costume images, extracting edge and contour features of ethnic costume images to automatically generate ethnic costume sketches. Before input into the generator network, the source image undergoes bilateral filtering for noise reduction and color quantization to optimize the source image quality and facilitate the generation of the final ethnic costume sketch. The present invention can effectively and automatically generate ethnic costume sketches, achieving automatic generation of ethnic costume grayscale image sketches.

Description

Technical Field

[0001] The present invention relates to a method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network, and belongs to the technical field of image translation. Background Art

[0002] Ethnic minority clothing culture holds a crucial position in the traditional culture of the Chinese nation and is a crucial component of the preservation and inheritance of ethnic minority culture. As a crucial means of digitizing ethnic minority culture, the generation of ethnic minority clothing sketches has numerous practical applications. A sketch is a collection of lines, organized according to certain rules, that can reflect the contour features of an image and, to a certain extent, embody the primary characteristics of the depicted object. Ethnic minority clothing is relatively complex, particularly some with numerous patterns and designs. This necessitates that detailed information within the clothing, particularly important lines, must be fully considered when extracting edge contours. Only by preserving this detailed information as much as possible can edge features of ethnic minority clothing images be effectively extracted. Therefore, research on edge contour feature extraction methods for ethnic minority clothing images is of great significance in the field of computer vision. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network, so as to solve the problem that the existing sketch generation method has poor effect on ethnic costume images.

[0004] The technical solution of the present invention is a method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network. This method automatically generates sketches of ethnic costume images using a cycle-consistent generative adversarial network, extracting edge and contour features from ethnic costume images to automatically generate ethnic costume sketches. Before inputting the source image into the generator network, bilateral filtering is used for noise reduction and color quantization to optimize the source image quality and facilitate the generation of the final ethnic costume sketch.

[0005] The specific steps are:

[0006] Step 1: Image preprocessing;

[0007] Step 1.1: Extract edge and contour information of ethnic costume images and convert them into ethnic costume sketches, thereby obtaining the target image for the automatic sketch generation task and forming the ethnic costume sketch dataset.

[0008] The dimension of the transformed ethnic costume sketch image data has been changed from the original 3D to 1D. The use of a 1D representation method can reduce the parameters of the entire cycle-consistent generative adversarial network model, reduce computational complexity, and improve the effect of generating sketch images.

[0009] Step 1.2: De-noise the original ethnic costume image using a bilateral filtering algorithm. This maintains edge details and smoothes the image, resulting in a clearer edge and more complete preservation of image detail, improving the quality of subsequent sketch generation.

[0010] Step 1.3: Use the image optimized in Step 1.2 as input for this step. Perform color quantization on the image and cluster the colors of the entire image. Use the K-means clustering algorithm with a clustering level of four, representing the four color class centers. After color quantization, generate a color patch map of the image. This four-color patch map simplifies the overall complexity of the image and facilitates subsequent sketch generation.

[0011] Step 2: Image generation and discrimination;

[0012] Step 2.1: Use the image color block map processed in Step 1.3 as the sketch generator G AB The sketch image generated by the sketch generator is obtained by taking the input data of . The input data dimension of the sketch generator is 3, which is an RGB color space image, and the output dimension is 1, which is a grayscale image without color information.

[0013] Step 2.2: Alternately input the generated sketches in Step 2.1 and the sketch images annotated in Step 1.1 into the sketch discriminator, judge each part of the image block, and then summarize the judgment results. This makes the discriminator output a comprehensive result of every part of the entire image and more interpretable.

[0014] Step 2.3: Use the generated sketch generated in Step 2.1 as the clothing image generator G BA The input of the clothing image generator is 1, and the output dimension is 3, which is an RGB color space image.

[0015] Step 2.4: Alternately input the processed images in Step 1.3 and the generated clothing images generated by the generator in Step 2.3 into the clothing image discriminator. The input image data is judged by the clothing image discriminator to determine whether the clothing image is real or fake.

[0016] Step 2.5: Sketch Generator G AB With clothing image generator G BA Perform iterative training and repeat Step 2.1, Step 2.2, Step 2.3, and Step 2.4. During the continuous iteration process, the final discrimination results of the two sets of discriminators will assist and influence the generator to generate better sketch images.

[0017] In Step 1.3, the quantization level used in the color quantization process is four.

[0018] The two image generators G in Step 2.1 and Step 2.3 AB With G BA , have the same network model, but different inputs and outputs.

[0019] The sketch discriminator in Step 2.2 and the clothing image discriminator in Step 2.4 both use a block judgment method to perform image discrimination.

[0020] This paper designs a method for automatically generating ethnic clothing sketches, applying image translation technology to the sketch generation task and optimizing it using image noise reduction and image color quantization algorithms. Two sets of generators with identical structures are used to iterate and learn the sketch generation and sketch restoration of clothing images, respectively. The two image generators are used to convert clothing images into clothing sketches, and then to convert the generated clothing sketches into clothing images. The difference between the generated clothing images and the original clothing images is used to continuously optimize the image quality of the generators. Simultaneously, the two sets of discriminators enable the clothing generator to better generate the final image, optimizing the final image quality.

[0021] Compared to general images, the task of automatically generating sketches of ethnic costumes is more difficult. Due to the more complex styles and colors of ethnic costumes, traditional methods cannot effectively complete the automatic generation of ethnic costume sketches. However, using a cycle-consistent generative adversarial network model, the resulting sketches are more outstanding.

[0022] The present invention has the beneficial effect of effectively and automatically generating sketches of ethnic costumes, achieving automatic generation of grayscale image sketches of ethnic costumes. Compared with existing automatic sketch generation methods, the present invention can better generate sketches of ethnic costume image objects with complex structures, resulting in final results that surpass mainstream sketch generation methods. Using two sets of generators and discriminators, it can better generate high-resolution ethnic costume sketch images and preserve ethnic costume details, which is of great significance for the generation of ethnic costume sketches. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the present invention;

[0024] Figure 2 This is a diagram of the generator network architecture used in Step 2.1 of Example 1 of the present invention;

[0025] Figure 3 is a diagram of the sketch discriminator network architecture of the present invention;

[0026] Figure 4 This is a diagram of the network architecture of the clothing image discriminator of the present invention;

[0027] Figure 5 This is a diagram of the generator network architecture used in Step 2.3 of Example 1 of the present invention;

[0028] Figure 6 This is the colorized output result of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0030] Example 1: Figure 1 As shown in the figure, a method for extracting ethnic costume sketches based on cycle-consistent generative adversarial networks is proposed. The specific steps are as follows:

[0031] Step 1: Image preprocessing;

[0032] Step 1.1: Extract and annotate the edge and contour information of ethnic costume images. The extraction and annotation methods can be manual extraction and manual annotation to obtain ethnic costume sketches, thus forming an ethnic costume sketch dataset. The dataset consists of two parts: ethnic costume images and corresponding ethnic costume sketches.

[0033] Step 1.2: The first step in preprocessing the ethnic costume image is to use the bilateral filtering noise reduction algorithm to process the ethnic costume image, so that the image maintains the edge and reduces the noise smoothly, thereby obtaining clear edge information of the ethnic costume image. Therefore, the edge detail information of the ethnic costume image is more prominent, which is conducive to the processing of subsequent steps.

[0034] Step 1.3: Perform color quantization on the image after denoising in Step 1.2. A key parameter in K-means color quantization is the quantization level, which produces different quantization effects. Experimental analysis shows that a quantization level of 4 effectively captures image detail, evenly distributes color, and enhances the appearance of outlines.

[0035] Step 2: Image generation and discrimination;

[0036] Step 2.1: Input the image processed in Step 1.3 into the sketch generator G AB In the example, we get the sketch image generated by the sketch generator. Generator G AB The function of is to convert the pre-processed ethnic costume image into the ethnic costume sketch image. Generator G AB The network model structure is as follows Figure 2 shown.

[0037] Step 2.2: Alternately input the generated sketches in Step 2.1 and the manually annotated sketch images in Step 1.1 into the sketch discriminator to continuously train the discriminative ability of the sketch discriminator. The role of the sketch discriminator is to judge whether the sketch generator G AB Generate image effects to assist the sketch generator G AB Generate better results. The network model structure of the sketch discriminator is as follows Figure 3 shown.

[0038] Step 2.3: Input the generated sketch generated in Step 2.1 into the clothing image generator G BA In the example above, we get the generated clothing image generated by the clothing image generator. Generator G BA The function of the generator G is to AB The generated result is converted from the ethnic costume sketch image to the ethnic costume image. BA The network model structure is as follows Figure 5 shown.

[0039] Step 2.4: Alternately input the processed images in Step 1.3 and the generated clothing images in Step 2.3 into the clothing image discriminator to continuously train the discrimination ability of the clothing image discriminator. The function of the clothing image discriminator is to judge whether the clothing image generator G BA The effect of the generated image can also be compared with the original ethnic costumes to indirectly update the iterative generator G AB The network model parameters of the clothing image discriminator are as follows: Figure 4 shown.

[0040] Step 2.5: Sketch Generator G AB With clothing map generator G BA Perform iterative training and repeat Step 2.1, Step 2.2, Step 2.3, and Step 2.4.

[0041] Furthermore, in Step 1.3, the quantization level used in the color quantization process is four.

[0042] Furthermore, the two image generators G in Step 2.1 and Step 2.3AB With G BA , have the same network model, but different inputs and outputs.

[0043] Furthermore, the sketch discriminator in Step 2.2 and the clothing image discriminator in Step 2.4 both use a block judgment method to perform image discrimination.

[0044] Figure 1 The architecture of the cycle-consistent generative adversarial network is shown. First, the ethnic costume image is preprocessed. The first step of preprocessing is image denoising, which is performed using the bilateral filtering denoising algorithm. The second step of preprocessing is color quantization, which is performed using the K-means clustering algorithm with a quantization level of 4. The preprocessed image is then input into the sketch generator G AB In the process, the generated sketch is output, and the real sketch and the generated sketch are alternately input into the sketch discriminator, which outputs the authenticity of the image, thereby judging the sketch generator G AB The generated image effect. Then the generated sketch is input into the clothing image generator G BA In the process, the generated clothing image is output, and the preprocessed clothing image and the generated clothing image are alternately input into the clothing image discriminator, and the judgment result of the discriminator is output to judge the clothing image generator G BA Generates image effects.

[0045] In specific implementation, the sketch generator G AB The network structure is as follows: the input image data is 3D, and after downsampling, the data dimensions are converted to 64, 128, 256, and 512 dimensions in sequence. Then, 9 residual blocks are used to further represent the image features. Finally, the data dimensions are restored through upsampling to finally generate the clothing sketch. The model dimension conversion is achieved through image convolution and deconvolution. Clothing image generator G BA The network structure and sketch generator G AB The network structure is the same, but the input and output settings are different.

[0046] The parameters of the two sets of generators and discriminators are continuously iterated and updated, and finally the training of the ethnic costume sketch generator is completed. The effect of converting the ethnic costume picture into the ethnic costume sketch is as follows: Figure 6 shown.

[0047] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network, comprising a preprocessing stage and a generation and discrimination stage, characterized by: The pre-processing stage is specifically as follows: Step 1.1: Extract edge and contour information from the ethnic costume image to obtain an ethnic costume sketch, which is used as the target image for the ethnic costume image sketch generation task; Step 1.2: Perform bilateral filtering to reduce noise on the ethnic clothing image, maintain the edges and reduce noise smoothly, obtain clearer edge information, and preserve the image details more completely; Step 1.3: Perform color quantization on the image processed in Step 1.2 to obtain the image color block effect; The generation and discrimination stages are specifically as follows: Step 2.1: Input the processed image in Step 1.3 into the sketch generator G AB , get the sketch image generated by the sketch generator; Step 2.2: Alternately input the sketch images generated by the sketch generator in Step 2.1 and the sketch images annotated in Step 1.1 into the sketch discriminator. The sketch discriminator then determines whether the sketches are real or fake. Step 2.3: Input the sketch image generated by the sketch generator in Step 2.1 into the clothing image generator G BA , obtain the generated clothing image generated by the clothing image generator; Step 2.4: Alternately input the processed images from Step 1.3 and the generated clothing images from Step 2.3 into the clothing image discriminator. The input image data is used by the clothing image discriminator to determine whether the clothing images are real or fake. Step 2.5: Sketch Generator G AB With clothing map generator G BA Perform iterative training and repeat Step 2.1, Step 2.2, Step 2.3, and Step 2.

4.

2. The method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network according to claim 1 is characterized in that: In Step 1.3, the quantization level used in the color quantization process is four.

3. The method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network according to claim 1 is characterized in that: The two image generators G in Step 2.1 and Step 2.3 AB With G BA , have the same network model, but different inputs and outputs.

4. The method for extracting ethnic costume sketches based on a cycle-consistent generative adversarial network according to claim 1, wherein: The sketch discriminator in Step 2.2 and the clothing image discriminator in Step 2.4 both use a block judgment method to perform image discrimination.

Images