Method, device, terminal device and storage medium for generating effect diagram

Abstract

The embodiment of the present invention discloses a rendering generation method, apparatus, terminal device, and storage medium. The method includes the following steps: obtaining a hand-drawn sketch, inputting the hand-drawn sketch into a pre-trained stereogram generation network, and correspondingly generating a stereogram, wherein the stereogram includes pixel brightness information; receiving a region selection instruction in the stereogram, and confirming a target region according to the region selection instruction; receiving display parameters, wherein the display parameters are used to confirm the display effect of the stereogram; and processing the stereogram according to the pixel brightness information, the target region, and the display parameters to generate a rendering. In this embodiment, the user only needs to manually draw a hand-drawn sketch and select the display parameters to be displayed to generate a rendering that meets the requirements, thereby eliminating a large amount of light and shade information and texture detail drawing work, greatly improving the drawing efficiency of clothing renderings, and solving the technical problem of low drawing efficiency of clothing renderings in the prior art.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of clothing, and in particular to a method, apparatus, terminal device and storage medium for generating an effect diagram. Background Art

[0002] During the conceptual design phase of clothing, designers often need to draw numerous hand-drawn sketches to validate their design ideas. These sketches are then used to select appropriate directions for further 3D modeling and rendering, ultimately guiding sample production. In the clothing industry, the representation of details like garment component shape, structure, material, drapes, and lighting and shadow is essentially standardized and replicable (for example, the same material is often rendered using the same techniques, and if the same component structure is drawn, the resulting appearance is nearly identical). However, because existing drawing software only provides "drawing technique assistance" and doesn't participate in the actual content generation, the techniques used by designers to create renderings using drawing software are not significantly different from those used to create hand-drawn sketches using pen and paper. Consequently, designers spend the majority of their time on "drawing the basic shape" of the renderings, and the repetitive drawing of details can be time-consuming. Since the presentation of details is closely related to the designer's technical proficiency, the designer's experience directly influences the final rendering.

[0003] In summary, how to improve the efficiency of drawing clothing renderings has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for generating an effect diagram, which solve the technical problem of low efficiency in drawing clothing effect diagrams in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for generating an effect diagram, comprising the following steps:

[0006] Obtaining a hand-drawn sketch, inputting the hand-drawn sketch into a pre-trained stereogram generation network, and correspondingly generating a stereogram, wherein the stereogram includes pixel brightness information;

[0007] receiving an area selection instruction in the stereoscopic image, and confirming a target area according to the area selection instruction;

[0008] receiving display parameters, wherein the display parameters are used to confirm a display effect of the stereogram;

[0009] The stereoscopic image is processed to generate an effect image according to the pixel brightness information, the target area and the display parameters.

[0010] Preferably, the stereogram generation network includes an encoder and a generator, the training set includes training hand-drawn sketches and training stereograms corresponding to the training hand-drawn sketches, and the stereogram generation network is trained by the following steps:

[0011] Initializing encoder parameters of the encoder and generator parameters of the generator;

[0012] Inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training to obtain optimal generator parameters of the generator;

[0013] Fixing the optimal generator parameters of the generator, inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtaining the optimal encoder parameters of the encoder;

[0014] A trained stereogram generation network is obtained according to the optimal generator parameters and the optimal encoder parameters.

[0015] Preferably, the specific process of inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training to obtain the optimal generator parameters of the generator is:

[0016] A first training hand-drawn sketch is selected from the training hand-drawn sketches, and a first training stereogram is selected from the training stereograms, and a first training step is performed. The first training step is specifically:

[0017] Inputting the first training hand-drawn sketch into the encoder to obtain latent variables, and inputting the latent variables into the generator so that the generator generates a first hand-drawn sketch and a first stereogram based on the generator parameters and the latent variables;

[0018] Inputting the first hand-drawn sketch, the first stereogram, the first training hand-drawn sketch, the first training stereogram, and the generator parameters into a preset first loss function to obtain a first loss value;

[0019] Inputting the first stereo image, the first training stereo image, and the generator parameters into a preset second loss function to obtain a second loss value;

[0020] Obtaining a total loss value according to the first loss value and the second loss value;

[0021] Resetting the generator parameters according to the total loss value to complete the first training step;

[0022] A new training hand-drawn sketch is selected from the training hand-drawn sketch, and a new training stereo image is selected from the training stereo image as the first training hand-drawn sketch and the first training stereo image respectively, and the first training step is re-executed until the total loss value is within a preset range, thereby obtaining the optimal generator parameters.

[0023] Preferably, the specific process of fixing the optimal generator parameters of the generator, inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtaining the optimal encoder parameters of the encoder is:

[0024] Determining a conversion relationship between a stereogram output by the generator and the generator parameters and the encoder parameters;

[0025] Initialize the encoder parameters, fix the optimal generator parameters of the generator, select a second training hand-drawn sketch and a second training stereogram from the training hand-drawn sketch, and perform a second training step. The second training step is specifically as follows:

[0026] Inputting the second training hand-drawn sketch into the stereogram generation network to obtain a second stereogram and a second hand-drawn sketch;

[0027] Inputting the second stereo image, the second training stereo image, the encoder parameters, and the optimal generator parameters into a preset third loss function, and converting the second stereo image into the optimal generator parameters and the encoder parameters in the third loss function according to the conversion relationship;

[0028] Calculating a distance between the encoder parameters and the optimal generator parameters according to the converted third loss function, and adjusting the encoder parameters according to the distance;

[0029] A new training hand-drawn sketch and a new training stereogram are selected from the training hand-drawn sketch and the training stereogram as the second training hand-drawn sketch and the second training hand-drawn sketch respectively, and the second training step is re-executed until the distance is within the preset distance range to obtain the optimal encoder parameters.

[0030] Preferably, the specific process of determining the conversion relationship between the stereogram output by the generator and the generator parameters and the encoder parameters is:

[0031] Determining a first relationship between the encoder parameters and the input training hand-drawn sketch and the latent variable;

[0032] Determining a second relationship between the latent variable and the generator parameter and the hand-drawn sketch and the generated stereogram generated by the generator;

[0033] Determining a third relationship among the generator parameters, the input training hand-drawn sketch, the generated hand-drawn sketch, and the generated stereogram according to the first relationship and the second relationship;

[0034] According to the third relationship, a conversion relationship between the generated stereogram and the generator parameters and the encoder parameters is determined.

[0035] Preferably, the specific process of inputting the hand-drawn sketch into a pre-trained stereogram generation network to generate the stereogram is as follows:

[0036] Inputting the hand-drawn sketch into a pre-trained stereogram generation network to obtain the encoder output initial latent variables;

[0037] Inputting the initial latent variables into the generator of the pre-trained stereogram generation network to obtain a third stereogram and a third hand-drawn sketch;

[0038] determining a fourth loss function of the initial latent variables according to the third hand-drawn sketch, the hand-drawn sketch, the encoder parameters, and the optimal generator parameters, and optimizing the initial latent variables according to the fourth loss function to obtain optimal latent variables;

[0039] The optimal latent variables are input into the generator so that the generator outputs a stereogram.

[0040] Preferably, the specific process of receiving the area selection instruction in the stereoscopic image and confirming the target area according to the area selection instruction is:

[0041] The stereogram is input into a pre-trained annotation network so that the annotation network marks different regions of the stereogram, receives a region selection instruction in the marked stereogram, and selects a target region from the different regions according to the region selection instruction.

[0042] Preferably, the display parameters include fabric parameters, pattern parameters and synthesis parameters;

[0043] Accordingly, the specific process of processing the stereoscopic image to generate the effect image according to the pixel brightness information, the target area and the display parameters is as follows:

[0044] Generate a texture tiling unit according to the fabric parameters and the pattern parameters;

[0045] Obtaining a three-dimensional component image according to the pixel brightness information, the synthesis parameter, the target area, and the texture tiling unit;

[0046] The three-dimensional component image is used to replace the pixels of the corresponding target area in the three-dimensional image to obtain an effect image.

[0047] Preferably, the specific process of obtaining the three-dimensional component image according to the pixel brightness information, the synthesis parameters, the target area and the texture tiling unit is:

[0048] Obtaining a target texture according to the synthesis parameters and the texture tiling unit;

[0049] Obtaining a replacement texture according to the pixel brightness information, the synthesis parameters, and the target texture;

[0050] Obtaining a light and dark texture according to the pixel brightness information, the synthesis parameter, and the replacement texture;

[0051] The light and dark textures are cropped according to the target area to obtain a three-dimensional component image.

[0052] Preferably, the synthesis parameters include texture tiling parameters;

[0053] Correspondingly, the specific process of obtaining the target texture according to the synthesis parameters and the texture tiling unit is:

[0054] A tiling mode is determined according to the texture tiling parameters, and the texture tiling unit is extended according to the tiling mode to obtain a target texture, and the target texture fully covers the target area.

[0055] Preferably, the synthesis parameter includes a brightness weight parameter;

[0056] Accordingly, the specific process of obtaining the replacement texture according to the pixel brightness information, the synthesis parameters and the target texture is:

[0057] The target texture is distorted according to the pixel brightness information and the brightness weight parameter to obtain a replacement texture.

[0058] Preferably, the synthesis parameters include light and dark weight parameters;

[0059] Accordingly, the specific process of obtaining the light and dark texture according to the pixel brightness information, the synthesis parameters and the replacement texture is:

[0060] The replacement texture is adjusted according to the pixel brightness information and the light and dark weight parameters to obtain a light and dark texture.

[0061] In a second aspect, an embodiment of the present invention provides an effect diagram generation device, comprising a stereogram generation module, a target area selection module, a display parameter receiving module, and an effect diagram generation module:

[0062] The stereogram generation module is used to obtain a hand-drawn sketch, input the hand-drawn sketch into a pre-trained stereogram generation network, and generate a stereogram accordingly, wherein the stereogram includes pixel brightness information;

[0063] The target area selection module is used to receive an area selection instruction in the stereoscopic image and confirm the target area according to the area selection instruction;

[0064] The display parameter receiving module is used to receive display parameters, and the display parameters are used to confirm the display effect of the stereogram;

[0065] The effect image generation module is used to process the stereoscopic image according to the pixel brightness information, the target area and the display parameters to generate an effect image.

[0066] In a third aspect, an embodiment of the present invention provides a terminal device, the terminal device including a processor and a memory;

[0067] The memory is used to store a computer program and transmit the computer program to the processor;

[0068] The processor is configured to execute the method for generating an effect diagram as described in the first aspect according to instructions in the computer program.

[0069] In a fourth aspect, an embodiment of the present invention provides a storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to execute a method for generating an effect diagram as described in the first aspect.

[0070] As described above, an embodiment of the present invention provides a rendering generation method, comprising the following steps: obtaining a hand-drawn sketch, inputting the hand-drawn sketch into a pre-trained stereogram generation network, and correspondingly generating a stereogram, the stereogram including pixel brightness information; receiving a region selection instruction in the stereogram, and confirming a target region according to the region selection instruction; receiving display parameters, the display parameters being used to confirm the display effect of the stereogram; and processing the stereogram to generate a rendering based on the pixel brightness information, the target region, and the display parameters. In this embodiment, the user only needs to manually draw a hand-drawn sketch and select the display parameters to be displayed to generate a rendering that meets the requirements, eliminating the need for drawing a large amount of light and dark information and texture details, greatly improving the efficiency of drawing clothing renderings, and solving the technical problem of low efficiency in drawing clothing renderings in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A schematic flow chart of a method for generating an effect diagram provided by an embodiment of the present invention.

[0072] Figure 2A schematic diagram of a hand-drawn interface program provided by an embodiment of the present invention.

[0073] Figure 3 A schematic diagram of generating a stereogram provided by an embodiment of the present invention.

[0074] Figure 4 A schematic diagram of training a stereogram generation network provided by an embodiment of the present invention.

[0075] Figure 5 A schematic diagram of tuning latent variables provided by an embodiment of the present invention.

[0076] Figure 6 A schematic diagram of different regions of a stereogram obtained according to an embodiment of the present invention.

[0077] Figure 7 A schematic diagram of an interface for generating a rendering according to an embodiment of the present invention.

[0078] Figure 8 A schematic diagram of generating an effect image based on a stereogram provided in an embodiment of the present invention.

[0079] Figure 9 A schematic structural diagram of an effect diagram generating device provided by an embodiment of the present invention.

[0080] Figure 10 A schematic structural diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The following description and accompanying drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them. The examples represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments of the present application includes the entire scope of the claims, as well as all available equivalents of the claims. In this document, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is actually disclosed. In this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "include," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. The various embodiments herein are described in a progressive manner, with each embodiment focusing on the differences from the other embodiments. Reference can be made to the common and similar parts between the various embodiments. For structures, products, etc. disclosed in the embodiments, the description is relatively simple because they correspond to the parts disclosed in the embodiments. For relevant parts, refer to the method description.

[0082] Example 1

[0083] like Figure 1 As shown, Figure 1 This is a flowchart of a method for generating an effect diagram provided by an embodiment of the present invention. The method for generating an effect diagram provided by an embodiment of the present invention can be executed by an effect diagram generating device. The effect diagram generating device can be implemented by software and / or hardware. The effect diagram generating device can be composed of two or more physical entities or a single physical entity. For example, the effect diagram generating device can be a computer, a host computer, a tablet, or other device. The method includes the following steps:

[0084] Step 101: Obtain a hand-drawn sketch, input the hand-drawn sketch into a pre-trained stereogram generation network, and generate a stereogram accordingly. The stereogram includes pixel brightness information.

[0085] First, a user's hand-drawn sketch is obtained and fed into a pre-trained stereogram generation network to generate a stereogram. In this embodiment, the hand-drawn sketch includes a hand-drawn outline line drawing and a hand-drawn fold line drawing. The hand-drawn outline line drawing contains the stereogram's outline information, while the hand-drawn fold line drawing contains the stereogram's wrinkle texture information. After obtaining the hand-drawn outline line drawing and the hand-drawn fold line drawing, they are fed into the pre-trained stereogram generation network to generate a stereogram. The resulting stereogram's outline will strictly match the hand-drawn outline line drawing, and the shading of the folds in the stereogram will approximate the wrinkle morphology in the hand-drawn fold line drawing.

[0086] It should be further explained that the obtained stereoscopic image includes pixel brightness information of each pixel, and the pixel brightness information reflects the brightness and darkness of each pixel in the stereoscopic image.

[0087] For example, in one embodiment, Figure 2 As shown, the rendering generation device provides a hand-drawn interface program, and the user can draw hand-drawn sketches on the hand-drawn interface program through hand-drawn pens, touch screens, mice and other stroke input devices. The hand-drawn interface program provides different tools such as hand-drawn contour line drawing tools and hand-drawn fold line drawing tools. It can be understood that when the user clicks the hand-drawn contour line drawing tool, a hand-drawn contour line diagram can be drawn, and when the user clicks the hand-drawn fold line drawing tool, a hand-drawn fold line diagram can be drawn, so that the hand-drawn interface program can distinguish between the drawn hand-drawn contour line diagram and the hand-drawn fold line diagram (for example, lines drawn by different drawing tools have different colors, different layers or attribute data tags, etc.). After drawing, the user submits the hand-drawn contour line diagram and the hand-drawn fold line diagram, and the rendering generation device will input the hand-drawn contour line diagram and the hand-drawn fold line diagram into the pre-trained stereogram generation network to obtain a stereogram. The process is as follows Figure 3 shown.

[0088] Based on the above embodiment, the stereogram generation network includes an encoder and a generator, and the training set includes training hand-drawn sketches and training stereograms corresponding to the training hand-drawn sketches.

[0089] In one embodiment, a stereogram generation network includes an encoder and a generator, wherein the encoder is used to encode a hand-drawn sketch into a latent space, the encoder's input is a hand-drawn sketch, and its output is a latent variable, and the generator is used to map the latent variable to the stereogram and the hand-drawn sketch, the generator's input is a latent variable, and its output is a stereogram and a hand-drawn sketch. In this embodiment, the types of the encoder and generator can be set according to actual needs. For example, the backbone models of some existing classification networks in the industry can be used as encoders, such as the ResNet series, EfficientNet series, Transformer series, etc. The generator can select the StyleGAN series, etc. In this embodiment, the specific models of the encoder and generator are not limited.

[0090] The training set in this embodiment includes training hand-drawn sketches and training stereograms corresponding to the training hand-drawn sketches. Each training set data contains a training stereogram and multiple training hand-drawn sketches corresponding to the training stereogram. It can be understood that the training hand-drawn sketches are historical hand-drawn sketches, and the training hand-drawn sketches include historical hand-drawn contour line drawings and historical hand-drawn fold line drawings. The training stereogram can be obtained by rendering a 3D model or performing image processing on the image. The historical hand-drawn contour line drawings and historical hand-drawn fold line drawings can be obtained by manual drawing or image processing.

[0091] The stereogram generation network is trained through steps 1011 to 1014, specifically:

[0092] Step 1011: Initialize the encoder parameters of the encoder and the generator parameters of the generator.

[0093] In this embodiment, the stereogram generation network is first initialized, and the encoder parameters of the encoder and the generator parameters of the generator are initialized.

[0094] Step 1012: Input the training hand-drawn sketch and the training stereogram into the stereogram generation network for training to obtain the optimal generator parameters of the generator.

[0095] After initializing the stereogram generation network, the training hand-drawn sketches and training stereograms in the training set are input into the stereogram generation network for training. During the training process, the generator parameters of the generator are first tuned to obtain the optimal generator parameters. It can be understood that each time the training hand-drawn sketches and training stereograms input into the stereogram generation network are Figure 1One to one correspondence.

[0096] Based on the above embodiment, in step 1012, the training hand-drawn sketch and the training stereogram are input into the stereogram generation network for training. The specific process of obtaining the optimal generator parameters of the generator is performed by steps 10121 to 10127, specifically:

[0097] Step 10121: Select a first training hand-drawn sketch from the training hand-drawn sketches, select a first training stereogram from the training stereograms, and execute a first training step. The first training step is specifically as follows:

[0098] Step 10122: Input the first training hand-drawn sketch into the encoder to obtain latent variables, and input the latent variables into the generator so that the generator generates the first hand-drawn sketch and the first stereogram based on the generator parameters and the latent variables.

[0099] In the embodiment, the first training hand-drawn sketch is first input into the encoder, and the encoder outputs latent variables. Then, the latent variables are input into the generator, so that the generator generates the first hand-drawn sketch and the first stereogram based on the generator parameters and the latent variables. The formula is as follows:

[0100] [sketch_pred,shade_pred]=Generator(w;θ g )

[0101] Among them, θ g is the generator parameter, w is the hidden variable, w is randomly sampled during the training process, sketch_pred is the hand-drawn sketch generated by the generator, shade_pred is the stereogram generated by the generator, and generator is the generator.

[0102] Step 10123: Input the first hand-drawn sketch, the first stereogram, the first training hand-drawn sketch, the first training stereogram, and the generator parameters into a preset first loss function to obtain a first loss value.

[0103] Afterwards, the first hand-drawn sketch, the first stereogram, the first training hand-drawn sketch, the first training stereogram, and the generator parameters are input into the preset first loss function to obtain the first loss value. The formula of the first loss function loss1 is as follows:

[0104] loss1(θ g )=GAN_LOSS(sketch_real,shade_real,sketch_pred,shade_pred; θ g )

[0105] Among them, GAN_LOSS is the generative adversarial loss function, sketch_real is the input training hand-drawn sketch, and shade_real is the input training stereogram.

[0106] Step 10124: Input the first stereo image, the first training stereo image, and the generator parameters into a preset second loss function to obtain a second loss value.

[0107] The first stereo image, the first training stereo image, and the generator parameters are input into the preset second loss function to obtain the second loss value. The formula of the second loss function loss2 is as follows:

[0108] loss2(θ g )=GAN_LOSS(shade_real,shade_pred; θ g )

[0109] In this embodiment, the schematic diagram of the training process is as follows: Figure 4 As shown, the generator outputs a two-channel data, where the first channel is a stereogram, labeled shade_pred, and the second channel is a hand-drawn sketch matching the stereogram, labeled sketch_pred; Figure 4 In GAN_LOSS, the loss expression function of the stylegan2 series is selected. In order to supervise the matching relationship between the stereogram and the hand-drawn sketch, the two-channel data output by the generator is used as the input of loss1. Since loss1 inputs two-channel data at the same time, according to the principle of GAN technology, supervision of the two-channel data can be achieved; Figure 4 Loss2 only supervises the stereogram to supervise the generation quality of the stereogram. The function expression of loss2 is no different from loss1 except that the input is different.

[0110] Step 10125: Obtain a total loss value based on the first loss value and the second loss value.

[0111] like Figure 4 As shown, after obtaining the first loss value and the second loss value, the total loss function can be obtained. The total loss value can be calculated according to the total loss function, where the formula of the total loss function loss_toatal is as follows:

[0112] loss_total(θ g )=λ1loss1(θ g )+λ2loss2(θ g )

[0113] Here, λ1 and λ2 represent weights, which can be set in advance.

[0114] Step 10126: Reset the generator parameters according to the total loss value to complete the first training step.

[0115] After calculating the total loss value, the parameters of the generator are reset according to the total loss value to complete the first training step.

[0116] Step 10127: Select a new training hand-drawn sketch from the training hand-drawn sketches and a new training stereo image from the training stereo image as the first training hand-drawn sketch and the first training stereo image respectively, and re-execute the first training step until the total loss value is within the preset range to obtain the optimal generator parameters.

[0117] Afterwards, a new training hand-drawn sketch is selected from the training hand-drawn sketches as the first training hand-drawn sketch, and a training stereogram corresponding to the new training hand-drawn sketch is selected from the training stereogram as the first training stereogram, and the first training step is re-executed. The purpose of the training is to find the optimal generator parameters so that the total loss value loss_total is optimal. When the total loss value loss_total is within a preset range, the optimal generator parameters can be obtained.

[0118] Step 1013: fix the optimal generator parameters of the generator, input the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtain the optimal encoder parameters of the encoder.

[0119] After obtaining the optimal generator parameters of the generator, the optimal generator parameters of the generator are fixed so that the optimal generator parameters remain unchanged. The training hand-drawn sketch and the training stereogram are input into the stereogram generation network for training. The encoder parameters of the encoder are tuned to obtain the optimal encoder parameters of the encoder. In one embodiment, the encoder parameters are tuned using the Adam optimization method.

[0120] Based on the above embodiment, in step 1013, the optimal generator parameters of the generator are fixed, and the training hand-drawn sketch and the training stereogram are input into the stereogram generation network for training. The specific process of obtaining the optimal encoder parameters of the encoder is performed by steps 10131 to 10136, specifically:

[0121] Step 10131: Determine the conversion relationship between the stereogram output by the generator and the generator parameters and encoder parameters.

[0122] In this embodiment, it is first necessary to determine the conversion relationship between the stereogram output by the generator and the generator parameters and encoder parameters. In one embodiment, the conversion relationship between the training hand-drawn sketch input to the encoder and the latent variables and encoder parameters can be first determined. Then, the conversion relationship between the latent variables and generator parameters and the stereogram generated by the generator and the generated hand-drawn sketch can be further determined. Finally, the conversion relationship between the stereogram output by the generator and the generator parameters and encoder parameters can be obtained.

[0123] It should be further explained that, in this embodiment, the encoder includes a backbone network and a header part. The header part of the encoder only includes a convolutional layer and a fully connected layer. The header part is used to complete the dimensionality matching with the latent space. In this embodiment, during the process of training the encoder, the backbone network of the encoder is fixed, and only the header part is trained.

[0124] In one embodiment, the specific process of determining the conversion relationship between the stereogram output by the generator and the generator parameters and the encoder parameters in step 10131 is performed by steps 101311 to 101314, and the specific process is:

[0125] Step 101311: Determine the first relationship between the encoder parameters and the input training hand-drawn sketch and the latent variable.

[0126] First, determine the first relationship between the encoder parameters and the input training hand-drawn sketch and the latent variable. In this embodiment, the encoder parameters θ e =(θ e,m ,θ e,h ), where θ e,m is the parameter of the backbone network, θ e,h is the parameter of the header part. After inputting the training hand-drawn sketch into the encoder, we can get:

[0127] w=Encoder(sketch_real;θ e )

[0128] Where encoder represents the encoder.

[0129] Step 101312: Determine the second relationship between the latent variables and generator parameters and the hand-drawn sketch generated by the generator and the generated stereogram.

[0130] Afterwards, a second relationship between the latent variables and the generator parameters and the hand-drawn sketch and the generated stereogram is determined. In this embodiment, the expression of the second relationship is:

[0131] [sketch_pred,shade_pred]=Generator(w;θg )

[0132] Step 101313: Determine the third relationship among the generator parameters, the input training hand-drawn sketch, the generated hand-drawn sketch, and the generated stereogram based on the first relationship and the second relationship.

[0133] After that, by bringing w into the generator, we can get the third relation, which is:

[0134] [sketch_pred,shade_pred]=Generator(Encoder(sketch_real;θ e );θ g )

[0135] Step 101314: Determine the conversion relationship between the generated stereogram and the generator parameters and encoder parameters based on the third relationship.

[0136] After obtaining the third relationship, the stereogram shade_pred generated by the generator and the encoder parameter θ can be determined e and the generator parameters θ g The conversion relationship between them.

[0137] Step 10132: Initialize the encoder parameters, fix the optimal generator parameters of the generator, select a second training hand-drawn sketch and a second training stereogram from the training hand-drawn sketches, and perform the second training step. The second training step is specifically as follows:

[0138] Step 10133: Input the second training hand-drawn sketch into the stereogram generation network to obtain a second stereogram and a second hand-drawn sketch.

[0139] After selecting a second training hand-drawn sketch and a second training stereogram corresponding to the second training hand-drawn sketch from the training set, the second training hand-drawn sketch is input into the encoder of the stereogram generation network, so that the generator in the stereogram generation network outputs the second stereogram and the second hand-drawn sketch.

[0140] Step 10134: input the second stereo image, the second training stereo image, the encoder parameters, and the optimal generator parameters into a preset third loss function. According to the conversion relationship, in the third loss function, the second stereo image is converted into the optimal generator parameters and the encoder parameters.

[0141] Afterwards, the second stereo image, the second training stereo image, the encoder parameters, and the optimal generator parameters are input into a preset third loss function. In this embodiment, the third loss function loss3 adopts a perceptual loss function, and the expression formula is as follows:

[0142] loss3(θg ,θ e )=perceptual_loss(shade_pred,shade_real; θ g ,θ e )

[0143] Among them, perceptual_loss is the perceptual loss function.

[0144] Afterwards, according to the conversion relationship, the second stereogram is converted into optimal generator parameters and encoder parameters in the third loss function.

[0145] Step 10135: Calculate the distance between the encoder parameters and the optimal generator parameters based on the converted third loss function, and adjust the encoder parameters based on the distance.

[0146] In this embodiment, optimizing encoder parameters is to optimize the following issues:

[0147]

[0148] Among them, θ g It is fixed to the optimal generator parameter and does not participate in the optimization. h represents the header part of the encoder. The optimization method adopts the Adam method. The specific process of the Adam method has been recorded in many existing technologies and will not be repeated in this embodiment.

[0149] In one embodiment, the learning rate is lowered to 0.000001, and the encoder parameters of the encoder are fine-tuned by calculating the distance between the encoder parameters and the optimal generator parameters; at this time, the mathematical description is:

[0150]

[0151] Step 10136: Select a new training hand-drawn sketch and a new training stereogram from the training hand-drawn sketch and the training stereogram as the second training hand-drawn sketch and the second training hand-drawn sketch respectively, and re-execute the second training step until the distance is within the preset distance range to obtain the optimal encoder parameters.

[0152] Afterwards, a new training hand-drawn sketch is selected from the training hand-drawn sketch as the second training hand-drawn sketch, and a training stereo image corresponding to the new training hand-drawn sketch is selected from the training stereo image as the second training stereo image, and the generator network is retrained to readjust the encoder parameters until the distance between the encoder parameters and the optimal generator parameters is within a preset distance range, thereby obtaining the optimal encoder parameters.

[0153] Step 1014: Obtain a trained stereogram generation network based on the optimal generator parameters and the optimal encoder parameters.

[0154] After obtaining the optimal generator parameters and the optimal encoder parameters, the optimal generator parameters and the optimal encoder parameters can be substituted into the stereogram generation network to obtain a trained stereogram generation network.

[0155] Based on the above embodiment, in step 1011, the hand-drawn sketch is input into the pre-trained stereogram generation network. The specific process of generating the stereogram is performed by steps 10111 to 10114, which are as follows:

[0156] Step 10111: Input the hand-drawn sketch into the encoder of the pre-trained stereogram generation network to obtain the encoder output initial latent variables;

[0157] After obtaining the trained stereogram generation network, the hand-drawn sketch is input into the pre-trained stereogram generation network. After passing through the encoder, the initial latent variable is generated, denoted as w0, and the expression formula is as follows:

[0158] w=Encoder(sketch_input;θ e )

[0159] Among them, sketch_input is a hand-drawn sketch.

[0160] Step 10112: Input the initial latent variables into the generator of the pre-trained stereogram generation network to obtain the third stereogram and the third hand-drawn sketch output by the generator.

[0161] In this step, we first set the optimization times of the initial latent variable w0, which is controlled within 50 times. Let w = w0, and enter the optimization process of the initial latent variable. After the initial latent variable is input into the generator, the generator generates the third stereogram and the third hand-drawn sketch. The expression formula is as follows:

[0162] [sketch_pred,shade_pred]=Generator(w;θ g )

[0163] Step 10113: Determine the fourth loss function of the initial latent variable based on the third hand-drawn sketch, the hand-drawn sketch, the encoder parameters, and the optimal generator parameters, and optimize the initial latent variable based on the fourth loss function to obtain the optimal latent variable.

[0164] Afterwards, the fourth loss function of the initial latent variable is determined based on the third hand-drawn sketch, the hand-drawn sketch, the encoder parameters, and the optimal generator parameters. In this embodiment, the expression formula of the fourth loss function loss4 is:

[0165] loss4(w)=perceptual_loss(sketch_pred,sketch_real; θ g ,θ e )

[0166] The process of optimizing the initial latent variables is: optimization Determine whether the number of iterations meets the set optimization number. If so, end the optimization process, store the optimized latent variable w, and obtain the optimal latent variable. If not, iterate again. The process is as follows: Figure 5 shown.

[0167] Step 10114: Input the optimal latent variable into the generator so that the generator outputs a stereogram.

[0168] After obtaining the optimal latent variables, the optimal latent variables are input into the generator, so that the generator outputs the final stereogram.

[0169] Step 102: Receive an area selection instruction in the stereoscopic image, and confirm the target area according to the area selection instruction.

[0170] After obtaining the stereoscopic image, an area selection instruction is received in the stereoscopic image, and the target area is determined according to the area selection instruction. For example, in one embodiment, the user sends the area selection instruction by drawing a circle on the stereoscopic image, and the target area in the stereoscopic image is determined according to the range circled by the user in the stereoscopic image.

[0171] Based on the above embodiment, in step 102, a region selection instruction is received in the stereoscopic image, and the process of determining the target region according to the region selection instruction is specifically as follows:

[0172] The stereo image is input into a pre-trained annotation network so that the annotation network marks different areas of the stereo image, receives area selection instructions in the marked stereo image, and selects the target area from different areas according to the area selection instructions.

[0173] In this embodiment, it is necessary to pre-train the annotation network to obtain a trained annotation network that can mark different areas of the stereogram. For example, in one embodiment, a labeling training set of the annotation network is first obtained. The labeling training set consists of multiple training stereograms and multiple region labeling images corresponding to each training stereogram. The region labeling images are generated by manually labeling different regions of the stereogram. The region images are binary bitmaps. The size of the region images (image resolution) is consistent with the stereogram. Figure 1The file name is used to mark the category (or component name) to which the region image belongs, and then the annotation training set is used to train the annotation network to obtain a trained annotation network.

[0174] After that, the stereogram is input into the trained annotation network. The trained annotation network will label the different areas of the stereogram according to whether the pixels in the stereogram belong to the corresponding area. The process is as follows: Figure 6 After marking different areas of the stereogram, a region selection instruction sent by a user is received in the marked stereogram, and a target area is selected from the different areas according to the region selection instruction.

[0175] Step 103: Receive display parameters, which are used to confirm the display effect of the stereogram.

[0176] After the target area is selected, the display parameters set by the user are received, and the display parameters are used to confirm the display effect of the stereogram.

[0177] Step 104: Process the stereoscopic image to generate an effect image according to the pixel brightness information, the target area, and the display parameters.

[0178] After receiving the display parameters set by the user, the stereogram is processed based on the pixel brightness information, the target area, and the display parameters to generate a rendering. For example, in one embodiment, a light and dark texture is first generated based on the display parameters and the pixel brightness information. Then, stereogram components are extracted from the light and dark texture based on the target area. Finally, the stereogram components are replaced in the corresponding area of ​​the stereogram to generate a rendering.

[0179] Based on the above embodiment, the display parameters include fabric parameters, pattern parameters and synthesis parameters;

[0180] In one embodiment, Figure 7 As shown, an effect diagram generation interface is provided on the effect diagram generation device. The effect diagram generation interface has a stereogram generated from step 101. The stereogram is input into the trained annotation network, and different areas of the stereogram are marked and displayed on the effect diagram generation interface. The user can edit and select an area as a designated target area through the component area editing control, and select designated fabric parameters, pattern parameters, and set synthesis parameters through the fabric pattern and parameter setting control.

[0181] Accordingly, in step 104, the specific process of processing the stereoscopic image to generate the effect image according to the pixel brightness information, the target area and the display parameters is performed by steps 1041 to 1043. The specific process is as follows:

[0182] Step 1041: Generate texture tiling units according to fabric parameters and pattern parameters.

[0183] After setting the fabric parameters and pattern parameters, the texture tiling units of the effect image can be generated according to the fabric parameters and pattern parameters.

[0184] Step 1042: Obtain a three-dimensional component image according to the pixel brightness information, synthesis parameters, target area, and texture tiling units.

[0185] After obtaining the texture tiling unit, the texture tiling unit is further extended according to the pixel brightness information and the synthesis parameter and the extended texture tiling unit is set. Then, the extended texture tiling unit is cropped using the target area to obtain a three-dimensional component image.

[0186] Step 1043: Use the 3D component image to replace the pixels of the corresponding target area in the 3D image to obtain an effect image.

[0187] Finally, the pixels of the corresponding target area in the stereo image are replaced with the stereo component image to obtain the final effect image.

[0188] Based on the above embodiment, the specific process of obtaining the three-dimensional component image according to the pixel brightness information, synthesis parameters, target area, and texture tiling unit in step 1042 is performed by steps 10421 to 10424, specifically as follows:

[0189] Step 10421: Obtain target texture according to synthesis parameters and texture tiling units.

[0190] First, the texture tiling unit is extended using the synthesis parameters so that the texture tiling unit can cover the target area and obtain the target texture.

[0191] Based on the above embodiment, the synthesis parameters include texture tiling parameters.

[0192] It should be further explained that the texture tiling parameters determine the tiling mode of the texture tiling unit, such as the rotation direction, translation position, and repetition mode (horizontal tiling, vertical tiling, horizontal and vertical tiling, or no tiling, etc.).

[0193] Accordingly, in step 10421, the specific process of obtaining the target texture according to the synthesis parameters and the texture tiling unit is as follows:

[0194] A tiling method is determined according to texture tiling parameters, and a target texture is obtained by extending the texture tiling unit according to the tiling method, and the target texture fully covers the target area.

[0195] like Figure 8As shown, after obtaining the texture tiling parameters set by the user, the tiling mode of the texture tiling unit can be determined, and the continuously extended target texture is generated by repeated drawing according to the tiling mode to cover the target area.

[0196] Step 10422: Obtain a replacement texture based on pixel brightness information, synthesis parameters, and target texture.

[0197] Afterwards, the target texture is subjected to displacement distortion according to the pixel brightness information and the synthesis parameters, and the positions of the pixels of the target texture are displaced to obtain a displacement texture.

[0198] Based on the above embodiment, the synthesis parameter includes a brightness weight parameter.

[0199] Accordingly, in step 10422, the specific process of obtaining the replacement texture according to the pixel brightness information, the synthesis parameters, and the target texture is as follows:

[0200] According to the pixel brightness information and the brightness weight parameter, the target texture is distorted to obtain the replacement texture.

[0201] like Figure 8 As shown, since the pixel brightness information of the stereo image is related to the surface convexity and concavity, in this embodiment, the pixel position of the target texture is re-displaced through a "displacement warp" method based on the pixel brightness information and the brightness weight parameter to generate a displaced texture. Since the distortion of the displaced texture is related to the pixel brightness information of the stereo image, a realistic texture fitting effect can be produced. The brightness weight parameter can control the ratio between the pixel brightness information of the stereo image and the strength of the displacement warp, thereby controlling the strength of the fitting effect. It can be understood that the larger the ratio, the greater the pixel displacement and the greater the degree of distortion.

[0202] Step 10423: Obtain light and dark textures based on pixel brightness information, synthesis parameters, and displacement textures.

[0203] Afterwards, the brightness and darkness changes of the pixels of the replacement texture are further set according to the pixel brightness information and the synthesis parameters to obtain the brightness and darkness texture.

[0204] Based on the above embodiment, the synthesis parameters include brightness and darkness weight parameters.

[0205] Accordingly, in step 10423, the specific process of obtaining the light and dark texture according to the pixel brightness information, synthesis parameters and displacement texture is as follows:

[0206] According to the pixel brightness information and the light and dark weight parameters, the displacement texture is adjusted to obtain the light and dark texture.

[0207] like Figure 8As shown, since the pixel brightness information of the stereogram provides the light and dark color information that produces the three-dimensional effect, the light and dark color information of the stereogram can be obtained from the pixel brightness information. Afterwards, the Overlay function in the CanvasRenderingContext2D.globalCompositeOperation of the HTML5 image synthesis API is used to mix the light and dark color information of the stereogram and the color information of the displacement texture. This can generate a light and dark texture with a three-dimensional effect based on the displacement texture. The light and dark weight parameter is used to control the intensity of the influence of the light and dark colors of the stereogram on the color of the displacement texture, so as to achieve the effect of controlling the light and dark contrast and brightness.

[0208] Step 10424: crop the light and dark textures according to the target area to obtain a three-dimensional component image.

[0209] like Figure 8 As shown, the target area is then used as a mask to crop the light and dark textures according to the outline of the target area, and the light and dark textures outside the target area are discarded, thereby obtaining a three-dimensional component image with component outline, texture pattern information, and light and dark and distortion changes.

[0210] As described above, an embodiment of the present invention provides a rendering generation method, comprising the following steps: obtaining a hand-drawn sketch, inputting the hand-drawn sketch into a pre-trained stereogram generation network, and correspondingly generating a stereogram, the stereogram including pixel brightness information; receiving a region selection instruction in the stereogram, and confirming a target region according to the region selection instruction; receiving display parameters, the display parameters being used to confirm the display effect of the stereogram; and processing the stereogram to generate a rendering based on the pixel brightness information, the target region, and the display parameters. In this embodiment, the user only needs to manually draw a hand-drawn sketch and select the display parameters to be displayed to generate a rendering that meets the requirements, eliminating the need for drawing a large amount of light and dark information and texture details, greatly improving the efficiency of drawing clothing renderings, and solving the technical problem of low efficiency in drawing clothing renderings in the prior art.

[0211] Example 2

[0212] like Figure 9 As shown, Figure 9 An effect diagram generation device provided by an embodiment of the present invention includes a stereogram generation module 201, a target area selection module 202, a display parameter receiving module 203, and an effect diagram generation module 204:

[0213] The stereogram generation module 201 is used to obtain a hand-drawn sketch, input the hand-drawn sketch into a pre-trained stereogram generation network, and generate a stereogram accordingly. The stereogram includes pixel brightness information;

[0214] The target area selection module 202 is used to receive an area selection instruction in the stereoscopic image and confirm the target area according to the area selection instruction;

[0215] The display parameter receiving module 203 is used to receive display parameters, and the display parameters are used to confirm the display effect of the stereogram;

[0216] The effect image generation module 204 is used to process the stereo image and generate an effect image according to the pixel brightness information, the target area and the display parameters.

[0217] Based on the above embodiment, the stereogram generation network includes an encoder and a generator, the training set includes training hand-drawn sketches and training stereograms corresponding to the training hand-drawn sketches, and the rendering generation device also includes a training module, which is used to train the stereogram generation network, specifically:

[0218] Encoder parameters for initializing the encoder and generator parameters for the generator;

[0219] Input the training hand-drawn sketches and the training stereograms into the stereogram generation network for training to obtain the optimal generator parameters of the generator;

[0220] The optimal generator parameters of the generator are fixed, and the training hand-drawn sketches and training stereograms are input into the stereogram generation network for training to obtain the optimal encoder parameters of the encoder;

[0221] According to the optimal generator parameters and the optimal encoder parameters, a trained stereo image generation network is obtained.

[0222] Based on the above embodiment, the training module is used to input the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and the optimal generator parameters of the generator are specifically:

[0223] It is used to select a first training hand-drawn sketch from the training hand-drawn sketches, select a first training stereogram from the training stereograms, and perform a first training step. The first training step is specifically as follows:

[0224] Inputting the first training hand-drawn sketch into the encoder to obtain latent variables, and inputting the latent variables into the generator, so that the generator generates the first hand-drawn sketch and the first stereogram based on the generator parameters and the latent variables;

[0225] Inputting the first hand-drawn sketch, the first stereogram, the first training hand-drawn sketch, the first training stereogram, and the generator parameters into a preset first loss function to obtain a first loss value;

[0226] Inputting the first stereo image, the first training stereo image, and the generator parameters into a preset second loss function to obtain a second loss value;

[0227] Obtaining a total loss value according to the first loss value and the second loss value;

[0228] According to the total loss value, the generator parameters are reset to complete the first training step;

[0229] A new training hand-drawn sketch is selected from the training hand-drawn sketch, and a new training stereo image is selected from the training stereo image as the first training hand-drawn sketch and the first training stereo image respectively. The first training step is re-executed until the total loss value is within the preset range, thereby obtaining the optimal generator parameters.

[0230] Based on the above embodiment, the training module is used to fix the optimal generator parameters of the generator, input the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtain the optimal encoder parameters of the encoder as follows:

[0231] Used to determine the conversion relationship between the stereogram output by the generator and the generator parameters and encoder parameters;

[0232] Initialize the encoder parameters, fix the optimal generator parameters of the generator, select the second training hand-drawn sketch and the second training stereogram from the training hand-drawn sketch, and perform the second training step. The second training step is as follows:

[0233] Inputting the second training hand-drawn sketch into the stereogram generation network to obtain a second stereogram and a second hand-drawn sketch;

[0234] Inputting the second stereogram, the second training stereogram, the encoder parameters, and the optimal generator parameters into a preset third loss function, and converting the second stereogram into the optimal generator parameters and the encoder parameters in the third loss function according to the conversion relationship;

[0235] Calculate the distance between the encoder parameters and the optimal generator parameters according to the converted third loss function, and adjust the encoder parameters according to the distance;

[0236] A new training hand-drawn sketch and a new training stereogram are selected from the training hand-drawn sketch and the training stereogram as the second training hand-drawn sketch and the second training hand-drawn sketch respectively, and the second training step is re-executed until the distance is within the preset distance range to obtain the optimal encoder parameters.

[0237] Based on the above embodiment, the training module is used to determine the conversion relationship between the stereogram output by the generator and the generator parameters and encoder parameters as follows:

[0238] for determining encoder parameters and a first relationship between the input training hand-drawn sketch and the latent variable;

[0239] Determining a second relationship between the latent variables and the generator parameters and the hand-drawn sketch generated by the generator and the generated stereogram;

[0240] Determining a third relationship among the generator parameters, the input training hand-drawn sketch, the generated hand-drawn sketch, and the generated stereogram based on the first relationship and the second relationship;

[0241] According to the third relationship, a conversion relationship between the generated stereogram and the generator parameters and the encoder parameters is determined.

[0242] Based on the above embodiment, the stereogram generation module 201 is used to input the hand-drawn sketch into the pre-trained stereogram generation network, and the corresponding stereogram is generated specifically as follows:

[0243] It is used to input the hand-drawn sketch into the pre-trained stereogram generation network to obtain the encoder output initial latent variables;

[0244] Input the initial latent variable into the generator of the pre-trained stereogram generation network to obtain a third stereogram and a third hand-drawn sketch;

[0245] determining a fourth loss function of the initial latent variable based on the third hand-drawn sketch, the hand-drawn sketch, the encoder parameters, and the optimal generator parameters, and optimizing the initial latent variable based on the fourth loss function to obtain the optimal latent variable;

[0246] The optimal latent variables are input into the generator so that the generator outputs a stereo image.

[0247] Based on the above embodiment, the target area selection module 202 is used to receive an area selection instruction in the stereoscopic image, and confirm that the target area is specifically:

[0248] It is used to input the stereo image into a pre-trained annotation network so that the annotation network marks different areas of the stereo image, receives area selection instructions in the marked stereo image, and selects the target area from different areas according to the area selection instructions.

[0249] Based on the above embodiment, the display parameters include fabric parameters, pattern parameters and synthesis parameters;

[0250] Accordingly, the effect image generation module 204 is used to process the stereo image according to the pixel brightness information, the target area and the display parameters to generate the effect image specifically as follows:

[0251] Used to generate texture tiling units according to fabric parameters and pattern parameters;

[0252] Obtaining a three-dimensional component image according to pixel brightness information, synthesis parameters, target area, and texture tiling units;

[0253] Use the three-dimensional component image to replace the pixels of the corresponding target area in the three-dimensional image to obtain the effect image.

[0254] Based on the above embodiment, the effect image generation module 204 is used to obtain the three-dimensional component image according to the pixel brightness information, synthesis parameters, target area and texture tiling unit, specifically:

[0255] Used to obtain the target texture according to the synthesis parameters and the texture tiling unit;

[0256] Obtaining a replacement texture based on pixel brightness information, synthesis parameters, and target texture;

[0257] According to the pixel brightness information, synthesis parameters and displacement texture, the light and dark texture is obtained;

[0258] The light and dark textures are cropped according to the target area to obtain a three-dimensional component image.

[0259] Based on the above embodiment, the synthesis parameters include texture tiling parameters;

[0260] Accordingly, the effect image generation module 204 is used to obtain the target texture according to the synthesis parameters and the texture tiling unit:

[0261] It is used to determine the tiling method according to the texture tiling parameters, extend the texture tiling unit according to the tiling method to obtain the target texture, and the target texture is covered with the target area.

[0262] Based on the above embodiment, the synthesis parameters include a brightness weight parameter;

[0263] Accordingly, the effect image generation module 204 is used to obtain the replacement texture according to the pixel brightness information, synthesis parameters and target texture, specifically:

[0264] It is used to distort the target texture according to the pixel brightness information and brightness weight parameters to obtain the displacement texture.

[0265] Based on the above embodiment, the synthesis parameters include light and dark weight parameters;

[0266] Accordingly, the effect image generation module 204 is used to obtain the light and dark texture according to the pixel brightness information, synthesis parameters and replacement texture, specifically:

[0267] It is used to adjust the displacement texture according to the pixel brightness information and the light and dark weight parameters to obtain the light and dark texture.

[0268] Example 3

[0269] This embodiment also provides a terminal device, such as Figure 10As shown, a terminal device 30 includes a processor 300 and a memory 301;

[0270] The memory 301 is used to store a computer program 302 and transmit the computer program 302 to the processor;

[0271] The processor 300 is configured to execute the steps in the embodiment of the above-mentioned method for generating an effect diagram according to the instructions in the computer program 302 .

[0272] Exemplarily, the computer program 302 may be divided into one or more modules / units, which are stored in the memory 301 and executed by the processor 300 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 302 in the terminal device 30.

[0273] The terminal device 30 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device 30 may include, but is not limited to, a processor 300 and a memory 301. Those skilled in the art will understand that Figure 10 It is only an example of the terminal device 30 and does not constitute a limitation of the terminal device 30. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 30 may also include input and output devices, network access devices, buses, etc.

[0274] The processor 300 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0275] The memory 301 may be an internal storage unit of the terminal device 30, such as a hard disk or memory of the terminal device 30. The memory 301 may also be an external storage terminal device of the terminal device 30, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 30. Furthermore, the memory 301 may include both an internal storage unit of the terminal device 30 and an external storage device. The memory 301 is used to store the computer program and other programs and data required by the terminal device 30. The memory 301 may also be used to temporarily store data that has been output or is about to be output.

[0276] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0278] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0279] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0280] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0281] Example 4

[0282] An embodiment of the present invention further provides a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform a method for generating an effect diagram, the method comprising the following steps:

[0283] Obtain a hand-drawn sketch and input it into a pre-trained stereogram generation network to generate a stereogram. The stereogram includes pixel brightness information.

[0284] receiving a region selection instruction in the stereogram, and confirming a target region according to the region selection instruction;

[0285] Receive display parameters, the display parameters are used to confirm the display effect of the stereogram;

[0286] The stereo image is processed to generate an effect image according to the pixel brightness information, the target area and the display parameters.

[0287] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the embodiments of the present invention are not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the embodiments of the present invention. Therefore, although the embodiments of the present invention are described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments. Without departing from the concept of the embodiments of the present invention, the embodiments of the present invention may also include more other equivalent embodiments, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims

1. A method for generating an effect diagram, characterized in that: The following steps are involved: Obtaining a hand-drawn sketch, inputting the hand-drawn sketch into a pre-trained stereogram generation network, and correspondingly generating a stereogram, wherein the stereogram includes pixel brightness information; receiving an area selection instruction in the stereoscopic image, and confirming a target area according to the area selection instruction; receiving display parameters, wherein the display parameters are used to confirm a display effect of the stereogram; Processing the stereoscopic image to generate an effect image according to the pixel brightness information, the target area, and the display parameters; Wherein, the display parameters include fabric parameters, pattern parameters and synthesis parameters; Accordingly, the specific process of processing the stereoscopic image to generate the effect image according to the pixel brightness information, the target area and the display parameters is as follows: Generate a texture tiling unit according to the fabric parameters and the pattern parameters; Obtaining a three-dimensional component image according to the pixel brightness information, the synthesis parameter, the target area, and the texture tiling unit; Using the three-dimensional component image to replace the pixels of the corresponding target area in the three-dimensional image, to obtain an effect image; The synthesis parameters include texture tiling parameters, brightness weight parameters, and light and dark weight parameters; the specific process of obtaining the three-dimensional component image according to the pixel brightness information, the synthesis parameters, the target area, and the texture tiling unit is as follows: Determining a tiling mode according to the texture tiling parameters, extending the texture tiling unit according to the tiling mode to obtain a target texture, and tiling the target texture to fully cover the target area; distorting the target texture according to the pixel brightness information and the brightness weight parameter to obtain a replacement texture; Adjusting the replacement texture according to the pixel brightness information and the light and dark weight parameter to obtain a light and dark texture; The light and dark textures are cropped according to the target area to obtain a three-dimensional component image.

2. The method for generating an effect diagram according to claim 1, wherein: The stereogram generation network includes an encoder and a generator, the training set includes training hand-drawn sketches and training stereograms corresponding to the training hand-drawn sketches, and the stereogram generation network is trained by the following steps: Initializing encoder parameters of the encoder and generator parameters of the generator; Inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training to obtain optimal generator parameters of the generator; Fixing the optimal generator parameters of the generator, inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtaining the optimal encoder parameters of the encoder; A trained stereogram generation network is obtained according to the optimal generator parameters and the optimal encoder parameters.

3. The method for generating an effect diagram according to claim 2, wherein: The specific process of inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training to obtain the optimal generator parameters of the generator is as follows: A first training hand-drawn sketch is selected from the training hand-drawn sketches, and a first training stereogram is selected from the training stereograms, and a first training step is performed. The first training step is specifically: Inputting the first training hand-drawn sketch into the encoder to obtain latent variables, and inputting the latent variables into the generator so that the generator generates a first hand-drawn sketch and a first stereogram based on the generator parameters and the latent variables; Inputting the first hand-drawn sketch, the first stereogram, the first training hand-drawn sketch, the first training stereogram, and the generator parameters into a preset first loss function to obtain a first loss value; Inputting the first stereo image, the first training stereo image, and the generator parameters into a preset second loss function to obtain a second loss value; Obtaining a total loss value according to the first loss value and the second loss value; Resetting the generator parameters according to the total loss value to complete the first training step; A new training hand-drawn sketch is selected from the training hand-drawn sketch, and a new training stereo image is selected from the training stereo image as the first training hand-drawn sketch and the first training stereo image respectively, and the first training step is re-executed until the total loss value is within a preset range, thereby obtaining the optimal generator parameters.

4. The method for generating an effect diagram according to claim 3, wherein: The specific process of fixing the optimal generator parameters of the generator, inputting the training hand-drawn sketch and the training stereogram into the stereogram generation network for training, and obtaining the optimal encoder parameters of the encoder is as follows: Determining a conversion relationship between a stereogram output by the generator and the generator parameters and the encoder parameters; Initialize the encoder parameters, fix the optimal generator parameters of the generator, select a second training hand-drawn sketch and a second training stereogram from the training hand-drawn sketch, and perform a second training step. The second training step is specifically as follows: Inputting the second training hand-drawn sketch into the stereogram generation network to obtain a second stereogram and a second hand-drawn sketch; Inputting the second stereo image, the second training stereo image, the encoder parameters, and the optimal generator parameters into a preset third loss function, and converting the second stereo image into the optimal generator parameters and the encoder parameters in the third loss function according to the conversion relationship; Calculating a distance between the encoder parameters and the optimal generator parameters according to the third loss function, and adjusting the encoder parameters according to the distance; A new training hand-drawn sketch and a new training stereogram are selected from the training hand-drawn sketch and the training stereogram as the second training hand-drawn sketch and the second training hand-drawn sketch respectively, and the second training step is re-executed until the distance is within the preset distance range to obtain the optimal encoder parameters.

5. The method for generating an effect diagram according to claim 4, wherein: The specific process of determining the conversion relationship between the stereogram output by the generator and the generator parameters and the encoder parameters is as follows: Determining a first relationship between the encoder parameters and the input training hand-drawn sketch and the latent variable; Determining a second relationship between the latent variable and the generator parameter and the hand-drawn sketch and the generated stereogram generated by the generator; Determining a third relationship among the generator parameters, the input training hand-drawn sketch, the generated hand-drawn sketch, and the generated stereogram according to the first relationship and the second relationship; According to the third relationship, a conversion relationship between the generated stereogram and the generator parameters and the encoder parameters is determined.

6. The method for generating an effect diagram according to claim 5, wherein: The specific process of inputting the hand-drawn sketch into the pre-trained stereogram generation network to generate the stereogram is as follows: Inputting the hand-drawn sketch into a pre-trained stereogram generation network to obtain the encoder output initial latent variables; Inputting the initial latent variables into the generator of the pre-trained stereogram generation network to obtain a third stereogram and a third hand-drawn sketch; determining a fourth loss function of the initial latent variables according to the third hand-drawn sketch, the hand-drawn sketch, the encoder parameters, and the optimal generator parameters, and optimizing the initial latent variables according to the fourth loss function to obtain optimal latent variables; The optimal latent variables are input into the generator so that the generator outputs a stereogram.

7. The method for generating an effect diagram according to claim 1, wherein: The specific process of receiving the area selection instruction in the stereoscopic image and confirming the target area according to the area selection instruction is as follows: The stereogram is input into a pre-trained annotation network so that the annotation network marks different regions of the stereogram, receives a region selection instruction in the marked stereogram, and selects a target region from the different regions according to the region selection instruction.

8. A rendering generation device, characterized in that: It includes a stereogram generation module, a target area selection module, a display parameter receiving module, and an effect diagram generation module: The stereogram generation module is used to obtain a hand-drawn sketch, input the hand-drawn sketch into a pre-trained stereogram generation network, and generate a stereogram accordingly, wherein the stereogram includes pixel brightness information; The target area selection module is used to receive an area selection instruction in the stereoscopic image and confirm the target area according to the area selection instruction; The display parameter receiving module is used to receive display parameters, and the display parameters are used to confirm the display effect of the stereogram; The effect image generation module is used to process the stereoscopic image to generate an effect image according to the pixel brightness information, the target area and the display parameters; Wherein, the display parameters include fabric parameters, pattern parameters and synthesis parameters; Accordingly, the effect diagram generation module is specifically used for: Generate a texture tiling unit according to the fabric parameters and the pattern parameters; Obtaining a three-dimensional component image according to the pixel brightness information, the synthesis parameter, the target area, and the texture tiling unit; Using the three-dimensional component image to replace the pixels of the corresponding target area in the three-dimensional image, to obtain an effect image; The synthesis parameters include texture tiling parameters, brightness weight parameters, and light and dark weight parameters; the effect map generation module is specifically used to: Determining a tiling mode according to the texture tiling parameters, extending the texture tiling unit according to the tiling mode to obtain a target texture, and tiling the target texture to fully cover the target area; distorting the target texture according to the pixel brightness information and the brightness weight parameter to obtain a replacement texture; Adjusting the replacement texture according to the pixel brightness information and the light and dark weight parameter to obtain a light and dark texture; The light and dark textures are cropped according to the target area to obtain a three-dimensional component image.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute a method for generating an effect diagram according to any one of claims 1 to 7 according to instructions in the computer program.

10. A storage medium storing computer executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are used to execute a rendering generation method according to any one of claims 1 to 7.

Images