Virtual costume generation method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a virtual costume generation method, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining a model grid of an initial virtual costume model; determining mapping information set by a user; based on the map information, generating a model map corresponding to the map information through a pre-trained map generation model; and according to the model chartlet, performing chartlet pasting on the model grid to obtain a chartlet virtual clothing model. According to the scheme provided by the invention, the user can more flexibly carry out virtual dressing change, and the interactivity and the participation sense of the virtual dressing change of the user are improved, so that the user experience in the dressing change process is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for generating virtual clothing, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of computer technology, virtual dressing technology is increasingly used in people's daily lives. For example, virtual dressing allows users to design clothes, and virtual dressing allows users to try on clothes to choose their favorite clothing styles.

[0003] When performing virtual dressing, the related technology usually divides the virtual clothing into multiple areas, and the user can adjust the color and pattern of each area of ​​the virtual clothing to obtain the clothing effect he wants.

[0004] In the related art, since users change their virtual clothes by adjusting the color or pattern of local areas of virtual clothing, users can only adjust local colors or patterns based on current clothing, which greatly limits the user's dressing process, and the user's interactivity and sense of participation in dressing are weak, resulting in a poor user experience. Summary of the invention

[0005] The present application provides a method, device, electronic device and computer-readable storage medium for generating virtual clothing, which enables users to perform virtual dressing more flexibly, improves the interactivity and sense of participation of users in performing virtual dressing, and thus improves the user experience during the dressing process. The specific scheme is as follows.

[0006] In a first aspect, an embodiment of the present application provides a method for generating virtual clothing, the method comprising:

[0007] Obtaining a model mesh of an initial virtual clothing model;

[0008] Determine the sticker information set by the user;

[0009] Based on the map information, a model map corresponding to the map information is generated by a pre-trained map generation model;

[0010] The model grid is mapped according to the model map to obtain a mapped virtual clothing model.

[0011] Optionally, the sticker information includes at least one of the following: a sticker image, and a description text corresponding to the sticker.

[0012] Optionally, determining the map information set by the user includes:

[0013] Acquire information of a cloth map input by a user, wherein the cloth map is used to be pasted on the model grid by continuous paving;

[0014] The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes:

[0015] Based on the information of the cloth map, a four-way continuous graph corresponding to the cloth map is generated by a pre-trained first map generation model, wherein the first map generation model is used to convert an input image into a four-way continuous graph;

[0016] Determine the four-square continuous graph corresponding to the cloth texture as the model texture corresponding to the texture information;

[0017] The step of mapping the model grid according to the model map includes:

[0018] The model map is pasted on the model grid by continuous paving.

[0019] Optionally, the first map generation model includes a Stable Diffusion model, the convolution layer of the Stable Diffusion model includes a seamless convolution network, and the seamless convolution network has a receptive field for an opposite side edge when performing image feature processing on a side edge.

[0020] Optionally, the cloth map information includes a cloth map image;

[0021] Before generating a four-sided continuous graph corresponding to the cloth map by using a pre-trained first map generation model based on the information of the cloth map, the method further includes:

[0022] Extracting image description text of the cloth texture image from the cloth texture image;

[0023] The step of generating a four-sided continuous graph corresponding to the cloth map by using a pre-trained first map generation model based on the information of the cloth map comprises:

[0024] Based on the cloth texture image and the image description text, a four-sided continuous graph corresponding to the cloth texture image is generated by a pre-trained first texture generation model.

[0025] Optionally, the cloth map information includes a cloth map image;

[0026] The step of generating a four-sided continuous graph corresponding to the cloth texture by using a pre-trained first texture generation model based on the information of the cloth texture includes:

[0027] Performing noise processing on the cloth map image to obtain cloth image information after noise addition;

[0028] Based on the noisy cloth image information, a quadrilateral continuous graph corresponding to the cloth texture is generated by a pre-trained first texture generation model.

[0029] Optionally, the performing noise processing on the cloth map image to obtain the noisy cloth image information includes:

[0030] Performing multiple noise addition processes on the cloth map image to obtain multiple pieces of noisy cloth image information;

[0031] The method of generating a four-sided continuous graph corresponding to the cloth texture based on the noisy cloth image information by using a pre-trained first texture generation model comprises:

[0032] Based on the multiple pieces of noisy cloth image information, a plurality of four-sided continuous graphs corresponding to the cloth texture are generated by a pre-trained first texture generation model.

[0033] Optionally, determining the map information set by the user includes:

[0034] Acquire a graffiti sticker of a user, and determine the graffiti sticker as sticker information set by the user;

[0035] The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes:

[0036] Based on the graffiti map, an optimized model map corresponding to the graffiti map is generated through a pre-trained second map generation model, and the second map generation model is used to convert the graffiti map into an image after image optimization.

[0037] Optionally, before obtaining the graffiti sticker of the user's graffiti, the method further includes:

[0038] Extracting an initial front texture and an initial back texture of the initial virtual clothing model, and using the initial front texture and the initial back texture as graffiti templates;

[0039] The step of obtaining a graffiti sticker of a user's graffiti comprises:

[0040] Acquire a graffiti sticker of graffiti made by the user within the graffiti template, wherein the graffiti sticker includes a front graffiti sticker and a back graffiti sticker;

[0041] The optimized model map includes an optimized front model map and an optimized back model map.

[0042] Optionally, before generating an optimized model map corresponding to the graffiti map by using a pre-trained second map generation model based on the graffiti map, the method further comprises:

[0043] Determine a front outer contour image of the initial front map and a back outer contour image of the initial back map according to the graffiti template;

[0044] The step of generating an optimized model map corresponding to the graffiti map by using a pre-trained second map generation model based on the graffiti map includes:

[0045] Based on the graffiti map, the front outer contour map and the back outer contour map, an optimized model map corresponding to the graffiti map and consistent with the contour of the model grid is generated through a pre-trained second map generation model.

[0046] Optionally, the second map generation model includes a Stable Diffusion model and a ControlNet network, and the ControlNet network is used to control the Stable Diffusion model to generate a map within a front contour range of the front contour map and within a back contour range of the back contour map.

[0047] Optionally, before inputting the front profile image and the back profile image into the ControlNet network and inputting the graffiti map into the Stable Diffusion model, the method further includes:

[0048] Obtain a multi-view consistency description text, where the multi-view consistency description text is used to indicate that each model map generated by the StableDiffusion model is consistent at multiple viewpoints;

[0049] The step of inputting the front profile image and the back profile image into the ControlNet network and inputting the graffiti map into the Stable Diffusion model comprises:

[0050] The front contour map and the back contour map are input into the ControlNet network, and the graffiti map and the multi-view consistency description text are input into the Stable Diffusion model to obtain an optimized model map corresponding to the graffiti map, consistent with the contour of the model grid and consistent with multiple perspectives.

[0051] Optionally, mapping the model grid according to the model map to obtain a mapped virtual clothing model includes:

[0052] Determine the pattern corresponding to the model map;

[0053] Get target color information;

[0054] According to the target color information, the color of the pattern is adjusted to match the target color information to obtain an adjusted pattern;

[0055] The model grid is mapped according to the adjusted pattern and the model map to obtain a mapped virtual clothing model.

[0056] Optionally, determining the pattern corresponding to the model map includes:

[0057] Obtaining a target pattern selected by a user as a pattern corresponding to the model map;

[0058] Alternatively, a pattern included in the model map is extracted as the pattern corresponding to the model map.

[0059] Optionally, acquiring target color information includes:

[0060] Get the target color information selected by the user;

[0061] Alternatively, the background color of the model map is determined as the target color information.

[0062] Optionally, adjusting the color of the pattern to match the target color information according to the target color information to obtain an adjusted pattern includes:

[0063] Extracting a pattern outline line drawing of the pattern;

[0064] The color of the pattern outline line drawing is adjusted to match the target color information to obtain an adjusted pattern pattern.

[0065] Optionally, adjusting the color of the pattern outline line drawing to match the target color information to obtain an adjusted pattern includes:

[0066] Based on the target color information and the pattern outline line drawing, an adjusted pattern pattern corresponding to the pattern outline line drawing is generated through a pre-trained third map generation model, and the color of the adjusted pattern pattern matches the target color information.

[0067] Optionally, the third map generation model includes a Stable Diffusion model, a ControlNet network and a text-to-image adapter T2I-Adapter network, the ControlNet network is used to guide the Stable Diffusion model to generate a map according to the contour in the pattern contour line drawing, and the T2I-Adapter network is used to guide the Stable Diffusion model to generate a map according to the target color information.

[0068] Optionally, extracting the pattern outline line drawing of the pattern includes:

[0069] The pattern contour line drawing of the pattern is extracted using the Canny edge detection method.

[0070] Optionally, the texture information includes a texture image;

[0071] The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes:

[0072] Separating the foreground pattern and the background in the map image to obtain a multi-channel separation map, wherein the foreground pattern and the background in the multi-channel separation map are located in different layers;

[0073] Determine the foreground texture map and the background texture map;

[0074] The foreground texture map, the background texture map and the multi-channel separation texture are input into a pre-trained fourth texture generation model to generate a model material map corresponding to the texture information, wherein the material of the foreground pattern in the model material map is consistent with the foreground texture map. Figure 1 The material of the background pattern in the model texture map is consistent with the background texture map. Figure 1 To.

[0075] Optionally, the step of obtaining a model grid of an initial virtual clothing model includes:

[0076] Acquire a target clothing model selected by a user from a plurality of candidate clothing models, and determine the target clothing model as an initial virtual clothing model, wherein each of the candidate clothing models has a different model grid structure;

[0077] extracting a model mesh of the initial virtual clothing model;

[0078] Alternatively, a target model grid selected by a user from a plurality of candidate model grids corresponding to a plurality of candidate clothing models is obtained, and the target model grid is determined as the model grid of the initial virtual clothing model, and the model grid structures of the candidate clothing models are different.

[0079] Optionally, before mapping the model grid according to the model map to obtain the mapped virtual clothing model, the method further includes:

[0080] UV unfolding the model mesh according to a preset unfolding method to obtain a model UV mapping corresponding to the model mesh;

[0081] The step of mapping the model grid according to the model map to obtain a mapped virtual clothing model includes:

[0082] According to the model texture and the model UV mapping, the model grid is textured to obtain a textured virtual clothing model.

[0083] In a second aspect, an embodiment of the present application further provides a method for generating virtual clothing, which is applied to a client, and the method includes:

[0084] Displaying a clothing generation interface, wherein the clothing generation interface is used to allow the user to perform clothing design operations;

[0085] In response to the mapping information set by the user on the clothing generation interface, the mapping information is sent to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method according to any one of the first aspects, and sends the mapped virtual clothing model to the client;

[0086] The virtual clothing model after the mapping is displayed on the clothing generation interface.

[0087] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor; when the processor executes the computer program instructions, it implements the method described in any one of the first aspect or the second aspect.

[0088] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspects.

[0089] Compared with the prior art, this application has the following advantages:

[0090] The method for generating virtual clothing provided by the present application obtains the model grid of the initial virtual clothing model, determines the mapping information set by the user, that is, the mapping information is set by the user, and the user can set the specific content of the mapping information according to his or her own design requirements, so that the model grid can be mapped according to his or her own design requirements to obtain the clothing appearance he or she likes; the electronic device then generates a model map corresponding to the above-mentioned map information based on the map information set by the user through a pre-trained map generation model, and maps the model grid according to the generated model map to obtain a virtual clothing model after mapping. The pre-trained map generation model can efficiently, quickly and accurately obtain a model map that matches the map information set by the user.

[0091] It can be seen that the solution provided by the present application allows users to flexibly set the map information according to their own design requirements, so as to cover the model grid with a map that matches the map designed by the user, and generate a clothing model that is consistent with the map style set by the user. Therefore, the solution provided by the present application allows users to flexibly design a virtual clothing model that matches the style they set according to their own preferences and styles, and users can creatively design the appearance of clothing as a whole, so that users can perform virtual cross-dressing more flexibly, improve the interactivity and participation of users in virtual cross-dressing, thereby improving the user experience in the cross-dressing process, and significantly shorten the time of model generation, and improve the quality of the generated model.

[0092] This application is based on the cutting-edge artificial intelligence generated content (Artificial Intelligence Generated Content, AIGC for short) technology and has developed a 3D model generation solution based on AIGC, that is, the model texture that the user wants is generated through the texture generation model, thereby generating the clothing model appearance that the user wants. This application has created a 3D clothing model generation platform. Users only need to give lightweight input (such as pictures, text, graffiti, etc. about the texture information of the texture) to easily obtain high-quality, creative and beautiful 3D clothing models. The solution provided in this application can assist designers in achieving low-cost, large-scale, and efficient 3D clothing creation. When the solution provided in this application is applied to a virtual cross-dressing scene, it can also provide users with a DIY creative generation solution, which not only brings new creative experiences to users, but also alleviates the 3D clothing supply pressure of developers of virtual cross-dressing games. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a schematic diagram of the application scenario provided by this application;

[0094] Figure 2 It is a flowchart of an example of a method for generating virtual clothing provided in the first embodiment of the present application;

[0095] Figure 3 This is an example diagram of the process of generating a virtual clothing model after a texture is generated through a cloth texture in the method for generating a virtual clothing provided in an embodiment of the present application;

[0096] Figure 4 It is an example diagram of the four-sided continuous graph generated in the embodiment of the present application and the stitching effect of the four-sided continuous graph;

[0097] Figure 5 It is a comparison diagram of the effect of splicing the texture before optimization and the four-square continuous texture after optimization in the embodiment of the present application;

[0098] Figure 6 This is an example diagram of the process of generating a virtual clothing model after a graffiti sticker is used to generate a sticker in the method for generating virtual clothing provided in an embodiment of the present application;

[0099] Figure 7 This is an example diagram of a process of adjusting a pattern on a model map in a method for generating a virtual garment provided in an embodiment of the present application;

[0100] Figure 8 This is an example diagram of the process of generating a virtual clothing model after a texture is generated through a material texture in the method for generating a virtual clothing provided in an embodiment of the present application;

[0101] Fig. 9 is a flowchart of an example of a method for generating virtual clothing provided in the second embodiment of the present application;

[0102] Fig.10 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0103] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described clearly and completely below in conjunction with the drawings in the embodiments of the present application. However, the present application can be implemented in many other ways different from the following description. Therefore, based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present application.

[0104] It should be noted that the terms "first", "source domain", "third", etc. in the claims, description and drawings of the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. The data used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having" and their variants are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0105] In order to facilitate understanding of the various embodiments of the present application, the application background of the embodiments is described.

[0106] With the rapid development of computer technology, virtual dressing technology is increasingly used in people's daily lives. For example, virtual dressing allows users to design clothes, and virtual dressing allows users to try on clothes to choose their favorite clothing styles.

[0107] When performing virtual dressing, the related technology usually divides the virtual clothing into multiple areas, and the user can adjust the color and pattern of each area of ​​the virtual clothing to obtain the clothing effect he wants.

[0108] In the related art, since users change their virtual clothes by adjusting the color or pattern of local areas of virtual clothing, users can only adjust local colors or patterns based on current clothing, which greatly limits the user's dressing process, and the user's interactivity and sense of participation in dressing are weak, resulting in a poor user experience.

[0109] In order to solve the above problems, the embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for generating virtual clothing, which enables users to perform virtual dressing more flexibly, improves the interactivity and participation of users in virtual dressing, and thus improves the user experience during the dressing process.

[0110] The virtual clothing generation solution provided in this application can be used in clothing design platforms, virtual dressing platforms, virtual try-on platforms, etc., but is not limited to these.

[0111] In order to facilitate understanding of the method embodiments of the present application, the application scenarios are introduced. Figure 1 , Figure 1 The following is a schematic diagram of an application scenario of the method embodiment of the present application. It can be applied to a virtual dressing system. This application scenario is a schematic illustration and is not intended to be a specific description of its application scenario. Figure 1As shown, in this application scenario, a client for generating virtual clothing and a server for clothing are provided. In this embodiment, a connection is established between the client 101 and the server 102 via network communication.

[0112] The client 101 may be a mobile phone, tablet computer (pad), smart watch, desktop computer, smart TV, VR device, vehicle-mounted device, wearable device or other electronic device with display and data processing functions. The client 101 may display three-dimensional models and images.

[0113] A specific communication connection needs to be established between the client 101 and the server 102 to perform image transmission. For example, the server transmits an image including a verification code and a verification code prompt to the client.

[0114] The server 102 has high computing power. The server 102 can be a server, which has high-speed processor (central processing unit, CPU) computing power, long-term reliable operation, powerful input / output (input / output, I / O) external data throughput capacity and better scalability. The server 102 can be a single server or a server cluster. The server 102 is used to generate corresponding model maps according to the map information provided by the client, and to paste the model map on the model grid to obtain the mapped virtual clothing model required by the user, and send the mapped virtual clothing model to the client 101, so that the client can display the mapped virtual clothing model. The server 102 can also provide other specific services to the client, such as user information access, website access, application access, etc., which are not specifically limited in this application.

[0115] The client 101 can communicate with the server 102. The client 101 and the server 102 can communicate using various communication systems, such as a wired communication system or a wireless communication system. The wireless communication system can be, for example, a global system for mobile communications (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) system, a general packet radio service (GPRS), a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, a universal mobile telecommunication system (UMTS), a worldwide interoperability for microwave access (WiMAX) communication system, a future fifth generation (5G) system or a new radio (NR), a satellite communication system, etc.

[0116] In this embodiment, the server is mainly used to generate a model texture corresponding to the texture information set by the user through a pre-trained texture generation model, and the client 101 is mainly used to obtain the texture from the server and display the virtual clothing model for the user to view.

[0117] Embodiment 1

[0118] The first embodiment of the present application provides a method for generating virtual clothing. The execution subject of the method may be an electronic device, which may be a desktop computer, a laptop computer, a smart mobile terminal, a smart TV, a wearable device, a VR device, a server, etc., but is not limited thereto. Since the intelligent model is usually deployed on the server and can run smoothly, the embodiment of the present application uses the server as the execution subject to illustrate the method for generating virtual clothing.

[0119] like Figure 2 As shown, the method for generating virtual clothing provided in the first embodiment of the present application includes the following steps S110 to S140.

[0120] Step S110: obtaining a model mesh of an initial virtual clothing model.

[0121] In the embodiment of the present application, an initial virtual clothing model can be generated in advance. Specifically, the initial virtual clothing model can be manually pre-made and generated, or an existing clothing model or a clothing model generated by an intelligent model can be determined as the initial virtual clothing model. The initial virtual clothing model is a basic model, which can be understood as a model that has not been finely designed. The texture of the initial virtual clothing model can be a texture of a basic color, such as a white texture, a gray texture, etc., so that the initial virtual clothing model shows a basic model appearance.

[0122] The virtual clothing model in the embodiment of the present application is a three-dimensional model (i.e., a 3D model), which includes a model mesh and a model map (such as a physically based rendering PBR map). The mesh defines the geometric shape structure of the 3D model, thereby determining the basic style of the garment, and the PBR map includes a variety of map information such as texture maps and material maps, which is a key factor in determining the specific design, style, pattern and material of the clothing model.

[0123] In the embodiment of the present application, the model mesh and the texture of the initial virtual clothing model may be separated, thereby obtaining the model mesh corresponding to the separated initial virtual clothing model.

[0124] In a specific embodiment, step S110 can be implemented according to the following steps S111 to S112.

[0125] Step S111: obtaining a target clothing model selected by a user from a plurality of candidate clothing models, and determining the target clothing model as an initial virtual clothing model.

[0126] Step S112: extracting the model mesh of the initial virtual clothing model.

[0127] Alternatively, step S110 may also be implemented according to the following steps: obtaining a target model grid selected by a user from a plurality of candidate model grids corresponding to a plurality of candidate clothing models, and determining the target model grid as the model grid of the initial virtual clothing model.

[0128] The model grid structures of the candidate clothing models are different.

[0129] In this embodiment, multiple candidate clothing models can be pre-generated for users to choose from. Each candidate clothing model can be a basic model. The model structure of each candidate clothing model is different, that is, the model grid structure corresponding to each candidate clothing model is different, resulting in different clothing patterns corresponding to each candidate clothing model. Users can select the target clothing model they want from multiple candidate clothing models according to their own preferences, or select the target model grid they want from multiple candidate model grids to select the clothing pattern they want. By setting multiple candidate clothing models, this embodiment enables users to flexibly select the clothing pattern they want, thereby improving the flexibility of users in clothing design.

[0130] Step S120: Determine the sticker information set by the user.

[0131] The above-mentioned map information is used to guide the generation of the model map of the model mesh. The map information may include at least one of the following: a map image, a description text corresponding to the map. The map image may be a graffiti image drawn by the user, an existing image input by the user, or another image selected by the user, which is not specifically limited in this application. The description text corresponding to the map is used to describe the content of the map. For example, the description text corresponding to the map may be "an image containing pink flowers, green leaves, and a light orange background".

[0132] The texture information is usually a texture roughly set by the user, which can reflect the user's design concept. It is necessary to generate a model texture that can be used for model covering based on the texture information.

[0133] Step S130: Based on the above-mentioned texture information, a model texture corresponding to the above-mentioned texture information is generated by a pre-trained texture generation model.

[0134] Specifically, the map information can be input into a pre-trained map generation model to generate a model map corresponding to the above map information. The map generation model is used to generate a map that meets the model map requirements. The map generation model can be a stable diffusion Stable Diffusion model, wherein the Stable Diffusion model is a generative model for generating high-quality images. It gradually diffuses and refines the initial noise sequence to eventually generate a realistic image. This process is achieved through a series of step-by-step deconvolution operations. Each step adds more details and information to the noise sequence, so that the generated image gradually approaches a real and detailed image. The input of the Stable Diffusion model can be text information or an image. The map generation model can also be a generative adversarial network GAN, a variational autoencoder VAE, a random variational generation network SVG and other models. Those skilled in the art can choose according to actual needs, and this application does not specifically limit it.

[0135] Optionally, the map information can also be processed and input into the map generation model, for example, after noise addition processing, encoding processing, text extraction, contour extraction and other processing, and then input into the map generation model. The specific processing method can be determined according to the actual output scenario. This application does not specifically limit it. The following text will give examples of the processing methods corresponding to each scenario.

[0136] When training the map generation model, the map generation model can be trained through supervised learning algorithms, unsupervised learning algorithms, reinforcement learning algorithms, generative adversarial network algorithms and other model training algorithms. The training process of the model can adopt the training method in the relevant technology, and this application will no longer introduce the specific training process in detail.

[0137] When the sticker information is the description text corresponding to the sticker, the model sticker corresponding to the sticker information can be the generated model sticker that meets the requirements of the description text. When the sticker information is the sticker image corresponding to the sticker, the model sticker corresponding to the sticker information can be the sticker processed by the sticker image. For example, the graffiti sticker image can be realistically processed, the blurred sticker image can be clarified, the initial sticker image can be color corrected, etc. The conditions satisfied by the model sticker corresponding to the sticker information can be determined according to the actual scene requirements. The following text will explain the model stickers corresponding to different types of sticker information through several examples. The sticker generation model can generate model stickers that meet the scene requirements.

[0138] Step S140: mapping the model grid according to the generated model map to obtain a mapped virtual clothing model.

[0139] Specifically, the model map may be pasted on the model grid. To facilitate model mapping and to perform model mapping more accurately, before step S140, the method may further include the following step S140a.

[0140] Step S140a: UV unfolding the model mesh according to a preset unfolding method to obtain a model UV mapping corresponding to the model mesh.

[0141] In the field of 3D model design, UV mapping is used to indicate the correspondence between the surface of a 3D object (i.e., model mesh) and a 2D map. Since the UV mapping of many original models is often chaotic and disordered, insufficient to express the semantics of the model as a "clothing", it poses an obstacle to the AI-driven generation process. Therefore, UV mapping can be unfolded to make the model mesh display a more orderly structural layout on the 2D plane that conforms to the semantics of clothing design, laying the foundation for subsequent AI generation.

[0142] Model UV unfolding refers to the process of mapping each vertex of the 3D model mesh surface to a 2D plane (usually a rectangular UV space). The main purpose of UV unfolding is to enable texture mapping to be correctly and seamlessly mapped to the surface of the model. For example, in 3D modeling software, modelers can perform UV unfolding manually or using automated tools so that they can accurately edit and draw texture details in subsequent texture production.

[0143] In the embodiment of the present application, the model mesh can be manually UV-unwrapped according to a preset unwrapping method. The specific unwrapping method can be manually set according to the mapping effect, and the present application does not specifically limit it. Figure 3 As shown in the virtual clothing model after mapping, if it is necessary to display the mapping of the upper body part and the skirt part of the dress at different scales, then when performing UV unfolding, the upper body part and the skirt part can be unfolded and scaled at different scales in the UV coordinates. The specific unfolding method is not specifically limited in this application.

[0144] Accordingly, the above step S140 can be implemented according to the following steps S141:

[0145] Step S141: mapping the model mesh according to the model mapping and the model UV mapping to obtain a mapped virtual clothing model.

[0146] Specifically, the model map may be placed in the UV coordinates to correspond in position with the model grid unfolded in the UV coordinates, thereby pasting the model map on the model grid.

[0147] This embodiment can more accurately map the model by unfolding the UV of the model mesh, so that the obtained mapped model is more accurate.

[0148] The method for generating virtual clothing provided by the present application obtains the model grid of the initial virtual clothing model, determines the mapping information set by the user, that is, the mapping information is set by the user, and the user can set the specific content of the mapping information according to his or her own design requirements, so that the model grid can be mapped according to his or her own design requirements to obtain the clothing appearance he or she likes; the electronic device then generates a model map corresponding to the above-mentioned map information based on the map information set by the user through a pre-trained map generation model, and maps the model grid according to the generated model map to obtain a virtual clothing model after mapping. The pre-trained map generation model can efficiently, quickly and accurately obtain a model map that matches the map information set by the user.

[0149] It can be seen that the solution provided by the present application allows users to flexibly set the map information according to their own design requirements, so as to cover the model grid with a map that matches the map designed by the user, and generate a clothing model that is consistent with the map style set by the user. Therefore, the solution provided by the present application allows users to flexibly design a virtual clothing model that matches the style they set according to their own preferences and styles, and users can creatively design the appearance of clothing as a whole, so that users can perform virtual cross-dressing more flexibly, improve the interactivity and participation of users in virtual cross-dressing, thereby improving the user experience in the cross-dressing process, and significantly shorten the time of model generation, and improve the quality of the generated model.

[0150] In terms of the 3D clothing model generation solution, the embodiment of the present application has constructed a new and complete 3D clothing model generation solution, which can maximize the satisfaction of users' different creative needs, allowing users to gain more sense of participation and better playability experience, and can also give full play to the creativity of artificial intelligence in clothing design to produce creative and high-quality 3D clothing models.

[0151] This application is based on the cutting-edge artificial intelligence generated content (Artificial Intelligence Generated Content, AIGC for short) technology and has developed a 3D model generation solution based on AIGC, that is, the model texture that the user wants is generated through the texture generation model, thereby generating the clothing model appearance that the user wants. This application has created a 3D clothing model generation platform. Users only need to give lightweight input (such as pictures, text, graffiti, etc. about the texture information of the texture) to easily obtain high-quality, creative and beautiful 3D clothing models. The solution provided in this application can assist designers in achieving low-cost, large-scale, and efficient 3D clothing creation. When the solution provided in this application is applied to a virtual cross-dressing scene, it can also provide users with a DIY creative generation solution, which not only brings new creative experiences to users, but also alleviates the 3D clothing supply pressure of developers of virtual cross-dressing games.

[0152] In one implementation, the above step S120 can be implemented according to the following step S121.

[0153] Step S121: obtaining information of a cloth map input by a user, wherein the cloth map is used to be pasted on the model grid by continuous paving.

[0154] The cloth map is used to represent the cloth of the clothing model, that is, the cloth map is the cloth unit of the clothing model, for example, it can be a square or rectangular cloth map. By continuously paving the cloth map on the model grid, the virtual clothing model can be covered with the map. Figure 3As shown, the cloth map can be a square map, so as to facilitate continuous tiling in the UV coordinates and also make the tiling on the clothing model more uniform.

[0155] The cloth texture information input by the user may be a cloth texture image or a description text of the cloth texture.

[0156] Correspondingly, the above step S130 can be implemented according to the following steps S131 to S132.

[0157] Step S131: Based on the information of the cloth texture, a quadrilateral continuous graph corresponding to the cloth texture is generated by a pre-trained first texture generation model.

[0158] Step S132: determining the four-square continuous graph corresponding to the above cloth texture as the model texture corresponding to the above texture information.

[0159] The first map generation model is used to convert an input image into a four-dimensional continuous map.

[0160] The four-sided continuous image is a rectangular image, such as a rectangular or square image, and the opposite side edges of the four-sided continuous image can be seamlessly spliced, that is, the edges of the opposite sides of the four-sided continuous image can be naturally transitioned without splicing seams after splicing. For example, the upper edge and the lower edge of the four-sided continuous image can be seamlessly spliced ​​after being connected, and the left edge and the right edge can be seamlessly spliced ​​after being connected. Figure 4 As shown, Figure 4 The first picture in is a single square continuous picture. Figure 4 The effect of 2×2 tiling and 3×3 tiling of a single square continuous picture is as follows Figure 4 As shown in the second and third figures in the figure, it can be seen that the pattern formed by continuously paving the four-square continuous pattern has no joints. Figure 5 The following is a comparison of the stitching effect of ordinary cloth textures and the stitching effect of four-sided continuous cloth textures. Figure 5 It can be seen that after being processed into a four-square continuous image by the first mapping generation model, the splicing seams are significantly eliminated and the splicing transition is more natural.

[0161] The first map generation model is used to convert the input image into a four-sided continuous image. The first map generation model is the map generation model in step S130. The first map generation model can be a neural network model, a generative adversarial network model, etc., which is not specifically limited in this application. The first map generation model can be obtained by training with sample maps based on a training algorithm such as a supervised learning algorithm and a semi-supervised learning algorithm. The specific training method can refer to the relevant technology, and this application will not describe the training process in detail.

[0162] In a specific embodiment, the first map generation model may include a Stable Diffusion model, the convolution layer of the Stable Diffusion model includes a Seamless Convolution network, and the Seamless Convolution network has a receptive field for the opposite side edge when processing image features on one side edge. In this way, when processing one side edge of the image through the seamless convolution network, the image features of the opposite side edge of the side edge can be considered at the same time. Therefore, when generating an image, the image of the opposite side edge of the generated image can meet the seamless splicing requirements. This embodiment improves the convolution layer structure in the U-Net network of the Stable Diffusion model, that is, the original convolution network in the U-Net network of the Stable Diffusion model is modified to a Seamless Convolution network, thereby improving the generation process of edge pixels.

[0163] Specifically, Figure 3 As shown, the Stable Diffusion model includes a U-Net network, and the U-Net network includes various convolution networks, and each convolution network is a Seamless Convolution network in this application.

[0164] Correspondingly, step S140 can be implemented according to the following step S142.

[0165] Step S142: Paste the model texture on the model grid by continuous paving.

[0166] The continuous tiling may be continuous tiling in a matrix shape until the model grid is fully tiled.

[0167] The effect of continuous tiling on the model grid is as follows Figure 3 As shown in the virtual clothing model after mapping, it can be seen that Figure 3 The cloth map in the model is processed into a four-dimensional continuous map by the Stable Diffusion model and then pasted on the model grid. The effect is more natural and without seams, and the mapping effect is better.

[0168] In a specific embodiment, when the cloth map information includes a cloth map image, before step S131, the method may further include the following step S131a.

[0169] Step S131a: extracting image description text of the cloth texture image from the cloth texture image.

[0170] Specifically, the image description text of the cloth map image can be extracted through a pre-trained image-to-text conversion model. The image-to-text conversion model is used to convert the image into the corresponding description text. The image-to-text conversion model can be a neural network model, an adversarial network model, etc. The training process of the image-to-text conversion model can refer to related technologies and will not be described in detail in this application.

[0171] For example, Figure 3 As shown, before inputting into the first map to generate the model cloth map image, the image description text "a pattern of flowers and leaves on apinkbackground" corresponding to the cloth map image can be extracted first. The description text here can be expressed in English or in Chinese, which is not specifically limited in this application.

[0172] Accordingly, the above step S131 can be implemented by the following steps S1311:

[0173] Step S1311: Based on the cloth texture image and the image description text, a four-sided continuous graph corresponding to the cloth texture image is generated by a pre-trained first texture generation model.

[0174] Specifically, the cloth map image and the above-mentioned image description text can be directly input into the first map generation model to obtain a four-sided continuous graph, or the cloth map image and the image description text can be vector-encoded to obtain corresponding vectors and then input into the first map generation model to obtain a four-sided continuous graph. Technical personnel in this field can handle it flexibly, and this application does not specifically limit it.

[0175] This embodiment extracts the image description text corresponding to the image of the cloth map and guides the generation process through the image description text. The text description can provide more semantic information and context to help the model more accurately understand the image content that should be generated. For example, the text description can include the features of the object, the background information of the scene, etc., so as to generate high-quality, consistent and more realistic images.

[0176] In a specific embodiment, when the cloth map information includes a cloth map image, step S131 can be implemented according to the following steps S1312 to S1313.

[0177] Step S1312: performing noise processing on the cloth map image to obtain noisy cloth image information.

[0178] Specifically, Figure 3 As shown, the noise addition process can be performed in the following manner: the above-mentioned cloth map image is encoded to obtain the cloth map vector code, and then the noise map code and the cloth map vector code are fused through a transformation model to obtain the noisy cloth image information.

[0179] Step S1313: Based on the above-mentioned noisy cloth image information, a quadrilateral continuous graph corresponding to the cloth texture is generated by a pre-trained first texture generation model.

[0180] Specifically, the noisy cloth image information can be input into the first texture generation model to generate a four-sided continuous graph corresponding to the cloth texture.

[0181] This embodiment can simulate the diffusion effect in the generation process through noise addition processing. By controlling and increasing the noise, the generator can have sufficient information at each stage to gradually improve the quality and realism of the generated image, thereby making the quality of the generated four-square continuous image higher.

[0182] In a specific embodiment, the above step S1312 can be implemented according to the following steps: performing multiple noise addition processes on the cloth map image to obtain multiple pieces of noisy cloth image information.

[0183] Step S1313 can be implemented by the following steps: based on a plurality of noisy cloth image information, a plurality of square continuous images corresponding to the cloth texture are generated by a pre-trained first texture generation model.

[0184] Step S140 can be implemented by the following steps: a plurality of square continuous images are respectively pasted on the above-mentioned model grid to obtain a plurality of corresponding virtual clothing models after pasting. This embodiment can generate a plurality of square continuous images by adding noise multiple times, so as to generate virtual clothing models after pasting in multiple styles, so that users can have more choices and provide users with more diverse design effects.

[0185] In one implementation, step S120 can be implemented by following step S122. Step S130 can be implemented by following step S133.

[0186] Step S122: Acquire a graffiti sticker drawn by the user, and determine the graffiti sticker as sticker information set by the user.

[0187] The graffiti sticker can be a graffiti sticker drawn by the user on the clothing generation interface of the client, or a graffiti sticker drawn in advance by the user. The outer contour of the graffiti sticker can match the contour of the model grid, or it can not match, for example, it can be square, circular, etc., which is not specifically limited in this application. The user can draw graffiti arbitrarily according to his own design ideas, and send the drawn graffiti sticker to the server for model sticker generation.

[0188] In a specific embodiment, before step S122, the above method may further include the following step S122a.

[0189] Step S122a: extracting an initial front texture and an initial back texture of the initial virtual clothing model, and using the initial front texture and the initial back texture as graffiti templates.

[0190] The initial virtual clothing model includes a model mesh and an initial texture, so the initial texture of the initial virtual clothing model can be extracted. Specifically, the extracted initial texture includes a front graffiti texture and a back graffiti texture. When the initial virtual clothing model is UV-unwrapped, the initial front texture and the initial back texture of the virtual clothing model can be extracted.

[0191] Correspondingly, step S122 can be implemented according to the following step S1221.

[0192] Step S1221: obtaining a graffiti sticker of the user graffitied within the graffiti template, wherein the graffiti sticker includes a front graffiti sticker and a back graffiti sticker.

[0193] Correspondingly, the optimized model map includes an optimized front model map and an optimized back model map.

[0194] This embodiment limits the user to graffiti within the range of the graffiti template, so that the obtained front graffiti map and the back graffiti map match the outline of the model grid, so that the subsequently generated model map also matches the outline of the model grid, which makes it easier for the subsequently generated model map to better match the structure of the model grid, and can achieve higher-quality mapping of the grid model, with better effects after mapping.

[0195] Step S133: Based on the graffiti map, an optimized model map corresponding to the graffiti map is generated by a pre-trained second map generation model.

[0196] The image after image optimization can be a realistic image, and the image after image optimization can be an image after various optimization processes such as image enhancement, denoising, super-resolution enhancement, style conversion, lighting and shadow adjustment, etc. Figure 6 As shown, Figure 6 The front graffiti map and the back graffiti map in the image are optimized by the second map generation model, and the following is obtained: Figure 6 The front and back model textures shown in .

[0197] The second sticker generation model is used to convert the graffiti sticker into an image after image optimization. The second sticker generation model is the sticker generation model in step S130. The second sticker generation model can be a neural network model, a generative adversarial network model, a Markov random field (MRF) model, etc., which is not specifically limited in this application. The second sticker generation model can be obtained by training with sample stickers based on a training algorithm such as a supervised learning algorithm and a semi-supervised learning algorithm. The specific training method can refer to the relevant technology, and this application will not describe the training process in detail.

[0198] When the graffiti map includes a front graffiti map and a back graffiti map, the front graffiti map and the back graffiti map can be spliced ​​to obtain a spliced ​​map, and based on the spliced ​​map, a pre-trained second map generation model is used to generate an optimized model map corresponding to the above graffiti map.

[0199] This embodiment enables users to doodle and obtain optimized maps that match the user's doodle image. In this way, users can flexibly express their design ideas through doodles, thereby designing virtual clothing that is consistent with their doodle ideas, enriching the user's design methods for clothing design. Using the solution provided by this embodiment can not only maximize the user's creative needs and obtain a better playability experience, but also give full play to the creativity of artificial intelligence in clothing design and produce high-quality and creative 3D clothing models.

[0200] In a specific embodiment, before step S133, the above method may further include the following step S133a.

[0201] Step S133a: determining a front outer contour image of an initial front texture and a back outer contour image of an initial back texture according to the graffiti template.

[0202] Specifically, the scribble contours of the initial front map and the initial back map can be extracted respectively to obtain the front outer contour map and the back outer contour map. The obtained front outer contour map and the back outer contour map are as follows: Figure 6 shown.

[0203] Correspondingly, step S133 can be implemented according to the following step S1331.

[0204] Step S1331: Based on the graffiti map, the front outer contour map and the back outer contour map, a pre-trained second map generation model is used to generate an optimized model map corresponding to the graffiti map and consistent with the contour of the model grid.

[0205] Among them, the second texture generation model can control the generated front model texture within the contour range of the front outer contour map when producing the model texture, and control the generated back model texture within the contour range of the back outer contour map.

[0206] Specifically, the graffiti map, the front outer contour map and the back outer contour map may be input into a second map generation model to obtain the optimized model map.

[0207] This embodiment can control the generated mapping to better match the contour of the model grid through the front outer contour map and the back outer contour map, so that the mapped model is more accurate and meets the design needs of the user.

[0208] In a specific embodiment, the second map generation model may include a Stable Diffusion model and a ControlNet network, and the ControlNet network is used to control the Stable Diffusion model to generate maps within the front contour range of the front contour map and the back contour range of the back contour map.

[0209] In this embodiment, the ControlNet network can be regarded as a plug-in of the Stable Diffusion model. When generating images, the ControlNet network and the Stable Diffusion model work together. The ControlNet network can control the neural network structure of the diffusion model (such as Stable Diffusion) by adding additional conditions, and can realize conditional image generation. Figure 6 The model structure in is the structure of the second map generation model, from Figure 6 It can be seen that the Stable Diffusion model and the ControlNet network of the second map generation model work together to generate a map that meets the requirements within the range of the extracted front contour and back contour.

[0210] Correspondingly, step S1331 can be implemented by the following steps: inputting the above-mentioned front contour map and the above-mentioned back contour map into the ControlNet network, and inputting the graffiti map into the Stable Diffusion model, so that the Stable Diffusion model outputs an optimized model map corresponding to the graffiti map and consistent with the contour of the model grid under the control of the ControlNet network.

[0211] This embodiment can better control the Stable Diffusion model to generate a map within the front contour and the back contour by setting the ControlNet network, ensuring that the generated map can be better mapped back to the model grid of the 3D virtual clothing model.

[0212] In a specific embodiment, before step S1331, the method may further include the following step S1331a.

[0213] Step S1331a: Obtain a multi-view consistency description text, where the multi-view consistency description text is used to indicate that each model map generated by the Stable Diffusion model is consistent at multiple viewpoints.

[0214] For example, when a Stable Diffusion model generates a front model texture and a back model texture, the front model texture and the back model texture should be consistent in front and back perspectives.

[0215] Step S1331 can be implemented by the following steps: input the front contour map and the back contour map into the ControlNet network, input the graffiti map and the multi-view consistency description text into the StableDiffusion model, and obtain an optimized model map corresponding to the graffiti map, consistent with the contour of the model mesh and consistent with multiple perspectives.

[0216] Optionally, when the second map generation model includes a Stable Diffusion model and a ControlNet network, when generating the model map in step S1331, the graffiti map can be denoised to obtain a noisy graffiti map, and the noisy graffiti map, the front outer contour map and the back outer contour map are input into the second map generation model to generate an optimized model map corresponding to the graffiti map and consistent with the contour of the model grid. When denoising, the graffiti map can also be denoised multiple times to obtain multiple noisy graffiti maps, so that multiple model maps can be obtained for the user to choose from.

[0217] In this embodiment, inputting multi-view consistency description text can better ensure that the generated front view and back view maps are consistent as a whole, thereby generating higher quality model maps and virtual clothing models.

[0218] In one implementation, step S140 may be implemented according to the following steps S143 to S146.

[0219] Step S143: Determine the pattern corresponding to the model map.

[0220] Specifically, the target pattern selected by the user can be obtained as the pattern corresponding to the model map. Alternatively, the pattern included in the above model map can be extracted as the pattern corresponding to the model map. When the target pattern selected by the user is used as the pattern corresponding to the model map, the target pattern can be subsequently integrated into the generated model map and then pasted on the model grid to meet the user's need to add patterns. When the pattern included in the model map is extracted as the pattern corresponding to the model map, the existing pattern in the model map can be adjusted to meet the user's needs.

[0221] The user may select a pattern in the clothing generation interface, wherein the clothing generation interface may display various candidate pattern patterns, or the user may input a target pattern pattern, which is not specifically limited in the present application.

[0222] When extracting the pattern included in the model map, it can be extracted through a pre-trained pattern extraction model. The training method of the pattern extraction model adopts supervised training, semi-supervised training in related technologies to obtain other training algorithms, which will not be described in detail here.

[0223] like Figure 7 Shown is the target pattern selected by the user.

[0224] Step S144: Obtain target color information.

[0225] Specifically, the target color information selected by the user can be obtained. The user can select or input the target color information in the clothing generation interface, so that the user can set the color of the pattern according to his or her preference; or, the background color of the model map can be determined as the target color information, so that the pattern color in the map can be coordinated and matched with the background color, ensuring that the color matching of the pattern is harmonious with the main color, thereby making the final generated model more beautiful as a whole.

[0226] Step S145: According to the target color information, the color of the pattern is adjusted to match the target color information to obtain an adjusted pattern.

[0227] The color of the pattern is adjusted to match the target color information. The color of the pattern can be the same as the target color, or similar to the target color, or harmonious with the target color. Specifically, the color of the pattern can be harmoniously matched with the target color according to the image design configuration.

[0228] Specifically, the color of the pattern can be adjusted to match the target color information through a pre-trained color adjustment model. The color adjustment model can be trained based on supervised training, semi-supervised training, or unsupervised training in related technologies. The specific training process will not be described in detail. Figure 7 shown.

[0229] In a specific embodiment, step S145 can be implemented according to the following steps S1451 to S1452.

[0230] Step S1451: extracting the pattern outline line drawing of the pattern.

[0231] Specifically, the pattern contour line diagram of the pattern can be extracted by using the Canny edge detection method. The extracted pattern contour line diagram is, for example, Figure 7 The pattern outline line diagram shown in the figure is used to clearly mark the outline lines of the pattern in the pattern. The pattern outline line diagram can be a diagram formed by two colors with relatively large differences, one color is used to represent the background, and the other color is used to represent the pattern outline lines, for example, black is the line, and white is the pattern outline line, so as to facilitate accurate distinction. Optionally, the pattern outline line diagram of the pattern pattern can also be extracted by Sobel algorithm, Prewitt algorithm, etc.

[0232] The pattern outline includes the outlines of each line of the pattern, including the outer outline, detail outline, etc. The pattern outline can fully represent the pattern.

[0233] Step S1452: adjusting the color of the pattern outline line drawing to match the target color information to obtain an adjusted pattern pattern.

[0234] Specifically, based on the target color information and the pattern outline line drawing, an adjusted pattern pattern corresponding to the pattern pattern line drawing can be generated through a pre-trained third map generation model, and the color of the adjusted pattern pattern matches the target color information.

[0235] The third map generation model is used to convert the pattern contour line map into an image whose color matches the target color. The third map generation model can be a neural network model, a generative adversarial network model, a Markov random field (MRF) model, etc., which is not specifically limited in this application. The third map generation model can be obtained by training the sample contour map based on a supervised learning algorithm, a semi-supervised learning algorithm, and other training algorithms. The specific training method can refer to the relevant technology, and this application will not describe the training process in detail.

[0236] In a specific embodiment, the third map generation model may include a Stable Diffusion model, a ControlNet network and a text-to-image adapter T2I-Adapter network, wherein the ControlNet network is used to guide the Stable Diffusion model to generate a map according to the contour in the pattern contour line drawing, and the T2I-Adapter network is used to guide the Stable Diffusion model to generate a map according to the target color information.

[0237] Correspondingly, the step of inputting the target color information and the pattern outline line drawing into a pre-trained third texture generation model to generate an adjusted pattern corresponding to the pattern outline line drawing includes:

[0238] The target color information is input into the T2I-Adapter network, and the pattern contour line diagram is input into the ControlNet network, so that the Stable Diffusion model outputs an adjusted pattern pattern corresponding to the pattern contour line diagram and matching the target color information under the control of the ControlNet network and the T2I-Adapter network.

[0239] The T2I-Adapter network is a lightweight, plug-in neural network structure that controls the diffusion model (such as Stable Diffusion) by adding additional conditions, and can achieve lightweight conditional image generation. This embodiment controls the color of the image generated by the Stable Diffusion model through the T2I-Adapter network, and can generate natural and harmonious color matching for the pattern.

[0240] Optionally, a noise image and text prompt information may be input into the Stable Diffusion model, or the noised image and text prompt information after the above pattern outline line drawing is noised may be input into the Stable Diffusion model. The text prompt information is used to prompt the Stable Diffusion model to generate the image. For example, the text prompt information may be "set the color of the pattern to match the target color information, and set the background of the pattern to be the same as the preset color." The text prompt information may also be other prompt information, which is not specifically limited in this application. In the process of adding noise, multiple times of adding noise may be performed, so that multiple adjusted pattern patterns can be obtained for the user to flexibly choose. By setting the adding noise and text prompt information, the Stable Diffusion model can generate more accurate pattern patterns.

[0241] This embodiment uses the ControlNet+T2I method to recolor the pattern, which not only ensures the accuracy of the pattern style, but also ensures the harmony of the pattern color matching. Through the intelligent algorithm, the creative generation of clothing maps is further simplified, thereby improving the creation efficiency.

[0242] This embodiment can clearly show the pattern outline of the pattern by extracting the pattern outline line diagram, ensuring that the pattern has high accuracy and clarity when applied to clothing, laying the foundation for color matching, so that the adjusted pattern obtained after adjusting the color can be more matched with the original pattern.

[0243] Step S146: mapping the model grid according to the adjusted pattern and the model mapping to obtain a mapped virtual clothing model.

[0244] This implementation allows the user to adjust the color of the pattern of the model map, and can adjust the disharmonious pattern color in the model map designed and generated by the user to a more harmonious one, so as to produce more rich and harmonious color schemes for the user to choose, thereby making the model map more beautiful; patterns can also be superimposed according to user needs, and the superimposed pattern should match harmoniously with the map background.

[0245] In one implementation, when the above-mentioned sticker information includes a sticker image, step S130 can be implemented according to the following steps S134 to S136.

[0246] Step S134: Separate the foreground pattern and the background in the above-mentioned texture image to obtain a multi-channel separation texture, in which the foreground pattern and the background are located in different layers.

[0247] Specifically, the foreground pattern and background in the above-mentioned map image can be separated by a pre-trained pattern separation model to obtain a multi-channel separation map. The pattern separation model can be a neural network model, a regression model, etc., which is not specifically limited in this application. The pattern separation model can separate the background and foreground patterns of the image into different layers. The specific training process of the pattern separation model refers to the relevant technology, which is not specifically limited in this application.

[0248] like Figure 8 As shown, Figure 8 The multi-channel separation image corresponding to the map image is shown in FIG.

[0249] Step S135: Determine the foreground texture map and the background texture map.

[0250] Specifically, the foreground material selected by the user and the background material selected by the user may be determined, or the preset foreground material and the preset background material may be determined. The determined foreground material and background material are the foreground material map and the background material map. Among them, the material determines the texture and effect of the 3D clothing model. The material is mainly determined by the material map. The material map usually includes basic material maps such as metalness map, roughness map, transparency map, special effect map, etc., and may also include advanced material maps such as dark patterns, hollowing, laser, etc., but is not limited to these.

[0251] Optionally, the above basic materials may also include: cotton, linen, silk, leather, wool, denim, yarn and other material maps for users to freely choose. These basic materials can be intelligently generated with the help of expert knowledge, designer experience and the like.

[0252] Users can select a background material map from various basic material maps, select a foreground material map from various advanced material maps, and can also flexibly select foreground and background material maps according to their own design effect requirements, which is not specifically limited in this application.

[0253] Step S136: Input the foreground texture map, the background texture map and the multi-channel separation texture into a pre-trained fourth texture generation model to generate a model material texture corresponding to the texture information, wherein the material of the foreground pattern in the model texture is consistent with the foreground texture map. Figure 1 The material of the background pattern in the model map is consistent with the background material map Figure 1 To.

[0254] The fourth texture generation model is used to superimpose the foreground texture and the background texture on the foreground pattern layer and the background pattern layer of the multi-channel separation texture, respectively, so as to generate the required model texture. Figure 8 As shown, the fourth map can be used to generate a model (i.e. Figure 8 In the texture generation model in the figure), multiple background textures and corresponding multiple foreground textures are inputted, so that the fourth texture generation model generates multiple model textures. Subsequently, the multiple model textures can be respectively pasted on the model grid to generate multiple textures and then the virtual clothing model is provided for the user to select. Figure 8 As shown in FIG. 1 , four model material maps can be generated, and these four model material maps can be pasted on the model grid respectively. After generating four maps, the virtual clothing model is generated. The model material map generated in this embodiment is the model map. Since the model map may include other types of maps in addition to the model material map, such as color maps, transparency maps, etc., various maps can be superimposed and pasted on the model grid. The final effect is as follows: Figure 8 The virtual clothing model after mapping is shown in .

[0255] The fourth texture generation model can be a neural network model, a generative adversarial network model, etc., which is not specifically limited in this application. The fourth texture generation model can be obtained by training a sample foreground material texture, a sample background material texture, a sample multi-channel separation texture and a supervised learning algorithm, a semi-supervised learning algorithm and other training algorithms. The specific training method can refer to the relevant technology, and this application will not describe the training process in detail.

[0256] This implementation uses multi-channel separation maps to represent the foreground and background areas respectively, and can assign different materials to the foreground and background areas respectively. It can not only simulate the texture of real materials (such as cotton, linen, silk, leather, wool, denim, yarn, etc.), but also achieve a variety of advanced material effects (such as printing, hidden patterns, hollowing, laser, etc.), thereby improving the appearance of the produced virtual clothing models and giving users more design space.

[0257] Embodiment 2

[0258] The second embodiment of the present application provides a method for generating virtual clothing. The executor of the method is a client, which can be a desktop computer, a laptop computer, a smart mobile terminal, a smart TV, a wearable device, a VR device, etc., but is not limited thereto.

[0259] like Fig. 9 As shown, the method for generating virtual clothing provided in the second embodiment of the present application includes the following steps S210 to S230.

[0260] Step S210: displaying a clothing generation interface, wherein the clothing generation interface is used to enable the user to perform clothing design operations.

[0261] The clothing generation interface can be used to enable users to select textures, draw textures, select materials, select colors, select initial models, and other design interfaces related to clothing design, which are not specifically limited in this application.

[0262] Step S220: In response to the mapping information set by the user on the clothing generation interface, the mapping information is sent to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method described in any one of the first embodiments, and sends the mapped virtual clothing model to the client.

[0263] The sticker information set by the user may be a sticker selected by the user, a sticker drawn by the user, a sticker description text input by the user, etc., and this application does not specifically limit this.

[0264] Step S230: displaying the mapped virtual clothing model on the clothing generation interface.

[0265] In one embodiment, step S220 may include: in response to the user inputting the cloth map information, sending the cloth map information to the server, so that the server generates a virtual clothing model after mapping according to the method of generating the map by the first map generation model described in the first embodiment.

[0266] Specifically, the clothing generation interface may display information of candidate fabric textures, and the user may select information of a desired fabric texture from information of each candidate fabric texture, or the user may input information of the fabric texture.

[0267] In one implementation, step S220 may include: in response to obtaining a graffiti map of a user, sending the graffiti map to a server, so that the server generates a virtual clothing model after the map is mapped according to the method of generating the map by the second map generation model as described in the first embodiment.

[0268] Specifically, a graffiti interface may be displayed in the clothing generation interface, and the user may graffiti in the graffiti interface to generate a graffiti sticker.

[0269] The method for generating virtual clothing provided in this embodiment is an explanation of the method provided in the first embodiment on the client side. The relevant contents and beneficial effects have been described in detail in the first embodiment and will not be described in detail here.

[0270] Embodiment 3

[0271] The third embodiment of the present application also provides a virtual clothing generation device corresponding to the virtual clothing generation method embodiment provided in the first embodiment. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the virtual clothing generation method embodiment provided above. The virtual clothing generation device provided in the present application includes:

[0272] An acquisition unit, used for acquiring a model mesh of an initial virtual clothing model;

[0273] A determination unit, used to determine the map information set by the user;

[0274] A generating unit, configured to generate a model map corresponding to the map information based on the map information by using a pre-trained map generation model;

[0275] The pasting unit is used to paste the model grid according to the model map to obtain a virtual clothing model after pasting.

[0276] Embodiment 4

[0277] The fourth embodiment of the present application also provides a virtual clothing generation device corresponding to the virtual clothing generation method embodiment provided in the second embodiment above. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the virtual clothing generation method embodiment provided above. The virtual clothing generation device provided in the present application includes:

[0278] A display unit, used to display a clothing generation interface, wherein the clothing generation interface is used to allow the user to perform clothing design operations;

[0279] A sending unit, configured to respond to the mapping information set by the user on the clothing generation interface, and send the mapping information to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method described in any one of the first embodiments, and sends the mapped virtual clothing model to the client;

[0280] The display unit is also used to display the mapped virtual clothing model on the clothing generation interface.

[0281] Embodiment 5

[0282] The fifth embodiment of the present application also provides an electronic device embodiment corresponding to the virtual clothing generation method embodiment provided in the first embodiment above. The following description of the electronic device embodiment is only illustrative. The electronic device embodiment is as follows:

[0283] Please refer to Fig.10 Understand the above electronic devices, Fig.10 The electronic device provided in this embodiment includes: a processor 1001, a memory 1002, a communication bus 1003, and a communication interface 1004;

[0284] The memory 1002 is used to store computer instructions for data processing. When the computer instructions are read and executed by the processor 1001, the following steps are performed:

[0285] Obtaining a model mesh of an initial virtual clothing model;

[0286] Determine the sticker information set by the user;

[0287] Based on the map information, a model map corresponding to the map information is generated by a pre-trained map generation model;

[0288] The model grid is mapped according to the model map to obtain a mapped virtual clothing model.

[0289] The sixth embodiment of the present application also provides an electronic device embodiment corresponding to the virtual clothing generation method embodiment provided in the second embodiment above. The following description of the electronic device embodiment is only illustrative. The electronic device embodiment is as follows:

[0290] The electronic device provided in this embodiment includes: a processor, a memory, a communication bus, and a communication interface;

[0291] The memory is used to store computer instructions for data processing, and when the computer instructions are read and executed by the processor, the following steps are performed:

[0292] Displaying a clothing generation interface, wherein the clothing generation interface is used to allow the user to perform clothing design operations;

[0293] In response to the mapping information set by the user on the clothing generation interface, the mapping information is sent to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method described in any one of the first embodiments, and sends the mapped virtual clothing model to the client;

[0294] The virtual clothing model after the mapping is displayed on the clothing generation interface.

[0295] The seventh embodiment of the present application also provides a computer-readable storage medium for implementing the above virtual clothing generation method. The computer-readable storage medium embodiment provided in this application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The following described embodiment is only exemplary.

[0296] The computer readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the following steps are implemented:

[0297] Obtaining a model mesh of an initial virtual clothing model;

[0298] Determine the sticker information set by the user;

[0299] Based on the map information, a model map corresponding to the map information is generated by a pre-trained map generation model;

[0300] The model grid is mapped according to the model map to obtain a mapped virtual clothing model.

[0301] The eighth embodiment of the present application also provides a computer-readable storage medium for implementing the above virtual clothing generation method. The computer-readable storage medium embodiment provided in the present application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The following described embodiment is only illustrative.

[0302] The computer readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the following steps are implemented:

[0303] Displaying a clothing generation interface, wherein the clothing generation interface is used to allow the user to perform clothing design operations;

[0304] In response to the mapping information set by the user on the clothing generation interface, the mapping information is sent to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method described in any one of the first embodiments, and sends the mapped virtual clothing model to the client;

[0305] The virtual clothing model after the mapping is displayed on the clothing generation interface.

[0306] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0307] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0308] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0309] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0310] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for generating virtual clothing, characterized in that: The method comprises: Obtaining a model mesh of an initial virtual clothing model; Determine the sticker information set by the user; Based on the map information, a model map corresponding to the map information is generated by a pre-trained map generation model; The model grid is mapped according to the model map to obtain a mapped virtual clothing model.

2. The method for generating virtual clothing according to claim 1, characterized in that: The sticker information includes at least one of the following: a sticker image and a description text corresponding to the sticker.

3. The method for generating virtual clothing according to claim 1, characterized in that: The step of determining the map information set by the user includes: Acquire information of a cloth map input by a user, wherein the cloth map is used to be pasted on the model grid in a continuous paving manner; The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes: Based on the information of the cloth map, a four-way continuous graph corresponding to the cloth map is generated by a pre-trained first map generation model, wherein the first map generation model is used to convert an input image into a four-way continuous graph; Determine the four-square continuous graph corresponding to the cloth texture as the model texture corresponding to the texture information; The step of mapping the model grid according to the model map includes: The model map is pasted on the model grid by continuous paving.

4. The method for generating virtual clothing according to claim 3, characterized in that: The first map generation model includes a Stable Diffusion model, the convolution layer of the Stable Diffusion model includes a seamless convolution network, and the seamless convolution network has a receptive field for the opposite side edge when performing image feature processing on one side edge.

5. The method for generating virtual clothing according to claim 4, characterized in that: The cloth texture information includes a cloth texture image; Before generating a four-sided continuous graph corresponding to the cloth map by using a pre-trained first map generation model based on the information of the cloth map, the method further includes: Extracting image description text of the cloth texture image from the cloth texture image; The step of generating a four-sided continuous graph corresponding to the cloth map by using a pre-trained first map generation model based on the information of the cloth map comprises: Based on the cloth texture image and the image description text, a four-sided continuous graph corresponding to the cloth texture image is generated by a pre-trained first texture generation model.

6. The method for generating virtual clothing according to claim 4, characterized in that: The cloth texture information includes a cloth texture image; The step of generating a four-sided continuous graph corresponding to the cloth map by using a pre-trained first map generation model based on the information of the cloth map comprises: Performing noise processing on the cloth map image to obtain cloth image information after noise addition; Based on the noisy cloth image information, a quadrilateral continuous graph corresponding to the cloth texture is generated by a pre-trained first texture generation model.

7. The method for generating virtual clothing according to claim 6, characterized in that: The step of performing noise processing on the cloth map image to obtain the cloth image information after noise addition includes: Performing multiple noise addition processes on the cloth map image to obtain multiple pieces of noisy cloth image information; The method of generating a four-sided continuous graph corresponding to the cloth texture by using a pre-trained first texture generation model based on the noisy cloth image information includes: Based on the multiple pieces of noisy cloth image information, a plurality of four-sided continuous graphs corresponding to the cloth texture are generated by a pre-trained first texture generation model.

8. The method for generating virtual clothing according to claim 1, characterized in that: The step of determining the map information set by the user includes: Acquire a graffiti sticker of a user, and determine the graffiti sticker as sticker information set by the user; The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes: Based on the graffiti map, an optimized model map corresponding to the graffiti map is generated through a pre-trained second map generation model, and the second map generation model is used to convert the graffiti map into an image after image optimization.

9. The method for generating virtual clothing according to claim 8, characterized in that: Before obtaining the graffiti sticker of the user, the method further includes: Extracting an initial front texture and an initial back texture of the initial virtual clothing model, and using the initial front texture and the initial back texture as graffiti templates; The step of obtaining a graffiti sticker of a user's graffiti comprises: Acquire a graffiti sticker of graffiti made by the user within the graffiti template, wherein the graffiti sticker includes a front graffiti sticker and a back graffiti sticker; The optimized model map includes an optimized front model map and an optimized back model map.

10. The method for generating virtual clothing according to claim 9, characterized in that: Before generating an optimized model map corresponding to the graffiti map by using a pre-trained second map generation model based on the graffiti map, the method further includes: Determine a front outer contour image of the initial front map and a back outer contour image of the initial back map according to the graffiti template; The step of generating an optimized model map corresponding to the graffiti map by using a pre-trained second map generation model based on the graffiti map includes: Based on the graffiti map, the front outer contour map and the back outer contour map, an optimized model map corresponding to the graffiti map and consistent with the contour of the model grid is generated through a pre-trained second map generation model.

11. The method for generating virtual clothing according to claim 10, characterized in that: The second map generation model includes a Stable Diffusion model and a ControlNet network, and the ControlNet network is used to control the Stable Diffusion model to generate a map within a front contour range of the front contour map and within a back contour range of the back contour map.

12. The method for generating virtual clothing according to claim 11, characterized in that: Before inputting the front profile image and the back profile image into the ControlNet network and inputting the graffiti map into the StableDiffusion model, the method further includes: Obtain a multi-view consistency description text, where the multi-view consistency description text is used to indicate that each model map generated by the StableDiffusion model is consistent at multiple viewpoints; The step of inputting the front profile image and the back profile image into the ControlNet network and inputting the graffiti map into the Stable Diffusion model comprises: The front contour map and the back contour map are input into the ControlNet network, and the graffiti map and the multi-view consistency description text are input into the Stable Diffusion model to obtain an optimized model map corresponding to the graffiti map, consistent with the contour of the model grid and consistent with multiple perspectives.

13. The method for generating virtual clothing according to claim 1, characterized in that: The step of mapping the model grid according to the model map to obtain a mapped virtual clothing model includes: Determine the pattern corresponding to the model map; Get target color information; According to the target color information, the color of the pattern is adjusted to match the target color information to obtain an adjusted pattern; The model grid is mapped according to the adjusted pattern and the model map to obtain a mapped virtual clothing model.

14. The method for generating virtual clothing according to claim 13, characterized in that: The step of determining the pattern corresponding to the model map comprises: Obtaining a target pattern selected by a user as a pattern corresponding to the model map; Alternatively, a pattern included in the model map is extracted as the pattern corresponding to the model map.

15. The method for generating virtual clothing according to claim 13, characterized in that: The obtaining of target color information includes: Get the target color information selected by the user; Alternatively, the background color of the model map is determined as the target color information.

16. The method for generating virtual clothing according to claim 13, characterized in that: The step of adjusting the color of the pattern to match the target color information according to the target color information to obtain the adjusted pattern includes: Extracting a pattern outline line drawing of the pattern; The color of the pattern outline line drawing is adjusted to match the target color information to obtain an adjusted pattern pattern.

17. The method for generating virtual clothing according to claim 16, characterized in that: The step of adjusting the color of the pattern outline line drawing to match the target color information to obtain an adjusted pattern includes: Based on the target color information and the pattern outline line drawing, an adjusted pattern pattern corresponding to the pattern outline line drawing is generated through a pre-trained third map generation model, and the color of the adjusted pattern pattern matches the target color information.

18. The method for generating virtual clothing according to claim 17, characterized in that: The third map generation model includes a Stable Diffusion model, a ControlNet network and a text-to-image adapter T2I-Adapter network. The ControlNet network is used to guide the Stable Diffusion model to generate a map according to the contour in the pattern contour line drawing, and the T2I-Adapter network is used to guide the Stable Diffusion model to generate a map according to the target color information.

19. The method for generating virtual clothing according to claim 16, characterized in that: The step of extracting the pattern outline line drawing of the pattern comprises: The pattern contour line drawing of the pattern is extracted using the Canny edge detection method.

20. The method for generating virtual clothing according to claim 1, characterized in that: The texture information includes a texture image; The step of generating a model map corresponding to the map information by using a pre-trained map generation model based on the map information includes: Separating the foreground pattern and the background in the map image to obtain a multi-channel separation map, wherein the foreground pattern and the background in the multi-channel separation map are located in different layers; Determine the foreground texture map and the background texture map; The foreground material map, the background material map and the multi-channel separation map are input into a pre-trained fourth texture generation model to generate a model material map corresponding to the texture information, wherein the material of the foreground pattern in the model material map is consistent with the foreground material map, and the material of the background pattern in the model material map is consistent with the background material map.

21. The method for generating virtual clothing according to any one of claims 1 to 20, characterized in that: The step of obtaining a model grid of an initial virtual clothing model includes: Acquire a target clothing model selected by a user from a plurality of candidate clothing models, and determine the target clothing model as an initial virtual clothing model, wherein each of the candidate clothing models has a different model grid structure; extracting a model mesh of the initial virtual clothing model; Alternatively, a target model grid selected by a user from a plurality of candidate model grids corresponding to a plurality of candidate clothing models is obtained, and the target model grid is determined as the model grid of the initial virtual clothing model, and the model grid structures of the candidate clothing models are different.

22. The method for generating virtual clothing according to any one of claims 1 to 20, characterized in that: Before mapping the model grid according to the model map to obtain the mapped virtual clothing model, the method further includes: UV unfolding the model mesh according to a preset unfolding method to obtain a model UV mapping corresponding to the model mesh; The step of mapping the model grid according to the model map to obtain a mapped virtual clothing model includes: According to the model texture and the model UV mapping, the model grid is textured to obtain a textured virtual clothing model.

23. A method for generating virtual clothing, characterized in that: Applied to a client, the method comprises: Displaying a clothing generation interface, wherein the clothing generation interface is used to allow the user to perform clothing design operations; In response to the mapping information set by the user on the clothing generation interface, the mapping information is sent to the server, so that the server generates a mapped virtual clothing model according to the virtual clothing generation method according to any one of claims 1 to 22, and sends the mapped virtual clothing model to the client; The virtual clothing model after the mapping is displayed on the clothing generation interface.

24. An electronic device, characterized in that: include: A processor, a memory, and computer program instructions stored on the memory and executable on the processor; When the processor executes the computer program instructions, the method according to any one of claims 1 to 23 is implemented.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method described in any one of claims 1 to 23.

Images