Image generation method, apparatus, device, and storage medium

Abstract

The present disclosure provides an image generation method, device and equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, deep learning, large model, and can be applied to virtual reality, 3D clothing generation and other scenes. The specific implementation scheme is: obtaining image features of a target input image, wherein the target input image at least contains a target object, and the image features at least contain contour features and posture features of the target object; obtaining clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by the target object in the target input image; and performing feature fusion on the image features of the target input image and the clothing features to obtain a target clothing image of the clothing worn by the target object.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of computer vision, deep learning, and large models, and can be applied to scenarios such as virtual reality and three-dimensional (3D) clothing generation. Background Technology

[0002] With technological advancements, virtual digital human technology is rapidly developing and demonstrating its strong application potential across multiple fields. As technology continues to progress and applications deepen, virtual digital humans are expected to play a role in more fields and scenarios, becoming an important part of future digital life. Therefore, research on virtual digital humans has become a current hot topic. Summary of the Invention

[0003] This disclosure provides an image generation method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, an image generation method is provided, comprising:

[0005] Obtain image features of the target input image, wherein the target input image contains at least a target object, and the image features contain at least: contour features and pose features of the target object;

[0006] Obtain clothing features of at least one clothing feature image; wherein, the at least one clothing feature image is related to the clothing worn by the target object in the target input image;

[0007] The image features of the target input image and the clothing features are fused to obtain a target clothing image for the clothing worn by the target object.

[0008] According to another aspect of this disclosure, an image generation apparatus is provided, comprising:

[0009] The first feature extraction unit is used to obtain image features of the target input image, wherein the target input image contains at least a target object, and the image features contain at least: contour features and pose features of the target object;

[0010] The second feature extraction unit is used to obtain clothing features of at least one clothing feature image; wherein, the at least one clothing feature image is related to the clothing worn by the target object in the target input image;

[0011] The feature fusion unit is used to fuse the image features of the target input image and the clothing features to obtain a target clothing image for the clothing worn by the target object.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0018] This disclosed solution generates a target clothing image by fusing the image features of the target input image with the clothing features of at least one clothing feature image. In this way, the clothing worn by the target object can be accurately identified from the target input image, and a tiled image of the clothing (such as the target clothing image) can be generated. Furthermore, because the features are effectively fused, the generated tiled image can retain clothing details to the greatest extent, thus providing more accurate and natural image data for subsequent clothing display, design, and 3D modeling.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is an illustrative flow diagram of an image generation method according to an embodiment of this application. Figure 1 ;

[0022] Figure 2 This is an illustrative flow diagram of an image generation method according to an embodiment of this application. Figure 2 ;

[0023] Figure 3 This is a schematic diagram of the model structure of an image generation method according to an embodiment of this application in a specific example;

[0024] Figure 4 This is a schematic diagram showing the effect of applying the image generation method according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of an image generation apparatus according to an embodiment of this application;

[0026] Figure 6 This is a block diagram of an electronic device used to implement the image generation method of the embodiments of this disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] In this document, the term "and / or" merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document indicates any combination of at least two of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this document refer to and distinguish between multiple similar technical terms, not to restrict the order or to limit there to only two. For example, "first feature" and "second feature" refer to two categories / two features; the first feature can be one or more, and the second feature can also be one or more.

[0029] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can still be practiced even without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0030] In the field of virtual try-on, a large amount of research and applications have focused on combining clothing images with human images to demonstrate the try-on effect. However, current technology mainly focuses on the "try-on" function, that is, how to virtually "wear" clothing on a person. In contrast, how to "take off" the clothing worn on a person and restore its appearance in a flat lay-out manner is a problem that has not been fully studied.

[0031] In practical applications, attempts to accurately extract the original appearance of clothing or headdresses from images of people's attire often encounter problems such as inaccurate image segmentation, occlusion of clothing parts, and uneven lighting. These issues hinder high-quality clothing displays and their further application in fields such as 3D clothing generation, modeling, and design.

[0032] Based on this, the present disclosure provides an image generation method that can accurately separate the clothing worn by the target object in the image from any captured image, such as tops, bottoms, or headwear, and generate a complete tiled image of the clothing. In this way, it effectively overcomes the problem that it is difficult to extract complete clothing images due to the low segmentation accuracy of traditional segmentation techniques. At the same time, it improves the extraction accuracy of the clothing image of the target object, providing more accurate and natural image data for subsequent clothing display, design, and 3D modeling.

[0033] Furthermore, this disclosed solution can also be widely applied to the following scenarios:

[0034] Scenario 1: 3D clothing (such as garments) generation and modeling scenario. For example, by generating tiled images of clothing, realistic texture support can be provided for the 3D model of clothing.

[0035] Scenario 2: Clothing display scenario on e-commerce platforms, for example, restoring the clothing in an image of a person to a flat lay image to better showcase the details of the clothing.

[0036] Scenario 3: Virtual fitting room scenario. For example, through this technology, users can not only see how the clothes look on them, but also see the full picture of the clothing, thus improving the user experience.

[0037] Scenario 4: Apparel design and production scenario, for example, using the flat lay image of the restored apparel for subsequent design and production.

[0038] Specifically, Figure 1 This is an illustrative flow diagram of an image generation method according to an embodiment of this application. Figure 1 This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0039] Furthermore, the method includes at least a portion of the following: For example... Figure 1 As shown, it includes:

[0040] Step S101: Obtain the image features of the target input image.

[0041] Here, the target input image must at least contain a target object; for example, the target input image can be an image of a person wearing clothing, an image of a mannequin wearing clothing, or an image of a pet wearing clothing, etc. In other words, the aforementioned target input image can be any captured image, a video screenshot, or an image containing a virtual digital human, etc., and this disclosed solution does not impose specific limitations on it.

[0042] Furthermore, the image features include at least: the contour features of the target object (such as the contour features of clothing) and the pose features of the target object. This provides data support for subsequently identifying and extracting complete clothing details (such as patterns and textures).

[0043] Step S102: Obtain clothing features from at least one clothing feature image.

[0044] Here, the at least one clothing feature image is related to the clothing worn by the target object in the target input image; for example, each input clothing feature image is related to the clothing worn by the target object in the target input image.

[0045] For example, in one instance, the clothing feature image is an image associated with or matching the features of the clothing worn by the target object in the target input image. This provides data support for subsequently accurately identifying the details (such as patterns and textures) of the clothing worn by the target object.

[0046] Step S103: Perform feature fusion on the image features of the target input image and the clothing features (e.g., the clothing features of each input clothing feature image) to obtain a target clothing image for the clothing worn by the target object.

[0047] In this way, the present invention generates a target clothing image for the clothing worn by the target object by fusing the image features of the target input image and the clothing features of at least one clothing feature image. In this way, the clothing worn by the target object can be accurately identified from the target input image and a tiled image (such as the target clothing image) can be generated for the clothing. Furthermore, since the features are effectively fused, the generated tiled image can retain the clothing details to the greatest extent. Thus, more accurate and natural image data is provided for subsequent clothing display, design and 3D modeling.

[0048] Furthermore, since the input image of this disclosed solution has rich dimensions, such as including the target input image and clothing feature image, it is convenient to integrate input features of multiple dimensions to extract clothing features, thereby enabling this disclosed solution to process images with complex backgrounds and diverse poses.

[0049] Furthermore, this disclosed solution, on the one hand, reduces the complexity of user operations, for example, only requiring the input of the target image and a rough image of clothing features, while improving the flexibility and convenience of the "try on" operation, thereby effectively enhancing the user experience; on the other hand, because the input image has richer dimensions, it enhances the ability to handle the problem of clothing occlusion in the image, effectively avoiding the problem of missing information when extracting clothing due to occlusion, thereby improving the accuracy of clothing extraction from the image.

[0050] Furthermore, in a specific example, after fusing the image features of the target input image and the clothing features of at least one clothing feature image, the fused features can be input into the color pattern generation network module to obtain a colored target clothing image, thus further improving the user experience.

[0051] Furthermore, in a specific example, the image features of the target input image can be obtained in the following manner; specifically, obtaining the image features of the target input image as described above includes:

[0052] The target input image is input into a first network to extract features from at least the pose and body contour of the target object in the target input image, so as to obtain image features containing the contour features (e.g., the body contour features of the target object) and the pose features (e.g., the body pose features) of the target object.

[0053] For example, in one instance, this disclosure proposes a "try-off" model based on the reverse application of a virtual try-on model. This try-off model can effectively generate flat images of clothing, thereby providing more accurate and natural image data for subsequent clothing display, design, and 3D modeling.

[0054] Furthermore, in one example, the Try-off model can be specifically a diffusion model.

[0055] Furthermore, the Try-off model includes a first network (e.g., a "TryoffNet" module) that encodes the input target image and extracts high-level features such as the target object's pose, body contour, and image background from the encoded data, thereby obtaining image features containing the target object's contour and pose features. This lays the foundation for accurately generating target clothing images of the clothing worn by the target object.

[0056] Here, the TryoffNet module in this example can be implemented using a Convolutional Neural Network (CNN).

[0057] It should be noted that the input to the TryoffNet module can be any captured image, such as an everyday image, and this arbitrary image does not require preprocessing. This makes the present invention highly flexible and capable of processing images with complex backgrounds and diverse poses.

[0058] In this way, the disclosed solution can use the first network to extract detailed features related to the target object from the target input image. Thus, by accurately capturing the pose, contour and other features of the target object, it provides a reference for the subsequent generation of the target clothing image. Moreover, since the extracted image features are more granular, such as including pose and contour features, it can also effectively reduce the problem of information loss caused by clothing being occluded or deformed, thereby providing strong support for improving the accuracy and realism of clothing generation.

[0059] Furthermore, in a specific example, to overcome the problem of uneven lighting and shadows on clothing in an image due to different lighting conditions, the target input image can be pre-lit to ensure that the lighting and shadows in the target input image meet the lighting requirements, thus providing strong support for overcoming the problem of uneven lighting and shadows on clothing in an image due to different lighting conditions. Further, the pre-lit target input image is then input into the first network, thereby obtaining image features containing the contour features of the target object (e.g., the body contour features of the target object) and the pose features of the target object (e.g., body pose features).

[0060] For example, in one example, the Try-off model also includes a lighting inpainting module, which can perform lighting inpainting on the input target image, thereby facilitating the restoration of the true color and texture of clothing under different lighting conditions, and further improving the quality of clothing image generation.

[0061] In a specific example of the disclosed solution, the clothing features for obtaining at least one clothing feature image (e.g., step S102) described above can be obtained in any of the following ways:

[0062] Method 1: Obtain the garment pattern features from a two-dimensional pattern image.

[0063] Here, the two-dimensional pattern image is obtained based on the two-dimensional pattern of the clothing worn by the target object. For example, it is a pattern structure diagram of the clothing worn by the target object in the target input image, extracted using other tools (such as other models), such as a 2D pattern mask image.

[0064] Specifically, in one example, the clothing pattern features of the two-dimensional pattern image can characterize the clothing structure and outline of the clothing worn by the target object, so as to control the overall shape of the clothing in the image generation when extracting the clothing worn by the target object, so as to generate a clothing image that meets the requirements of 2D pattern, thereby effectively solving the problem that the clothing images generated by the prior art are distorted, blurry, and do not match the actual clothing images due to clothing occlusion or deformation.

[0065] Furthermore, in one example, obtaining the clothing pattern features from the two-dimensional pattern image described above can specifically include: inputting the two-dimensional pattern image into a third network to extract features from the clothing pattern outline using the third network, thereby obtaining the clothing pattern features.

[0066] For example, the Try-off model also includes a third network (e.g., a pattern processing network module). This pattern processing network module can extract features from the garment's outline, thus obtaining garment pattern features. This provides strong support for controlling the overall shape of the garment in image generation when extracting the garment worn by the target object, and also provides data support for accurately generating target garment images. Simultaneously, it provides strong support for effectively solving the problems of distortion, blurring, and mismatch with actual garment images caused by garment occlusion or deformation in existing technologies.

[0067] Method 2: Obtain the clothing detail features from the clothing mask image.

[0068] Here, the clothing mask image is obtained by masking the clothing worn by the target object in the target input image. For example, it is a rough clothing image obtained by using a simple image processing tool (cutout software) and cropping along the outline of the clothing in the target input image. In other words, the clothing mask image is a rough mask image, which may not contain precise clothing edges or details.

[0069] Here, since the clothing mask image provided by this disclosure can be a crude mask image, this disclosure further avoids cumbersome operations for users, thereby further improving the user experience.

[0070] Furthermore, in one example, obtaining the clothing detail features from the clothing mask image described above can specifically include: inputting the clothing mask image into a second network to extract features of at least one of the following aspects of the clothing using the second network, thereby obtaining clothing detail features: color, texture, and pattern. That is, inputting the clothing mask image into the second network to extract features of at least one of the clothing's color, texture, and pattern using the second network, thereby obtaining clothing detail features.

[0071] For example, the Try-off model includes a second network (e.g., a GarmentNet module) that extracts low-level details about the clothing worn by the target object from the clothing mask image, such as color, texture, and pattern. This facilitates a better understanding of the clothing details during subsequent image generation, resulting in more realistic clothing images and laying the foundation for improved user experience.

[0072] Here, in one example, the GarmentNet module described above can adopt a fully convolutional network (U-Net) architecture. This facilitates the extraction of low-level features of clothing, such as color, texture, and pattern, through the convolutional layers in the U-Net architecture. These features can effectively ensure that the clothing in the generated target clothing image is consistent in detail with the clothing in the output target input image.

[0073] Method 3: Obtain the garment pattern features from the 2D pattern image, and obtain the garment detail features from the garment mask image.

[0074] For details regarding 2D pattern images and clothing mask images, please refer to the examples above; they will not be repeated here.

[0075] For example, in one example, the Try-off model includes a TryoffNet module, a pattern processing network module, and a GarmentNet module. Through this network structure, a 2D pattern mask image can be effectively combined with a coarse clothing mask image to generate a colored clothing image of the garment under a specified 2D pattern. This effectively preserves the details of the pattern, color, and texture of the generated target clothing image under the 2D pattern, thereby further improving the user experience.

[0076] In this way, the disclosed solution can utilize the multi-dimensional features of clothing to accurately identify the clothing worn by the target object from the target input image and generate a target clothing image for that clothing. This improves the flexibility and convenience of the "try on and off" operation, thereby effectively enhancing the user experience. In addition, because the input image has richer dimensions, it enhances the ability to handle the problem of clothing occlusion in the image, effectively avoiding the problem of missing information when extracting clothing due to occlusion, thereby improving the accuracy of clothing extraction from the image.

[0077] Figure 2 This is an illustrative flow diagram of an image generation method according to an embodiment of this application. Figure 2 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figure 1 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0078] Furthermore, the method includes at least a portion of the following: For example... Figure 2 As shown, it includes:

[0079] Step S201: Obtain the image features of the target input image.

[0080] Here, the target input image contains at least a target object; the image features include at least the contour features (such as the contour features of clothing) and pose features of the target object.

[0081] It should be noted that the relevant content regarding the target input image can be found in the example above, and will not be repeated here.

[0082] Step S202: Obtain clothing features from at least one clothing feature image.

[0083] Here, the at least one clothing feature image is related to the clothing worn by the target object in the target input image. For example, each input clothing feature image is related to the clothing worn by the target object in the target input image.

[0084] It should be noted that the relevant content regarding the clothing features in the clothing feature images can be found in the examples above, and will not be repeated here.

[0085] Step S203: Perform cross-attention processing on the image features of the target input image with at least one of the clothing pattern features and clothing detail features to obtain a target clothing image for the clothing worn by the target object.

[0086] In other words, the image features of the target input image are subjected to cross-attention processing with at least one of the following to obtain a target clothing image for the clothing worn by the target object:

[0087] Clothing pattern features and clothing details.

[0088] For example, in one instance, the image features of the target input image can be cross-attentioned with the clothing detail features to obtain a target clothing image for the clothing worn by the target object. In other words, the image features and clothing detail features can be used to transfer information, thus ensuring that the details of the clothing are preserved during the generation process.

[0089] Alternatively, in another example, the image features of the target input image can be cross-attentioned with the clothing pattern features to obtain a target clothing image for the clothing worn by the target object. In other words, the image features and clothing pattern features can be used to transfer information, thus ensuring that the details and pattern of the clothing are preserved during the generation process.

[0090] Alternatively, in another example, the image features, clothing pattern features, and clothing detail features of the target input image can be cross-attention processed to obtain a target clothing image of the clothing worn by the target object. In other words, the image features, clothing pattern features, and clothing detail features can be used to transfer information, thus ensuring that the details of the generated clothing image can match the details of the clothing in the target input image. At the same time, it accurately matches the pattern shape of the clothing in the target input image (for example, accurately matching the outline and structure of the clothing worn by the target object in the target input image), thereby making the generated clothing image more accurate.

[0091] In this way, the disclosed solution can utilize the different dimensional features of clothing and use the cross-attention mechanism to generate a tiled image (i.e., a target clothing image) of the clothing worn by the target object. This effectively improves the accuracy of extracting the clothing worn by the target object from the target input image. Moreover, by using the cross-attention mechanism for information transmission, it is also possible to effectively control the details or pattern of the clothing image, thereby meeting the user's needs for clothing details or pattern and thus effectively improving the user experience.

[0092] For example, such as Figure 3As shown, in this example, the Try-off model mainly includes: the TryoffNet module, the GarmentNet module, the pattern processing network module, the Cross Attention module, and the color pattern generation network module. Furthermore, in this example, the input information includes three categories: an arbitrary target input image, a two-dimensional pattern image, and a rough clothing mask image. Further, after inputting these three types of images into the Try-off model, the TryoffNet module processes the target input image to obtain image features containing at least the contour and pose features of the target object. The pattern processing network module processes the two-dimensional pattern image to extract features of the clothing contour and structure, obtaining clothing pattern features. Finally, the GarmentNet module processes the clothing mask image to extract features using the convolutional layers in the GarmentNet module, obtaining clothing detail features.

[0093] Furthermore, the image features, clothing pattern features, and clothing detail features of the target input image are input into the cross-attention module to transfer information between the three types of features and obtain fused features. Then, a color pattern generation network module processes the fused features to generate a colored, detailed clothing image based on the specified pattern. Here, the cross-attention mechanism effectively combines the clothing detail information, pattern information, and pose and contour features from the target input image, ensuring that the generated clothing image not only maintains an accurate pattern contour but also preserves as much of the clothing detail as possible (such as color, texture, and pattern), thereby effectively improving the user experience.

[0094] It should be noted that, in order to overcome the problem of uneven lighting and shadow on clothing in the image due to different lighting conditions, the Try-off model of this disclosure can also be set up with a lighting restoration module. The lighting restoration module can be used to process the input target image, thereby facilitating the restoration of the real color and texture of clothing under different lighting conditions, and further improving the quality of clothing image generation.

[0095] Figure 4 This is a schematic diagram demonstrating the effect of applying the solution disclosed herein. In summary, compared to the prior art, the solution of this invention has the following advantages:

[0096] First, it supports multiple input data modes: Existing technologies usually require input of precise clothing images and portrait images of people in specific poses, while the present invention allows users to input any captured image, as well as a rough mask image, thus greatly improving the flexibility and convenience of operation.

[0097] Second, the information transmission of the cross-attention mechanism: In existing technologies, network modules often operate independently, resulting in low information transmission efficiency. However, the solution disclosed in this paper deeply integrates the different features proposed by different modules through a cross-attention mechanism, thereby effectively ensuring that the generated image not only meets the pattern requirements but also retains the details of the clothing.

[0098] Third, the balance between 2D pattern control and detail preservation: Existing technologies tend to lose the original details of clothing when the clothing is deformed, while the disclosed solution can use the structure of the model to ensure the consistency of the pattern while ensuring that the details of the clothing (such as color, texture and pattern) are preserved, thereby generating realistic color clothing images.

[0099] Fourth, the network model structure has been optimized: Based on the diffusion model (i.e., the try-off model), this disclosed solution adds a cross-attention mechanism for image features, clothing detail features, and clothing pattern features. This allows the generated images to better adapt to different patterns while preserving the detailed features of the clothing.

[0100] This disclosure also provides an image generation apparatus, such as... Figure 5 As shown, it includes:

[0101] The first feature extraction unit 501 is used to obtain image features of the target input image, wherein the target input image contains at least a target object, and the image features contain at least: contour features and pose features of the target object;

[0102] The second feature extraction unit 502 is used to obtain clothing features of at least one clothing feature image; wherein, the at least one clothing feature image is related to the clothing worn by the target object in the target input image;

[0103] The feature fusion unit 503 is used to fuse the image features of the target input image and the clothing features to obtain a target clothing image for the clothing worn by the target object.

[0104] In a specific example of the scheme disclosed herein, the second feature extraction unit is specifically used to perform at least one of the following:

[0105] Obtain the clothing pattern features from the two-dimensional pattern image; wherein, the two-dimensional pattern image is obtained based on the two-dimensional pattern of the clothing worn by the target object;

[0106] The clothing detail features of the clothing mask image are obtained; where the clothing mask image is obtained by masking the clothing worn by the target object in the target input image.

[0107] In a specific example of the scheme disclosed herein, the feature fusion unit is specifically used for:

[0108] The image features of the target input image are subjected to cross-attention processing with at least one of the clothing pattern features and clothing detail features.

[0109] In a specific example of the scheme disclosed herein, the first feature extraction unit is specifically used for:

[0110] The target input image is input into a first network to extract features of at least the pose and body contour of the target object in the target input image, so as to obtain image features containing the contour features and pose features of the target object.

[0111] In a specific example of the scheme disclosed herein, the first feature extraction unit is specifically used for:

[0112] The target input image is subjected to illumination restoration so that the light and shadow in the target input image meet the lighting requirements; the illumination-restored target input image is then input into the first network.

[0113] In a specific example of the scheme disclosed herein, the second feature extraction unit is specifically used for:

[0114] The clothing mask image is input into a second network to extract features from at least one of the clothing's color, texture, and pattern, thereby obtaining the clothing's detailed features.

[0115] In a specific example of the scheme disclosed herein, the second feature extraction unit is specifically used for:

[0116] The two-dimensional pattern image is input into a third network to extract features of the garment pattern outline using the third network, thereby obtaining the garment pattern features.

[0117] For a description of the specific functions and examples of each unit of the apparatus in this disclosure embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0118] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0120] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0121] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0122] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0123] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as image generation methods. For example, in some embodiments, the image generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the image generation method by any other suitable means (e.g., by means of firmware).

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0129] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0130] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image generation method, comprising: Based on the trial-off network module in the trial-off model and combined with the target input image, the image features of the target input image are obtained. The trial-off model includes three modules set in parallel: a trial-off network module, a pattern processing network module, and a clothing network module, as well as a cross-attention module set in sequence with the three parallel modules. The target input image contains at least a target object. The image features contain at least the contour features and pose features of the target object. Obtain clothing features related to the clothing worn by the target object in the target input image, including: Based on the pattern processing network module in the trial-and-off model, and combined with the two-dimensional pattern image, the clothing pattern features of the two-dimensional pattern image are obtained; wherein, the two-dimensional pattern image is obtained based on the two-dimensional pattern of the clothing worn by the target object; the clothing pattern features of the two-dimensional pattern image represent the clothing structure and outline of the clothing worn by the target object; wherein, the trial-and-off model can generate clothing images that meet the 2D pattern requirements based on the clothing structure and outline of the clothing worn by the target object in image generation; Based on the clothing network module in the trial-and-drop model, and combined with the clothing mask image, the clothing detail features of the clothing mask image are obtained; wherein, the clothing mask image is obtained by masking the clothing worn by the target object in the target input image; wherein, the trial-and-drop model can generate clothing images that match the clothing worn by the target object based on the clothing detail features in image generation. The image features of the target input image and the clothing features are fused to obtain a target clothing image for the clothing worn by the target object, including: Based on the cross-attention module in the trial-and-drop model, the image features of the target input image are cross-attention processed with the clothing pattern features and clothing detail features to obtain the target clothing image for the clothing worn by the target object.

2. The method according to claim 1, wherein, The trial-and-drop network module in the trial-and-drop model, combined with the target input image, obtains the image features of the target input image, including: The target input image is input to the trial-and-drop network module, so that the trial-and-drop network module can extract features of at least the pose and body contour of the target object in the target input image to obtain image features containing the contour features and pose features of the target object.

3. The method according to claim 2, wherein, The method further includes: The target input image is subjected to illumination restoration so that the light and shadow in the target input image meet the lighting requirements; The step of inputting the target input image into the trial-and-drop network module includes: The target input image after illumination restoration is input into the trial network module.

4. The method according to any one of claims 1-3, wherein, The clothing network module in the trial-and-drop model, combined with the clothing mask image, obtains the clothing detail features of the clothing mask image, including: The clothing mask image is input into the clothing network module, and the clothing network module is used to extract features from at least one of the clothing's color, texture, and pattern to obtain the clothing's detailed features.

5. The method according to any one of claims 1-3, wherein, The pattern processing network module based on the trial-and-wear model, combined with the two-dimensional pattern image, obtains the clothing pattern features of the two-dimensional pattern image, including: The two-dimensional pattern image is input into the pattern processing network module to extract features of the garment pattern outline using the pattern processing network module, so as to obtain the garment pattern features.

6. An image generation apparatus, comprising: The first feature extraction unit is used to obtain the image features of the target input image based on the trial-off network module in the trial-off model and in combination with the target input image. The trial-off model includes three modules set in parallel: a trial-off network module, a pattern processing network module, and a clothing network module, as well as a cross-attention module set in sequence with the three modules set in parallel. The target input image contains at least a target object. The image features contain at least the contour features and pose features of the target object. The second feature extraction unit is used to obtain clothing features related to the clothing worn by the target object in the target input image; The feature fusion unit is used to fuse the image features of the target input image and the clothing features to obtain a target clothing image for the clothing worn by the target object. The second feature extraction unit is specifically used for: Based on the pattern processing network module in the trial-and-off model, and combined with the two-dimensional pattern image, the clothing pattern features of the two-dimensional pattern image are obtained; wherein, the two-dimensional pattern image is obtained based on the two-dimensional pattern of the clothing worn by the target object; the clothing pattern features of the two-dimensional pattern image represent the clothing structure and outline of the clothing worn by the target object; wherein, the trial-and-off model can generate clothing images that meet the 2D pattern requirements based on the clothing structure and outline of the clothing worn by the target object in image generation; Based on the clothing network module in the trial-and-drop model, and combined with the clothing mask image, the clothing detail features of the clothing mask image are obtained; wherein, the clothing mask image is obtained by masking the clothing worn by the target object in the target input image; wherein, the trial-and-drop model can generate clothing images that match the clothing worn by the target object based on the clothing detail features in image generation. Specifically, the feature fusion unit is used for: Based on the cross-attention module in the trial-and-drop model, the image features of the target input image are cross-attention processed with the clothing pattern features and clothing detail features to obtain the target clothing image for the clothing worn by the target object.

7. The apparatus according to claim 6, wherein, The first feature extraction unit is specifically used for: The target input image is input to the trial-and-drop network module, so that the trial-and-drop network module can extract features of at least the pose and body contour of the target object in the target input image to obtain image features containing the contour features and pose features of the target object.

8. The apparatus according to claim 7, wherein, The first feature extraction unit is specifically used for: The target input image is subjected to illumination restoration so that the light and shadow in the target input image meet the lighting requirements; the illumination-restored target input image is then input into the trial-and-error network module.

9. The apparatus according to any one of claims 6-8, wherein, The second feature extraction unit is specifically used for: The clothing mask image is input into the clothing network module, and the clothing network module is used to extract features from at least one of the clothing's color, texture, and pattern to obtain the clothing's detailed features.

10. The apparatus according to any one of claims 6-8, wherein, The second feature extraction unit is specifically used for: The two-dimensional pattern image is input into the pattern processing network module to extract features of the garment pattern outline using the pattern processing network module, so as to obtain the garment pattern features.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images