Image generation method and device, equipment and storage medium

Abstract

The invention provides an image generation method and device, equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as virtual reality and 3D clothing generation. According to the specific implementation scheme, image features of a target input image are obtained, the target input image at least comprises a target object, and the image features at least comprise contour features and posture features of the target object; obtaining costume features of the at least one costume feature image; wherein the at least one clothing feature image is related to clothing worn by a target object in the target input image; and carrying out feature fusion on the image features of the target input image and the costume features to obtain a target costume image for the costume worn by the target object.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the technical fields of computer vision, deep learning, large models, etc., and can be applied to scenes such as virtual reality and three-dimensional (3D) clothing generation. Background Art

[0002] With the development of technology, virtual digital human technology is developing rapidly and showing its strong application potential in many fields. With the continuous advancement of technology and the deepening of application, virtual digital humans are expected to play a role in more fields and more scenarios, becoming an important part of future digital life. Based on this, the research on virtual digital humans has become a hot topic. Summary of the invention

[0003] The present disclosure provides an image generation method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image generation method, comprising:

[0005] Obtaining image features of a target input image, wherein the target input image at least includes a target object, and the image features at least include: contour features and posture features of the target object;

[0006] Obtaining clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by a target object in the target input image;

[0007] The image features of the target input image and the clothing features are feature fused to obtain a target clothing image corresponding to the clothing worn by the target object.

[0008] According to another aspect of the present disclosure, there is provided an image generating device, comprising:

[0009] A first feature extraction unit is used to obtain image features of a target input image, wherein the target input image at least includes a target object, and the image features at least include: contour features and posture features of the target object;

[0010] a second feature extraction unit, configured to obtain clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by a target object in the target input image;

[0011] The feature fusion unit is used to perform feature fusion on the image features of the target input image and the clothing features to obtain a target clothing image for the clothing worn by the target object.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0018] The disclosed solution generates a target clothing image for clothing worn by a target object by fusing image features of a target input image and clothing features of at least one clothing feature image. In this way, the clothing worn by the target object can be accurately identified from the target input image, and a tiled image for the clothing (such as a target clothing image) can be generated. Furthermore, since the features can be effectively fused, the generated tiled image can retain clothing details to the greatest extent. In this way, more accurate and natural image data is provided for subsequent display, design, and 3D modeling of clothing.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0021] Figure 1 This is a schematic flow chart of an image generation method according to an embodiment of the present application. Figure 1 ;

[0022] Figure 2 This is a schematic flow chart of an image generation method according to an embodiment of the present application. Figure 2 ;

[0023] Figure 3 is a schematic diagram of a model structure in a specific example of an image generation method according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of the effect after applying the image generation method according to an embodiment of the present application;

[0025] Figure 5 is a structural schematic diagram of an image generating device according to an embodiment of the present application;

[0026] Figure 6 It is a block diagram of an electronic device used to implement the image generating method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0029] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0030] In the field of virtual clothing try-on, a lot of research and applications have been devoted to combining clothing images with human images to show the effect of trying on. However, current technologies mainly focus on the "try-on" function, that is, how to virtually "wear" clothing on a person. Relatively speaking, how to "take off" clothing worn on a person and restore the appearance of the clothing in a flat form is a problem that has not been fully studied.

[0031] In practical applications, attempts to accurately extract the original appearance of clothing or headdresses from images of people wearing them often encounter problems such as inaccurate image segmentation, partial occlusion of clothing, and uneven lighting. These problems hinder high-quality clothing display and its further application in the fields of 3D clothing generation, modeling, and design.

[0032] Based on this, the disclosed scheme provides an image generation method, which can accurately separate the clothing worn by the target object in the image, such as tops, bottoms or headwear, from any taken image, and generate a complete tiled image of the clothing. In this way, the problem of difficulty in extracting complete clothing images due to the low segmentation accuracy of traditional segmentation technology is effectively overcome. At the same time, the extraction accuracy of the clothing image worn by the target object is improved, providing more accurate and natural image data for subsequent clothing display, design and 3D modeling.

[0033] Furthermore, the disclosed solution can also be widely applied to the following scenarios:

[0034] Scene 1: 3D clothing (such as clothing) generation and modeling scene. For example, by generating a tiled image of clothing, real texture support can be provided for the three-dimensional model of clothing.

[0035] Scenario 2: Clothing display scenario on e-commerce platforms. For example, the clothing in an image worn by a person can be restored to a tiled image to better display clothing details.

[0036] Scene 3: Virtual fitting room scene. For example, through this technology, users can not only see the effect of wearing the clothes on their bodies, but also see the overall appearance of the clothes, which improves the user experience.

[0037] Scenario 4: Clothing design and production scenario, for example, using the restored tiled images of clothing for subsequent design and production.

[0038] Specifically, Figure 1 This is a schematic flow chart of an image generation method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0039] Further, the method includes at least part of the following contents. Figure 1 As shown, including:

[0040] Step S101: Obtain image features of a target input image.

[0041] Here, the target input image at least includes the target object; for example, the target input image may be an image of a person wearing clothing, an image of a mannequin wearing clothing, or an image of a pet wearing clothing, etc. In other words, the target input image may be any captured image, or a video screenshot, or an image including a virtual digital person, etc., and the disclosed solution does not impose any specific restrictions on this.

[0042] Furthermore, the image features at least include: contour features of the target object (such as contour features of clothing) and posture features of the target object, thus providing data support for subsequent recognition and extraction of complete clothing details (such as patterns and textures).

[0043] Step S102: obtaining clothing features of at least one clothing feature image.

[0044] Here, the at least one clothing feature image is related to the clothing worn by the target object in the target input image; for example, each input clothing feature image is related to the clothing worn by the target object in the target input image.

[0045] For example, in one example, the clothing feature image is an image associated with or matched with the features of the clothing worn by the target object in the target input image, thus providing data support for subsequent accurate identification of clothing details (such as patterns and textures) of the clothing worn by the target object.

[0046] Step S103: feature fusion of the image features of the target input image and the clothing features (eg, clothing features of each input clothing feature image) to obtain a target clothing image corresponding to the clothing worn by the target object.

[0047] In this way, the disclosed scheme generates a target clothing image for the clothing worn by the target object by fusing the image features of the target input image and the clothing features of at least one clothing feature image. In this way, the clothing worn by the target object can be accurately identified from the target input image, and a tiled image for the clothing (such as the target clothing image) can be generated. Moreover, since the features can be effectively fused, the generated tiled image can retain the clothing details to the greatest extent. In this way, more accurate and natural image data is provided for subsequent clothing display, design, and 3D modeling.

[0048] Furthermore, since the input image of the disclosed solution has rich dimensions, for example, including a target input image and a clothing feature image, it is convenient to integrate input features of multiple dimensions to extract clothing features, thereby enabling the disclosed solution to process images with complex backgrounds and diverse postures.

[0049] Furthermore, the disclosed solution, on the one hand, reduces the complexity of user operations. For example, only the target input image and a rough clothing feature image need to be input. At the same time, the flexibility and convenience of the "try-on" operation are improved, thereby effectively improving the user experience. On the other hand, since the dimensions of the input image are richer, the ability to handle occlusion problems of clothing in the image is enhanced, effectively avoiding the problem of missing information when extracting clothing due to occlusion of clothing, thereby improving the accuracy of clothing extraction in the image.

[0050] Furthermore, in a specific example, after the image features of the target input image and the clothing features of at least one clothing feature image are fused, the fused features obtained after the feature fusion can also be input into the color pattern generation network module to obtain a colored target clothing image, thereby further improving the user experience.

[0051] Furthermore, in a specific example, the image features of the target input image may be obtained in the following manner; specifically, the above-mentioned obtaining of the image features of the target input image specifically includes:

[0052] The target input image is input into a first network, so as to use the first network to perform feature extraction on at least the posture and body contour of the target object in the target input image, so as to obtain image features including contour features of the target object (e.g., body contour features of the target object) and posture features of the target object (e.g., body posture features).

[0053] For example, in one example, the disclosed solution proposes a "try-off" model based on the reverse application of a virtual try-on (Try-on) model. The try-off model can effectively generate a tiled image of clothing, thereby providing more accurate and natural image data for subsequent clothing display, design, and 3D modeling.

[0054] Furthermore, in one example, the try-off model may be specifically a diffusion model.

[0055] Furthermore, the try-off model includes a first network (e.g., a "try-off" network (TryoffNet) module), which can encode the input target input image and extract high-level features such as the target object's posture, body contour, and image background from the encoded data, thereby obtaining image features including the contour features and posture features of the target object. This lays the foundation for the subsequent accurate generation of a target clothing image for the clothing worn by the target object.

[0056] Here, the TryoffNet module in this example can be implemented using a convolutional neural network (CNN).

[0057] Here, it should be noted that the input of the TryoffNet module can be any captured image, such as an image of daily life, and the arbitrary image does not need to be preprocessed. This makes the disclosed solution extremely flexible and capable of processing images with complex backgrounds and diverse postures.

[0058] In this way, the disclosed solution can utilize the first network to extract detailed features related to the target object from the target input image, thereby providing a reference basis for the subsequent generation of the target clothing image by accurately capturing the posture, contour and other features of the target object; moreover, since the granularity of the extracted image features is finer, for example, including posture, contour and other features, it can also effectively reduce the problem of information loss caused by occlusion or deformation of clothing, thereby providing strong support for the subsequent improvement of the accuracy and authenticity of clothing generation.

[0059] Furthermore, in a specific example, in order to overcome the problem of uneven light and shadow of clothing in the image due to different lighting conditions, before inputting the target input image into the first network, the target input image may be firstly subjected to light restoration, so that the light and shadow in the target input image meets the light and shadow requirements, thereby providing strong support for overcoming the problem of uneven light and shadow of clothing in the image due to different lighting conditions. Furthermore, the target input image after light restoration is input into the first network, so as to obtain image features including the contour features of the target object (for example, the body contour features of the target object) and the posture features of the target object (for example, the body posture features).

[0060] For example, in one example, the try-off model also includes a lighting restoration module, which can perform lighting restoration on the input target input image, thereby facilitating the restoration of the true clothing color and texture under different lighting conditions, further improving the quality of clothing image generation.

[0061] In a specific example of the disclosed solution, the above-mentioned obtaining of clothing features of at least one clothing feature image (for example, step S102) can be obtained specifically by any of the following methods:

[0062] Method 1: Obtain clothing pattern features of a two-dimensional pattern image.

[0063] Here, the two-dimensional pattern image is obtained based on the two-dimensional pattern of the clothing worn by the target object. For example, it is a pattern structure diagram of the clothing worn by the target object in the target input image extracted using other tools (for example, other models), such as a 2D pattern mask diagram.

[0064] Specifically, in one example, the clothing pattern features of the two-dimensional pattern image can characterize the clothing structure and clothing outline of the clothing worn by the target object, so that when extracting the clothing worn by the target object, the overall shape of the clothing in image generation can be controlled to generate a clothing image that meets the 2D pattern requirements, thereby effectively solving the problem of distortion, blurring, and mismatch with the actual clothing image caused by clothing occlusion or deformation in the prior art.

[0065] Furthermore, in one example, the above-mentioned obtaining clothing pattern features of the two-dimensional pattern image may specifically include: inputting the two-dimensional pattern image into a third network, so as to use the third network to extract features of the pattern contour of the clothing to obtain the clothing pattern features.

[0066] For example, the try-off model also includes a third network (e.g., a pattern processing network module). Furthermore, the pattern processing network module can extract features of the pattern outline of the clothing, and then obtain clothing pattern features. This provides strong support for controlling the overall shape of the clothing in the image generation when extracting the clothing worn by the target object, and also provides data support for the subsequent accurate generation of the target clothing image for the clothing worn by the target object. At the same time, it also provides strong support for effectively solving the problem of distortion, blurring, and mismatching of clothing images generated by the existing technology due to clothing occlusion or deformation.

[0067] Method 2: Obtain clothing detail features of clothing mask image.

[0068] Here, the clothing mask image is obtained by masking the clothing worn by the target object in the target input image. For example, it is a rough clothing image obtained by using a simple image processing tool (cutout software) and cutting out along the outline of the clothing in the target input image. In other words, the clothing mask image is a rough mask image, and the clothing mask image may not contain precise clothing edges or details.

[0069] Here, since the clothing mask image provided by the disclosed solution can be a crude mask image, the disclosed solution further avoids the user's tedious operations, thereby further improving the user experience.

[0070] Further, in an example, the above-mentioned obtaining clothing detail features of the clothing mask image may specifically include: inputting the clothing mask image into the second network, so as to use the second network to extract features of at least one of the following clothing to obtain clothing detail features: color, texture, pattern. That is, inputting the clothing mask image into the second network, so as to use the second network to extract features of at least one of the color, texture and pattern of the clothing to obtain clothing detail features.

[0071] For example, the try-off model includes a second network (e.g., a GarmentNet module), which can extract low-level detail features of the clothing worn by the target object from the clothing mask image, such as color, texture, pattern, etc. This facilitates a better understanding of the details of the clothing during subsequent image generation, making the generated clothing image more realistic, thus laying the foundation for improving the user experience.

[0072] Here, in one example, the GarmentNet module described above can adopt a U-Net (fully convolutional network) architecture, so that it is convenient to extract low-level features of clothing, such as color, texture, and pattern, through the convolutional layer in the U-Net architecture. These features can effectively ensure that the clothing of the generated target clothing image is consistent with the clothing of the output target input image in details.

[0073] Method three: obtaining clothing pattern features of a two-dimensional pattern image, and obtaining clothing detail features of a clothing mask image.

[0074] Here, for the relevant contents of the two-dimensional pattern image and the clothing mask image, please refer to the above examples, which will not be repeated here.

[0075] For example, in one example, the Try-off model includes a TryoffNet module, a pattern processing network module, and a GarmentNet module. Thus, through the above network structure, the 2D pattern mask image can be effectively combined with the rough clothing mask image to generate a color clothing image of the clothing under the specified 2D pattern. In this way, the details of the pattern, color, and texture of the generated target clothing image under the 2D pattern are effectively retained, thereby further improving the user experience.

[0076] In this way, the disclosed solution can utilize the multi-dimensional features of clothing to accurately identify the clothing worn by the target object from the target input image, and generate a target clothing image for the clothing, thereby improving the flexibility and convenience of the "try-on" operation, thereby effectively improving the user experience; in addition, since the dimensions of the input image are richer, the ability to handle occlusion problems of clothing in the image is enhanced, effectively avoiding the problem of missing information on clothing during extraction due to occlusion of clothing, thereby improving the accuracy of clothing extraction in the image.

[0077] Figure 2 This is a schematic flow chart of an image generation method according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0078] Further, the method includes at least part of the following contents. Figure 2 As shown, including:

[0079] Step S201: Obtain image features of a target input image.

[0080] Here, the target input image at least includes a target object; the image features at least include: contour features (such as contour features of clothing) and posture features of the target object.

[0081] It should be noted that, for the relevant content of the target input image, please refer to the above example, which will not be repeated here.

[0082] Step S202: obtaining clothing features of at least one clothing feature image.

[0083] Here, the at least one clothing feature image is related to the clothing worn by the target object in the target input image. For example, each input clothing feature image is related to the clothing worn by the target object in the target input image.

[0084] It should be noted that, for the content related to the clothing features of the clothing feature image, reference may be made to the above examples, which will not be repeated here.

[0085] Step S203: cross-attention processing is performed on the image features of the target input image and at least one of the clothing pattern features and clothing detail features to obtain a target clothing image for the clothing worn by the target object.

[0086] That is, the image features of the target input image are cross-attention processed with at least one of the following to obtain a target clothing image for the clothing worn by the target object:

[0087] Clothing pattern characteristics and clothing detail characteristics.

[0088] For instance, in one example, the image features of the target input image and the clothing detail features may be cross-attention processed to obtain a target clothing image for the clothing worn by the target object. In other words, the image features and the clothing detail features may be transferred to ensure that the clothing details are retained during the generation process.

[0089] Or, for example, in another example, the image features of the target input image and the clothing pattern features can be cross-attention processed to obtain a target clothing image for the clothing worn by the target object. In other words, the image features and the clothing pattern features can be transferred to ensure that the details and pattern of the clothing are retained during the generation process.

[0090] Alternatively, in another example, the image features, clothing pattern features, and clothing detail features of the target input image may be cross-attended to obtain a target clothing image of the clothing worn by the target object. In other words, the image features, clothing pattern features, and clothing detail features may be information-transmitted, thereby ensuring that the details of the generated clothing image can match the details of the clothing in the target input image, and at the same time, accurately match the pattern shape of the clothing in the target input image (for example, accurately match the outline and structure of the clothing worn by the target object in the target input image), thereby making the generated clothing image more accurate.

[0091] In this way, the disclosed scheme can utilize the different dimensional features of clothing and, with the help of the cross-attention mechanism, generate a tiled image of the clothing worn by the target object (i.e., the target clothing image). In this way, the extraction accuracy of the clothing worn by the target object in the target input image is effectively improved. Moreover, by transmitting information through the cross-attention mechanism, the details or style of the clothing image can be effectively controlled, thereby meeting the user's needs for clothing details or style, thereby effectively improving the user experience.

[0092] For example, if Figure 3As shown, in this example, the Try-off model mainly includes: a TryoffNet module, a GarmentNet module, a pattern processing network module, a cross attention module and a color pattern generation network module; further, in this example, the input information includes three categories, namely, an arbitrary target input image, a two-dimensional pattern image and a rough clothing mask image; further, after the above three types of images are input into the Try-off model, the target input image is processed by the TryoffNet module to obtain image features that at least include contour features and posture features of the target object, the two-dimensional pattern image is processed by the pattern processing network module to extract features of the clothing contour and clothing structure of the clothing to obtain clothing pattern features, and the clothing mask image is processed by the GarmentNet module to extract features of the clothing mask image by using the convolutional layer in the GarmentNet module to obtain clothing detail features.

[0093] Furthermore, the image features, clothing pattern features, and clothing detail features of the target input image are input into the Cross Attention module to transfer information of the above three types of features and obtain the fused features after fusion; further, the fused features after fusion are processed by the color pattern generation network module to generate a colorful clothing image with complete details based on the specified pattern. Here, through the cross attention mechanism, the clothing detail information, pattern information, and the posture and contour features in the target input image are effectively combined, so that the generated clothing image not only maintains the accurate pattern outline, but also retains the details of the clothing (such as color, texture, and pattern, etc.) as much as possible, thereby effectively improving the user experience.

[0094] It should be noted that in order to overcome the problem of uneven light and shadow of clothing in the image due to different lighting conditions, a lighting restoration module can also be set in the try-off model of the disclosed solution to use the lighting restoration module to process the input target input image, thereby facilitating the restoration of the real clothing color and texture under different lighting conditions, thereby further improving the quality of clothing image generation.

[0095] Figure 4 The schematic diagram is a diagram showing the effect after applying the disclosed solution. In summary, compared with the prior art, the solution of the present invention has the following advantages:

[0096] First, it supports multiple input data modes: the existing technology usually requires the input of accurate clothing images and portrait images of specific postures, while the disclosed solution allows users to input any captured images and rough mask images, which greatly improves the flexibility and convenience of the operation.

[0097] Second, information transmission of the cross-attention mechanism: The network modules in the prior art often operate independently, and the information transmission efficiency is not high. However, the disclosed solution deeply integrates the different features proposed by different modules through the cross-attention mechanism, thus effectively ensuring that the generated image meets the pattern requirements while retaining the details of the clothing.

[0098] Third, the balance between 2D pattern control and detail retention: the existing technology easily loses the original details of the clothing when the clothing is deformed, while the disclosed solution can use the structure of the model to ensure the consistency of the pattern while ensuring that the details of the clothing (such as color, texture and pattern) are retained, thereby generating realistic color clothing pictures.

[0099] Fourth, the network model structure is optimized: Based on the diffusion model (also known as the Try-off model), the disclosed solution adds a cross-attention mechanism for image features, clothing detail features, and clothing pattern features, so that the generated images can better adapt to different patterns while retaining the detailed features of the clothing.

[0100] The disclosed solution also provides an image generating device, such as Figure 5 As shown, including:

[0101] A first feature extraction unit 501 is used to obtain image features of a target input image, wherein the target input image at least includes a target object, and the image features at least include: contour features and posture features of the target object;

[0102] A second feature extraction unit 502 is used to obtain clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by the target object in the target input image;

[0103] The feature fusion unit 503 is used to perform feature fusion on the image features of the target input image and the clothing features to obtain a target clothing image corresponding to the clothing worn by the target object.

[0104] In a specific example of the disclosed solution, the second feature extraction unit is specifically configured to perform at least one of the following:

[0105] Obtaining clothing pattern features of a two-dimensional pattern image; wherein the two-dimensional pattern image is obtained based on the two-dimensional pattern of clothing worn by the target object;

[0106] The clothing detail features of the clothing mask image are obtained; wherein the clothing mask image is obtained by masking the clothing worn by the target object in the target input image.

[0107] In a specific example of the disclosed solution, the feature fusion unit is specifically used to:

[0108] The image features of the target input image are cross-attention processed with at least one of the clothing pattern features and the clothing detail features.

[0109] In a specific example of the disclosed solution, the first feature extraction unit is specifically used to:

[0110] The target input image is input into the first network, so as to use the first network to extract features of at least the posture and body contour of the target object in the target input image, so as to obtain image features including contour features and posture features of the target object.

[0111] In a specific example of the disclosed solution, the first feature extraction unit is specifically used to:

[0112] Performing lighting restoration on the target input image so that the light and shadow in the target input image meet the lighting and shadow requirements; and inputting the target input image after lighting restoration into the first network.

[0113] In a specific example of the disclosed solution, the second feature extraction unit is specifically used to:

[0114] The clothing mask image is input into the second network, so as to use the second network to extract features of at least one of the color, texture and pattern of the clothing to obtain clothing detail features.

[0115] In a specific example of the disclosed solution, the second feature extraction unit is specifically used to:

[0116] The two-dimensional pattern image is input into the third network, so as to use the third network to extract features of the pattern contour of the clothing to obtain clothing pattern features.

[0117] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0118] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0120] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0121] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0123] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as image generation methods. For example, in some embodiments, the image generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the image generation method in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0129] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating an image, comprising: Obtaining image features of a target input image, wherein the target input image at least includes a target object, and the image features at least include: contour features and posture features of the target object; Obtaining clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by a target object in the target input image; The image features of the target input image and the clothing features are feature fused to obtain a target clothing image corresponding to the clothing worn by the target object.

2. The method according to claim 1, wherein: The obtaining of clothing features of at least one clothing feature image includes at least one of the following: Obtaining clothing pattern features of a two-dimensional pattern image; wherein the two-dimensional pattern image is obtained based on the two-dimensional pattern of clothing worn by the target object; The clothing detail features of the clothing mask image are obtained; wherein the clothing mask image is obtained by masking the clothing worn by the target object in the target input image.

3. The method according to claim 2, wherein: The step of fusing the image features of the target input image and the clothing features comprises: The image features of the target input image are cross-attention processed with at least one of the clothing pattern features and the clothing detail features.

4. The method according to claim 2 or 3, wherein: The step of obtaining the image features of the target input image includes: The target input image is input into the first network, so as to use the first network to extract features of at least the posture and body contour of the target object in the target input image, so as to obtain image features including contour features and posture features of the target object.

5. The method according to claim 4, wherein: The method further comprises: Performing lighting restoration on the target input image so that the light and shadow in the target input image meets the light and shadow requirements; Wherein, inputting the target input image into the first network includes: The target input image after illumination restoration is input to the first network.

6. The method according to claim 2 or 3, wherein: The clothing detail features of the clothing mask image are obtained, including: The clothing mask image is input into the second network, so as to use the second network to extract features of at least one of the color, texture and pattern of the clothing to obtain clothing detail features.

7. The method according to claim 2 or 3, wherein: The clothing pattern features of the two-dimensional pattern image are obtained, including: The two-dimensional pattern image is input into the third network, so as to use the third network to extract features of the pattern contour of the clothing to obtain clothing pattern features.

8. An image generating device, comprising: A first feature extraction unit is used to obtain image features of a target input image, wherein the target input image at least includes a target object, and the image features at least include: contour features and posture features of the target object; a second feature extraction unit, configured to obtain clothing features of at least one clothing feature image; wherein the at least one clothing feature image is related to clothing worn by a target object in the target input image; The feature fusion unit is used to perform feature fusion on the image features of the target input image and the clothing features to obtain a target clothing image for the clothing worn by the target object.

9. The device according to claim 8, wherein: The second feature extraction unit is specifically configured to perform at least one of the following: Obtaining clothing pattern features of a two-dimensional pattern image; wherein the two-dimensional pattern image is obtained based on the two-dimensional pattern of clothing worn by the target object; The clothing detail features of the clothing mask image are obtained; wherein the clothing mask image is obtained by masking the clothing worn by the target object in the target input image.

10. The device according to claim 9, wherein: The feature fusion unit is specifically used for: The image features of the target input image are cross-attention processed with at least one of the clothing pattern features and the clothing detail features.

11. The device according to claim 9 or 10, wherein: The first feature extraction unit is specifically used to: The target input image is input into the first network, so as to use the first network to extract features of at least the posture and body contour of the target object in the target input image, so as to obtain image features including contour features and posture features of the target object.

12. The device according to claim 11, wherein The first feature extraction unit is specifically used to: Performing lighting restoration on the target input image so that the light and shadow in the target input image meet the lighting and shadow requirements; and inputting the target input image after lighting restoration into the first network.

13. The device according to claim 9 or 10, wherein: The second feature extraction unit is specifically used to: The clothing mask image is input into the second network, so as to use the second network to extract features of at least one of the color, texture and pattern of the clothing to obtain clothing detail features.

14. The device according to claim 9 or 10, wherein: The second feature extraction unit is specifically used to: The two-dimensional pattern image is input into the third network, so as to use the third network to extract features of the pattern contour of the clothing to obtain clothing pattern features.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images