Virtual fitting method and device, model training method and device and electronic equipment

Abstract

The invention discloses a virtual try-on method, a model training method and device, electronic equipment and a computer readable storage medium. The virtual try-on method comprises the steps of obtaining a human body image to be subjected to virtual try-on and a target clothing image; expanding a region corresponding to the to-be-fitted clothing part in the human body image to a non-fitting clothing part to obtain a fitting region for fitting the target clothing in the human body image; and on the basis of a principle of keeping the tried-on target clothing in the original clothing model, trying on the target clothing in the target clothing image into the try-on area, and repairing a gap area in which the target clothing is not tried on in the try-on area according to a non-try-on clothing part to obtain a virtual try-on image. Through the scheme provided by the invention, the generated virtual try-on image can better maintain the original version of the target clothing, so that the virtual try-on better displays the original effect of the target clothing, and the virtual try-on effect is improved.

Description

Technical Field

[0001] The present application relates to the technical field of virtual try-on, and in particular to a virtual try-on method, a model training method, a device, an electronic device and a computer-readable storage medium. Background Art

[0002] In today's digital age, e-commerce platforms are rapidly becoming the preferred channel for consumers to shop. This trend has created an urgent need for online shopping experience technology. For apparel products, virtual try-on technology has attracted much attention because it allows customers to preview the effects of clothing without actually wearing them.

[0003] In the related art, when performing virtual try-on, the clothing area that needs to be replaced in the user image uploaded by the user is usually replaced with the target clothing that needs to be tried on.

[0004] However, since the clothing styles of the original clothing in the user image and the target clothing to be tried on are usually different, for example, the original clothing is a long top, and the style of the target clothing is a short top, when the target clothing is replaced into the clothing area to be replaced in the user image, due to the different styles of the two, in order to make the generated virtual try-on image more complete and natural, related technologies usually perform deformation such as stretching or compressing on the target clothing to replace the original clothing area in the user image. The virtual try-on image obtained in this way may change the overall style of the target clothing during the virtual try-on because the target clothing has been deformed, so that the virtual try-on cannot well display the original effect of the target clothing, and the virtual try-on effect is poor. Summary of the invention

[0005] The present application provides a virtual try-on method, a model training method, an apparatus, an electronic device, and a computer-readable storage medium, which can enable the generated virtual try-on image to better maintain the original pattern of the target clothing, so that the virtual try-on can better show the original effect of the target clothing, thereby improving the virtual try-on effect. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a virtual try-on method, the method comprising:

[0007] Acquire a human body image to be virtually tried on and a target clothing image;

[0008] Expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part to obtain a try-on area in the human body image for trying on the target clothing;

[0009] Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image.

[0010] In a second aspect, the present application provides a virtual try-on device, the device comprising:

[0011] An information acquisition unit, used to acquire an image of a human body to be virtually tried on and an image of a target garment;

[0012] An area determination unit, used for expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part, to obtain a try-on area for trying on the target clothing in the human body image;

[0013] The try-on generation unit is used to try on the target clothing in the target clothing image in the try-on area based on the principle of keeping the original clothing pattern of the target clothing to be tried on, and to repair the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to obtain a virtual try-on image.

[0014] In a third aspect, the present application further provides an electronic device comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor; when the processor executes the computer program instructions, the method as described in any one of the first aspects is implemented.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any of the methods described in the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0017] Compared with the prior art, this application has the following advantages:

[0018] The virtual try-on method provided in the embodiment of the present application obtains a human body image to be virtually tried on and a target clothing image, then expands the area corresponding to the clothing part to be tried on in the human body image toward the non-try-on clothing part, and obtains a try-on area in the human body image for trying on the target clothing. In this way, the obtained try-on area is expanded toward the non-try-on clothing part compared to the clothing part to be tried on in the human body image. For example, when the clothing part to be tried on is an upper garment, the try-on area is expanded downward on the basis of the upper garment area of ​​the human body image, that is, the try-on area is larger than the area of ​​the clothing part to be tried on in the human body image. Then, based on the principle of keeping the original clothing version of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area. Since the try-on area is expanded toward the non-try-on clothing part, the try-on area can provide a reference for trying on the target clothing. A larger try-on space is provided so that when trying on the target clothing in the try-on area, the original shape of the target clothing can be better maintained without longitudinally stretching or compressing the target clothing. That is to say, when trying on the target clothing, the try-on area is large, so the target clothing is not deformed in shape, but the original shape of the target clothing is maintained, and then the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part. Since the try-on area is extended to the non-try-on clothing part, the try-on area is often larger than the target clothing, so that after trying on the target clothing in the try-on area, there may be a gap area between the target clothing tried on in the try-on area and the non-try-on clothing part in the human body image, and the gap area needs to be repaired to make the try-on image more natural. Therefore, after repairing the gap area according to the non-try-on clothing part in the human body image, the virtual try-on image can be made more natural.

[0019] It can be seen that the virtual try-on method provided by the present application can make the generated virtual try-on image better maintain the original shape of the target clothing by trying on the target clothing in the extended try-on area, without stretching the target clothing during the try-on but keeping the original shape of the target clothing, so that the virtual try-on can better show the original effect of the target clothing, thereby improving the virtual try-on effect, and repairing the gap area between the target clothing and the untried clothing part in the human body image through the content in the human body image, so that the virtual try-on image can be more natural and complete, thereby ensuring the overall try-on effect of the virtual try-on image.

[0020] When training a virtual try-on model using the model training method provided in the present application, since the try-on area of ​​the sample mask in the training sample is the area obtained by extending the area corresponding to the part of the clothing to be tried on in the sample human body image to the part of the clothing not to be tried on, and since the sample target clothing tried on in the sample try-on image maintains the original style, the virtual try-on model obtained by subsequently adjusting the parameters of the model to be trained according to the difference between the output result and the sample try-on image can generate a virtual try-on image that better maintains the original style of the target clothing and is more natural and complete. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a rendering of the try-on image generated by relevant technology;

[0022] Figure 2 It is a schematic diagram of the application scenario of the virtual try-on solution provided by this application;

[0023] Figure 3 It is a flowchart of an example of a virtual try-on method provided in an embodiment of the present application;

[0024] Figure 4 is an example diagram for determining a try-on area in an embodiment of the present application;

[0025] Figure 5 It is a comparison diagram of a virtual try-on image generated by the virtual try-on method provided in the embodiment of the present application and a virtual try-on image generated by a related technology;

[0026] Figure 6 It is a flowchart of an example of a model training method provided in an embodiment of the present application;

[0027] Figure 7 It is a flowchart of another example of the model training method provided in the embodiment of the present application;

[0028] Figure 8 It is a structural block diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described clearly and completely below in conjunction with the drawings in the embodiments of the present application. However, the present application can be implemented in many other ways different from the following description. Therefore, based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "source domain", "third", etc. in the claims, description and drawings of the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. The data used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having" and their variants are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In order to facilitate understanding of the various embodiments of the present application, the application background of the embodiments is described.

[0032] In today's digital age, e-commerce platforms are rapidly becoming the preferred channel for consumers to shop. This trend has created an urgent need for online shopping experience technology. For apparel products, virtual try-on technology has attracted much attention because it allows customers to preview the effects of clothing without actually wearing them.

[0033] In the related art, when performing virtual try-on, the clothing area that needs to be replaced in the user image uploaded by the user is usually replaced with the target clothing that needs to be tried on.

[0034] However, since the clothing styles of the original clothing in the user image and the target clothing to be tried on are usually different, the original clothing in the human body image is a long top, and the style of the target clothing is a short top. When the target clothing is replaced into the clothing area to be replaced in the user image, due to the different styles of the two, in order to make the generated virtual try-on image more complete and natural, related technologies usually perform deformation such as stretching or compression on the target clothing to replace the original clothing area in the user image. The virtual try-on image obtained in this way may change the overall style of the target clothing during the virtual try-on because the target clothing has been deformed, so that the virtual try-on cannot well display the original effect of the target clothing, and the virtual try-on effect is poor.

[0035] For example, Figure 1 As shown in the figure, the clothing originally worn in the human body image is a long top, and the target clothing is a short top. In the virtual fitting image obtained by the solution in the related art, the short target clothing is lengthened to obtain Figure 1 The virtual try-on effect shown in Figure 1 It can be seen that this method of related technology changes the original short version of the target clothing into a long version, resulting in the virtual try-on image not accurately showing the original version of the target clothing, and the try-on effect is not good.

[0036] In order to solve the above problems, the embodiments of the present application provide a virtual try-on method, a model training method, an apparatus, an electronic device and a computer-readable storage medium, which are intended to enable the generated virtual try-on image to better maintain the original pattern of the target clothing, so that the virtual try-on can better show the original effect of the target clothing, thereby improving the virtual try-on effect.

[0037] The virtual try-on method provided in this application can be used to generate virtual try-on images, for example, to generate virtual try-on images of human images trying on target clothing based on human images taken and uploaded by users. The solution provided in this application can be used to try on clothing for sale on e-commerce platforms, and can also be used for virtual try-ons in other scenarios, such as video production platforms, virtual game platforms, clothing design platforms, etc. This application does not specifically limit the specific application scenarios for generating virtual try-on videos.

[0038] In order to facilitate understanding of the method embodiments of the present application, the application scenarios are introduced. Figure 2 , Figure 2 Schematic diagram of the application scenario of the solution provided in the embodiment of the present application. For the convenience of description, the following describes the embodiment of the present application by taking the e-commerce platform as an example. The application scenario is a schematic illustration and is not intended to be a specific description of the application scenario. Figure 2 As shown, in this application scenario, a server 102 and a client 101 are provided. In this embodiment, a connection is established between the client 101 and the server 102 via network communication, so as to perform data transmission.

[0039] The client 101 can be an electronic device with display and data processing functions, such as a mobile phone, a tablet computer (pad), a smart watch, a desktop computer, a smart TV, a VR device, a vehicle-mounted device, a wearable device, a laptop computer, etc. The client 101 is used to send the human body image uploaded by the user and the target clothing image selected by the user to the server 102, and obtain and display the virtual try-on image generated by the server 102 from the server 102.

[0040] The server 102 has high computing power. The server 102 can be a server, which has high-speed processor (central processing unit, CPU) computing power, long-term reliable operation, powerful input / output (input / output, I / O) external data throughput capacity and better scalability. The server 102 can be a single server or a server cluster. The server 102 is used to receive human body images and target clothing images from the client 101, and generate corresponding virtual try-on images based on the human body images and target clothing images. The server 102 can also provide other specific services to the client 101, such as user information access, website access, application access, etc., which are not specifically limited in this application.

[0041] The client 101 and the server 102 may communicate with each other using various communication systems, such as a wired communication system or a wireless communication system. The wireless communication system may be, for example, a global system for mobile communications (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) system, a general packet radio service (GPRS), a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, a universal mobile telecommunication system (UMTS), a worldwide interoperability for microwave access (WiMAX) communication system, a future fifth generation (5G) system or a new radio (NR), a satellite communication system, etc.

[0042] In other application scenarios, only a client may be provided, and the client is used to generate a corresponding virtual try-on image according to a human body image uploaded by a user and a target clothing image selected by the user, and display the virtual try-on image.

[0043] Embodiment 1

[0044] The first embodiment of the present application provides a virtual try-on method, which is applied to an electronic device, which may be a server, a laptop, a tablet computer, a desktop computer, a mobile phone, a smart TV, a smart hand, a VR device, a client, and other electronic devices with data processing functions.

[0045] like Figure 3 As shown, the virtual try-on method provided in the first embodiment of the present application includes the following steps S110 to S130.

[0046] Step S110: Acquire a human body image to be virtually tried on and a target clothing image.

[0047] The above-mentioned human body image can be an image containing a human body taken and uploaded by a user who needs to perform a virtual try-on, or it can be a human body image containing a model or other human body stored in an electronic device, or it can be other images containing a human body, which is not specifically limited in this application.

[0048] The target clothing image can be a clothing image selected by the user who needs to perform a virtual try-on through a selection operation, or a clothing image uploaded by the user to the electronic device, or a clothing image pre-set by the electronic device, which is not specifically limited in this application. The above-mentioned target clothing image can be an upper clothing image or a lower clothing image. The upper clothing image can include images of tops, dresses, windbreakers, etc., which are used to wear clothing on the upper body of the human body. The lower clothing image can include images of trousers or shorts, skirts, etc., which are used to wear clothing on the lower body of the human body.

[0049] In order to further ensure the fitting effect, the clothing worn in the human image and the target clothing can be the same type of clothing. For example, if the target clothing is a T-shirt, the clothing worn in the human image is also a T-shirt. If the target clothing is a windbreaker, the clothing worn in the human image is also a windbreaker. Of course, the clothing worn in the human image and the target clothing can also be different types of clothing. For example, if the target clothing is a coat, the clothing worn in the human image can be a vest. This application does not specifically limit this.

[0050] Step S120: expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part, to obtain a try-on area in the human body image for trying on the target clothing.

[0051] In the embodiment of the present application, the clothing part to be tried on in the human body image includes one clothing part, that is, the clothing part to be tried on can be an upper garment or a lower garment. When the clothing part to be tried on is an upper garment, the non-try-on clothing part is a lower garment; when the clothing part to be tried on is a lower garment, the non-try-on clothing part is an upper garment.

[0052] Specifically, the electronic device can determine the clothing part to be tried on according to the target clothing included in the target clothing image. For example, if the target clothing image is an upper clothing image, the clothing part to be tried on in the human body image is the upper clothing. If the target clothing image is a lower clothing image, the clothing part to be tried on in the human body image is the lower clothing. In actual applications, since users usually have more requirements for virtual try-on of upper clothing, the following embodiments are mainly described by taking the upper clothing part to be tried on as an example, and the virtual try-on of lower clothing can refer to the upper clothing.

[0053] In an embodiment of the present application, the electronic device may first determine the area corresponding to the part of the clothing to be tried on in the human body image, and then expand the area to the non-try-on clothing part. Specifically, the area corresponding to the part of the clothing to be tried on in the human body image may be determined by a pre-trained area recognition model, or the area corresponding to the part of the clothing to be tried on in the human body image may be determined by other methods, such as detecting the edge corresponding to the part of the clothing to be tried on in the human body image by an edge detection method, and then determining the area corresponding to the part of the clothing to be tried on based on the edge, or, by using a region generation algorithm to divide pixels with similar attributes in the human body image into the same area, and determining the area corresponding to the part of the clothing to be tried on based on the divided areas. The present application does not limit the specific method of determining the area corresponding to the part of the clothing to be tried on in the human body image.

[0054] The area corresponding to the determined part of the clothing to be tried on can be expanded toward the periphery by a first preset pixel value compared to the area where the part of the clothing to be tried on in the human body image is located. The first preset pixel value is usually relatively small, so that clothing of different degrees of looseness can be accommodated in the area, and the relevant details of the human body and background in the human body image can be well preserved, so that the generated virtual try-on image can accurately try on the target image and maintain good consistency with the original human body image.

[0055] In a specific embodiment, when the above-mentioned clothing part to be tried on is an upper garment, the area corresponding to the clothing part to be tried on in the human body image is expanded to the non-try-on clothing part, that is, the area corresponding to the upper garment part is expanded downward. Specifically, the area corresponding to the clothing part to be tried on in the human body image is expanded to the non-try-on clothing part to a preset position, a preset pixel value, or a preset pixel ratio value, etc., which is not specifically limited in this application. For example, the area corresponding to the upper garment part to be tried on in the human body image is expanded downward to the crotch position, downward by 50 pixel values, downward by 10% of the vertical pixel value of the human body image, etc. The specific expansion size and expansion method can be set according to actual needs, which is not specifically limited in this application.

[0056] For example, Figure 4 As shown, Figure 4The second and third rows in FIG. 1 show schematic diagrams of the fitting area obtained by downwardly expanding the area corresponding to the upper garment to be fitted. Figure 4 The mid-fit area is indicated by the gray pixel coverage.

[0057] In an embodiment of the present application, the try-on area can be replaced in the human body image by a preset pixel value to mark the try-on area, or the try-on area can be marked by a contour line, which is not specifically limited in the present application.

[0058] In one implementation, step S120 may be implemented according to the following steps S121 to S122.

[0059] Step S121: Detect whether the non-try-on clothing part in the human body image covers the clothing part to be tried on.

[0060] For example, when the garment to be tried on is an upper garment, it will be detected whether the lower garment covers the upper garment. If the lower garment covers the upper garment, it means that the lower edge of the upper garment is stuffed into the lower garment, so that the lower garment covers the bottom of the upper garment. If the garment to be tried on is a lower garment, it will be detected whether the upper garment covers the lower garment. If the upper garment covers the lower garment, it means that the length of the upper garment is greater than the upper edge of the lower garment and the upper garment is not stuffed into the lower garment, so that the upper garment covers the top of the lower garment.

[0061] Optionally, the crotch key point in the human body image can be identified, and whether the non-try-on clothing part in the human body image covers the clothing part to be tried on can be determined based on the clothing located at the crotch key point in the human body image. Specifically, whether the non-try-on clothing part in the human body image covers the clothing part to be tried on can be determined based on the clothing located at the crotch key point in the human body image and whether there is a gap between the upper and lower clothing in the human body image. For example, Figure 4 As shown in , when there is a gap between the top and bottom, it is determined that the non-try-on clothing part in the human body image covers the clothing part to be tried; when there is no gap between the top and bottom, if the clothing located at the key point of the crotch is the bottom, such as Figure 4 As shown in the first row of , it is determined that the bottom covers the top; if the clothing at the key point of the crotch is the top, such as Figure 4 As shown in the third row, it is determined that the upper garment covers the lower garment.

[0062] Exemplarily, determining whether the non-try-on clothing part in the human body image covers the clothing part to be tried based on the clothing located at the key point of the crotch in the human body image includes: determining the length and short version of the clothing part to be tried in the human body image, and determining whether the non-try-on clothing part in the human body image covers the clothing part to be tried based on the length and short version and the clothing located at the key point of the crotch in the human body image. Figure 4As shown in FIG. 1 , when the clothing part to be tried on is an upper garment, if the clothing at the key point of the crotch is a lower garment, and the upper garment is a short version, it is determined that the non-try-on clothing part (lower garment) does not cover the clothing part to be tried on (upper garment). Figure 4 As shown in the first row; if the clothing at the key point of the crotch is bottoms, and the top is a long version, it is determined that the non-try-on clothing part (bottoms) covers the clothing part to be tried (tops), such as Figure 4 The second row is shown.

[0063] Optionally, the integrity of the lower boundary of the upper garment and the upper boundary of the lower garment in the human body image may also be detected, and based on the integrity detection results of the lower boundary of the upper garment and the upper boundary of the lower garment, it is determined whether the non-try-on garment part in the human body image covers the garment part to be tried. Specifically, it is determined whether the non-try-on garment part in the human body image covers the garment part to be tried based on the above integrity detection results and whether there is a gap between the upper and lower garments in the human body image. For example, when there is no gap between the upper garment and the lower garment, if the lower boundary of the upper garment is complete, it is determined that the upper garment covers the lower garment, and if the upper boundary of the lower garment is complete, it is determined that the lower garment covers the upper garment.

[0064] Those skilled in the art may also use other methods to detect whether the non-try-on clothing parts in the human body image cover the clothing parts to be tried on, which is not specifically limited in this application.

[0065] Step S122: If the detection result of step S121 is that no covering is caused, the area corresponding to the part of the clothing to be tried on in the human body image is expanded to the part of the clothing not to be tried on.

[0066] In this embodiment, the area corresponding to the clothing part to be tried on is expanded only when it is detected that the non-try-on clothing part in the human body image does not cover the clothing part to be tried on. For example, Figure 4 As shown, when the part of the clothing to be tried on is a top, the area corresponding to the top will be expanded toward the bottom when it is detected that the bottom does not cover the top. This is because when the bottom does not cover the top, the top in the human body image is complete, so the area corresponding to the top in the human body image is the complete area of ​​the original top, and this area is usually different from the pattern of the target clothing, so the area can be expanded downward to make the try-on area larger, so as to better provide a larger space for the target clothing to be tried on.

[0067] Optionally, the virtual try-on method may further include the following step S140.

[0068] Step S140: If the detection result of step S121 is that the target garment is covered, the area corresponding to the part of the garment to be tried on in the human body image is determined as a fitting area for trying on the target garment.

[0069] For example, Figure 4 As shown in the first row, when the clothing part to be tried on is the top, if the bottom covers the top, the connection between the top and the bottom can be cut off, and the area corresponding to the top is determined as the fitting area for trying on the target clothing. When the bottom covers the top, the top is stuffed into the bottom. In this way, when trying on the top, the top can be virtually tried on by stuffing it into the bottom. In this case, there is no need to stretch the target clothing, but the part of the target clothing that needs to be stuffed into the bottom can be directly cut off. Therefore, there is no need to expand the area downward. In addition, when the top is stuffed into the bottom, the bottom is not a part that needs to be tried on, so the original image of the bottom can be kept as much as possible. Therefore, not expanding the top area downward can avoid the loss of the bottom, so as to better maintain the clothing parts that do not need to be tried on in the human body image.

[0070] This embodiment determines whether to expand the try-on area based on whether the non-try-on clothing parts in the human body image cover the clothing parts to be tried on. Accurate virtual try-on can be performed on human body images with different covering relationships between upper and lower garments, thereby generating more accurate virtual try-on images for different types of human body images.

[0071] Step S130: Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image.

[0072] Optionally, in step S130, the target clothing in the target clothing image may be tried on in the try-on area in the following manner: Determine the ratio relationship between the sizes of the target clothing, and try on the target clothing in the target clothing image in the try-on area in a manner that keeps the ratio relationship between the sizes unchanged.

[0073] When the target garment is a top, each size may include at least two of the top's length, width, shoulder width, chest circumference, waist circumference, hip circumference, and sleeve length, but not limited thereto; when the target garment is a bottom, each size may include at least two of the bottom's length, waist circumference, hip circumference, hip width, thigh width, knee circumference, and trouser leg width, but not limited thereto. The ratio relationship between each size of the target garment can well reflect the pattern of the target top view. During the virtual try-on process, if the ratio relationship between each size of the target garment remains unchanged, it can well ensure that the original clothing pattern of the target garment is maintained during the virtual try-on. For example, for a top, if the ratio between the length and width of the top is kept unchanged during the virtual try-on, the pattern of the top being tried on can be well guaranteed.

[0074] Specifically, the above virtual try-on image can be generated by a pre-trained virtual try-on model. The virtual try-on model is used to try on the target clothing in the target clothing image in the original clothing pattern to the try-on area, and repair the gap area in the try-on area that has not tried on the target clothing according to the non-try-on clothing parts, generating a virtual try-on image with the target clothing tried on, and the target clothing in the generated virtual try-on image maintains the original clothing pattern. Specifically, the target clothing image, the human body image, and the above try-on area can be input into the virtual try-on model to obtain the above virtual try-on image.

[0075] The training process of the virtual try-on model can be trained by training algorithms such as supervised learning algorithms, semi-supervised learning algorithms, and unsupervised learning algorithms in related technologies, which is not specifically limited in this application. The specific training process will be described in the embodiments below. Through training, a virtual try-on model that meets the requirements can be obtained, so that the virtual try-on model can accurately and efficiently generate virtual try-on images that meet the requirements.

[0076] In one implementation, before step S130, the above virtual try-on method may further include the following step S130a.

[0077] Step S130a: Determine the clothing mask corresponding to the human body image according to the try-on area. The clothing mask is used to distinguish and represent the try-on area and the non-try-on area outside the try-on area in the human body image through different pixel values.

[0078] Among them, the clothing mask can be an image of the same size as the human body image. The pixel values of the area corresponding to the try-on area in the human body image and other areas in the clothing mask can be a binary image. For example, the pixel value of the area corresponding to the above try-on area in the clothing mask can be set to 1, and the pixel values of other areas can be set to 0, so that a black and white clothing mask corresponding to the human body image can be obtained.

[0079] Correspondingly, in step S130, the target clothing in the target clothing image can be tried on to the try-on area according to the following step S131.

[0080] Step S131: According to the clothing mask, and based on the principle of keeping the tried-on target clothing in the original clothing pattern, try on the target clothing in the target clothing image to the try-on area.

[0081] Since the try-on area is marked in the clothing mask corresponding to the human body image, therefore, through the clothing mask, the try-on area can be conveniently obtained, so that the target clothing in the target clothing image can be accurately tried on to the try-on area.

[0082] This embodiment sets a clothing mask corresponding to the human body image and marking the try-on area, so that the specific position of the try-on area can be accurately located during the virtual try-on, thereby improving the accuracy of the virtual try-on.

[0083] In a specific embodiment, step S130 can be implemented according to the following steps S132 to S133.

[0084] Step S132: Determine input information of a pre-trained virtual try-on model according to the clothing mask, the target clothing image and the human body image.

[0085] Specifically, the clothing mask, the target clothing image and the human body image can be determined as the input information of the virtual try-on model, or the clothing mask, the target clothing image or the human body image can be processed, such as feature extraction processing, clarity processing, etc., to obtain the above-mentioned input information, or other information can be added as the above-mentioned input information, which is not specifically limited in the present application.

[0086] In a specific embodiment, before step S132, the above method may further include the following step S132a.

[0087] Step S132a: Determine the human body posture information corresponding to the human body image.

[0088] The human body posture information is used to display the posture of the human body in the human body image. In the embodiment of the present application, the human body posture information corresponding to the human body image can be identified by a pre-trained posture recognition model. The posture recognition model can be trained using a supervised learning algorithm, a semi-supervised learning algorithm, an unsupervised learning algorithm, or other training algorithms in the relevant technology, which is not specifically limited in the present application.

[0089] Correspondingly, step S132 can be implemented according to the following steps S132-1.

[0090] Step S132 - 1 : Determine input information of a pre-trained virtual fitting model according to the clothing mask, the target clothing image, the human body image, and the human body posture information.

[0091] For example, the clothing mask, the target clothing image, the human body image and the human body posture information may be determined as input information of a virtual try-on model, or the information may be subjected to feature extraction or other processing to obtain input information.

[0092] This embodiment uses human body posture information as the input of the virtual try-on model, so that the virtual try-on model can accurately capture the human body posture in the human body image, so that the subsequently generated virtual try-on image is consistent with the human body posture in the human body image; in addition, the human body posture information can also represent the body shape of the human body in the human body image, so that the virtual try-on model can generate a more accurate virtual try-on image according to the body shape of the human body in the human body posture information.

[0093] Step S133: inputting the input information into the virtual try-on model, so that the virtual try-on model tries on the target clothing in the target clothing image into the try-on area, and repairs the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to generate a virtual try-on image in which the target clothing to be tried on maintains the original clothing pattern.

[0094] In this embodiment, the virtual try-on model can accurately obtain which areas in the human body image are used to try on the target clothing based on the clothing mask, so that the virtual try-on image can be generated more accurately.

[0095] In a specific embodiment, the virtual try-on model may be a diffusion model.

[0096] Among them, the diffusion model can convert the input data into random noise by gradually adding noise, and then learn to reverse this process to recover a clear image from the noise. Due to its progressive denoising characteristics, the diffusion model can generate very detailed and realistic images, which is suitable for image generation of complex scenes or multimodal data.

[0097] Correspondingly, step S132-1 can be implemented according to the following steps A to C.

[0098] Step A: adding the noise to be added to the human body image to obtain a noisy image.

[0099] The noise to be added may be random noise or preset noise, which is not specifically limited in this application.

[0100] Step B: replacing the area in the human body image corresponding to the fitting area with a preset pixel value to obtain a clothing-independent human body image.

[0101] like Figure 4As shown, the try-on area in the human body image can be replaced by gray, white or other color pixel values, and the obtained clothing-independent human body image can clearly mark the area in the human body image where the target clothing needs to be tried on, thereby facilitating the virtual try-on model to perform virtual try-on in the try-on area. In addition, the area in the clothing-independent human body image that is not replaced by the preset pixel value can provide a repair reference for the subsequent repair of the gap area, so that the virtual try-on image generated by the virtual try-on model is better consistent with the content in the human body image, thereby improving the accuracy and naturalness of the virtual try-on image.

[0102] Step C: Determine the noisy image, the clothing-independent human body image, the clothing mask, the target clothing image, and the human body posture information as input information of a pre-trained virtual try-on model.

[0103] After the input information determined in step C is input into the virtual try-on model, the virtual try-on model can, under the guidance of the clothing-irrelevant human body image, the clothing mask, the target clothing image and the human body posture information, perform denoising step by step from the noisy image to obtain a virtual try-on image. The target clothing in the obtained virtual try-on image maintains the original clothing pattern, and the clothing worn in the virtual try-on image is natural as a whole and has better consistency with the non-try-on clothing parts in the human body image.

[0104] The virtual try-on method provided in the embodiment of the present application obtains a human body image to be virtually tried on and a target clothing image, then expands the area corresponding to the clothing part to be tried on in the human body image toward the non-try-on clothing part, and obtains a try-on area in the human body image for trying on the target clothing. In this way, the obtained try-on area is expanded toward the non-try-on clothing part compared to the clothing part to be tried on in the human body image. For example, when the clothing part to be tried on is an upper garment, the try-on area is expanded downward on the basis of the upper garment area of ​​the human body image, that is, the try-on area is larger than the area of ​​the clothing part to be tried on in the human body image. Then, based on the principle of keeping the original clothing version of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area. Since the try-on area is expanded toward the non-try-on clothing part, the try-on area can provide a reference for trying on the target clothing. A larger try-on space is provided so that when trying on the target clothing in the try-on area, the original shape of the target clothing can be better maintained without longitudinally stretching or compressing the target clothing. That is to say, when trying on the target clothing, the try-on area is large, so the target clothing is not deformed in shape, but the original shape of the target clothing is maintained, and then the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part. Since the try-on area is extended to the non-try-on clothing part, the try-on area is often larger than the target clothing, so that after trying on the target clothing in the try-on area, there may be a gap area between the target clothing tried on in the try-on area and the non-try-on clothing part in the human body image, and the gap area needs to be repaired to make the try-on image more natural. Therefore, after repairing the gap area according to the non-try-on clothing part in the human body image, the virtual try-on image can be made more natural.

[0105] It can be seen that the virtual try-on method provided by the present application can make the generated virtual try-on image better maintain the original version of the target clothing by trying on the target clothing in the extended try-on area, and the target clothing is not stretched but the original version is maintained during the try-on, so that the virtual try-on can better show the original effect of the target clothing, thereby improving the virtual try-on effect, and the gap area between the target clothing and the untried clothing part in the human body image is repaired through the content in the human body image, so that the virtual try-on image can be more natural and complete, thereby ensuring the overall try-on effect of the virtual try-on image, such as Figure 5 Shown is a comparison chart of the virtual try-on image generated by the solution provided by the present application and the virtual try-on image generated by related technologies. It can be seen that the solution provided by the present application can maintain the original style of the target clothing and ensure the overall try-on effect of the virtual try-on image.

[0106] Embodiment 2

[0107] The second embodiment of the present application also provides a model training method, which is applied to an electronic device, which can be a desktop computer, a laptop computer, a server, a mobile terminal, a gateway, or other electronic device with data processing capabilities. This embodiment is at least used to train the virtual try-on model required for the virtual try-on in the first embodiment, and other models in the first embodiment (such as the area recognition model, the posture recognition model, etc.) can be trained during the training of the virtual try-on model, or can be obtained by pre-training.

[0108] like Figure 6 As shown, the model training method provided in this embodiment includes the following steps S210 to S250.

[0109] Step S210: Obtain training samples.

[0110] The training samples include a sample human body image, a sample target clothing image, a sample try-on image in which the sample target clothing is tried on, and a sample mask corresponding to the sample human body image, wherein the sample mask is used to distinguish between a try-on area in the sample human body image and a non-try-on area outside the try-on area through different pixel values, and the try-on area in the sample mask is an area obtained by extending an area corresponding to a part of the clothing to be tried on in the sample human body image to a non-try-on clothing part, the sample target clothing tried on in the sample try-on image maintains the original style, and the human body in the sample try-on image is consistent with the human body in the corresponding sample human body image, that is, the contents of the sample try-on image and the corresponding sample human body image are the same except for the part to be tried on.

[0111] The determination method and specific content of the above-mentioned sample mask can refer to the clothing mask in the first embodiment, and the specific related content of the above-mentioned sample human body image and sample target clothing image can refer to the human body image and target image in the first embodiment, which will not be described in detail here. The above-mentioned sample fitting image of the sample target clothing can be an image taken after the human body in the sample human body image tries on the target clothing, or it can be a fitting image with a good fitting evaluation result in the historical fitting data. Those skilled in the art can also use other means to collect sample fitting images.

[0112] Step S220: Determine sample input information of the model to be trained according to the sample human body image, the sample target clothing image, and the sample mask.

[0113] The execution method of step S220 may refer to step S132 in the first embodiment, and will not be described in detail here.

[0114] Step S230: input the sample input information into the model to be trained to obtain an output result.

[0115] Step S240: adjusting the parameters of the model to be trained according to the difference between the output result and the sample fitting image, so as to obtain a trained virtual fitting model.

[0116] In a specific embodiment, step S240 can also be implemented according to the following steps: according to the difference between the output result and the sample try-on image, and the consistency of the style between the clothing tried on in the output result and the sample target clothing, the parameters of the model to be trained are adjusted to obtain a trained virtual try-on model.

[0117] Step S240 uses a supervised training algorithm to adjust the parameters of the training image generation model. For example, the gradient descent method or other optimization algorithms can be used to update the model parameters through back propagation. The specific parameter adjustment method can use the supervised training algorithm of related technology, which will not be described in detail here.

[0118] In a specific embodiment, the sample human body images may include sample covered human body images in which the non-trying clothing parts cover the clothing parts to be tried and sample uncovered human body images in which the non-trying clothing parts do not cover the clothing parts to be tried. That is, the samples for model training are divided into two categories, one is the sample covered human body images, and the other is the sample uncovered human body images.

[0119] The sample mask corresponding to the sample uncovered human body image includes a sample expansion mask and a sample non-expanded mask. The try-on area in the sample expansion mask is the area obtained by expanding the area corresponding to the clothing part to be tried in the sample uncovered human body image to the non-try-on clothing part, and the try-on area in the sample non-expanded mask is the area corresponding to the clothing part to be tried in the sample uncovered human body image. In other words, for the sample uncovered human body image in which the non-try-on clothing part does not cover the clothing part to be tried, two masks are set accordingly. One is the conventional sample non-expanded mask that marks the area corresponding to the clothing part to be tried, and the other is the sample non-expanded mask that expands the area corresponding to the clothing part to be tried and then marks it. Both masks are used for model training.

[0120] The try-on area in the sample mask corresponding to the sample covered human body image is the area corresponding to the clothing part to be tried on in the sample covered human body image. In other words, the sample covered human body image is set with a conventional mask for model training.

[0121] This embodiment sets two types of samples, namely, sample covered human body images and sample uncovered human body images, so that the trained virtual try-on model can accurately try on human body images in which non-try-on clothing parts cover the clothing parts to be tried on, and can also accurately try on human body images in which non-try-on clothing parts do not cover the clothing parts to be tried on, thereby improving the applicability and accuracy of the virtual try-on model.

[0122] In addition, the present embodiment sets a sample extension mask and a sample non-extension mask for the sample uncovered human body image. The sample extension mask can increase the diversity and quantity of samples, so that the model to be trained converges quickly and improves the training efficiency. The sample non-extension mask can enable the model to accurately try on human body images whose non-try-on clothing parts do not cover the clothing parts to be tried, avoid deformation such as stretching of the target clothing, and improve the accuracy of the model in virtual try-on.

[0123] In a specific embodiment, before step S220, the above method may further include the following step S220a.

[0124] Step S220a: When the selected sample is the sample uncovered human body image, the sample expanded mask is selected as the corresponding sample mask from the sample expanded mask and the sample unexpanded mask corresponding to the selected sample uncovered human body image with a preset probability.

[0125] The preset probability may be any probability between 30% and 70%, or may be other smaller or larger probabilities, which is not specifically limited in the present application.

[0126] In a specific embodiment, before step S220, the following steps may be further included: determining sample human body posture information corresponding to the sample human body image. Accordingly, step S220 may be implemented according to the following steps: determining sample input information of the model to be trained according to the sample human body image, the sample target clothing image, the sample mask and the sample human body posture information.

[0127] In a specific embodiment, when the model to be trained is a diffusion model, step S220 can be implemented according to the following steps: adding preset noise to the sample human body image to obtain a sample noisy image; replacing the area in the sample human body image corresponding to the try-on area with a preset pixel value to obtain a sample clothing-independent human body image; and determining the sample noisy image, the sample clothing-independent human body image, the sample clothing mask, the sample target clothing image, and the sample human body posture information as input information of a pre-trained virtual try-on model.

[0128] When training a virtual try-on model using the model training method provided in the present application, since the try-on area of ​​the sample mask in the training sample is the area obtained by extending the area corresponding to the part of the clothing to be tried on in the sample human body image to the part of the clothing not to be tried on, and since the sample target clothing tried on in the sample try-on image maintains the original style, the virtual try-on model obtained by subsequently adjusting the parameters of the model to be trained according to the difference between the output result and the sample try-on image can generate a virtual try-on image that better maintains the original style of the target clothing and is more natural and complete.

[0129] The following is an introduction to the model training method provided by this embodiment through a specific example. Figure 7 As shown, the model training method provided in this example includes the following steps 1 to 2.

[0130] Step 1: Obtain the input information of the semantic alignment test network to be trained.

[0131] The semantic alignment fitting network to be trained is the model to be trained, and can also be understood as a virtual fitting model to be trained.

[0132] like Figure 7 As shown, the input information of the semantic alignment trial network to be trained includes Figure 7 The first row is the covered samples, and the second row is the uncovered samples. The covered samples refer to the samples corresponding to the human body images covered by the samples, and the uncovered samples refer to the samples corresponding to the human body images not covered by the samples.

[0133] The masked samples in the first row include: a sample noisy image Z with preset noise added to the sample human image, sample human posture information D, sample target clothing image C, sample unexpanded mask M, and the corresponding sample clothing-independent human image P. a .

[0134] The unmasked samples in the second row include: a sample noisy image Z with preset noise added to the sample human image, sample human posture information D, sample target clothing image C, sample unexpanded mask M and corresponding sample clothing-independent human image P a , sample unexpanded mask M Adp Human body images that have nothing to do with the corresponding sample clothing

[0135] For the uncovered samples in the second row, we select the samples from M and P with probability p. a 、M Adp and Select M from the two sets of data Adp and As the input information of the semantic alignment trial network to be trained.

[0136] Step 2: Input the input information of step 1 into the semantic alignment trial network to be trained to obtain the output result, and adjust the parameters in the semantic alignment trial network to be trained according to the output result to obtain the trained semantic alignment trial network.

[0137] During the training process, the process of using the masked samples in the first row for training is an interference training process, and the process of using the uncovered samples in the second row for training is a non-interference training process.

[0138] The process of step 2 can refer to steps S210 to S240 in the model training method provided above, which will not be described in detail here.

[0139] The relevant contents of the model training method provided in this embodiment are similar to the relevant contents of the virtual try-on method provided in the first embodiment. The specific instructions, execution process and beneficial effects can refer to the first embodiment, and this embodiment will not be described in detail.

[0140] Embodiment 3

[0141] The first embodiment of the present application also provides a virtual try-on device corresponding to the virtual try-on method embodiment provided in the first embodiment. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the virtual try-on method embodiment provided above. The virtual try-on device provided in this embodiment includes:

[0142] An information acquisition unit, used to acquire an image of a human body to be virtually tried on and an image of a target garment;

[0143] An area determination unit, used for expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part, to obtain a try-on area for trying on the target clothing in the human body image;

[0144] The try-on generation unit is used to try on the target clothing in the target clothing image in the try-on area based on the principle of keeping the original clothing pattern of the target clothing to be tried on, and to repair the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to obtain a virtual try-on image.

[0145] Optionally, the area determination unit is specifically used to: detect whether the non-try-on clothing part in the human body image covers the clothing part to be tried; if the detection result is that it does not cover, expand the area corresponding to the clothing part to be tried in the human body image to the non-try-on clothing part.

[0146] Optionally, the area determination unit is further configured to:

[0147] If the detection result is that the target clothing is covered, the area corresponding to the part of the clothing to be tried on in the human body image is determined as a fitting area for trying on the target clothing.

[0148] Optionally, the clothing part to be tried on is an upper garment, and the clothing part not to be tried on is a lower garment; or, the clothing part to be tried on is a lower garment, and the clothing part not to be tried on is an upper garment.

[0149] Optionally, based on the principle of keeping the target garment to be tried on in its original garment version, the device further comprises:

[0150] The mask determination unit is used to determine the clothing mask corresponding to the human body image according to the try-on area, and the clothing mask is used to distinguish the try-on area and the non-try-on area outside the try-on area in the human body image through different pixel values; the try-on generation unit is specifically used to: according to the clothing mask and based on the principle of keeping the target clothing to be tried on with the original clothing pattern, try on the target clothing in the target clothing image into the try-on area.

[0151] Optionally, the try-on generation unit is specifically used to: determine input information of a pre-trained virtual try-on model according to the clothing mask, the target clothing image and the human body image; input the input information into the virtual try-on model so that the virtual try-on model tries on the target clothing in the target clothing image into the try-on area, and repairs the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to generate a virtual try-on image in which the target clothing to be tried on maintains the original clothing pattern.

[0152] Optionally, the device further comprises: a posture determination unit, configured to determine human posture information corresponding to the human image;

[0153] The try-on generation unit is specifically used to determine input information of a pre-trained virtual try-on model according to the clothing mask, the target clothing image, the human body image and the human body posture information.

[0154] Optionally, the virtual try-on model is a diffusion model;

[0155] The try-on generation unit is specifically used to: add the noise to be added to the human body image to obtain a noisy image; replace the area in the human body image corresponding to the try-on area with a preset pixel value to obtain a clothing-independent human body image; and determine the noisy image, the clothing-independent human body image, the clothing mask, the target clothing image, and the human body posture information as input information of a pre-trained virtual try-on model.

[0156] Optionally, the area determination unit is specifically used to: identify the crotch key point in the human body image, and determine whether the non-try-on clothing part in the human body image covers the clothing part to be tried based on the clothing located at the crotch key point in the human body image; or, detect the integrity of the lower boundary of the upper clothing and the upper boundary of the lower clothing in the human body image, and determine whether the non-try-on clothing part in the human body image covers the clothing part to be tried based on the detection result.

[0157] Embodiment 4

[0158] The fourth embodiment of the present application also provides a model training device corresponding to the model training method embodiment provided in the second embodiment. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the model training method embodiment provided above. The model training device provided in this embodiment includes:

[0159] A sample acquisition unit, used to acquire training samples, wherein the training samples include a sample human image, a sample target clothing image, a sample try-on image in which the sample target clothing is tried on, and a sample mask corresponding to the sample human image, wherein the sample mask is used to distinguish between a try-on area in the sample human image and a non-try-on area outside the try-on area through different pixel values, wherein the try-on area in the sample mask is an area obtained by extending an area corresponding to a part of the clothing to be tried on in the sample human image to a non-try-on clothing part, and the sample target clothing tried on in the sample try-on image maintains the original pattern;

[0160] A sample information determination unit, used to determine sample input information of a model to be trained according to the sample human image, the sample target clothing image, and the sample mask;

[0161] A sample output unit, used to input the sample input information into the model to be trained to obtain an output result;

[0162] The adjustment unit is used to adjust the parameters of the model to be trained according to the difference between the output result and the sample fitting image to obtain a trained virtual fitting model.

[0163] The fifth embodiment of the present application also provides an electronic device embodiment corresponding to the virtual try-on method provided in the first embodiment. The following description of the electronic device embodiment is only illustrative. The electronic device embodiment is as follows:

[0164] Please refer to Figure 8 Understand the above electronic devices, Figure 8The electronic device provided in this embodiment includes: a processor 1001, a memory 1002, a communication bus 1003, and a communication interface 1004;

[0165] The memory 1002 is used to store computer instructions for data processing. When the computer instructions are read and executed by the processor 1001, the following steps are performed:

[0166] Acquire a human body image to be virtually tried on and a target clothing image;

[0167] Expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part to obtain a try-on area in the human body image for trying on the target clothing;

[0168] Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image.

[0169] The sixth embodiment of the present application also provides an electronic device embodiment corresponding to the model training method provided in the second embodiment. The following description of the electronic device embodiment is only illustrative. The electronic device embodiment is as follows:

[0170] The electronic device provided in this embodiment includes: a processor, a memory, a communication bus, and a communication interface;

[0171] The memory is used to store computer instructions for data processing, and when the computer instructions are read and executed by the processor, the following steps are performed:

[0172] Acquire training samples, the training samples including a sample human body image, a sample target clothing image, a sample try-on image in which the sample target clothing is tried on, and a sample mask corresponding to the sample human body image, the sample mask being used to distinguish between a try-on area in the sample human body image and a non-try-on area outside the try-on area through different pixel values, the try-on area in the sample mask being an area obtained by extending an area corresponding to a part of the clothing to be tried on in the sample human body image to a non-try-on clothing part, and the sample target clothing tried on in the sample try-on image retains the original pattern;

[0173] Determine sample input information of a model to be trained according to the sample human body image, the sample target clothing image, and the sample mask;

[0174] Inputting the sample input information into the model to be trained to obtain an output result;

[0175] According to the difference between the output result and the sample fitting image, the parameters of the model to be trained are adjusted to obtain a trained virtual fitting model.

[0176] The seventh embodiment of the present application also provides a computer-readable storage medium for implementing the method described in the first embodiment. The computer-readable storage medium embodiment provided in this application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The embodiment described below is only illustrative.

[0177] The computer readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the following steps are implemented:

[0178] Acquire a human body image to be virtually tried on and a target clothing image;

[0179] Expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part to obtain a try-on area in the human body image for trying on the target clothing;

[0180] Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image.

[0181] The eighth embodiment of the present application also provides a computer-readable storage medium for implementing the method described in the second embodiment. The computer-readable storage medium embodiment provided in the present application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The embodiment described below is only illustrative.

[0182] The computer readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the following steps are implemented:

[0183] Acquire training samples, the training samples including a sample human body image, a sample target clothing image, a sample try-on image in which the sample target clothing is tried on, and a sample mask corresponding to the sample human body image, the sample mask being used to distinguish between a try-on area in the sample human body image and a non-try-on area outside the try-on area through different pixel values, the try-on area in the sample mask being an area obtained by extending an area corresponding to a part of the clothing to be tried on in the sample human body image to a non-try-on clothing part, and the sample target clothing tried on in the sample try-on image retains the original pattern;

[0184] Determine sample input information of a model to be trained according to the sample human body image, the sample target clothing image, and the sample mask;

[0185] Inputting the sample input information into the model to be trained to obtain an output result;

[0186] According to the difference between the output result and the sample fitting image, the parameters of the model to be trained are adjusted to obtain a trained virtual fitting model.

[0187] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0188] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0189] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0190] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0191] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A virtual try-on method, characterized in that: The method comprises: Acquire a human body image to be virtually tried on and a target clothing image; Expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part to obtain a try-on area in the human body image for trying on the target clothing; Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image.

2. The virtual try-on method according to claim 1, characterized in that: The step of extending the area corresponding to the clothing to be tried on part of the human body image to the non-clothing to be tried on part includes: Detecting whether the non-try-on clothing part in the human body image covers the clothing part to be tried on; If the detection result shows that no covering is caused, the area corresponding to the clothing part to be tried on in the human body image is expanded to the non-clothing part.

3. The virtual try-on method according to claim 2, characterized in that: The method further comprises: If the detection result is that the target clothing is covered, the area corresponding to the part of the clothing to be tried on in the human body image is determined as a fitting area for trying on the target clothing.

4. The virtual try-on method according to claim 3, characterized in that: The clothing part to be tried on is an upper garment, and the clothing part not to be tried on is a lower garment; Alternatively, the clothing portion to be tried on is bottom clothing, and the clothing portion not to be tried on is top clothing.

5. The virtual try-on method according to any one of claims 1 to 4, characterized in that: Before trying on the target garment in the target garment image in the try-on area based on the principle of keeping the original garment pattern of the target garment to be tried on, the method further includes: Determine a clothing mask corresponding to the human body image according to the fitting area, wherein the clothing mask is used to distinguish between the fitting area and a non-fitting area outside the fitting area in the human body image through different pixel values; The step of trying on the target clothing in the target clothing image in the try-on area based on the principle of keeping the original clothing pattern of the target clothing to be tried on comprises: According to the clothing mask and based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area.

6. The virtual try-on method according to claim 5, characterized in that: Based on the principle of keeping the original clothing pattern of the target clothing to be tried on, the target clothing in the target clothing image is tried on in the try-on area, and the gap area in the try-on area where the target clothing is not tried on is repaired according to the non-try-on clothing part to obtain a virtual try-on image, including: Determining input information of a pre-trained virtual try-on model according to the clothing mask, the target clothing image, and the human body image; The input information is input into the virtual try-on model so that the virtual try-on model tries on the target clothing in the target clothing image into the try-on area, and repairs the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to generate a virtual try-on image in which the target clothing to be tried on maintains the original clothing pattern.

7. The virtual try-on method according to claim 6, characterized in that: Before determining the input information of the pre-trained virtual try-on model, the method further includes: Determining human body posture information corresponding to the human body image; Determining input information of a pre-trained virtual fitting model according to the clothing mask, the target clothing image, and the human body image includes: Input information of a pre-trained virtual try-on model is determined according to the clothing mask, the target clothing image, the human body image, and the human body posture information.

8. The virtual try-on method according to claim 7, characterized in that: The virtual fitting model is a diffusion model; The step of determining input information of a pre-trained virtual fitting model according to the clothing mask, the target clothing image, the human body image, and the human body posture information includes: Adding the noise to be added to the human body image to obtain a noisy image; The area in the human body image corresponding to the fitting area is replaced by a preset pixel value to obtain a clothing-independent human body image; The noisy image, the clothing-independent human body image, the clothing mask, the target clothing image, and the human body posture information are determined as input information of a pre-trained virtual try-on model.

9. The virtual try-on method according to claim 2, characterized in that: The step of trying on the target clothing in the target clothing image in the try-on area based on the principle of keeping the original clothing pattern of the target clothing to be tried on comprises: Determining the ratio relationship between various sizes of the target clothing; The target clothing in the target clothing image is tried on in the fitting area in a manner that the ratio relationship between the sizes is kept unchanged.

10. The virtual try-on method according to claim 2, characterized in that: The detecting whether the non-try-on clothing part in the human body image covers the clothing part to be tried on comprises: Identify the key point of the crotch in the human body image, and determine whether the non-try-on clothing part in the human body image covers the clothing part to be tried on according to the clothing located at the key point of the crotch in the human body image; Alternatively, the integrity of the lower boundary of the upper garment and the upper boundary of the lower garment in the human body image is detected, and it is determined based on the detection result whether the non-try-on garment part in the human body image covers the garment part to be tried-on.

11. A model training method, characterized in that: The method comprises: Acquire training samples, the training samples including a sample human body image, a sample target clothing image, a sample try-on image in which the sample target clothing is tried on, and a sample mask corresponding to the sample human body image, the sample mask being used to distinguish between a try-on area in the sample human body image and a non-try-on area outside the try-on area through different pixel values, the try-on area in the sample mask being an area obtained by extending an area corresponding to a part of the clothing to be tried on in the sample human body image to a non-try-on clothing part, and the sample target clothing tried on in the sample try-on image retains the original pattern; Determine sample input information of a model to be trained according to the sample human body image, the sample target clothing image, and the sample mask; Inputting the sample input information into the model to be trained to obtain an output result; According to the difference between the output result and the sample fitting image, the parameters of the model to be trained are adjusted to obtain a trained virtual fitting model.

12. The model training method according to claim 11, characterized in that: The sample human body images include sample covered human body images in which the non-try-on clothing parts cover the clothing parts to be tried on and sample uncovered human body images in which the non-try-on clothing parts do not cover the clothing parts to be tried on; The sample mask corresponding to the sample uncovered human body image includes a sample extended mask and a sample unextended mask, wherein the try-on area in the sample extended mask is an area obtained by extending the area corresponding to the clothing part to be tried on in the sample uncovered human body image to the non-try-on clothing part, and the try-on area in the sample unextended mask is an area corresponding to the clothing part to be tried on in the sample uncovered human body image; The try-on area in the sample mask corresponding to the sample covered human body image is an area corresponding to the clothing part to be tried on in the sample covered human body image.

13. The model training method according to claim 12, characterized in that: Before determining the sample input information of the model to be trained according to the sample human image, the sample target clothing image, and the sample mask, the method further includes: When the selected sample is the sample uncovered human body image, the sample expanded mask is selected as the corresponding sample mask from the corresponding sample expanded mask and the sample unexpanded mask with a preset probability.

14. A virtual try-on device, characterized in that: The device comprises: An information acquisition unit, used to acquire an image of a human body to be virtually tried on and an image of a target garment; An area determination unit, used for expanding the area corresponding to the clothing part to be tried on in the human body image to the non-try-on clothing part, to obtain a try-on area for trying on the target clothing in the human body image; The try-on generation unit is used to try on the target clothing in the target clothing image in the try-on area based on the principle of keeping the original clothing pattern of the target clothing to be tried on, and to repair the gap area in the try-on area where the target clothing is not tried on according to the non-try-on clothing part, so as to obtain a virtual try-on image.

15. An electronic device, characterized in that: include: A processor, a memory, and computer program instructions stored on the memory and executable on the processor; When the processor executes the computer program instructions, the method according to any one of claims 1 to 13 is implemented.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method described in any one of claims 1 to 13 when executed by a processor.

Images