Virtual fitting method, virtual fitting device, electronic equipment and storage medium

Abstract

The invention discloses a virtual fitting method, a virtual fitting device, electronic equipment and a storage medium, and the virtual fitting method comprises the steps: obtaining a current user image and a current to-be-fitted clothes image; generating a current three-dimensional human body mask according to the current user image; and inputting the current three-dimensional human body mask and the current to-be-tried-on clothing image to a reloading model to obtain a target reloading image, the reloading model being obtained by training a deep learning neural network model based on the historical to-be-tried-on clothing image and the historical three-dimensional human body mask. According to the method, on the basis of the user image, the to-be-tried-on clothing image and the clothing changing model, the user is simulated to try on the to-be-tried-on clothing, the user can select the online shopping clothing according to the try-on effect, and the online shopping efficiency of the online shopping clothing can be improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a virtual fitting method, a virtual fitting device, an electronic device and a storage medium. Background Art

[0002] When buying clothes online, you cannot try them on before buying like you can in offline physical stores. When the clothes you receive do not meet your expectations, you need to negotiate with the merchant to return or exchange the clothes. The return and exchange process is relatively cumbersome, which reduces the efficiency of online clothing shopping. Summary of the invention

[0003] In view of this, embodiments of the present application provide a virtual fitting method, a virtual fitting device, an electronic device, and a storage medium to overcome or at least partially solve the above problems of the prior art.

[0004] In a first aspect, an embodiment of the present application provides a virtual fitting method, comprising: obtaining a current user image and a current image of clothing to be tried on; generating a current three-dimensional human body mask based on the current user image; inputting the current three-dimensional human body mask and the current image of clothing to be tried on into a dressing model to obtain a target dressing image, wherein the dressing model is obtained by training a deep learning neural network model based on historical images of clothing to be tried on and historical three-dimensional human body masks.

[0005] In a second aspect, the embodiment of the present application provides a virtual fitting device, which includes a first acquisition module, a generation module, and a first input module. The first acquisition module is used to acquire the current user image and the current image of the clothing to be tried on; the generation module is used to generate the current three-dimensional human body mask according to the current user image; the first input module is used to input the current three-dimensional human body mask and the current image of the clothing to be tried on to the dressing model to obtain the target dressing image, and the dressing model is obtained by training the deep learning neural network model based on the historical clothing images and the historical three-dimensional human body masks.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the virtual fitting method provided in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the virtual fitting method provided in the first aspect above.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the virtual fitting method provided in the first aspect above.

[0009] The solution provided by the present application obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, and inputs the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model to obtain the target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical images of the clothing to be tried on and historical three-dimensional human body masks. It realizes the simulation of the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A flow chart of a virtual fitting method provided in an embodiment of the present application is shown.

[0012] Figure 2 A structural schematic diagram of a target human body mask generation model in a virtual fitting method provided in an embodiment of the present application is shown.

[0013] Figure 3 A structural schematic diagram of a receptive field adaptive attention model in a virtual fitting method provided in an embodiment of the present application is shown.

[0014] Figure 4 Another flowchart of the virtual fitting method provided in an embodiment of the present application is shown.

[0015] Figure 5 Another flowchart of the virtual fitting method provided in the embodiment of the present application is shown.

[0016] Figure 6 A structural block diagram of a virtual fitting device provided in an embodiment of the present application is shown.

[0017] Figure 7 A functional block diagram of an electronic device provided in an embodiment of the present application is shown.

[0018] Figure 8A computer-readable storage medium provided in an embodiment of the present application for storing or carrying a program code for implementing a virtual fitting method provided in an embodiment of the present application is shown.

[0019] Fig. 9 A computer program product provided in an embodiment of the present application and used for storing or carrying program codes for implementing a virtual fitting method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0022] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0023] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0024] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0025] When buying clothes online, you cannot try them on before buying like you can in offline physical stores. When the clothes you receive do not meet your expectations, you need to negotiate with the merchant to return or exchange the clothes. The return and exchange process is relatively cumbersome, which reduces the efficiency of online clothing shopping.

[0026] In response to the above problems, the virtual fitting method, virtual fitting device, electronic device and storage medium provided in the embodiments of the present application obtain the current user image and the current image of the clothing to be tried on, generate the current three-dimensional human body mask according to the current user image, and input the current three-dimensional human body mask and the current image of the clothing to be tried on to the dressing model to obtain the target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical images of the clothing to be tried on and historical three-dimensional human body masks, so as to simulate the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0028] The virtual fitting method provided in the embodiment of the present application can be applied to electronic devices, and the electronic devices can be servers or terminal devices, etc., which are not limited here and can be specifically configured according to actual needs.

[0029] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), big data, and artificial intelligence platforms.

[0030] The terminal device can be a mobile terminal device (for example, a mobile phone, a car terminal, a PDA (Personal Digital Assistant), a tablet PC (Tablet Personal Computer, Tablet PC), a laptop computer, a smart watch, a smart bracelet, etc.), or a fixed terminal device (a desktop computer, a smart panel, an all-in-one computer, etc.).

[0031] See also Figure 1 , which shows a flowchart of a virtual fitting method provided by an embodiment of the present application. In a specific embodiment, the virtual fitting method can be applied to electronic devices. Figure 1 The process shown is described in detail, and the virtual fitting method may include the following steps S110 to S130.

[0032] Step S110: Acquire the current user image and the current clothing image to be tried on.

[0033] In an embodiment of the present application, when a user needs to virtually try on clothes, he or she may send a clothes-changing instruction to an electronic device, and the electronic device receives and responds to the clothes-changing instruction to obtain an image of the current user and an image of the current clothing to be tried on.

[0034] The current user image may be a human body image of the user or another person, and is not limited here.

[0035] Regarding the process of the electronic device acquiring the current user image, in some implementations, the electronic device may pre-store the current user image, and the electronic device may read the pre-stored current user image.

[0036] Regarding the process of the electronic device acquiring the current user image, in some implementations, the electronic device may be configured with a camera, and the electronic device may capture a body image of the user through the camera, and determine the captured body image as the current user image.

[0037] Among them, the camera can be any one of a white light camera, an infrared camera, a laser camera, etc., which is not limited here.

[0038] Regarding the process of the electronic device acquiring the current user image, in some implementations, the electronic device may generate first prompt information and receive the current user image uploaded by the user according to the first prompt information.

[0039] Among them, the first prompt information can be used to prompt the user to upload the current user image to the electronic device. The first prompt information can be at least any one of text prompt information, sound prompt information or light prompt information, etc., which is not limited here.

[0040] Regarding the process of the electronic device acquiring the current clothing image to be tried on, in some embodiments, the electronic device may be pre-configured with a clothing list, which may include multiple clothing images. The electronic device may display the clothing list, receive a target clothing image selected by the user from the multiple clothing images in the clothing list, and determine the target clothing image as the current clothing image to be tried on.

[0041] Regarding the process of the above-mentioned electronic device acquiring the image of the current clothing to be tried on, in some embodiments, the electronic device may be pre-configured with a clothing information list, and the clothing information list may include multiple clothing information, and each clothing information may include at least any one of clothing material, clothing style, clothing size, and clothing color, etc., which is not limited here.

[0042] The electronic device can display a clothing information list, receive target clothing information selected by the user from multiple clothing information in the clothing information list, and input the target clothing information into the clothing generation model. The clothing generation model receives and responds to the target clothing information and outputs the current clothing image to be tried on.

[0043] The target clothing information may include at least any one of target clothing material, target clothing style, target clothing size, target clothing color, etc., which is not limited here.

[0044] The clothing generation model can be obtained by training a deep learning neural network model based on historical clothing information. The historical clothing information may include at least any one of historical clothing material, historical clothing size, and historical clothing color, etc., which is not limited here.

[0045] The deep learning neural network model can be a Convolutional Neural Networks (CNN) model, a Deep Belief Networks (DBN) model, a Stacked Auto Encoder Networks (SAE) model, a Recurrent Neural Networks (RNN) model, a Deep Neural Networks (DNN) model, a Long Short-Term Memory (LSTM) network model or a Gated Recurring Units (GRU) model, etc. The type of the deep learning neural network model is not limited here and can be set according to actual needs.

[0046] Regarding the process of the electronic device acquiring the current image of the garment to be tried on, in some implementations, the electronic device may generate a second prompt message, and receive the current image of the garment to be tried on uploaded by the user according to the second prompt message.

[0047] Among them, the second prompt information can be used to prompt the user to upload the current image of the clothing to be tried on to the electronic device. The second prompt information can be at least any one of text prompt information, sound prompt information or light prompt information, etc., which is not limited here.

[0048] In some implementations, the electronic device may detect the user's operation, and when it is determined based on the detected user operation that the user has input a clothes-changing instruction, the clothes-changing instruction is received.

[0049] For example, when a user needs to try on virtual clothes, he or she can perform touch operations on the operation panel of the electronic device. The electronic device responds to the user's touch operations, generates corresponding touch signals, and analyzes the touch signals. When it is determined that the touch signal is a preset clothes-changing signal used to represent virtual clothes-trying, it is determined that a clothes-changing instruction has been received.

[0050] In some embodiments, the electronic device may be provided with a voice recognition module. When a user needs virtual trying on, the user may send a voice message within the voice collection range of the voice recognition module. The voice recognition module collects the voice message sent by the user, and performs voice recognition on the collected voice message. Based on the recognition result of the voice recognition, it is determined that the recognition result contains keywords for indicating virtual trying on, such as "virtual trying on", "virtual" and "changing clothes", etc., and then it is determined that the changing clothes instruction has been received.

[0051] In some embodiments, when a user needs to try on clothes virtually, a clothes-changing instruction may be sent to a client. The client receives and responds to the clothes-changing instruction, forwards the clothes-changing instruction to an electronic device via a network, and the electronic device receives the clothes-changing instruction forwarded by the client.

[0052] The client is connected to the electronic device through a network and exchanges data with the electronic device through the network.

[0053] The client can be any one of a mobile client (for example, a mobile phone client, a PDA client, a Tablet PC client, a laptop client, a smart watch client, a smart bracelet client or a wearable client, etc.) or a fixed client (for example, a desktop computer client, a smart panel client, etc.). The type of client is not limited here and can be set according to actual needs.

[0054] The network may be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN), or a Wireless Personal Area Network (WPAN), etc., without limitation here.

[0055] Step S120: generating a current three-dimensional human body mask according to the current user image.

[0056] In the embodiment of the present application, after the electronic device obtains the current user image and the current image of the clothing to be tried on, it can generate a current three-dimensional human body mask according to the current user image.

[0057] The current 3D human body mask may be used to represent the 3D human body contour corresponding to the current user image.

[0058] In some embodiments, after the electronic device obtains the current user image and the current image of the clothing to be tried on, it can determine the current human body posture based on the current user image, and input the current human body posture and the current user image into the target human body mask generation model. The target human body mask generation model receives and responds to the current human body posture and the current user image, and outputs the current three-dimensional human body mask. The target human body mask generation model is obtained based on the Segment Anything model and the receptive field adaptive attention model. The receptive field adaptive attention model is used to extract the boundary features of the human body image in the current user image, which can improve the perception ability of the target human body mask generation model, improve the generation accuracy of the current three-dimensional human body mask, and make the virtual fitting effect closer to the real fitting effect, which is beneficial to improving the user's fitting experience.

[0059] Among them, the current human body posture can be used to represent the human body posture and human body shape corresponding to the current user image.

[0060] The Segment Anything model may include an image encoder, a prompt encoder, and a mask decoder, where the mask decoder is connected to the image encoder and the prompt encoder.

[0061] The receptive field adaptive attention model can be connected between the Image Encoder and the Mask Decoder, such as Figure 2 The receptive field adaptive attention model can include a convolution module, a receptive field attention module, a normalization module, a reshape broadcast module, and a rectified linear unit (ReLU) activation function module connected in sequence, as shown in Figure 3 The convolution module can also be connected to the ReLU activation function module through a residual (shortcut). Through the shortcut connection, the receptive field adaptive attention model can learn deeper features, which is beneficial to improving the generation accuracy of the target human mask generation model.

[0062] The convolution module can be used to extract features of the current user image to obtain an initial feature map; the receptive field attention module can be used to extract boundary features of the human body image in the initial feature map to obtain a receptive field attention map, and the attention value range of the receptive field attention map is [0, 1]; the normalization module can be used to normalize the receptive field attention map to obtain a normalized feature map; the reshaping broadcast module can be used to reshape the dimension of the attention weight of the normalized feature map to the same dimension as the initial feature map to obtain a reshaped feature map, and broadcast the reshaped feature map; the ReLU activation function module can be used to perform rectified linear unit activation function processing on the reshaped feature map.

[0063] The electronic device can input the current user image into the Densepose model. The Densepose model receives and responds to the current user image and outputs the current human joint point positions. The electronic device generates the current human body posture according to the current human body joint point positions and generates the human body posture based on the human body joint point positions detected by the Densepose model, thereby improving the accuracy of human body posture generation.

[0064] In some embodiments, after the electronic device obtains the current user image and the current image of the clothing to be tried on, it can determine the current human body posture based on the current user image, obtain historical user images, determine historical human body postures based on the historical user images, construct a receptive field adaptive attention model, and fuse the Segment Anything model and the receptive field adaptive attention model to obtain an initial human body mask generation model, and input the historical human body postures and historical user images into the initial human body mask generation model for training to obtain a target human body mask generation model, and input the current human body posture and the current user image into the target human body mask generation model, the target human body mask generation model receives and responds to the current human body posture and the current user image, and outputs the current three-dimensional human body mask.

[0065] The historical user images may be human images including character information such as different body shapes, different postures, different skin colors, different genders, and different ages.

[0066] The electronic device can input historical user images to the Densepose model. The Densepose model receives and responds to the historical user images and outputs historical human joint point positions. The electronic device generates historical human body postures based on the historical human body joint point positions.

[0067] The electronic device can fuse the convolution module, the receptive field attention module, the normalization module, the reshaping broadcast module and the ReLU activation function module to obtain the receptive field adaptive attention model.

[0068] Step S130: input the current 3D human body mask and the current clothing image to be tried on into the dressing model to obtain a target dressing image.

[0069] In an embodiment of the present application, after the electronic device generates a current three-dimensional human body mask based on the current user image, the current three-dimensional human body mask and the current image of the clothing to be tried on can be input into the dressing model. The dressing model receives and responds to the current three-dimensional human body mask and the current image of the clothing to be tried on, and outputs a target dressing image. This realizes the simulation of the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose online shopping clothing based on the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0070] The target dressing image may be used to simulate a fitting image of a current user corresponding to the current user image trying on a current garment to be tried on corresponding to a current garment image to be tried on.

[0071] The dressing model can be obtained by training a deep learning neural network model based on historical images of clothing to be tried on and historical three-dimensional human body masks.

[0072] The historical clothing images to be tried on may be clothing images including clothing information such as different clothing materials, different clothing styles, different clothing colors, and different clothing sizes.

[0073] The historical three-dimensional human body masks can be generated by training an initial human body mask generation model based on historical human body postures and historical user images.

[0074] In some embodiments, the electronic device inputs the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model. After obtaining the target dressing image, the target dressing image can be sent to the social platform through the network. The social platform receives and responds to the target dressing image and shares the target dressing image, which is conducive to improving the user's fitting experience.

[0075] The solution provided by the present application obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, and inputs the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model to obtain the target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical images of the clothing to be tried on and historical three-dimensional human body masks. It realizes the simulation of the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0076] See also Figure 4 , which shows a flowchart of a virtual fitting method provided by another embodiment of the present application. In a specific embodiment, the virtual fitting method can be applied to electronic devices. Figure 4 The process shown is described in detail, and the virtual fitting method may include the following steps S210 to S250.

[0077] Step S210: Acquire the current user image and the current clothing image to be tried on.

[0078] Step S220: Generate a current three-dimensional human body mask according to the current user image.

[0079] Step S230: input the current 3D human body mask and the current clothing image to be tried on into the dressing model to obtain a target dressing image.

[0080] In this embodiment, step S210, step S220 and step S230 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be described in detail here.

[0081] Step S240: obtaining the clothing material corresponding to the current clothing image to be tried on.

[0082] In this embodiment, the electronic device inputs the current 3D human body mask and the current image of clothing to be tried on into the dressing model, and after obtaining the target dressing image, the clothing material corresponding to the current image of clothing to be tried on can be obtained.

[0083] In some embodiments, the electronic device inputs the current three-dimensional human body mask and the current clothing image to be tried on into the dressing model. After obtaining the target dressing image, the current clothing image to be tried on can be analyzed to obtain the clothing material corresponding to the current clothing image to be tried on.

[0084] Among them, the electronic device can input the current clothing image to be tried on into the clothing material analysis model, and the clothing material analysis model receives and responds to the current clothing image to be tried on, and outputs the clothing material corresponding to the current clothing image to be tried on.

[0085] The clothing material analysis model can be obtained by training a deep learning neural network model based on historical clothing images to be tried on that are labeled with clothing materials.

[0086] In some embodiments, the electronic device inputs the current three-dimensional human body mask and the current image of clothing to be tried on into the dressing model. After obtaining the target dressing image, it can generate a third prompt information and receive the clothing material corresponding to the current image of clothing to be tried on uploaded by the user according to the third prompt information.

[0087] Among them, the third prompt information can be used to prompt the user to upload the clothing material corresponding to the current clothing image to be tried on to the electronic device. The third prompt information can be at least any one of text prompt information, sound prompt information or light prompt information, etc., which is not limited here.

[0088] Step S250: performing wrinkle processing and / or stretching processing on the target clothing-changing image according to the mask vertices of the current three-dimensional human body mask and the clothing material.

[0089] In this embodiment, after the electronic device obtains the clothing material corresponding to the current clothing image to be tried on, it can perform wrinkle processing and / or stretching processing on the clothing image in the target dressing image according to the mask vertices and clothing material of the current three-dimensional human body mask, so that the wrinkled and / or stretched clothing is more adapted to the human body posture, which is conducive to improving the user's fitting effect.

[0090] The solution provided in this embodiment obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, inputs the current three-dimensional human body mask and the current image of the clothing to be tried on to the dressing model, obtains the target dressing image, obtains the clothing material corresponding to the current image of the clothing to be tried on, and performs wrinkle processing and / or stretching processing on the target dressing image according to the mask vertices and clothing material of the current three-dimensional human body mask, thereby simulating the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model, and the user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of the clothing.

[0091] Furthermore, according to the mask vertices and clothing materials of the current three-dimensional human body mask, the clothing image in the target dressing image is wrinkled and / or stretched so that the wrinkled and / or stretched clothing is more adapted to the human body posture, which is beneficial to improving the user's fitting effect.

[0092] See also Figure 5 , which shows a flowchart of a virtual fitting method provided by another embodiment of the present application. In a specific embodiment, the virtual fitting method can be applied to electronic devices. Figure 5 The process shown is described in detail, and the virtual fitting method may include the following steps S310 to S350.

[0093] Step S310: Acquire the current user image and the current clothing image to be tried on.

[0094] Step S320: Generate a current three-dimensional human body mask according to the current user image.

[0095] Step S330: input the current 3D human body mask and the current clothing image to be tried on into the dressing model to obtain a target dressing image.

[0096] In this embodiment, step S310, step S320 and step S330 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be described in detail here.

[0097] Step S340: Obtain the human skin color corresponding to the current user image.

[0098] In this embodiment, the electronic device inputs the current three-dimensional human body mask and the current image of clothing to be tried on into the dressing model, and after obtaining the target dressing image, the human skin color corresponding to the current user image can be obtained.

[0099] In some embodiments, the electronic device inputs the current three-dimensional human body mask and the current image of clothing to be tried on into the dressing model. After obtaining the target dressing image, the current user image can be analyzed to obtain the human skin color corresponding to the current user image.

[0100] Among them, the electronic device can input the current user image into the human skin color detection model, and the human skin color detection model receives and responds to the current user image and outputs the human skin color corresponding to the current user image.

[0101] The human skin color detection model can be obtained by training a deep learning neural network model based on historical user images labeled with human skin color.

[0102] In some embodiments, the electronic device inputs the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model. After obtaining the target dressing image, it can generate a fourth prompt message and receive the human skin color corresponding to the current user image uploaded by the user according to the fourth prompt message.

[0103] Among them, the fourth prompt information can be used to prompt the user to upload the human skin color corresponding to the current user image to the electronic device. The fourth prompt information can be at least any one of text prompt information, sound prompt information or light prompt information, etc., which is not limited here.

[0104] In some implementations, after the electronic device obtains the human skin color corresponding to the current user image, it can obtain the target clothing color in the target clothing-changing image and determine whether the target clothing color matches the human skin color.

[0105] The target clothing color in the target dressing image may be the same as the clothing color of the current clothing image to be tried on, or may be different from the clothing color of the current clothing image to be tried on, etc., which is not limited here.

[0106] The electronic device can match the target clothing color with the human skin color to obtain a color matching degree, and determine whether the target clothing color matches the human skin color according to the color matching degree.

[0107] When the color matching degree is greater than or equal to the matching degree threshold, it is determined that the target clothing color matches the human skin color; when the color matching degree is less than the matching degree threshold, it is determined that the target clothing color matches the human skin color.

[0108] The matching threshold can be used to characterize the minimum color matching degree between the target clothing color and the human skin color. The matching threshold can be a color matching degree preset by the user, or a color matching degree automatically generated by the electronic device according to multiple virtual fitting processes.

[0109] Regarding the process of the electronic device acquiring the target clothing color in the target clothing changing image, as an implementation mode, the electronic device may input the target clothing changing image into a clothing color extraction model, and the clothing color extraction model receives and responds to the target clothing changing image and outputs the target clothing color.

[0110] Among them, the clothing color extraction model can be obtained by training a deep learning neural network model based on historical clothing change images annotated with historical clothing colors.

[0111] Regarding the process of the electronic device acquiring the target clothing color in the target dressing image, as an implementation mode, the electronic device may generate fifth prompt information and receive the target clothing color in the target dressing image uploaded by the user according to the fifth prompt information.

[0112] Among them, the fifth prompt information can be used to prompt the user to upload the target clothing color in the target dressing image to the electronic device. The fifth prompt information can be at least any one of text prompt information, sound prompt information or light prompt information, etc., which is not limited here.

[0113] Step S350: When it is determined that the target clothing color in the target clothing-changing image does not match the human skin color, the target clothing color is adjusted to match the human skin color.

[0114] In this embodiment, when the electronic device determines that the target clothing color in the target dressing image does not match the human skin color, the target clothing color is adjusted to match the human skin color, so that the clothing in the target dressing image is more compatible with the human skin color, which is beneficial to improving the user's fitting effect.

[0115] The solution provided in this embodiment obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, inputs the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model, obtains the target dressing image, obtains the human skin color corresponding to the current user image, and when it is determined that the target clothing color in the target dressing image does not match the human skin color, adjusts the target clothing color to match the human skin color, thereby simulating the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model, and the user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0116] Furthermore, when the electronic device determines that the target clothing color in the target dressing image does not match the human skin color, the target clothing color is adjusted to match the human skin color, so that the clothing in the target dressing image is more compatible with the human skin color, which is beneficial to improving the user's fitting effect.

[0117] See also Figure 6 , which shows a virtual fitting device 500 provided by an embodiment of the present application. The virtual fitting device 500 can be applied to electronic devices. The following takes electronic devices as an example. Figure 6 The virtual fitting device 500 shown in FIG. 1 is described in detail. The virtual fitting device 500 may include a first acquisition module 510 , a generation module 520 and a first input module 530 .

[0118] The first acquisition module 510 can be used to acquire the current user image and the current image of the clothing to be tried on; the generation module 520 can be used to generate the current three-dimensional human body mask based on the current user image; the first input module 530 can be used to input the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model to obtain the target dressing image. The dressing model can be obtained by training a deep learning neural network model based on historical clothing images and historical three-dimensional human body masks.

[0119] In some implementations, the generation module 520 may include a determination unit and an input unit.

[0120] The determination unit can be used to determine the current human body posture based on the current user image; the input unit can be used to input the current human body posture and the current user image into the target human body mask generation model to obtain the corresponding current three-dimensional human body mask. The target human body mask generation model can be obtained based on the Segment Anything model and the receptive field adaptive attention model. The receptive field adaptive attention model can be used to extract the boundary features of the human body image in the current user image.

[0121] In some implementations, the virtual fitting device 500 may further include a second acquisition module, a determination module, a construction module, a fusion module, and a second input module.

[0122] The second acquisition module can be used to input the current human body posture and the current user image into the target human body mask generation model through the input unit, and obtain the historical user image before obtaining the corresponding current three-dimensional human body mask; the determination module can be used to determine the historical human body posture according to the historical user image; the construction module can be used to construct the receptive field adaptive attention model; the fusion module can be used to fuse the Segment Anything model and the receptive field adaptive attention model to obtain the initial human body mask generation model; the second input module can be used to input the historical human body posture and the historical user image into the initial human body mask generation model for training to obtain the target human body mask generation model.

[0123] In some embodiments, a building block can include a fusion unit.

[0124] The fusion unit can be used to fuse the convolution module, the receptive field attention module, the normalization module, the reshape broadcast module, and the rectified linear unit activation function module to obtain the receptive field adaptive attention model.

[0125] The convolution module can be used to extract features of the current user image to obtain an initial feature map; the receptive field attention module can be used to extract boundary features of the human body image in the initial feature map to obtain a receptive field attention map, and the attention value range of the receptive field attention map can be [0, 1]; the normalization module can be used to normalize the receptive field attention map to obtain a normalized feature map; the reshaping broadcast module can be used to reshape the dimension of the attention weight of the normalized feature map to the same dimension as the initial feature map to obtain a reshaped feature map, and broadcast the reshaped feature map; the rectified linear unit activation function module can be used to perform rectified linear unit activation function processing on the reshaped feature map.

[0126] In some embodiments, the determination unit may input a subunit as well as a generation subunit.

[0127] The input subunit can be used to input the current user image into the DensePose model to obtain the current human joint point position; the generation subunit can be used to generate the current human body posture according to the current human body joint point position.

[0128] In some implementations, the virtual fitting device 500 may further include a third acquisition module and a processing module.

[0129] The third acquisition module can be used to obtain the clothing material corresponding to the current clothing image to be tried on; the processing module can be used to perform wrinkle processing and / or stretching processing on the target clothing change image according to the mask vertices and clothing material of the current three-dimensional human body mask.

[0130] In some implementations, the virtual fitting device 500 may further include a fourth acquisition module and an adjustment module.

[0131] The fourth acquisition module can be used to obtain the human skin color corresponding to the current user image; the adjustment module can be used to adjust the target clothing color to match the human skin color when it is determined that the target clothing color in the target dressing image does not match the human skin color.

[0132] The solution provided in this embodiment obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, and inputs the current three-dimensional human body mask and the current image of the clothing to be tried on to the dressing model to obtain the target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical images of the clothing to be tried on and historical three-dimensional human body masks. It realizes the simulation of the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0133] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0134] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0135] See also Figure 7 , which shows a functional block diagram of an electronic device 600 provided by an embodiment of the present application, and the electronic device 600 may include one or more of the following components: a memory 610, a processor 620, and one or more applications, wherein the one or more applications may be stored in the memory 610 and configured to be executed by the one or more processors 620, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0136] The memory 610 may include a random access memory (RAM) or a read-only memory (ROM). The memory 610 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 610 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a current user image, obtaining a current image of clothing to be tried on, generating a current three-dimensional human body mask, inputting a current three-dimensional human body mask, inputting a current image of clothing to be tried on, obtaining a target dressing image, training a deep learning neural network model, determining a current human body posture, inputting a current human body posture, inputting a current user image, obtaining a current three-dimensional human body mask, extracting boundary features, obtaining historical user images, determining historical human body postures, constructing a receptive field adaptive attention model, fusing a Segment Anything model and a receptive field adaptive attention model, obtaining an initial human body mask generation model, inputting historical human body postures, inputting historical user images, training an initial human body mask generation model, obtaining a target human body mask generation model, obtaining the current human body joint point position, obtaining clothing material, wrinkle processing, stretching processing, obtaining human skin color, and adjusting the target clothing color, etc.), instructions for implementing the following various method embodiments, etc. The data storage area can also store data created by the electronic device 600 during use (such as current user image, current image of clothing to be tried on, current three-dimensional human body mask, dressing model, target dressing image, historical clothing images to be tried on, historical three-dimensional human body masks, deep learning neural network model, current human body posture, target human body mask generation model, Segment Anything model, receptive field adaptive attention model, human body image, boundary features, historical user images, historical human body posture, initial human body mask generation model, convolution module, receptive field attention module, normalization module, reshaping broadcast module, rectified linear unit activation function module, initial feature map, receptive field attention map, attention value range, [0, 1], normalized feature map, attention weight, reshaping feature map, DensePose model, current human joint point position, clothing material, mask vertices, human skin color and target clothing color), etc.

[0137] The processor 620 may include one or more processing cores. The processor 620 uses various interfaces and lines to connect various parts of the entire electronic device 600, and executes various functions and processes data of the electronic device 600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 610, and calling data stored in the memory 610. Optionally, the processor 620 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), etc. The processor 620 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 620, but may be implemented separately through a communication chip.

[0138] Please refer to Figure 8 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 700 stores a program code 710, which can be called by a processor to execute the method described in the above method embodiment.

[0139] The computer-readable storage medium 700 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has storage space for program code 710 for executing any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 710 can be compressed, for example, in an appropriate form.

[0140] Please refer to Fig. 9, which shows a structural block diagram of a computer program product 800 provided in an embodiment of the present application. The computer program product 800 includes a computer program / instruction 810, and the computer program / instruction 810 is stored in a computer-readable storage medium of a computer device. When the computer program product 800 is run on a computer device, a processor of the computer device reads the computer program / instruction 810 from the computer-readable storage medium, and the processor executes the computer program / instruction 810, so that the computer device performs the method described in the above method embodiment.

[0141] The solution provided in this embodiment obtains the current user image and the current image of the clothing to be tried on, generates the current three-dimensional human body mask according to the current user image, and inputs the current three-dimensional human body mask and the current image of the clothing to be tried on to the dressing model to obtain the target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical images of the clothing to be tried on and historical three-dimensional human body masks. It realizes the simulation of the user trying on the clothing to be tried on based on the user image, the image of the clothing to be tried on and the dressing model. The user can choose the online shopping clothing according to the trying-on effect, which is conducive to improving the online shopping efficiency of clothing.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual fitting method, characterized in that: include: Get the current user image and the current clothing image to be tried on; Generate a current three-dimensional human body mask according to the current user image; The current three-dimensional human body mask and the current clothing image to be tried on are input into the dressing model to obtain a target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical clothing images to be tried on and historical three-dimensional human body masks.

2. The virtual fitting method according to claim 1, characterized in that: The generating a current three-dimensional human body mask according to the current user image comprises: Determine a current human body posture according to the current user image; The current human body posture and the current user image are input into the target human body mask generation model to obtain the corresponding current three-dimensional human body mask. The target human body mask generation model is obtained based on the Segment Anything model and the receptive field adaptive attention model. The receptive field adaptive attention model is used to extract the boundary features of the human body image in the current user image.

3. The virtual fitting method according to claim 2, characterized in that: Before inputting the current human body posture and the current user image into the target human body mask generation model to obtain the corresponding current three-dimensional human body mask, the virtual fitting method further includes: Get historical user images; Determining historical human body postures based on the historical user images; Constructing the receptive field adaptive attention model; The Segment Anything model and the receptive field adaptive attention model are integrated to obtain an initial human mask generation model; The historical human body postures and the historical user images are input into the initial human body mask generation model for training to obtain the target human body mask generation model.

4. The virtual fitting method according to claim 3, characterized in that: The constructing of the receptive field adaptive attention model includes: The receptive field adaptive attention model is obtained by fusing the convolution module, the receptive field attention module, the normalization module, the reshaping broadcast module and the rectified linear unit activation function module; The convolution module is used to extract features of the current user image to obtain an initial feature map; the receptive field attention module is used to extract the boundary features of the human body image in the initial feature map to obtain a receptive field attention map, and the attention value range of the receptive field attention map is [0, 1]; the normalization module is used to normalize the receptive field attention map to obtain a normalized feature map; the reshaping broadcast module is used to reshape the dimension of the attention weight of the normalized feature map to the same dimension as the initial feature map to obtain a reshaped feature map, and broadcast the reshaped feature map; the rectified linear unit activation function module is used to perform rectified linear unit activation function processing on the reshaped feature map.

5. The virtual fitting method according to claim 2, characterized in that: The determining a current human body posture according to the current user image comprises: Input the current user image into the DensePose model to obtain the current human body joint position; The current human body posture is generated according to the current human body joint point positions.

6. The virtual fitting method according to any one of claims 1 to 5, characterized in that: Also includes: Obtaining the clothing material corresponding to the current clothing image to be tried on; According to the mask vertices of the current three-dimensional human body mask and the clothing material, the target clothing-changing image is subjected to wrinkle processing and / or stretching processing.

7. The virtual fitting method according to any one of claims 1 to 5, characterized in that: Also includes: Obtaining the human skin color corresponding to the current user image; When it is determined that the target clothing color in the target clothing-changing image does not match the human skin color, the target clothing color is adjusted to match the human skin color.

8. A virtual fitting device, characterized in that: include: The first acquisition module is used to acquire the current user image and the current image of the garment to be tried on; A generating module, used for generating a current three-dimensional human body mask according to the current user image; The first input module is used to input the current three-dimensional human body mask and the current image of the clothing to be tried on into the dressing model to obtain a target dressing image. The dressing model is obtained by training a deep learning neural network model based on historical clothing images and historical three-dimensional human body masks.

9. An electronic device, characterized in that: include: Memory; One or more processors coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the virtual fitting method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the virtual fitting method according to any one of claims 1 to 7.

Images