Virtual fitting method and device, storage medium and computer program product

Abstract

The invention provides a virtual fitting method and device, a storage medium and a computer program product. The method comprises the steps that a human body-body splicing vector corresponding to a target human body image uploaded through a display interface is acquired; in response to a selection touch operation for a first clothing template image in a clothing template display area of the display interface, obtaining a first clothing code vector and a first clothing opening coefficient of the first clothing template image; obtaining first information input in an open coefficient input field of the display interface, and responding to a first touch operation for a generation button of the display interface, and displaying a first clothing migration image generated according to the human body-body splicing vector, the first clothing coding vector, the first clothing opening coefficient and the first information in a clothing migration display area of the display interface. The user experience can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a virtual try-on method, device, storage medium, and computer program product. Background Technology

[0002] Clothing replacement typically uses a single garment and person as the encoding, which is then fed into a generator network for TPS interpolation and deformation to obtain the final result. Since the deformation is done on the clothing, it does not take into account the adaptability of different characteristics of the wearer (height, weight, body shape), making it look more like a texture, resulting in a poor user experience. Summary of the Invention

[0003] This application provides a virtual try-on method, device, storage medium, and computer program product that can improve the user experience.

[0004] To achieve the above objectives, the technical solution of this application is implemented as follows: Firstly, this application proposes a virtual try-on method, the method comprising: Obtain the human body-shape stitching vector corresponding to the target human body image uploaded through the display interface; In response to a touch operation that selects a first garment template image in the garment template display area of ​​the display interface, the first garment encoding vector and the first garment opening coefficient of the first garment template image are obtained; The system acquires the first information entered in the opening coefficient input field of the display interface and responds to the first touch operation of the generate button on the display interface, and displays the first garment migration image generated based on the human body-shape splicing vector, the first garment code vector, the first garment opening coefficient, and the first information in the garment migration display area of ​​the display interface.

[0005] Secondly, this application proposes an electronic device, comprising: a processor and a memory for storing a computer program capable of running on the processor. When the processor runs the computer program, it performs the steps of any of the methods described above.

[0006] Thirdly, this application proposes a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described above.

[0007] Fourthly, this application proposes a computer program product, including a computer program that, when executed by a processor, performs the steps of any of the methods described above.

[0008] This application proposes a virtual fitting method, device, storage medium, and computer program product. The method includes: acquiring a human-body stitching vector corresponding to a target human image uploaded via a display interface; in response to a touch operation selecting a first clothing template image in a clothing template display area of ​​the display interface, acquiring a first clothing code vector and a first clothing opening coefficient of the first clothing template image; acquiring first information entered in the opening coefficient input field of the display interface and, in response to a first touch operation of a generate button on the display interface, displaying a first clothing migration image generated based on the human-body stitching vector, the first clothing code vector, the first clothing opening coefficient, and the first information in a clothing migration display area of ​​the display interface. Using the above scheme, a first clothing transfer image is generated based on the first clothing encoding vector and the first clothing opening coefficient corresponding to the first clothing template image, the human body-shape stitching vector corresponding to the target human body image, and the first input information in the opening coefficient input field. It takes into account the human body and shape corresponding to the person in the target human body image, generates adapted clothing transfer images of different sizes for people with different shapes, and can generate first clothing transfer images with different opening coefficients in the opening coefficient input field according to different input information. That is, the opening coefficient of the first clothing transfer image can be adaptively adjusted, thereby improving the user experience. Attached Figure Description

[0009] Figure 1 A flowchart illustrating a virtual try-on method provided in an embodiment of this application; Figure 2 A schematic diagram of an exemplary display interface provided for an embodiment of this application; Figure 3 A flowchart illustrating an exemplary adaptive method for replacing a target person with given clothing, provided in an embodiment of this application; Figure 4 A schematic diagram of an exemplary garment migration image under different exposure coefficients, provided for embodiments of this application; Figure 5 This is a schematic diagram of the structure of a virtual fitting device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0012] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first," "second," etc., used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," etc., may be interchanged in a specific order or sequence where permissible, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0013] Clothing swapping typically involves encoding and decoding image information such as clothing via the network and then interpolating and deforming it using Transmission Parameter Signalling (TPS). This method often results in inaccurate reproduction of the original clothing details. Because it deforms the clothing, it doesn't consider the adaptability to different wearers (height, weight, body type). Furthermore, current clothing swapping technologies can only transfer detailed features of the clothing and cannot adaptively adjust these details according to user needs, lacking personalized adjustment methods. This leads to the generated image altering the clothing itself (color, details, texture); and resulting in inconsistencies when the generated image is combined with the target person. For example, if the clothing doesn't match the target person's body type, adaptive dressing is not possible; and the resulting image combined with the target person appears rigid, simply replacing and blending, resembling a static texture rather than a flexible application.

[0014] Conventional clothing replacement methods typically encode individual garments and characters, then feed them into a generative network to perform the replacement and obtain a complete result. However, this technique suffers from limited training datasets, failing to cover all real-world examples. During network inference, the limited input information, lacking sufficient and effective information units, leads to significant network divergence, resulting in greater randomness in the generated results, insufficient controllability, and a low satisfaction rate.

[0015] Based on this, embodiments of this application provide a virtual try-on method. Figure 1 A flowchart illustrating a virtual try-on method provided in this application embodiment; as shown Figure 1 As shown, the method includes: S101. Obtain the human body-shape splicing vector corresponding to the target human body image uploaded through the display interface.

[0016] It should be noted that the display interface can be understood as the display interface corresponding to the virtual fitting application and / or virtual fitting device. The virtual fitting application can be an application on an electronic device or a virtual fitting device. The electronic device can be any device capable of displaying a picture; in practical applications, the electronic device can be a mobile phone, computer, etc. The display interface includes an image display area for the target human body image, a clothing template display area, an opening coefficient input field, a generate button, and a clothing migration display area. The uploaded target human body image is displayed in the image display area. The positional relationship of the various areas on the display interface can be determined according to the actual situation and is not limited here. As an example, the positions of the image display area and the target clothing display area are symmetrical with the positions of the opening coefficient input field, the generate button, and the clothing migration display area. The image display area and the target garment display area are located on the left side of the display interface, while the opening coefficient input field, the generate button, and the garment migration display area are located on the right side; or the image display area and the target garment display area are located on the right side of the display interface, while the opening coefficient input field, the generate button, and the garment migration display area are located on the left side. The positional relationships between the image display area and the target garment display area, and between the opening coefficient input field, the generate button, and the garment migration display area, can be determined based on the actual situation and are not limited here. For ease of understanding, Figure 2 A schematic diagram illustrating an exemplary display interface provided in an embodiment of this application, such as... Figure 2 As shown, the display interface includes an image display area, a garment template display area, an opening coefficient input field, a generate button, and a garment migration display area. It should be noted that any layout of the display interface that includes the image display area, garment template display area, opening coefficient input field, generate button, and garment migration display area is considered part of the technical solution protected by this application, and is not limited to this. Figure 2 The layout shown.

[0017] It should be noted that the target human image can be understood as the image corresponding to the person to be tried on, who is wearing a third garment. The third garment can be any garment, and the specific details can be determined based on the actual situation; no restrictions are imposed here.

[0018] In this embodiment, the human-shape stitching vector can be understood as the vector obtained by stitching together the human body encoding vector and the shape encoding vector corresponding to the target human body image. The process of obtaining the human-shape stitching vector corresponding to the target human body image uploaded via the display interface specifically includes: receiving the target human body image and inputting it into a human body encoding network for encoding to obtain a human body encoding vector; inputting the target human body image into a shape parameter encoding network for encoding to obtain a shape parameter encoding vector; and stitching the human body encoding vector and the shape parameter encoding vector together to obtain the human-shape stitching vector.

[0019] It should be noted that the human body encoding network is an encoding network trained on a large amount of human body data. The body shape parameter encoding network is a human body shape parameter encoding network trained on a large amount of body shape parameters. Receiving a target human body image can be understood as the electronic device receiving the target human body image, or it can be illustrated as uploading the target human body image to the electronic device.

[0020] In this embodiment of the application, the process of inputting the target human body image into the human body coding network for encoding to obtain the human body coding vector specifically includes: segmenting the target human body image to obtain a third clothing image and a human body image; stitching the target human body image, the third clothing image, and the human body image together to obtain the target human body stitched vector; and inputting the target human body stitched vector into the human body coding network for encoding to obtain the human body coding vector.

[0021] It should be noted that the third clothing image can be understood as the image corresponding to the third type of clothing worn by the person in the target human body image; the human body image can be understood as the image corresponding to the human body in the target human body image excluding the third clothing. Segmenting the target human body image to obtain the third clothing image and the human body image can be understood as follows: inputting the target human body image into a segmentation network, using the clothing portion as the segmentation condition, yields a segmentation map, which consists of the third clothing image and the human body image. In practical applications, the target human body image can be denoted as DST; the third clothing image and the human body image can be collectively denoted as dst_mask.

[0022] In this embodiment of the application, the process of stitching together the target human body image, the third clothing image, and the human body image to obtain the target human body stitching vector specifically includes: extracting the pose of the target human body image to obtain the target human body pose image; extracting the shape of the target human body image to obtain the target human body shape image; and stitching together the target human body image, the third clothing image, the human body image, the target human body pose image, and the target human body shape image to obtain the target human body stitching vector.

[0023] It should be noted that extracting the pose of the target human image to obtain the target human pose image can be understood as inputting the target human image into a pose extraction network to obtain the target human pose image. The pose extraction network can be any network capable of extracting poses, and the specific one can be determined according to the actual situation, without limitation here. Extracting the shape of the target human image to obtain the target human shape image can be understood as inputting the target human image into a shape parameter network model to obtain the target human shape image. The target human shape image can also be understood as the target human shape parameter information. The shape parameter network model can be any network capable of extracting shape parameters, and the specific one can be determined according to the actual situation, without limitation here. Concatenating the target human image, the third clothing image, the human image, the target human pose image, and the target human shape image to obtain the target human concatenation vector can be understood as concatenating the target human image, the third clothing image, the human image, the target human pose image, and the target human shape image along the channel dimension to obtain the target human concatenation vector.

[0024] In practical applications, the target human body pose image can be denoted as dst_pose; the target human body shape image can be denoted as dst_body; and the target human body stitching vector can be denoted as dst_all. The target human body image, the third clothing image, the human body image, the target human body pose image, and the target human body shape image are stitched together to obtain the target human body stitching vector. For example, dst, dst_mask, dst_pose, and dst_body are stitched together along the channel dimension, and the stitching result is denoted as dst_all.

[0025] In this embodiment, the target human body image is input into a body parameter encoding network for encoding to obtain a body parameter encoding vector; the human body encoding vector and the body parameter encoding vector are concatenated to obtain a human body-body concatenation vector; this can be understood as inputting the target human body image into a trained human body shape parameter encoding network to obtain body parameter encoding, and concatenating the human body encoding vector and the body parameter encoding vector to obtain a human body-body concatenation vector.

[0026] The embodiment of this application encodes the target human body image by inputting it into a human body encoding network and a shape parameter encoding network, and then concatenates the human body encoding vector and the shape parameter encoding vector to obtain a human body-shape concatenation vector. By taking into account the shape parameters of the human body, the first clothing transfer image generated subsequently is more in line with the human body shape, thereby improving the user experience.

[0027] S102, in response to a touch operation that selects a first garment template image in the garment template display area of ​​the display interface, obtain the first garment encoding vector and the first garment opening coefficient of the first garment template image.

[0028] It should be noted that the number of clothing template images in the clothing template display area can be determined based on the actual situation and is not limited here. The first clothing template image can be understood as the image that the user selects from the clothing template display area for clothing migration.

[0029] In this embodiment of the application, the first clothing template image includes multiple second clothing images with different resolutions and / or shooting angles; in response to a touch operation that selects the first clothing template image in the clothing template display area of ​​the display interface, the method further includes: encoding the multiple second clothing images to obtain multiple second clothing encoding vectors; and concatenating the multiple second clothing encoding vectors to obtain a first clothing encoding vector.

[0030] It should be noted that the first garment template image includes multiple second garment images with different resolutions and / or shooting angles; multiple second garment images can be understood as multiple images of the same garment at different resolutions and / or shooting angles. The number of second garment images can be determined according to the actual situation and is not limited here. As an example, the maximum number of second garment images is 10. Resolution and shooting angle can be determined according to the actual situation and are not limited here. As an example, resolution can include 256*256 resolution, 320*320 resolution, 512*512 resolution; shooting angle can include front shooting, side shooting, etc. Frequency and shooting angle can also be used in combination.

[0031] In this embodiment, multiple second clothing images are encoded to obtain multiple second clothing encoding vectors. This can be understood as inputting each second clothing image into a clothing encoding network to obtain multiple second clothing encoding vectors. This clothing encoding network is trained on a large clothing dataset and includes a cross-attention module.

[0032] It should be noted that concatenating multiple second clothing coding vectors to obtain the first clothing coding vector can be understood as concatenating multiple second clothing coding vectors along the same dimension to obtain the first clothing coding vector.

[0033] In this embodiment of the application, the process of concatenating multiple second clothing code vectors to obtain a first clothing code vector specifically includes: when the quantity information of multiple second clothing code vectors is equal to a first value, concatenating multiple second clothing code vectors to obtain a first clothing code vector; when the quantity information is less than the first value, repeatedly concatenating multiple second clothing code vectors until the quantity information of the concatenated second clothing code vectors is equal to the first value, thereby obtaining the first clothing code vector.

[0034] It should be noted that the first value can be determined based on the actual situation and is not limited here. As an example, the first value can be 10. The quantity information of multiple second garment coding vectors is the same as the quantity information of multiple second garments. When the quantity information of multiple second garment coding vectors is equal to the first value, the multiple second garment coding vectors are concatenated to obtain the first garment coding vector. This can be understood as concatenating 10 second garment coding vectors along the same dimension when the quantity of multiple second garment coding vectors is 10 to obtain the first garment coding vector.

[0035] It should be noted that the case where the quantity information is less than the first value can be understood as the case where the quantity information of the second garment coding vector is less than 10. When the quantity information is less than the first value, multiple second garment coding vectors are repeatedly concatenated until the quantity information of the concatenated second garment coding vector equals the first value, resulting in the first garment coding vector. This can be understood as follows: when the quantity information of multiple second garment coding vectors is less than 10, first determine the quantity of existing second garment coding vectors and the difference between that quantity and 10; then concatenate the existing multiple second garment coding vectors along the same dimension to obtain the third garment coding vector; finally, select the second garment coding vector with the difference in quantity from the existing multiple second garment coding vectors and concatenate it with the third garment coding vector along the same dimension to obtain the first garment coding vector. The process of selecting the second garment coding vector with the difference in quantity from the existing multiple second garment coding vectors can be arbitrary or can be based on the order of the multiple second garment coding vectors (the order in which the existing multiple second garment coding vectors are concatenated). The specific method can be determined according to the actual situation and is not limited here.

[0036] The solution in this application improves the color, detail, texture, and fit of the generated clothing by inputting multiple second clothing images (including images with different resolutions and / or angles). Since the clothing encoding network used to encode multiple second clothing images is a trained network and includes a cross-attention mechanism, it can enhance the ability to preserve clothing details.

[0037] In this embodiment of the application, the first garment template image includes multiple second garment images with different resolutions and / or shooting angles; in response to a touch operation that selects the first garment template image in the garment template display area of ​​the display interface, the method further includes: obtaining multiple second garment opening coefficients corresponding to the multiple second garment images; and using the multiple second garment opening coefficients as the first garment opening coefficient.

[0038] It should be noted that multiple second garment opening coefficients can be understood as the opening coefficients of zipper openings, cuff openings, etc., corresponding to multiple second garment images. Obtaining multiple second garment opening coefficients corresponding to multiple second garment images can be understood as inputting multiple second garment images into a regression network to obtain multiple second garment opening coefficients. The first garment opening coefficient is equivalent to multiple second garment opening coefficients.

[0039] It should be noted that after determining the first garment code vector and the first opening coefficient corresponding to the first garment template image, the first garment template image, the first garment code vector, and the first opening coefficient are saved to the garment template display area. The method for determining the garment code vector and opening coefficient corresponding to each garment template image in the garment template display area is the same as the method for determining the first garment template image, and will not be repeated here.

[0040] The solution in this application embodiment adds input conditions such as zipper opening and cuff opening, which makes it easier for the subsequently generated first garment migration image to fit the target human body better, and also makes it easier to adjust the opening in the first garment migration image according to the custom input opening information, thereby improving the display effect and enhancing the user experience.

[0041] S103. Obtain the first information entered in the opening coefficient input field of the display interface and respond to the first touch operation of the generate button of the display interface, and display the first clothing migration image generated based on the human body-shape splicing vector, the first clothing code vector, the first clothing opening coefficient and the first information in the clothing migration display area of ​​the display interface.

[0042] It should be noted that the first piece of information can be understood as a custom exposure coefficient. This coefficient can range from 0 to 1. As an example, when the custom exposure coefficient is 0, the zipper is closed; when the coefficient is 1, the zipper is at its maximum open state. The exposure sizes in the dataset are sorted, and a linear regression of the exposure distance from 0 to 1 is performed to obtain the vector values ​​corresponding to different exposure levels.

[0043] It should be noted that the "Generate" button can be understood as the "Generate Image" button. The first clothing transfer image can be understood as the image obtained by transferring the clothing from the first clothing template image to the human body in the target human body image. The first clothing transfer image generated based on the human body-shape stitching vector, the first clothing encoding vector, the first clothing opening coefficient, and the first information can be understood as inputting the human body-shape stitching vector, the first clothing encoding vector, the first clothing opening coefficient, and the first information into the image generation network, and outputting the first clothing transfer image.

[0044] In this embodiment of the application, after displaying the first garment migration image in the garment migration display area, the method further includes: obtaining second information input in the opening coefficient input field of the display interface and responding to a second touch operation on the generation button of the display interface, and displaying the second garment migration image generated based on the human body-shape splicing vector, the first garment code vector, the first garment opening coefficient and the first information in the garment migration display area of ​​the display interface.

[0045] It should be noted that the second information is different from the first information. As an example, the first information is 0.2, and the second information is 0.3. The first garment migration image is the garment migration image when the opening coefficient is 0.2; the second garment migration image is the garment migration image when the opening coefficient is 0.3. The first garment migration image and the second garment migration image are identical except for the opening state (such as the zipper opening state).

[0046] The solution in this application embodiment can generate first garment migration images with different opening coefficients based on different input information in the opening coefficient input field. That is, the opening coefficient of the first garment migration image can be adaptively adjusted, thereby improving the user experience.

[0047] For ease of understanding, in practical applications, the above virtual try-on method can be illustrated as an adaptive replacement method between the target person and the given clothing. Figure 3 A flowchart illustrating an exemplary adaptive method for replacing a target person with given clothing, provided in an embodiment of this application; as shown. Figure 3 As shown, images of the same garment from different angles are input into the garment coding network and the garment opening prediction network to obtain garment coding vectors and garment opening coefficients (N0, N1, ..., N10). Human body information (including the stitching of the target human body image, the garment part, and the human body part) is input into the human body coding network to obtain the human body coding vector. The target human body image is input into the shape parameter coding network to obtain the shape parameter coding vector. The vector obtained by stitching the human body coding vector and the shape parameter coding vector, the custom opening coefficients (0 to 1), the garment coding network, and the garment opening coefficients (N0, N1, ..., N10) are input into the generator network to obtain the garment migration image.

[0048] To make it easier to understand, the above process will be explained in detail here, with the specific steps as follows: 1. The user inputs an image containing the three primary colors (Red, Green, Blue, RGB) of the clothing to be used, denoted as SRC. A maximum of 10 clothing images are allowed, src0, src1, src2...src9.

[0049] Special handling for clothing input: (1) Key point alignment: Use key point detection algorithm to detect key points on clothing (such as collar, cuff) and align them according to the key points to make their posture and position consistent.

[0050] (2) Size normalization: scale the clothing image to the same size. Scale proportionally according to the actual size.

[0051] (3) Shake the colors of the clothing, such as increasing the contrast, saturation and hue to enhance the adaptability to different lighting conditions.

[0052] (4) Background removal and replacement. Using a segmentation algorithm, only the clothing part is retained, and the background part is uniformly replaced with the green screen color to reduce the impact of the background on the model.

[0053] 2. Input these multiple images into the clothing coding network (Encoder), which is the encoder part of the overall network architecture. It compresses the RGB image vectors into lower-dimensional coding vectors. The resulting image coding vectors are then concatenated along the same dimension. If the number of images is less than 10, the operation is repeated several times until all images have 10 elements in this dimension. This is denoted as src_embedding.

[0054] Clothing Encoding Network: An encoding network, also known as a compression network, can compress an input RGB image of clothing into a smaller-dimensional encoded vector. Combined with a decoding network, this encoded vector can completely reconstruct the original input RGB clothing image. Training Method: The compression and decoding processes are performed using a clothing dataset and a loss function. Training is considered complete when the test set achieves a preset reconstruction metric. This clothing encoding network is specifically trained on a large clothing dataset. Due to this characteristic, the encoded vector obtained by the encoding network is more accurate, which is beneficial for reconstructing clothing details in later stages.

[0055] 3. Input the zippers and other openings of the clothing into the regression network to obtain the opening coefficient src_zipper.

[0056] 4. Input an image with the target task, denoted as DST.

[0057] (1) Input the image of the target person into the segmentation network to obtain the segmentation map, with the clothing part as the segmentation condition. The segmentation map consists of the clothing part and other parts of the human body, denoted as dst_mask.

[0058] (2) Input the target person image into the pose extraction network to extract the person pose image, denoted as dst_pose.

[0059] (3) Input the target person image into the body parameter network model and extract the person's body parameter information, denoted as dst_body.

[0060] (4) Concatenate DST, dst_mask, dst_pose, and dst_body along the channel dimension, and denote the concatenation result as dst_all.

[0061] (5) Input the splicing result into the pre-trained human image coding network to obtain the coding vector, denoted as dst_embedding. The human image coding network is a coding network pre-trained on a large amount of human data.

[0062] (6) Input the DST into the human body shape parameter extraction neural network. Obtain the human body shape feature parameters. Denote it as dst_feature.

[0063] 5. Manually input and adjust the range of the zipper or opening of the garment. The range is 0 to 1, which represents the interval from open to closed, and is denoted as I0.

[0064] 6. Input the src_embedding, src_zipper, dst_embedding, and I0 obtained in the above steps into the generator network to obtain the clothing migration image.

[0065] For ease of understanding, Figure 4 This application provides an exemplary illustration of clothing migration images under different opening coefficients; as shown in the following diagram. Figure 4 As shown, Figure 4 (a) A diagram showing a collar completely open (i.e., without buttons); Figure 4 (b) A diagram showing that the bottom two buttons of the collar are fastened; Figure 4 (c) shows a diagram of the bottom button of the collar being fastened; Figure 4 (d) indicates the bottom button of the collar, but the collar opening angle is greater than... Figure 4 (c) Small diagram.

[0066] The scheme in this embodiment pre-trains the clothing encoding network on a large amount of clothing data. It retains clothing detail information well, and an attention mechanism is applied to the encoded vector to enhance the ability to preserve clothing details. During training, a multi-scale hybrid approach (supporting input and output at different resolutions) is used, combining multiple supervision methods. Self-supervised methods are used to improve the decoder's ability to reconstruct clothing, while another branch is used for multi-information fusion generation. Multiple clothing images at different angles and / or resolutions are input into the encoding network, further improving the ability to preserve clothing details. The network includes a cross-attention module, which helps the model focus on important content in the image. Furthermore, this network employs multi-scale training, thus preserving key information during transformation operations.

[0067] In the training process of the human body coding network in this application embodiment, the data processing will input images of different body types wearing the same clothing; the body parameter coding network will obtain the body parameters of the current human body in the image and use the human body parameters (dst_body) as the network input. Different body parameter values ​​will have a direct relationship with the size of the generated clothing, thereby improving the display effect.

[0068] This application provides a virtual fitting device. Figure 5 This is a schematic diagram of the structure of a virtual fitting device provided in an embodiment of this application; as shown below. Figure 5 As shown, the virtual fitting device 500 includes: The first acquisition unit 501 is used to acquire the human body-shape splicing vector corresponding to the target human body image uploaded through the display interface. The second acquisition unit 502 is used to acquire the first clothing code vector and the first clothing opening coefficient of the first clothing template image in response to a selection touch operation of the first clothing template image in the clothing template display area of ​​the display interface. Display unit 503 is used to acquire first information input in the opening coefficient input field of the display interface and respond to a first touch operation on the generate button of the display interface to display a first garment migration image generated based on the human body-shape splicing vector, the first garment code vector, the first garment opening coefficient and the first information in the garment migration display area of ​​the display interface.

[0069] Optionally, the first clothing template image includes multiple second clothing images with different resolutions and / or shooting angles; prior to the touch operation that responds to the selection of the first clothing template image in the clothing template display area of ​​the display interface, the virtual fitting device 500 further includes a first encoding unit and a first stitching unit; The first encoding unit is used to encode the plurality of second clothing images to obtain a plurality of second clothing encoding vectors; The first splicing unit is used to splice the multiple second garment code vectors to obtain the first garment code vector.

[0070] Optionally, the first clothing template image includes multiple second clothing images with different resolutions and / or shooting angles; prior to the touch operation responding to the selection of the first clothing template image in the clothing template display area of ​​the display interface, the virtual fitting device 500 further includes a third acquisition unit and a control unit. The third acquisition unit is used to acquire multiple second garment opening coefficients corresponding to multiple second garment images of the Haloxylon ammodendron tree; As a unit, it is used to use the plurality of second garment opening coefficients as the first garment opening coefficient.

[0071] Optionally, the first splicing unit is further configured to, when the quantity information of the plurality of second clothing coding vectors is equal to a first value, splice the plurality of second clothing coding vectors to obtain the first clothing coding vector; and when the quantity information is less than the first value, repeatedly splice the plurality of second clothing coding vectors until the quantity information of the spliced ​​second clothing coding vectors is equal to the first value to obtain the first clothing coding vector.

[0072] Optionally, the first acquisition unit 501 is further configured to receive the target human body image, input the target human body image into a human body coding network for encoding to obtain a human body coding vector; input the target human body image into a shape parameter coding network for encoding to obtain a shape parameter coding vector; and concatenate the human body coding vector and the shape parameter coding vector to obtain the human body-shape concatenation vector.

[0073] Optionally, the first acquisition unit 501 is further configured to segment the target human body image to obtain a third clothing image and a human body image; stitch the target human body image, the third clothing image and the human body image together to obtain a target human body stitching vector; and input the target human body stitching vector into the human body coding network for encoding to obtain the human body coding vector.

[0074] Optionally, the first acquisition unit 501 is further configured to extract the pose of the target human body image to obtain a target human body pose image; extract the shape of the target human body image to obtain a target human body shape image; and stitch the target human body image, the third clothing image, the human body image, the target human body pose image, and the target human body shape image together to obtain the target human body stitching vector.

[0075] This application also provides an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; as shown below. Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 603. Optionally, the electronic device 600 may also include a communication bus 602.

[0076] In specific embodiments, the processor 601 described above can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above processor function can also be other types, and this embodiment does not specifically limit it.

[0077] In this embodiment, the communication bus 602 is used to establish a connection between the processor 601 and the memory 603; when the processor 601 executes the running program stored in the memory 603, it implements the following virtual try-on method: Obtain the human-shape stitching vector corresponding to the target human image uploaded through the display interface; in response to a touch operation selecting a first clothing template image in the clothing template display area of ​​the display interface, obtain the first clothing code vector and the first clothing opening coefficient of the first clothing template image; obtain the first information entered in the opening coefficient input field of the display interface and in response to a first touch operation on the generate button of the display interface, display the first clothing migration image generated based on the human-shape stitching vector, the first clothing code vector, the first clothing opening coefficient, and the first information in the clothing migration display area of ​​the display interface.

[0078] Furthermore, the first clothing template image includes multiple second clothing images with different resolutions and / or shooting angles; before the touch operation in response to the selection of the first clothing template image in the clothing template display area of ​​the display interface, the processor 601 is further configured to encode the multiple second clothing images to obtain multiple second clothing encoding vectors; and to concatenate the multiple second clothing encoding vectors to obtain the first clothing encoding vector.

[0079] Furthermore, the first garment template image includes multiple second garment images with different resolutions and / or shooting angles; before the touch operation in response to the selection of the first garment template image in the garment template display area of ​​the display interface, the processor 601 is further configured to obtain multiple second garment opening coefficients corresponding to the multiple second garment images; and use the multiple second garment opening coefficients as the first garment opening coefficient.

[0080] Furthermore, the processor 601 is also configured to, when the quantity information of the plurality of second garment coding vectors is equal to the first value, concatenate the plurality of second garment coding vectors to obtain the first garment coding vector; and when the quantity information is less than the first value, repeatedly concatenate the plurality of second garment coding vectors until the quantity information of the concatenated second garment coding vectors is equal to the first value, to obtain the first garment coding vector.

[0081] Furthermore, the processor 601 is also configured to receive the target human body image, input the target human body image into a human body coding network for encoding to obtain a human body coding vector; input the target human body image into a shape parameter coding network for encoding to obtain a shape parameter coding vector; and concatenate the human body coding vector and the shape parameter coding vector to obtain the human body-shape concatenation vector.

[0082] Furthermore, the processor 601 is also used to segment the target human body image to obtain a third clothing image and a human body image; to stitch the target human body image, the third clothing image, and the human body image together to obtain a target human body stitching vector; and to input the target human body stitching vector into the human body coding network for encoding to obtain the human body coding vector.

[0083] Furthermore, the processor 601 is also used to extract the pose of the target human body image to obtain a target human body pose image; extract the shape of the target human body image to obtain a target human body shape image; and stitch the target human body image, the third clothing image, the human body image, the target human body pose image, and the target human body shape image together to obtain the target human body stitching vector.

[0084] This application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors. The computer program implements the virtual try-on method as described above.

[0085] Based on the above embodiments, this application provides a computer program product, including a computer program that can be executed by one or more processors, and the computer program implements the virtual try-on method as described above.

[0086] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0088] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A virtual fitting method, characterized by, The method comprises: acquiring a human-body-shape splicing vector corresponding to a target human body image uploaded through a display interface; in response to a selection touch operation on a first clothing template image in a clothing template display area of the display interface, acquiring a first clothing coding vector and a first clothing opening coefficient of the first clothing template image; acquiring first information input in an opening coefficient input column of the display interface and, in response to a first touch operation on a generation button of the display interface, displaying a first clothing migration image generated according to the human-body-shape splicing vector, the first clothing coding vector, the first clothing opening coefficient and the first information in a clothing migration display area of the display interface.

2. The method of claim 1, wherein, The first clothing template image comprises a plurality of second clothing images with different resolutions and / or shooting angles; before the response to the selection touch operation on the first clothing template image in the clothing template display area of the display interface, the method further comprises: coding the plurality of second clothing images to obtain a plurality of second clothing coding vectors; splicing the plurality of second clothing coding vectors to obtain the first clothing coding vector.

3. The method of claim 1, wherein, The first clothing template image comprises a plurality of second clothing images with different resolutions and / or shooting angles; before the response to the selection touch operation on the first clothing template image in the clothing template display area of the display interface, the method further comprises: acquiring a plurality of second clothing opening coefficients corresponding to the plurality of second clothing images; taking the plurality of second clothing opening coefficients as the first clothing opening coefficient.

4. The method of claim 2, wherein, The splicing of the plurality of second clothing coding vectors to obtain the first clothing coding vector comprises: in a case where the quantity information of the plurality of second clothing coding vectors is equal to a first value, splicing the plurality of second clothing coding vectors to obtain the first clothing coding vector; in a case where the quantity information is less than the first value, repeatedly splicing the plurality of second clothing coding vectors until the quantity information of the second clothing coding vectors subjected to splicing is equal to the first value, to obtain the first clothing coding vector.

5. The method of claim 1, wherein, The acquisition of the human-body-shape splicing vector corresponding to the target human body image uploaded through the display interface comprises: receiving the target human body image and inputting the target human body image into a human body coding network to code and obtain a human body coding vector; inputting the target human body image into a shape parameter coding network to code and obtain a shape parameter coding vector; splicing the human body coding vector and the shape parameter coding vector to obtain the human-body-shape splicing vector.

6. The method of claim 5, wherein, The inputting of the target human body image into the human body coding network to code and obtain a human body coding vector comprises: segmenting the target human body image to obtain a third clothing image and a human body image; splicing the target human body image, the third clothing image and the human body image to obtain a target human body splicing vector; inputting the target human body splicing vector into the human body coding network to code and obtain the human body coding vector.

7. The method of claim 6, wherein, The target human body image, the third clothing image and the human body image are spliced to obtain a target human body splicing vector, including: Performing posture extraction on the target human body image to obtain a target human body posture image; Performing shape extraction on the target human body image to obtain the target human body shape image; Splicing the target human body image, the third clothing image, the human body image, the target human body posture image and the target human body shape image to obtain the target human body splicing vector.

8. An electronic device, comprising: comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor is configured to execute the steps of the method of any one of claims 1 to 7 when running the computer program.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images