Virtual fitting system based on CP-VTON improvement

Abstract

The invention discloses a CP-VTON improvement-based virtual fitting system, and the system comprises a data input module which is used for inputting a human body image and a clothes image; a geometric matching module; the geometric matching module comprises two multi-dimensional feature extraction modules based on ELAN improvement, and the two multi-dimensional feature extraction modules are respectively used for extracting features of a human body image and a clothes image; the two ECA attention modules are respectively used for carrying out channel attention division on the features extracted by the ELAN improvement-based multi-dimensional feature extraction module; the similarity calculation module is used for performing similarity calculation on the outputs of the two ECA attention modules to form a series of corresponding relations; the regression network module is used for estimating parameters of the geometric model according to the corresponding relation; and the conversion module is used for performing distortion conversion processing on the clothes image through the obtained parameters. According to the CP-VTON-based improved virtual fitting system, feature extraction is enriched, the pattern texture memory ability is enhanced, and the degree of adaptation between the human body posture and the clothes is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of virtual dressing, and in particular relates to a virtual fitting system improved based on CP-VTON. Background Art

[0002] Virtual dressing technology is basically based on 3D models, and its biggest advantage is stable effect. However, 3D-based dressing technology is very expensive in hardware deployment and computing efficiency, and obtaining 3D data itself is a very difficult task. In order to reduce the cost of 3D computing, people's attention began to shift to virtual dressing technology based on 2D images.

[0003] The existing VITON algorithm has problems such as improper matching, excessive distortion of clothes and large rendering differences. Although the CP-VITON algorithm corrects the excessive distortion of some clothes based on the VITON algorithm, the final clothing texture pattern rendering effect still needs to be improved, and the fit between clothing and human body posture needs to be strengthened. Summary of the invention

[0004] The present invention provides a virtual fitting system based on CP-VTON to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0005] A virtual fitting system based on CP-VTON improvement, comprising:

[0006] A data input module, used for inputting human body images and clothing images;

[0007] The geometric matching module is used to perform learnable distortion on clothing images so that the distorted clothing is aligned with the human body;

[0008] The geometric matching module comprises:

[0009] Two multi-dimensional feature extraction modules based on ELAN improvement, used to extract features of human body images and clothing images respectively;

[0010] Two ECA attention modules, respectively used to perform channel attention division on the features extracted by the ELAN-based improved multi-dimensional feature extraction module;

[0011] A similarity calculation module, used to calculate the similarity of the outputs of the two ECA attention modules to form a series of corresponding relationships;

[0012] A regression network module, used for estimating the parameters of the geometric model according to the above correspondence;

[0013] The conversion module is used to perform distortion conversion processing on the clothing image through the obtained parameters.

[0014] Furthermore, the improved multidimensional feature extraction module based on ELAN includes a plurality of Blocks, and the improved multidimensional feature extraction module based on ELAN fuses coarse and fine granularity feature information of different extraction stages and dimensions through the plurality of Blocks.

[0015] Furthermore, the Block includes a two-dimensional convolution layer, a ReLU activation function layer and a BatchNorm layer.

[0016] Furthermore, each Block first extracts features from the image through a two-dimensional convolutional layer, the extracted features pass through a ReLU activation function layer, and finally are normalized through a BatchNorm layer.

[0017] Furthermore, the specific method of the ECA attention module performing channel attention division on the features extracted by the multidimensional feature extraction module based on the ELAN improvement is:

[0018] Perform global average pooling on each channel of the original feature map, pooling pixels into a single pixel to represent the channel information;

[0019] Learn the importance of each channel through one-dimensional convolution;

[0020] The learned channel importance components are weighted to the corresponding channels of the original feature map.

[0021] Furthermore, the ECA attention module performs global average pooling on each channel of the original feature map as follows:

[0022] y i =(GAP(c i ))

[0023] y=cat channel y i (i∈[0~C-1])

[0024] Among them, GAP() represents the global average pooling operation, c i represents the i-th channel feature map, y i represents the result of global average pooling of the i-th channel, C represents the total number of channels, cat channel Indicates splicing on the channel dimension.

[0025] Furthermore, the formula for the ECA attention module to learn the importance of each channel through one-dimensional convolution is as follows:

[0026] w=σ(C1D k (y))

[0027] Among them, w is the importance weight coefficient of each channel, σ() represents the sigmoid activation function, C1D() represents one-dimensional convolution, and k represents the convolution kernel size.

[0028] Furthermore, the similarity calculation module calculates the similarity of the outputs of the two ECA attention modules using the following formula:

[0029]

[0030] CM=TM (b,c,h×w) ×PM (b,w×h,c)

[0031] Among them, TMCi represents the i-th channel value of the object matrix, PMCi represents the i-th channel value of the person matrix, CM represents the channel influence matrix, and CMCi represents the sum of the i-th row of CM.

[0032] Furthermore, the virtual fitting system improved based on CP-VTON also includes:

[0033] The fitting module is used to fuse the distorted clothing image with the human body image.

[0034] The invention is beneficial in that the provided virtual fitting system based on CP-VTON improvement enriches feature extraction, enhances pattern texture memory capability, and improves the degree of adaptation between human body posture and clothing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 is a schematic diagram of the structure of the geometric matching module of the present invention;

[0037] Figure 2 It is a structural schematic diagram of the multi-dimensional feature extraction module based on ELAN improvement of the present invention;

[0038] Figure 3 is a schematic diagram of the structure of the ECA attention module of the present invention;

[0039] Figure 4 It is a similarity calculation structure diagram of the similarity calculation module of the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0041] The present application discloses a virtual fitting system based on CP-VTON improvement, comprising: a data input module and a geometric matching module.

[0042] The data input module is used to input human body images and clothing images. The geometric matching module is used to perform learnable distortion on clothing images so that the distorted clothing is aligned with the human body.

[0043] Specifically, if Figure 1 As shown, the geometric matching module includes: two multi-dimensional feature extraction modules based on ELAN improvement, two ECA attention modules, a similarity calculation module, a regression network module and a conversion module.

[0044] Two multi-dimensional feature extraction modules based on the improved ELAN are used to extract the features of human images and clothing images respectively.

[0045] In the implementation of this application, a powerful ELAN multi-dimensional feature extraction module is introduced. ELAN's unique network structure can integrate coarse and fine granular feature information of different extraction stages and dimensions, greatly improving the expression ability of the feature extraction network. Figure 2 As shown in FIG. 1 , the multidimensional feature extraction module improved based on ELAN includes multiple Blocks. The multidimensional feature extraction module improved based on ELAN fuses coarse and fine granular feature information of different extraction stages and dimensions through multiple Blocks. Specifically, Block includes a two-dimensional convolution layer, a ReLU activation function layer, and a BatchNorm layer. The convolution operation of the two-dimensional convolution layer is to perform element-by-element multiplication and accumulation operations on the filter and the input image to obtain each element in the feature map. The main function of the ReLU activation function is to limit the input value to a non-negative range, that is, when the input value is less than 0, the output value is 0, and when the input value is greater than or equal to 0, the output value is equal to the input value itself, thereby enhancing the neuron expression ability. The BatchNorm layer normalizes the data to a distribution with a mean of 0 and a variance of 1, which makes the data distribution consistent on the one hand and avoids the disappearance of the gradient on the other hand. Each Block first extracts features from the image by a two-dimensional convolution layer, and the extracted features pass through the ReLU activation function layer, and finally are normalized by the BatchNorm layer.

[0046] The two ECA attention modules are used to perform channel attention division on the features extracted by the multi-dimensional feature extraction module improved based on ELAN.

[0047] In the implementation manner of the present application, the specific method of the ECA attention module performing channel attention division on the features extracted by the multidimensional feature extraction module based on the ELAN improvement is:

[0048] First, perform global average pooling on each channel of the original feature map, and pool the pixels into a single pixel to represent the channel information.

[0049] In the implementation of the present application, the formula for the ECA attention module to perform global average pooling on each channel of the original feature map is as follows:

[0050] y i =(GAP(c i ))

[0051] y=cat channel y i (i∈[0~C-1])

[0052] Among them, GAP() represents the global average pooling operation, c i represents the i-th channel feature map, y i represents the result of global average pooling of the i-th channel, C represents the total number of channels, cat channel Indicates splicing on the channel dimension.

[0053] Then, the importance of each channel is learned through one-dimensional convolution. Figure 3 As shown, in an embodiment of the present application, the formula for the importance of each channel learned by the ECA attention module through one-dimensional convolution is as follows:

[0054] w=σ(C1D k (y))

[0055] Among them, w is the importance weight coefficient of each channel, σ() represents the sigmoid activation function, C1D() represents one-dimensional convolution, and k represents the convolution kernel size.

[0056] Finally, the learned channel importance components are weighted to the corresponding channels of the original feature map.

[0057] The similarity calculation module is used to calculate the similarity of the outputs of the two ECA attention modules to form a series of corresponding relationships. Figure 4 As shown, in the implementation of the present application, a channel influence matrix is ​​added to enrich the connotation of the similarity matrix so that the model can learn the mutual influence between different channels. Specifically, the similarity calculation module calculates the similarity of the outputs of the two ECA attention modules through the following formula:

[0058]

[0059] CM=TM (b,c,h×w) ×PM (b,w×h,c)

[0060] Among them, TMCi represents the i-th channel value of the object matrix, PMCi represents the i-th channel value of the person matrix, CM represents the channel influence matrix, and CMCi represents the sum of the i-th row of CM.

[0061] The regression network module is used to estimate the parameters of the geometric model according to the above correspondence relationship. The conversion module is used to perform distortion conversion processing on the clothing image according to the obtained parameters.

[0062] In the implementation manner of the present application, the virtual fitting system improved based on CP-VTON further comprises a fitting module. The fitting module is used to fuse the distorted clothing image with the human body image.

[0063] The model is trained by collecting data sets, and the trained model has the function of automatically matching fitting. After inputting human body and clothing images into the trained model, it can automatically input the effect picture of human body trying on clothes.

[0064] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A virtual fitting system based on CP-VTON, characterized in that: Include: A data input module, used for inputting human body images and clothing images; The geometric matching module is used to perform learnable distortion on clothing images so that the distorted clothing is aligned with the human body; The geometric matching module comprises: Two multi-dimensional feature extraction modules based on ELAN improvement, used to extract features of human body images and clothing images respectively; Two ECA attention modules, respectively used to perform channel attention division on the features extracted by the ELAN-based improved multi-dimensional feature extraction module; A similarity calculation module, used to calculate the similarity of the outputs of the two ECA attention modules to form a series of corresponding relationships; A regression network module, used for estimating the parameters of the geometric model according to the above correspondence; The conversion module is used to perform distortion conversion processing on the clothing image through the obtained parameters.

2. The virtual fitting system based on CP-VTON improvement according to claim 1 is characterized in that: The improved multidimensional feature extraction module based on ELAN includes multiple Blocks, and the improved multidimensional feature extraction module based on ELAN fuses coarse and fine granularity feature information of different extraction stages and dimensions through the multiple Blocks.

3. The virtual fitting system based on CP-VTON improvement according to claim 2 is characterized in that: The Block includes a two-dimensional convolution layer, a ReLU activation function layer and a BatchNorm layer.

4. The virtual fitting system based on CP-VTON improvement according to claim 3 is characterized in that: Each Block first extracts features from the image through a two-dimensional convolutional layer, the extracted features pass through a ReLU activation function layer, and finally are normalized through a BatchNorm layer.

5. The virtual fitting system based on CP-VTON improvement according to claim 1 is characterized in that: The specific method of the ECA attention module performing channel attention division on the features extracted by the multi-dimensional feature extraction module based on the ELAN improvement is: Perform global average pooling on each channel of the original feature map, pooling pixels into a single pixel to represent the channel information; Learn the importance of each channel through one-dimensional convolution; The learned channel importance components are weighted to the corresponding channels of the original feature map.

6. The virtual fitting system based on CP-VTON improvement according to claim 5 is characterized in that: The formula for the ECA attention module to perform global average pooling on each channel of the original feature map is as follows: the i =(GAP(c i )) y=cat channel y i (i∈[0~C-1]) Among them, GAP() represents the global average pooling operation, c i represents the i-th channel feature map, y i represents the result of global average pooling of the i-th channel, C represents the total number of channels, cat channel Indicates splicing on the channel dimension.

7. The virtual fitting system based on CP-VTON improvement according to claim 6 is characterized in that: The formula for the ECA attention module to learn the importance of each channel through one-dimensional convolution is as follows: in=σ(C1D k (y)) Among them, w is the importance weight coefficient of each channel, σ() represents the sigmoid activation function, C1D() represents one-dimensional convolution, and k represents the convolution kernel size.

8. The virtual fitting system based on CP-VTON improvement according to claim 1 is characterized in that: The similarity calculation module calculates the similarity of the outputs of the two ECA attention modules using the following formula: CM=TM (b,c,h×w) ×PM (b,w×h,c) Among them, TMCi represents the i-th channel value of the object matrix, PMCi represents the i-th channel value of the person matrix, CM represents the channel influence matrix, and CMCi represents the sum of the i-th row of CM.

9. The virtual fitting system based on CP-VTON improvement according to claim 1 is characterized in that: The virtual fitting system improved based on CP-VTON also includes: The fitting module is used to fuse the distorted clothing image with the human body image.

Images