Garment matching scheme generation method and system based on neural network
Through the neural network-based clothing matching solution generation method, the problem of users' difficulty in judging the clothing matching effect is solved, an efficient shopping experience and low return rate are achieved, and transportation costs are reduced.
Patent Information
- Application Number
- CN202411700775.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-18
AI Technical Summary
It is difficult for users to accurately judge the matching effect of different clothing when purchasing clothing online, resulting in low purchase efficiency and high return rate.
A clothing matching solution generation method based on neural network is adopted to obtain the user's full body photos and three-dimensional information, and a three-dimensional mannequin model is established, and a variety of clothing matching solutions are generated based on the user's historical shopping records and the color, size and shape information of the clothing, and these solutions are rendered into the three-dimensional mannequin model to show the dressing effect.
It improves users' experience and purchase efficiency during shopping, significantly reduces users' return rate, and reduces the transportation costs of shopping platforms.
Smart Images
Figure CN119648328A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to a field, and in particular, to a method for generating clothing matching schemes based on a neural network. Background Art
[0002] When users buy clothes online, they will roughly determine whether they need to buy clothes by observing the dressing effect of the clothes on Mote and the patterns of the clothes themselves on the store page. However, when it comes to clothing matching, this method is more convenient when buying single clothes. However, if it comes to clothing matching considerations, it is more difficult. It is difficult for users to accurately judge the dressing effect of different clothes through different pictures and models. For this reason, many users will buy a large number of clothes of different styles and different positions online, and then wait for all these clothes to be delivered, and then try them on at home, leaving only the clothes that users think the matching effect is better, and the rest of the clothes will be returned to the merchant by express delivery. In this way, users cannot intuitively obtain the matching effect of clothes in this way, and the purchasing efficiency is low. Summary of the invention
[0003] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0004] As a first aspect of the present application, in order to solve the technical problems mentioned in the above background technology section, the present application provides a method for generating clothing matching solutions based on a neural network, comprising the following steps:
[0005] Step 1: Obtain several full-body photos of the user and the user's three-dimensional information to establish a three-dimensional human body model of the user;
[0006] Step 2: Get the product selected by the user, extract the color information, size information and shape information of the clothing from the product page to obtain rendering information;
[0007] Step 3: Obtain the user's historical clothing shopping records and extract several clothing matching plans from the selected products based on the association algorithm;
[0008] Step 4: Obtain rendering information of each garment in the clothing matching scheme, render the rendering information into a three-dimensional human body model, and obtain a clothing matching effect model.
[0009] In the technical solution provided by the present application, the user's favorite clothing matching method will be automatically extracted according to the goods selected by the user, and then a variety of clothing matching schemes will be generated, and then these clothing matching schemes will be rendered into a three-dimensional human body model to obtain a clothing matching effect model. In this way, during the shopping process, the user will continuously refresh the clothing matching effect model as he continuously selects goods, and then the user does not need to obtain physical clothes for trial wearing, and can get the matching effect of different goods on the body, as well as the matching effect of different goods on the body, thus increasing the user's shopping experience and purchasing efficiency; and the use of this solution can significantly reduce the user's return rate and reduce the transportation cost of the shopping platform.
[0010] User uploaded full body Figure 1 Generally, it is not a picture with a single background. Therefore, after uploading a 3D picture, it is necessary to accurately identify the user from the background of the picture. Under the existing scheme, it is generally based on whether the texture boundary of the elements in the image is similar to the human shape, so as to extract the human body boundary by edge similarity. This extraction method is very rough. When the background is complex, it is often difficult to accurately extract the key information of the human body. To this end, this application provides the following technical solutions:
[0011] Step 1 includes the following steps:
[0012] Step 11: Obtain a full-body photo of the user, input the full-body photo into the pre-trained backbone network, and extract a multi-scale feature map from the backbone network;
[0013] Step 12: Input the multi-scale feature map into the region proposal network to obtain the feature representation of each candidate region and obtain the instance mask;
[0014] Step 13: Input the multi-scale feature map into several layers of convolutional layers to extract the semantic feature map, input the semantic feature map into the Sigmoid function, and filter out the semantic mask containing each limb;
[0015] Step 14: Align the instance mask and the semantic mask in space, check the value of the semantic mask and the value of the instance mask in each pixel in turn, determine whether the pixel is a foreground according to the instance mask, and determine the category of the pixel according to the semantic mask;
[0016] Step 15: Store the merged mask information in a new two-dimensional array;
[0017] Step 16: Extract the three-dimensional model of the human body from the two-dimensional array.
[0018] The technical solution provided in the present application can accurately extract the edges of human bodies in photos by fusing instance masks and semantic masks. When there are many instances and types of instances in photos and the background is complex, the human body mask can be accurately extracted.
[0019] Furthermore, the loss function used in step 1 is L ro ;
[0020] L ro =L cs +L bx +L mk +L ps +L sc ;
[0021] L cs It is used to measure the difference between the predicted category and the true category. bx It is used to measure the difference between the bounding box and the true bounding box. mk Used to measure the difference between the predicted instance mask and the true instance mask;
[0022] L ps Used to solve the class imbalance problem in target detection to classify similar samples in the image;
[0023]
[0024] Among them, L sc It is used to measure the difference between the probability distribution predicted by the model and the actual label. N is the total number of pixels, and y is i is the true label of pixel i, p i is the probability that the pixel is predicted to be a foreground class, α t is the class weight and γ is the modulation factor.
[0025] Because the panoramic pictures provided by users are not pictures with fixed postures, the body postures of users in the panoramic pictures are different. When the overall mask and approximate body category of the user are extracted, it is not possible to accurately determine each joint point. To this end, the present application provides the following technical solutions:
[0026] Step 16 includes the following steps:
[0027] Step 161: extracting the user profile from the original image according to the range of the two-dimensional array;
[0028] Step 162: Input the two-dimensional array and the user profile into the Resnet-50 network in dual channels to obtain fused features;
[0029] Step 163: Input the fused features into the regressor to obtain morphological features, which include: posture, shape parameters θ and β;
[0030] Step 164: Input the morphological features into the SMPL human body model to generate a three-dimensional human body model, and input the three-dimensional information into the three-dimensional human body model.
[0031] In the solution provided in the present application, by fusing the user's original image information with the user's outline, the exact position of each limb can be analyzed, and when generating a three-dimensional human body model, the position of each joint point and the shape of the joint can be accurately judged, thereby generating an accurate three-dimensional model.
[0032] There are many styles of clothes. Different styles will present different shape characteristics when worn on the limbs. To this end, this application provides the following technical solutions:
[0033] Furthermore, step 2 includes the following steps:
[0034] Step 21: Obtain the products selected by the user, and divide them into an inner wear set, an outer wear set, a lower body set, and a shoe set according to the product types;
[0035] Step 22: For each product, the color information, size information, and shape information of the clothing at the corresponding position of each torso are extracted to generate rendering information for each product.
[0036] In the technical solution provided in the present application, color information, size information and shape information corresponding to the torso are collected, so that the transformation characteristics of the pattern on the clothing under different deformation conditions can be obtained, thereby increasing the accuracy of the final clothing.
[0037] When users match clothes, they consider many factors. If we only summarize the historical purchase records, it is actually difficult to accurately obtain the user's interest in clothes purchase. To this end, this application provides the following technical solutions:
[0038] Step 3 includes the following steps:
[0039] Step 31: pre-define clothing elements, which include clothing color, clothing pattern, clothing type, clothing brand, and clothing style;
[0040] Step 32: Obtain the user's historical clothing shopping records, and generate the user's historical clothing combinations therefrom;
[0041] Step 33: Extract the matching information of the user's clothing elements from the historical clothing matching, and generate clothing matching based on the matching information.
[0042] In this solution, various dressing elements such as clothing color, clothing pattern, clothing style, clothing brand, etc. are taken into consideration, and a dressing matching plan is generated based on the matching information between the elements. Therefore, the user's corresponding clothing matching needs can be accurately obtained, increasing the accuracy and effectiveness of clothing matching.
[0043] When users match clothes, they consider many factors. If the elements that users like are directly highlighted as the user's favorite tags, it is actually impossible to accurately find the user's interests when matching clothes. Therefore, it is impossible to accurately provide a clothing matching plan that meets the user's wishes. For this purpose, this application provides the following technical solutions:
[0044] Step 33 includes the following steps:
[0045] Step 331: For each historical clothing combination, convert it into a clothing element representation to generate a clothing interest sequence m k ;
[0046] m k ={a 1,k 、a 2,k 、a 3,k …a o,k …}; where k represents the index of the clothing interest series; o represents the index of the corresponding clothing element in the clothing interest series, a o,k represents the oth clothing element in the kth clothing interest sequence;
[0047] Step 332: Collect all clothing interest series m k Generate clothing interest matrix M K ;
[0048] Among them, K represents the total number of interest series;
[0049] Step 333: Calculate the correlation SQ between the frequent candidate item sets of any two columns in the clothing interest series ol , o and l are the indexes of clothing elements, o≠l;
[0050] SQ ol =a 1，o ×a 1，l +……a K，o ×a K，l ;
[0051] Step 334: Obtain the relevance SQ of all frequent candidate item sets ol Get the correlation matrix P;
[0052]
[0053] Step 335: Obtain the products selected by the user, and extract a clothing combination therefrom so as to obtain the highest correlation score in the correlation matrix P.
[0054] Furthermore, in step 33, for each clothing interest sequence, the blank elements need to be filled so that the clothing elements of each blank element sequence are arranged in the same order.
[0055] The technical solution provided by the present application constructs a correlation matrix P, so there is correlation information between any two elements in the correlation matrix P, so that it is possible to accurately find out which clothing elements will be used together for clothing matching when the user matches clothes. In this way, on the premise of obtaining the best score in the correlation matrix P, the clothing matching with the best recommendation significance is screened out. In this way, during the shopping process, the user only needs to add the selected goods to the shopping cart to continuously generate the matching solutions that the user subconsciously likes the most, thereby increasing shopping efficiency.
[0056] Step 4 includes the following steps:
[0057] Step 41: Obtain a three-dimensional human body model and the lengths of limbs in the three-dimensional human body model;
[0058] Step 42: Obtain the selected clothing matching scheme, and obtain rendering information based on the products selected in the clothing matching scheme;
[0059] Step 43: Determine rendering information of different regions in the three-dimensional human body model based on limb lengths and clothing sizes;
[0060] Step 44: Input the 3D human body model and rendering information into DirectX to obtain a rendered clothing model.
[0061] In the technical solution provided by the present application, three-dimensional information and size information are acquired in advance, so that the length of the clothes on the human body can be determined, and the area of the human body where the clothes will be roughly distributed can be determined. Therefore, after rendering the effect model of clothing matching, it can also be intuitively reflected whether the clothes selected by the user are of appropriate size.
[0062] As a second aspect of the present application, the present application provides a clothing matching scheme generation system based on a neural network, which uses the aforementioned neural network clothing matching scheme generation method to generate a clothing matching effect model.
[0063] The beneficial effect of the present application is that during the shopping process, the user will continuously refresh the clothing matching effect model as he / she continuously selects products, so that the user does not need to obtain physical clothing for trying on, and can obtain the matching effects of different products on the body, as well as the matching effects of different products on the body, thereby increasing the user's shopping experience and increasing purchasing efficiency; and the use of this solution can significantly reduce the user's return rate and reduce the transportation costs of the shopping platform. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The illustrative embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application.
[0065] In addition, throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the components and elements are not necessarily drawn to scale.
[0066] In the attached picture:
[0067] Figure 1 Flowchart of the neural network-based clothing matching solution generation method.
[0068] Figure 2 Extract the mask image.
[0069] Figure 3 A three-dimensional model diagram. DETAILED DESCRIPTION
[0070] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.
[0071] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.
[0072] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.
[0073] Reference Figure 1 , a method for generating clothing matching scheme based on neural network, comprising the following steps:
[0074] Step 1: Obtain several full-body photos of the user and the user's three-dimensional information, and build a three-dimensional human body model of the user.
[0075] The three-dimensional human body model is not a three-dimensional model of the whole body, but a three-dimensional model representation in a two-dimensional image. That is, the outline of the human body is selected from the two-dimensional image, and the three-dimensional position relationship of each area in the human body outline is identified.
[0076] Specifically, step 1 includes the following steps:
[0077] Step 1 includes the following steps:
[0078] Step 11: Obtain a full-body photo of the user, input the full-body photo into the pre-trained backbone network, and extract a multi-scale feature map from the backbone network.
[0079] The user can have multiple full-body photos or just one. The reason for requiring full-body photos is, on the one hand, to ensure the matching effect of clothing, and on the other hand, to ensure that the picture input into the neural network has enough elements. This avoids the inability to accurately extract the range of the human body when key instances such as limbs are missing.
[0080] The backbone network is a backbone network used to extract features from images. These features can be used for subsequent instance recognition, feature recognition, etc.
[0081] Step 12: Input the multi-scale feature map into the region proposal network to obtain the feature representation of each candidate region and obtain the instance mask.
[0082] The Region Proposal Network (RPN) is a fully convolutional network that is used in object detection algorithms to generate candidate regions that may be targets. The Region Proposal Network (RPN) is used to process the multi-scale feature maps extracted from the backbone network. It generates feature representations and instance masks for each candidate region, which may contain different parts or target objects in the user's photo. Through subsequent processing and recognition steps, the specific content and location information of these candidate regions can be further determined.
[0083] Step 13: Input the multi-scale feature map into several layers of convolutional layers to extract the semantic feature map, input the semantic feature map into the Sigmoid function, and filter out the semantic mask containing each limb;
[0084] Step 14: Align the instance mask and the semantic mask in space, check the value of the semantic mask and the value of the instance mask in each pixel in turn, determine whether the pixel is a foreground according to the instance mask, and determine the category of the pixel according to the semantic mask;
[0085] Step 15: Store the merged mask information in a new two-dimensional array. Figure 2 shown.
[0086] Step 16: Extract the three-dimensional model of the human body from the two-dimensional array.
[0087] The semantic mask indicates the category to which the pixel belongs, and the instance indicates the instance to which the pixel belongs. Suppose there is an image containing two people, the semantic mask marks both of them as the "person" category, while the instance segmentation mask distinguishes the two different people (such as instance A and instance B).
[0088] In the merged mask, the two instances can be assigned different class labels or color codes, such as labeling instance A as person-A and instance B as person_B.
[0089] In the semantic mask of this solution, the limbs of the human body, such as the head, torso, and limbs, are roughly distinguished. However, when executing steps 14 and 15, they are not used to fuse the instance mask.
[0090] Furthermore, in steps 11 to 15, there are three neural networks whose training method is a common supervised training method. The specific training method is not described in detail. However, this application is used to guide the training direction of the neural network.
[0091] Specifically: the loss function is L ro ;
[0092] L ro =L cs +L bx +L mk +L ps +L sc ;
[0093] L cs It is used to measure the difference between the predicted category and the true category. bx It is used to measure the difference between the bounding box and the true bounding box. mk Used to measure the difference between the predicted instance mask and the true instance mask;
[0094] L ps Used to solve the class imbalance problem in target detection to classify similar samples in the image;
[0095]
[0096] Among them, L sc It is used to measure the difference between the probability distribution predicted by the model and the actual label. N is the total number of pixels, and y is i is the true label of pixel i, pi is the probability that the pixel is predicted to be a foreground class, α t is the class weight and γ is the modulation factor.
[0097] Step 16 includes the following steps:
[0098] Step 161: extracting the user profile from the original image according to the range of the two-dimensional array;
[0099] The two-dimensional array in step 15 is actually the range of the human body in the original picture, but the two-dimensional array lacks the details of the picture. Therefore, the user outline extracted from the original picture using the outline will contain the color information of the user in the original picture, so the three-dimensional features of the instance can be determined using this color information.
[0100] Step 162: Input the two-dimensional array and the user profile into the Resnet-50 network in dual channels to obtain fused features;
[0101] The ResNet-50 network is a deep residual network (Residual Network), which is a specific implementation of the ResNet series of networks. It has 50 convolutional layers (excluding the fully connected layer and the final classification layer). The two-dimensional array and the user contour are input into the Resnet-50 network in dual channels to obtain the fusion feature extraction, which can accurately reflect the three-dimensional information difference of each limb in the two-dimensional array.
[0102] Step 163: Input the fused features into the regressor to obtain morphological features, which include: posture, shape parameters θ and β;
[0103] The regressor includes: a first fully connected layer, a concatenation layer, and a second fully connected layer;
[0104] The fusion features are input into the first fully connected layer, and a feature vector Ψ of size 2048 is output. The feature vector Ψ is spliced with the initial posture parameters, shape parameters, and camera parameters. The size of the spliced vector is 2205 (2048+24×6+10+3), where 24×6 represents the posture parameters (assuming there are 24 joints, each with 6 degrees of freedom, such as rotation angle, etc.), 10 represents the shape parameters (which may be some proportions or size parameters of the human body), and 3 represents the camera parameters (such as position, direction, etc.). This spliced vector is then passed through the second fully connected layer to output a residual Delta Theta of size 157 (24×6+10+3). Then it is iterated to process the final output morphological features.
[0105] Step 164: Input the morphological features into the SMPL human body model to generate a three-dimensional human body model, and input the three-dimensional information into the three-dimensional human body model. Figure 3 shown.
[0106] The SMPL human body model is a three-dimensional model of the human body based on vertex offset modeling, and its specific design method will not be further described here.
[0107] Step 2: Get the product selected by the user, and extract the color information, size information, and shape information of the clothing from the product page to obtain rendering information.
[0108] Step 21: Get the products selected by the user, and divide them into inner wear set, outer wear set, lower body set and shoe set according to the product types.
[0109] The products selected by the user are not the products that the user pays for directly, but the products that the user is interested in, such as adding to the shopping cart or adding to the collection. The inner wear set, outer wear set, lower body set and shoe set are just the classification methods for the products. The specific number of classifications will not be repeated. +
[0110] Step 22: For each product, the color information, size information, and shape information of the clothing at the corresponding position of each torso are extracted to generate rendering information for each product.
[0111] The rendering information of the product is actually the pattern and color of each part of the clothing. In order to ensure that the clothing can be accurately rendered on the 3D model, it also needs to contain size information. For example, if a coat is red in front and black in the back, the color information will show that the front is red and the back is black. The specific division method will not be repeated here.
[0112] Step 3: Obtain the user's historical clothing shopping records and extract several clothing matching solutions from the selected products based on the association algorithm.
[0113] During the shopping process, users will not only select one or two products of interest, but will select many products of interest. So the selected products here are the products in the shopping cart and favorites. Therefore, during the shopping process, after the user adds the products to the shopping cart, the clothing matching schemes most relevant to the user's interests will be continuously generated. The method of extracting the clothing matching schemes is as follows:
[0114] Step 3 includes the following steps:
[0115] Step 31: Predefine clothing elements, which include clothing color, clothing pattern, clothing type, clothing brand, and clothing style.
[0116] This application does not limit the number of types of dressing elements. It is mainly related to the way the shopping platform divides the products. However, it can be foreseen that the more types there are and the more detailed the division, the easier it will be to find elements that users like. Furthermore, the number of dressing elements for each piece of clothing is the same, that is, each piece of clothing needs to be described by the same dressing elements.
[0117] Step 32: Obtain the user's historical clothing shopping records, and generate the user's historical clothing combinations therefrom;
[0118] The historical clothing matching here is actually the clothing matching scheme that the user is interested in during the shopping process. The specific historical clothing matching scheme will not be further explained here.
[0119] Step 33: Extract the matching information of the user's clothing elements from the historical clothing matching, and generate clothing matching based on the matching information.
[0120] Step 33 includes the following steps:
[0121] Step 331: For each historical clothing combination, convert it into a clothing element representation to generate a clothing interest sequence m k ;
[0122] m k ={a 1,k 、a 2,k 、a 3,k …a o,k …}; where k represents the index of the clothing interest series; o represents the index of the corresponding clothing element in the clothing interest series, a o,k Represents the oth clothing element in the kth clothing interest sequence.
[0123] Because this solution needs to provide clothing matching solutions. Therefore, the clothing interest series must include all inner wear sets, outer wear sets, lower body sets and shoe sets, and the arrangement of inner wear sets, outer wear sets, lower body sets and shoe sets must be the same. For clothing elements, digital labels are used to describe them.
[0124] For example, the label for red is 1, the label for black is 2, the label for duck pattern is 1, and the label for fish pattern is 2. If there is a combination of red coat, red inner wear, red pants, and red shoes, then the interest sequence of this clothing combination is 1, ...1, ...1, ...1, ...; the ellipsis after the first 1 represents the subsequent clothing element of the red coat, and the ellipsis after the second 1 represents the subsequent clothing element of the red inner wear, and the subsequent elements correspond one to one. In this way, it can be seen that each historical clothing combination can be converted into a standard clothing interest sequence.
[0125] In step 331, for each clothing interest sequence, the blank elements need to be filled so that the clothing element arrangement order of each blank element sequence is the same. That is, if an element in the inner wear set or the outer wear set is missing in a certain historical clothing combination, it also needs to be filled with the initial information.
[0126] In this way, this solution can not only reflect the matching solutions of multiple clothing, but also the matching solutions of two clothing.
[0127] Step 332: Collect all clothing interest series m k Generate clothing interest matrix M K ;
[0128] Among them, K represents the total number of interest series;
[0129] Step 333: Calculate the correlation SQ between the frequent candidate item sets of any two columns in the clothing interest series ol , o and l are the indexes of clothing elements, o≠l;
[0130] SQ ol =a 1，o ×a 1，l +……a K，o ×a K，l ;
[0131] Step 334: Obtain the relevance SQ of all frequent candidate item sets ol Get the correlation matrix P;
[0132]
[0133] In this way, in the correlation matrix P, the correlation between any two elements can be obtained. If the correlation is larger, it means that the correlation between the two elements is higher.
[0134] Step 335: Obtain the products selected by the user, and extract a clothing combination therefrom so as to obtain the highest correlation score in the correlation matrix P.
[0135] Since the correlation matrix has been determined, we only need to use the exhaustive method to calculate the correlation of each clothing matching scheme, then add them up and compare the total scores.
[0136] Step 4 includes the following steps:
[0137] Step 41: Obtain a three-dimensional human body model and the lengths of the limbs in the three-dimensional human body model; that is, in step 41, the three-dimensional information is used to mark the corresponding lengths in the three-dimensional human body model, for example, the arm is marked as 80 cm.
[0138] Step 42: Obtain the selected clothing matching scheme, and obtain rendering information based on the products selected in the clothing matching scheme.
[0139] The clothing matching scheme is the clothing matching scheme determined in step 335, and the rendering information only needs to extract the rendering information of the selected product.
[0140] Step 43: Determine rendering information of different regions in the three-dimensional human body model based on limb lengths and clothing sizes;
[0141] The determination method here is as follows, mainly determining the rendering range based on the size. For example, if the arm length is known to be 80cm, if the sleeve length of the selected clothing is 90cm, the entire arm needs to be rendered in the color of the sleeve.
[0142] Step 44: Input the 3D human body model and rendering information into DirectX to obtain a rendered clothing model.
[0143] DirectX is an image rendering software developed by Microsoft. Given a 3D human body model and rendering information of different regions, it can actively render the required 3D human body model.
[0144] Embodiment 2: A clothing matching scheme generation system based on a neural network, which uses the aforementioned clothing matching scheme generation method of the neural network to generate a clothing matching effect model.
[0145] The above description is only some preferred embodiments of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) the technical features with similar functions disclosed in the embodiments of the present application to form a technical solution.
Claims
1. A method for generating clothing matching schemes based on neural networks, characterized in that: The steps include: Step 1: Obtain several full-body photos of the user and the user's three-dimensional information to establish a three-dimensional human body model of the user; Step 2: Get the product selected by the user, extract the color information, size information and shape information of the clothing from the product page to obtain rendering information; Step 3: Obtain the user's historical clothing shopping records and extract several clothing matching plans from the selected products based on the association algorithm; Step 4: Obtain rendering information of each garment in the clothing matching scheme, render the rendering information into a three-dimensional human body model, and obtain a clothing matching effect model.
2. The method for generating clothing matching scheme based on neural network according to claim 1, characterized in that: Step 1 includes the following steps: Step 11: Obtain a full-body photo of the user, input the full-body photo into the pre-trained backbone network, and extract a multi-scale feature map from the backbone network; Step 12: Input the multi-scale feature map into the region proposal network to obtain the feature representation of each candidate region and obtain the instance mask; Step 13: Input the multi-scale feature map into several layers of convolutional layers to extract the semantic feature map, input the semantic feature map into the Sigmoid function, and filter out the semantic mask containing each limb; Step 14: Align the instance mask and the semantic mask in space, check the value of the semantic mask and the value of the instance mask in each pixel in turn, determine whether the pixel is a foreground according to the instance mask, and determine the category of the pixel according to the semantic mask; Step 15: Store the merged mask information in a new two-dimensional array; Step 16: Extract the three-dimensional model of the human body from the two-dimensional array.
3. The method for generating clothing matching scheme based on neural network according to claim 2, characterized in that: The loss function used in step 1 is L ro ; L ro =L cs +L bx +L mk +L ps +L sc ; L cs It is used to measure the difference between the predicted category and the true category. bx It is used to measure the difference between the bounding box and the true bounding box. mk Used to measure the difference between the predicted instance mask and the true instance mask; L ps Used to solve the class imbalance problem in target detection to classify similar samples in the image; Among them, L sc It is used to measure the difference between the probability distribution predicted by the model and the actual label. N is the total number of pixels, and y is i is the true label of pixel i, p i is the probability that the pixel is predicted to be a foreground class, α t is the class weight and γ is the modulation factor.
4. The method for generating clothing matching scheme based on neural network according to claim 3, characterized in that: Step 16 includes the following steps: Step 161: extracting the user profile from the original image according to the range of the two-dimensional array; Step 162: Input the two-dimensional array and the user profile into the Resnet-50 network in dual channels to obtain fused features; Step 163: Input the fused features into the regressor to obtain morphological features, which include: posture, shape parameters θ and β; Step 164: Input the morphological features into the SMPL human body model to generate a three-dimensional human body model, and input the three-dimensional information into the three-dimensional human body model.
5. The method for generating clothing matching scheme based on neural network according to claim 1, characterized in that: Step 2 includes the following steps: Step 21: Obtain the products selected by the user, and divide them into an inner wear set, an outer wear set, a lower body set, and a shoe set according to the product types; Step 22: For each product, the color information, size information, and shape information of the clothing at the corresponding position of each torso are extracted to generate rendering information for each product.
6. The method for generating clothing matching scheme based on neural network according to claim 2, characterized in that: Step 3 includes the following steps: Step 31: pre-define clothing elements, which include clothing color, clothing pattern, clothing type, clothing brand, and clothing style; Step 32: Obtain the user's historical clothing shopping records, and generate the user's historical clothing combinations therefrom; Step 33: Extract the matching information of the user's clothing elements from the historical clothing matching, and generate clothing matching based on the matching information.
7. The method for generating clothing matching scheme based on neural network according to claim 6, characterized in that: Step 33 includes the following steps: Step 331: For each historical clothing combination, convert it into a clothing element representation to generate a clothing interest sequence m k ; m k ={a 1,k 、a 2,k 、a 3,k …a o,k …}; where k represents the index of the clothing interest series; o represents the index of the corresponding clothing element in the clothing interest series, a o,k represents the oth clothing element in the kth clothing interest sequence; Step 332: Collect all clothing interest series m k Generate clothing interest matrix M K ; Among them, K represents the total number of interest series; Step 333: Calculate the correlation SQ between the frequent candidate item sets of any two columns in the clothing interest series ol , o and l are the indexes of clothing elements, o≠l; SQ ol =a 1，o ×a 1，l +……a K，o ×a K，l ; Step 334: Obtain the relevance SQ of all frequent candidate item sets ol Get the correlation matrix P; Step 335: Obtain the products selected by the user, and extract a clothing combination therefrom so as to obtain the highest correlation score in the correlation matrix P.
8. The method for generating clothing matching scheme based on neural network according to claim 7, characterized in that: In step 33, for each clothing interest sequence, the blank elements need to be filled so that the clothing elements of each blank element sequence are arranged in the same order.
9. The method for generating clothing matching scheme based on neural network according to claim 1, characterized in that: Step 4 includes the following steps: Step 41: Obtain a three-dimensional human body model and the lengths of limbs in the three-dimensional human body model; Step 42: Obtain the selected clothing matching scheme, and obtain rendering information based on the products selected in the clothing matching scheme; Step 43: Determine rendering information of different regions in the three-dimensional human body model based on limb lengths and clothing sizes; Step 44: Input the 3D human body model and rendering information into DirectX to obtain a rendered clothing model.
10. A clothing matching scheme generation system based on neural network, characterized in that: A clothing matching scheme generation method using a neural network as described in any one of claims 1 to 9 is used to generate a clothing matching effect model.