A multi-feature enhanced deep learning clothing classification method based on auxiliary information

Abstract

The application discloses a kind of multi-feature enhanced deep learning clothes classification method based on auxiliary information, including determining clothes and user information needing classification;Mu-Cloth Net model is constructed based on convolutional neural network;Mu-Cloth Net model is trained, optimal weight is saved, and Mu-Cloth Net model is determined;Actual clothes image and user information are input into Mu-Cloth Net model, and clothes are classified.This scheme greatly improves the classification efficiency and improves the accuracy of classification by integrating auxiliary information, reduces the harmful effects of clothes deformation on the recognition result.And this scheme is respectively convolved to two information, and different scale features are extracted, so that the network can more effectively utilize auxiliary information to establish effective constraints.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a multi-feature enhanced deep learning method for clothing classification based on auxiliary information. Background Technology

[0002] In shopping malls and laundromats, clothing is still sorted manually, requiring significant manpower and resources to serve customers promptly. Even so, manual identification is slow, labor costs are high, and errors are common. To automate these processes and reduce labor costs, unmanned automated identification and sorting systems are essential.

[0003] Deep learning technology is developing rapidly and is widely used in fields such as computer vision and natural language processing. Compared with traditional methods, deep learning technology has significant advantages in object recognition. Deep learning only requires initial training and can automatically recognize objects in the future. However, current deep learning methods also have limitations. They are purely data-driven and have a high degree of uninterpretability. Moreover, different types of clothing have similar shapes, and due to the softness of the materials, there is a high possibility that the clothing itself may fold, twist, or change shape during actual recognition. Considering practical needs, current technology still requires recognition and classification methods with higher recognition efficiency and reliability. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-feature enhanced deep learning clothing classification method based on auxiliary information, which solves the technical problem of insufficient efficiency and reliability in the recognition of clothing, an object that is easily deformed, in the existing technology.

[0005] The aforementioned multi-feature enhancement deep learning clothing classification method based on auxiliary information includes the following steps:

[0006] Determine the clothing items to be categorized and the user information, obtain several images of clothing items for each category, label the clothing items accordingly, and set the user information accordingly to form a training set;

[0007] A Mu-Cloth Net model was constructed based on a convolutional neural network.

[0008] Train the Mu-Cloth Net model, save the optimal weights, and determine the Mu-Cloth Net model;

[0009] The Mu-Cloth Net model is used to classify clothing by inputting actual images of clothing and user information.

[0010] Preferably, the auxiliary information includes three pieces of information corresponding to the clothing: user gender, user age, and current season. After extracting the auxiliary information, the values ​​of the above three pieces of information are converted into a 3*224*224 image information matrix. The image information matrix corresponds to the clothing image. In the Mu-Cloth Net model, the input image information matrix and the clothing image are convolved respectively. The output results of each convolution layer are fused. The fused information is convolved from high order to low order to construct a fully connected layer. Different final output layer sizes are set for different categories to achieve classification of six types of clothing.

[0011] Preferably, for the training set, before input, the images are labeled with a number from 0 to n according to the clothing type. At the same time, the absolute path of each image is matched with the corresponding number according to the clothing type. When setting auxiliary information, male is labeled as 0 and female as 1 for user gender. User age is normalized and labeled as 0 to 1. The current season is labeled as 0, 1, 2, and 3 for spring, summer, autumn, and winter, respectively. In this way, the training and testing datasets are constructed.

[0012] Preferably, the Mu-Cloth Net model has 5 convolutional layers with a kernel size of 3. These convolutional layers extract image information with different features. The hidden layers are set to 128, 256, and 512, 1024, and 2048 respectively. The outputs of the five convolutional layers are 128*112*112, 256*256*256, 512*28*28, 1024*14*14, and 2048*7*7 respectively. The outputs of each convolutional layer for the two images are fused, with convolution performed from higher to lower order, resulting in outputs of 4096*7*7, 2048*28*28, 1024*56*56, 512*112*112, and 256*224*224 respectively.

[0013] Preferably, two fully connected layers with different outputs are constructed after the convolutional layer, with the first layer outputting 128 and the second layer outputting 6.

[0014] Preferably, activation functions ReLU and Dropout are added between each layer of the neural network to prevent overfitting and accelerate fitting.

[0015] Preferably, the image data with corresponding auxiliary information in the training set is shuffled and then input into the Mu-Cloth Net model for training. The optimizer is "Adam" and the loss function is "MAE". The optimized weights are obtained through the optimizer and the loss function, and the trained model is saved.

[0016] This invention offers the following advantages: First, deep learning methods are data-driven, finding patterns within "regular" data. This method proposes incorporating user-aided information. Since a user's gender, age, and seasonality significantly influence the type of clothing, this approach effectively narrows down the selection by leveraging the correlation between clothing, the user, and the current season before making a choice, enabling the network to make better decisions. By integrating auxiliary information, this approach greatly improves classification efficiency and accuracy, while reducing the detrimental impact of clothing deformation on the recognition results.

[0017] Secondly, our approach differs from normal information fusion by proposing "post-fusion." Unlike other deep learning models that typically add information to channels, we convolve the two pieces of information separately to extract features at different scales, and then fuse them at the highest order. The advantage of this approach is that it avoids inappropriate information being mixed in when extracting low-order information, thus preventing the network from receiving an incorrect signal. This allows the network to more effectively utilize auxiliary information to establish effective constraints.

[0018] Therefore, this invention can achieve better results than standard deep learning methods with less training data, achieving a clothing classification accuracy of 94%, a 5% improvement compared to standard neural networks. Furthermore, this method is flexible in implementation, not limited by clothing type or quantity, allowing for arbitrary combinations and strong adaptability. Finally, this method can achieve good results with only one training iteration, and after successful deployment, it can achieve fully automated classification in the future, completely replacing manual labor. Attached Figure Description

[0019] Figure 1 This is a flowchart of a multi-feature enhancement deep learning clothing classification method based on auxiliary information according to the present invention.

[0020] Figure 2 This is a model structure diagram of the Multiple Features Cloth Net model used in this invention. Detailed Implementation

[0021] The following detailed description of the embodiments, with reference to the accompanying drawings, will further illustrate the specific implementation of the present invention, in order to help those skilled in the art to have a more complete, accurate, and thorough understanding of the inventive concept and technical solutions of the present invention.

[0022] like Figure 1-2 As shown, the present invention provides a multi-feature enhanced deep learning clothing classification method based on auxiliary information, which includes the following steps.

[0023] Determine the clothing items to be categorized and the user information. Obtain several images of clothing items for each category, label the clothing type accordingly, and set the corresponding user information to form a training set.

[0024] A Multiple Features Cloth Net (Mu-Cloth Net) model was built based on a convolutional neural network.

[0025] Train the Mu-Cloth Net model, save the optimal weights, and determine the Mu-Cloth Net model.

[0026] The actual images and user information are input into the Mu-Cloth Net model to classify clothing.

[0027] The following explanation uses the example of dividing clothing into six categories: skirts, pants, shirts, T-shirts, down jackets, and coats.

[0028] The clothing items requiring categorization are determined to include the six categories mentioned above. Several images of each category are obtained using the Scrapy open-source web crawler framework and manually taken images to build a training set. The "user information" also includes: user gender, user age, and current season, obtained based on the user's registration information.

[0029] For the six categories mentioned above, each category is labeled as 0, 1, 2, 3, 4, and 5 in the order of skirts, pants, shirts, T-shirts, down jackets, and coats. Simultaneously, the absolute path of each image is mapped one-to-one with its corresponding number. When setting auxiliary information, user gender is labeled as 0 for males and 1 for females; user age is normalized and labeled from 0 to 1; and the current season is labeled as 0, 1, 2, and 3 for spring, summer, autumn, and winter, respectively. Finally, the training and testing datasets are constructed.

[0030] First, auxiliary information, including user gender, age, and current season, is extracted, resulting in three values. Each value is then copied to a 224*224 matrix, yielding a 3*224*224 image information matrix. This matrix is ​​then mapped to images of various clothing items and input into the initially constructed Mu-Cloth Net model.

[0031] Next, convolutions are performed on the two image data points separately, for a total of five convolutions. The outputs of each convolution layer are then fused, with the fused information arranged from high-order to low-order, ensuring the network learns important information from the outset. Finally, two fully connected layers are constructed to classify six categories of clothing. ReLU and Dropout activation functions are added between each neural network layer to prevent overfitting and accelerate the fitting process.

[0032] Specifically, the Mu-Cloth Net model described above consists of five convolutional layers with a kernel size of 3. These convolutional layers extract image information from different features. The hidden layers are set to 128, 256, and 512, 1024, and 2048 respectively. The outputs of the five convolutional layers are 128*112*112, 256*256*256, 512*28*28, 1024*14*14, and 2048*7*7, respectively. The outputs of each convolutional layer for the two images are fused, performing convolutions from higher to lower orders, resulting in outputs of 4096*7*7, 2048*28*28, 1024*56*56, 512*112*112, and 256*224*224. ReLU and Dropout activation functions are added between each neural network layer to prevent overfitting and accelerate fitting; Dropout is set to 0.5. Two fully connected layers are built after the convolutional layer. The first layer outputs 128, and the second layer outputs 6. Different final output layer sizes can be set for different categories.

[0033] The six types of image data were shuffled as described above and then fed into the Mu-Cloth Net model for training. The optimizer was "Adam" and the loss function was "MAE". The optimized weights were obtained through the optimizer and the loss function, and the trained model was saved.

[0034] The actual images and user information are then input into the Mu-Cloth Net model trained in the above process to classify clothing. There is a correspondence between the actual images and the user information.

[0035] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A multi-feature augmentation deep learning clothing classification method based on auxiliary information, characterized in that: Includes the following steps: Determine the clothing items to be categorized and the user information, obtain several images of clothing items for each category, label the clothing items accordingly, and set the user information accordingly to form a training set; A Mu-Cloth Net model was constructed based on a convolutional neural network. Train the Mu-Cloth Net model, save the optimal weights, and determine the Mu-Cloth Net model; The actual clothing images and user information are input into the Mu-Cloth Net model to classify the clothing; The auxiliary information includes three pieces of information corresponding to the clothing: user gender, user age, and current season. The scope is narrowed by utilizing the correlation between clothing, users, and the current season. After extracting the auxiliary information, the values ​​of the above three pieces of information are converted into a 3*224*224 image information matrix. The image information matrix corresponds to the clothing image. In the Mu-Cloth Net model, the input image information matrix and the clothing image are convolved respectively. The output results of each convolution layer are fused. The fused information is convolved from high order to low order to construct a fully connected layer. Different final output layer sizes are set for different categories to achieve classification of six types of clothing. In the Mu-Cloth Net model, two pieces of information are convolved separately to extract features at different scales. The features are then fused at the highest order. The outputs of each convolutional layer are fused, and the fused information is convolved from high order to low order.

2. The multi-feature enhancement deep learning clothing classification method based on auxiliary information according to claim 1, characterized in that: For the training set, before input, the images are labeled with numbers from 0 to n according to the clothing type. At the same time, the absolute path of each image is matched with the corresponding number according to the clothing type. When setting auxiliary information, male is labeled as 0 and female as 1 for user gender. User age is normalized and labeled as 0 to 1. The current season is labeled as 0, 1, 2, and 3 for spring, summer, autumn, and winter, respectively. Thus, the training and testing datasets are constructed.

3. The multi-feature enhancement deep learning clothing classification method based on auxiliary information according to claim 1, characterized in that: The Mu-Cloth Net model consists of 5 convolutional layers with a kernel size of 3. These convolutional layers extract image information from different features. The hidden layers are set to 128, 256, and 512, 1024, and 2048 respectively. The outputs of the five convolutional layers are 128*112*112, 256*256*256, 512*28*28, 1024*14*14, and 2048*7*7. The outputs of each convolutional layer for the two images are then fused, with convolution performed from higher to lower order. The outputs are 4096*7*7, 2048*28*28, 1024*56*56, 512*112*112, and 256*224*224 respectively.

4. The multi-feature enhancement deep learning clothing classification method based on auxiliary information according to claim 3, characterized in that: Two fully connected layers with different outputs are constructed after the convolutional layer. The first layer outputs 128, and the second layer outputs 6.

5. The multi-feature enhancement deep learning clothing classification method based on auxiliary information according to claim 1, characterized in that: Add activation functions ReLU and Dropout between each layer of the neural network to prevent overfitting and speed up the fitting process.

6. A multi-feature enhancement deep learning clothing classification method based on auxiliary information according to claim 1, characterized in that: The image data with corresponding auxiliary information in the training set is shuffled and then fed into the Mu-Cloth Net model for training. The optimizer is "Adam" and the loss function is "MAE". The optimized weights are obtained through the optimizer and the loss function, and the trained model is saved.

Images