Clothing attribute recognition model construction method and construction system based on multi-label

Abstract

The present invention discloses a method and system for constructing a multi-label clothing attribute recognition model, belonging to the field of computer vision technology. The technical problem to be solved is how to implement multi-label clothing attribute recognition in a domestic AI accelerator card environment. The method comprises the following steps: obtaining multiple clothing images as samples to construct a dataset; performing dataset equalization processing on the dataset based on a sample label method of label reorganization, and performing data enhancement on each obtained equalized sub-dataset to obtain multiple enhanced sub-datasets, which are used as training sub-datasets; constructing a clothing attribute recognition model based on a Resnet50 network, the clothing attribute recognition model being used to take clothing images as input, predict and output clothing attributes; and training the clothing attribute recognition model using a multi-stage hot restart optimizer based on the training sub-datasets and corresponding labels to obtain a final clothing attribute recognition model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for constructing a clothing attribute recognition model based on multiple labels. Background Art

[0002] While there are established solutions for clothing attribute recognition, these solutions are all based on single-label models, each of which can only recognize a single attribute. Multiple models are required for multi-attribute recognition. Simultaneously using multiple models for real-time computing on domestic AI accelerator cards presents performance issues, necessitating a multi-label model for multi-attribute clothing recognition.

[0003] Deep learning algorithms typically make the prior assumption that the training dataset (traindata) and validation dataset (testdata) used in training are independent and identically distributed (IID) from the actual data. Therefore, they are generally required to be balanced. However, in real-world engineering applications, collecting datasets is expensive, requiring a balance between dataset quality and cost. This often makes it difficult to collect balanced datasets. In the clothing attribute dataset collected using this method, white and black far outnumber other colors, and trousers and T-shirts far outnumber other styles. Therefore, directly training a deep learning model with this dataset is not feasible; a balanced dataset is required.

[0004] Dataset balancing methods can be categorized into two types: data-level and algorithm-level. Data-level methods primarily involve oversampling and undersampling, while algorithm-level methods primarily utilize loss functions with weighted penalties. These methods primarily target single-label datasets. When applied to multi-label datasets, they suffer from issues such as decreased model accuracy and difficulty determining penalty weight functions.

[0005] How to realize multi-label clothing attribute recognition in the domestic AI acceleration card environment is a technical problem that needs to be solved. Summary of the Invention

[0006] The technical task of the present invention is to address the above shortcomings and provide a multi-label clothing attribute recognition model construction method and construction system to solve the technical problem of how to realize multi-label clothing attribute recognition in a domestic AI acceleration card environment.

[0007] The method for constructing a clothing attribute recognition model based on multiple labels according to an embodiment of the present invention includes the following steps:

[0008] Obtain multiple clothing images as samples to construct a data set, where each sample corresponds to at least one label identifying a clothing attribute;

[0009] Performing dataset balancing on the dataset based on a sample labeling method using label reorganization, and performing data enhancement on each obtained balanced sub-dataset to obtain multiple enhanced sub-datasets, wherein the enhanced sub-datasets are used as training sub-datasets;

[0010] Building a clothing attribute recognition model based on the Resnet50 network, the clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes;

[0011] Based on the training sub-datasets and corresponding labels, the clothing attribute recognition model is trained through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model.

[0012] Preferably, data enhancement is performed on each balanced sub-dataset using the MixUp data enhancement method.

[0013] Preferably, a sample labeling method based on label reorganization is used to perform data set balancing on the data set, including the following steps:

[0014] Calculate the number of samples for each label in the dataset;

[0015] Extract the samples with the least labels to form a set G, and the remaining samples to form a set R;

[0016] Calculate the number of samples of each label in set G, and randomly select samples from set R and add them to set G to obtain set G1. The number of samples of each label in set G1 is balanced.

[0017] Set the number of sets R to m, the number of samples added to the set G to n, and t to the number of times it needs to be repeated. Repeat t times to obtain t balanced sub-datasets, and randomly add the remaining samples to the t balanced sub-datasets.

[0018] Preferably, the clothing attribute recognition model includes:

[0019] a data preprocessing module, the data preprocessing module including an image scaling layer and a data normalization layer, the image scaling layer being used to scale the input clothing image to a uniform size, and the data normalization layer being used to map the range of each pixel value in the scaled clothing image from [0, 255] to [0, 1];

[0020] A feature extraction module, which is a Resnet50 backbone network, is used to extract graphic features and obtain a 2048-dimensional feature vector;

[0021] The data post-processing module includes a data feature vector classification layer and a classification result translation layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2-dimensional feature vector to obtain a prediction result. The classification result translation layer maps the 2-dimensional feature vector into two attributes of clothing style and color as model output.

[0022] Preferably, the clothing attribute recognition model is trained based on the training sub-dataset and the corresponding labels through a multi-stage hot restart optimizer, comprising the following steps:

[0023] Load the training data sets in sequence, and determine different numbers of iterations according to the number of samples and batch size of each training data set;

[0024] Each training dataset is trained in a loop, and a multi-stage restart optimization trainer is used on each training dataset to train the model. The learning rate is reset at the beginning of the loop until all training datasets are trained to obtain the final clothing attribute recognition model.

[0025] Preferably, the final clothing attribute recognition model is quantified and deployed on a domestic AI acceleration card.

[0026] In a second aspect, the present invention provides a system for constructing a clothing attribute recognition model based on multiple labels, comprising:

[0027] A data set acquisition module, the data set acquisition module is used to acquire multiple clothing images as samples to construct a data set, each sample corresponding to at least one label identifying a clothing attribute;

[0028] A data set balancing processing module, the data set balancing processing module is used to perform data set balancing processing on the data set based on a sample label method of label reorganization, and perform data enhancement on each obtained balanced sub-data set to obtain multiple enhanced sub-data sets, and the enhanced sub-data sets are used as training sub-data sets;

[0029] A model configuration module, wherein the model configuration module is used to build a clothing attribute recognition model based on the Resnet50 network, wherein the clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes;

[0030] A model training module is used to train the clothing attribute recognition model based on the training sub-dataset and the corresponding labels through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model.

[0031] Preferably, the data set balancing processing module is used to perform data set balancing processing on the data set through the following steps:

[0032] Calculate the number of samples for each label in the dataset;

[0033] Extract the samples with the least labels to form a set G, and the remaining samples to form a set R;

[0034] Calculate the number of samples of each label in set G, and randomly select samples from set R and add them to set G to obtain set G1. The number of samples of each label in set G1 is balanced.

[0035] Set the number of sets R to m, the number of samples added to the set G to n, and t to the number of times it needs to be repeated. Repeat t times to obtain t balanced sub-datasets, and randomly add the remaining samples to the t balanced sub-datasets;

[0036] And the data set balancing processing module is used to perform data enhancement on each balanced sub-data set through the MixUp data enhancement method.

[0037] Preferably, the clothing attribute recognition model includes:

[0038] a data preprocessing module, the data preprocessing module including an image scaling layer and a data normalization layer, the image scaling layer being used to scale the input clothing image to a uniform size, and the data normalization layer being used to map the range of each pixel value in the scaled clothing image from [0, 255] to [0, 1];

[0039] A feature extraction module, which is a Resnet50 backbone network, is used to extract graphic features and obtain a 2048-dimensional feature vector;

[0040] The data post-processing module includes a data feature vector classification layer and a classification result translation layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2-dimensional feature vector to obtain a prediction result. The classification result translation layer maps the 2-dimensional feature vector into two attributes of clothing style and color as model output.

[0041] Preferably, the model training module is used to train the clothing attribute recognition model through the following steps:

[0042] Load the training data sets in sequence, and determine different numbers of iterations according to the number of samples and batch size of each training data set;

[0043] Each training dataset is trained in a loop, and a multi-stage restart optimization trainer is used on each training dataset to train the model. The learning rate is reset at the beginning of the loop until all training datasets are trained to obtain the final clothing attribute recognition model.

[0044] The multi-label-based clothing attribute recognition model construction method and construction system of the present invention have the following advantages:

[0045] 1. For the acquired data set, a sample label method based on label reorganization is used to perform data set balancing on the data set. That is, the data balancing method fully utilizes all the data in the data set to avoid overfitting and underfitting problems caused by general data balancing methods.

[0046] 2. When training the clothing attribute recognition model, we use the hot restart technology to fully train on the dataset, preventing the model from falling into the local optimum and improving the model prediction accuracy;

[0047] 3. The clothing attribute recognition model constructed by this method fully considers the performance issues of the domestic environment and improves the operating efficiency of the algorithm in the domestic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] Figure 1 This is a flowchart of a method for constructing a clothing attribute recognition model based on multiple labels in Example 1;

[0051] Figure 2 This is a structural block diagram of the clothing recognition model in the method for constructing a clothing attribute recognition model based on multiple labels in Example 1. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments given are not intended to limit the present invention. Unless there is a conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0053] The embodiments of the present invention provide a method and system for constructing a multi-label clothing attribute recognition model, which are used to solve the technical problem of how to achieve multi-label clothing attribute recognition in a domestic AI accelerator card environment.

[0054] Example 1:

[0055] The present invention provides a method for constructing a clothing attribute recognition model based on multiple labels, comprising the following steps:

[0056] S100, obtaining multiple clothing images as samples to construct a data set, each sample corresponding to at least one label identifying clothing attributes;

[0057] S200, performing data set balancing on the data set based on a sample labeling method of label reorganization, and performing data enhancement on each obtained balanced sub-data set to obtain multiple enhanced sub-data sets, and the enhanced sub-data sets are used as training sub-data sets;

[0058] S300: Build a clothing attribute recognition model based on the Resnet50 network. The clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes;

[0059] S400 : Based on the training sub-dataset and the corresponding labels, the clothing attribute recognition model is trained through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model.

[0060] In this embodiment, step S100 obtains a data set, which includes multiple samples. Each sample corresponds to at least one label, and the label is used to identify clothing attributes, such as style and color.

[0061] Step S200 balances the data set. As a specific implementation, the method first calculates the number of samples of each label in the data set, extracts the samples with the least number of labels to form a set G, and records the remaining samples as set R; then calculates the number of samples of each label in set G, randomly selects samples from set R and adds them to set G, so that the number of samples of each label in set G is balanced, which is recorded as G1. Assuming that the number of set R is m, the number of samples added to set G is n, and t is the number of times to be repeated, then Repeat t times to obtain t balanced sub-datasets G1...Gt, and randomly add the remaining samples to the t sub-datasets; use the MixUp data enhancement method on the t sub-datasets to generate t enhanced sub-datasets.

[0062] Step S300 uses a deep convolutional network structure to construct a clothing attribute recognition model, consisting of a data preprocessing module, a classic Resnet50 backbone network, and a data post-processing module. Domestic CPUs have relatively weak performance, so adding data preprocessing and post-processing functions to the model network improves data parallelism and significantly increases prediction speed. The data preprocessing module includes an image scaling layer and a data normalization layer. The image scaling layer uniformly scales the input image to a 96-pixel length by 96-pixel width image. The data normalization layer maps each pixel value from [0, 255] to [0, 1]. A Resnet50 backbone network is used to extract image features, generating a 2048-dimensional feature vector. The data post-processing module includes a data feature vector classification layer and a classification result conversion layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2D feature vector, thereby obtaining the prediction result. The classification result conversion layer maps the 2D feature vector into two attributes: clothing style and color, which serve as the model output.

[0063] Step S400 trains the clothing attribute recognition model constructed above. As a specific implementation, the method uses a multi-stage hot restart optimizer for training. The t training sub-datasets obtained in step S200 are loaded sequentially, and the number of iterations is determined based on the number of samples in each sub-dataset and the batch size. The multi-stage restart optimizer trainer is used for training on each sub-dataset. A large learning rate is selected at the beginning of training to ensure rapid model convergence. After convergence, the learning rate is gradually reduced to find the optimal point. The training is repeated for each data set, with the learning rate reset at the beginning of the loop, until all data sets are trained to obtain the final model.

[0064] The trained clothing attribute recognition model is quantified and deployed on a domestic AI accelerator card. The clothing image is input, the model is used for prediction, and the predicted clothing attribute results are returned.

[0065] This method obtains multiple balanced sub-datasets by reorganizing samples; designs a deep learning model suitable for domestic AI accelerator cards, and uses a multi-stage hot restart optimizer to train the model on each sub-dataset to avoid the model falling into local optimal points, thereby improving the model's clothing attribute recognition accuracy; fully considers the performance issues of the domestic environment and improves the operating efficiency of the model in the domestic environment.

[0066] Example 2:

[0067] The present invention is based on a multi-label clothing attribute recognition model construction system, which includes a data set acquisition module, a data set equalization processing module, a model configuration module and a model training module. The data set acquisition module is used to acquire multiple clothing images as samples to construct a data set, and each sample corresponds to at least one label identifying a clothing attribute; the data set equalization processing module is used to perform data set equalization processing on the data set based on a sample label method of label reorganization, and perform data enhancement on each obtained equalized sub-data set to obtain multiple enhanced sub-data sets, and the enhanced sub-data sets are used as training sub-data sets; the model configuration module is used to construct a clothing attribute recognition model based on a Resnet50 network, and the clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes; the model training module is used to train the clothing attribute recognition model based on the training sub-data sets and corresponding labels through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model.

[0068] In this embodiment, the data set balancing processing module is used to perform data set balancing processing on the data set through the following steps:

[0069] (1) Calculate the number of samples for each label in the dataset;

[0070] (2) Extract the samples with the least labels to form a set G, and the remaining samples to form a set R;

[0071] (3) Calculate the number of samples of each label in set G, and randomly select samples from set R and add them to set G to obtain set G1. The number of samples of each label in set G1 is balanced;

[0072] (4) Set the number of sets R to m, the number of samples added to set G to n, and t to the number of times it needs to be repeated. Repeat t times to obtain t balanced sub-datasets, and randomly add the remaining samples to the t balanced sub-datasets.

[0073] The dataset balancing processing module is used to perform data enhancement on each balanced sub-dataset using the MixUp data enhancement method.

[0074] The clothing attribute recognition model consists of a data preprocessing module, a classic Resnet50 backbone network, and a data postprocessing module. Due to the relatively weak performance of domestically produced CPUs, data preprocessing and postprocessing were added to the model network to enhance data parallelism and significantly improve prediction speed. The data preprocessing module includes an image scaling layer and a data normalization layer. The image scaling layer scales the input image to a uniform length and width of 96 pixels, while the data normalization layer maps each pixel value from the range [0, 255] to [0, 1]. A Resnet50 backbone network is used to extract image features, generating a 2048-dimensional feature vector. The data postprocessing module includes a data feature vector classification layer and a classification result conversion layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2D feature vector, thereby generating the prediction result. The classification result conversion layer maps the 2D feature vector into two attributes: clothing style and color, which serve as the model output.

[0075] The model training module is used to train the clothing attribute recognition model through the following steps: loading the t sub-datasets obtained in the first step in sequence, and determining different numbers of iterations according to the number of samples in each sub-dataset and the batch size; using a multi-stage restart optimization trainer for training on each sub-dataset, selecting a larger learning rate at the beginning of training to make the model converge quickly, and gradually reducing the learning rate after the model converges to find the optimal point; looping training for each data set, resetting the learning rate at the beginning of the loop, until all data sets are trained to obtain the final model.

[0076] The system of this embodiment can execute the method disclosed in Example 1 to construct a clothing attribute recognition model.

[0077] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A method for constructing a clothing attribute recognition model based on multiple labels, characterized by The steps include: Obtain multiple clothing images as samples to construct a data set, where each sample corresponds to at least one label identifying a clothing attribute; The dataset is balanced based on a sample label method using label reorganization, and each balanced sub-dataset obtained is enhanced using a MixUp data enhancement method to obtain multiple enhanced sub-datasets, which are used as training sub-datasets. Building a clothing attribute recognition model based on the Resnet50 network, the clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes; Based on the training sub-dataset and the corresponding labels, the clothing attribute recognition model is trained through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model. The final clothing attribute recognition model is quantized and deployed on a domestic AI accelerator card; The sample label method based on label reorganization performs data set balancing on the data set, including the following steps: Calculate the number of samples for each label in the dataset; Extract the samples with the least labels to form a set G, and the remaining samples to form a set R; Calculate the number of samples of each label in set G, and randomly select samples from set R and add them to set G to obtain set G1. The number of samples of each label in set G1 is balanced. Set the number of sets R to m, the number of samples added to set G to n, and t to the number of repetitions required. Repeat t times to obtain t balanced sub-datasets, and randomly add the remaining samples to the t balanced sub-datasets. The clothing attribute recognition model includes: a data preprocessing module, the data preprocessing module including an image scaling layer and a data normalization layer, the image scaling layer being used to scale the input clothing image to a uniform size, and the data normalization layer being used to map the range of each pixel value in the scaled clothing image from [0, 255] to [0, 1]; Feature extraction module, which is a Resnet50 backbone network used to extract image features and obtain a 2048-dimensional feature vector; A data post-processing module, comprising a data feature vector classification layer and a classification result translation layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2-dimensional feature vector to obtain a prediction result. The classification result translation layer maps the 2-dimensional feature vector into two attributes of clothing style and color as model output; The clothing attribute recognition model is trained based on the training sub-dataset and the corresponding labels through a multi-stage hot restart optimizer, including the following steps: Load the training sub-datasets in sequence, and determine different numbers of iterations according to the number of samples and batch size of each training sub-dataset; Each training sub-dataset is trained in a loop, and a multi-stage restart optimization trainer is used on each training sub-dataset for model training. The learning rate is reset at the beginning of the loop until all training sub-datasets are trained to obtain the final clothing attribute recognition model.

2. A multi-label clothing attribute recognition model construction system, characterized by include: A data set acquisition module, wherein the data set acquisition module is used to acquire a plurality of clothing images as samples to construct a data set, wherein each sample corresponds to at least one label identifying a clothing attribute; A data set balancing processing module, the data set balancing processing module is used to perform data set balancing processing on the data set based on a sample label method of label reorganization, and perform data enhancement on each obtained balanced sub-data set to obtain multiple enhanced sub-data sets, and the enhanced sub-data sets are used as training sub-data sets; A model configuration module, wherein the model configuration module is used to build a clothing attribute recognition model based on the Resnet50 network, wherein the clothing attribute recognition model is used to take clothing images as input, predict and output clothing attributes; a model training module, the model training module being configured to train the clothing attribute recognition model based on the training sub-dataset and the corresponding labels through a multi-stage hot restart optimizer to obtain a final clothing attribute recognition model; The data set balancing processing module is used to perform data set balancing processing on the data set through the following steps: Calculate the number of samples for each label in the dataset; Extract the samples with the least labels to form a set G, and the remaining samples to form a set R; Calculate the number of samples of each label in set G, and randomly select samples from set R and add them to set G to obtain set G1. The number of samples of each label in set G1 is balanced. Set the number of sets R to m, the number of samples added to set G to n, and t to the number of repetitions required. Repeat t times to obtain t balanced sub-datasets, and randomly add the remaining samples to the t balanced sub-datasets. And the data set balancing processing module is used to perform data enhancement on each balanced sub-data set through the MixUp data enhancement method; The clothing attribute recognition model includes: a data preprocessing module, the data preprocessing module including an image scaling layer and a data normalization layer, the image scaling layer being used to scale the input clothing image to a uniform size, and the data normalization layer being used to map the range of each pixel value in the scaled clothing image from [0, 255] to [0, 1]; Feature extraction module, which is a Resnet50 backbone network used to extract image features and obtain a 2048-dimensional feature vector; A data post-processing module, comprising a data feature vector classification layer and a classification result translation layer. The feature vector classification layer maps the 2048-dimensional feature vector into a 2-dimensional feature vector to obtain a prediction result. The classification result translation layer maps the 2-dimensional feature vector into two attributes of clothing style and color as model output; The model training module is used to train the clothing attribute recognition model through the following steps: Load the training sub-datasets in sequence, and determine different numbers of iterations according to the number of samples and batch size of each training sub-dataset; Each training sub-dataset is trained in a loop, and a multi-stage restart optimization trainer is used on each training sub-dataset for model training. The learning rate is reset at the beginning of the loop until all training sub-datasets are trained to obtain the final clothing attribute recognition model.

Images