A clothing identification method, device, electronic device and storage medium

Abstract

The present invention discloses a clothing recognition method, device, electronic device and storage medium. The method extracts features of a clothing image to be recognized based on a neural network model to obtain a first feature vector, and determines the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; if a target similarity that satisfies the similarity condition exists in each similarity, it is determined that the clothing attribute in the clothing image to be recognized is the same as the target attribute. For the chef uniform recognition scenario, the embodiment of the present invention recognizes clothing based on the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library. Each time a new chef uniform is added, it only needs to be added to the feature vector library, and there is no need to retrain the model, so there will be no problem of sample imbalance. Therefore, the accuracy of clothing recognition in the embodiment of the present invention is relatively high.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a clothing recognition method, device, electronic equipment and storage medium. Background Art

[0002] For regulatory authorities, remote supervision of food production companies and catering units is undoubtedly a good choice, and one of the important modules is to identify whether the chef is wearing chef uniforms. Clothing recognition is the recognition of clothing attributes, and chef uniforms are one of the clothing attributes.

[0003] In recent years, as deep learning has gradually become a popular research direction, convolutional neural networks have been widely used in image tasks. The general method is to directly treat various chef uniforms and non-chef uniforms as multiple categories for target detection. However, as the number of stores increases, once a new category of chef uniforms is added, all previous data must be retrained and tested before deployment. Moreover, there are often very few chef uniform samples in newly added stores, which is prone to sample imbalance. As a result, the accuracy of the final trained model is difficult to guarantee, and the accuracy of clothing recognition is poor. Summary of the invention

[0004] The embodiments of the present invention provide a clothing recognition method, device, electronic device and storage medium, which are used to solve the problem of poor accuracy of existing clothing recognition.

[0005] An embodiment of the present invention provides a clothing recognition method, the method comprising:

[0006] Acquire a clothing image to be identified, and extract features of the clothing image to be identified based on a neural network model to obtain a first feature vector;

[0007] Determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model;

[0008] If there is a target similarity satisfying the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0009] Furthermore, the acquiring of the clothing image to be identified includes:

[0010] The collected image is input into a target detection model, a clothing recognition area in the image is determined based on the target detection model, and an image corresponding to the clothing recognition area is used as a clothing image to be recognized.

[0011] Further, the determining of the similarity between the first feature vector and each second feature vector in each pre-stored feature vector library comprises:

[0012] For each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library is determined respectively, and a preset number of cosine distances are selected in order of cosine distances from small to large, and the average value of the selected cosine distances is determined, and the corresponding similarity is determined according to the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

[0013] Furthermore, if there is a target similarity satisfying the similarity condition in each similarity, determining that the clothing attribute in the clothing image to be identified is the same as the target attribute includes:

[0014] The maximum similarity is selected. If the maximum similarity is greater than a preset similarity threshold, it is determined that there is a target similarity that satisfies the similarity condition, and it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0015] Furthermore, the feature extraction of the clothing image to be identified based on the neural network model to obtain the first feature vector includes:

[0016] The first branch network of the neural network model extracts features of the clothing image to be identified to obtain a first depth feature vector; the second branch network of the neural network model extracts features of the clothing image to be identified to obtain a first color feature vector;

[0017] Determining the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library includes:

[0018] For each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library is determined, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library is determined; the depth similarity and the color similarity are weightedly added to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

[0019] Furthermore, the training process of the neural network model includes:

[0020] The sample images in the training set are input into the neural network model, and the features of the sample images are extracted based on the first branch network of the neural network model to obtain a sample depth feature vector; the features of the sample images are extracted based on the second branch network of the neural network model to obtain a sample color feature vector; the sample clothing category information corresponding to the sample image is obtained based on the sample depth feature vector and the sample color feature vector; the first branch network and the second branch network in the neural network model are trained according to the sample clothing category information, the labeled clothing category information corresponding to the sample image and the loss function.

[0021] Furthermore, the first branch network includes a backbone network, a first average pooling layer, a first fully connected layer and a second fully connected layer connected in sequence; the second branch network includes a second average pooling layer and a third fully connected layer connected in sequence.

[0022] On the other hand, an embodiment of the present invention provides a clothing recognition device, the device comprising:

[0023] An acquisition module, used for acquiring a clothing image to be identified, performing feature extraction on the clothing image to be identified based on a neural network model, and obtaining a first feature vector;

[0024] A first determination module is used to determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model;

[0025] The second determination module is configured to determine that the clothing attribute in the clothing image to be identified is the same as the target attribute if there is a target similarity satisfying the similarity condition in each similarity.

[0026] The acquisition module is specifically used to input the collected image into the target detection model, determine the clothing recognition area in the image based on the target detection model, and use the image corresponding to the clothing recognition area as the clothing image to be recognized.

[0027] The first determination module is specifically used to determine, for each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library, select a preset number of cosine distances in order from small to large, determine the average value of the selected cosine distances, and determine the corresponding similarity based on the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

[0028] The second determination module is specifically used to select the maximum similarity. If the maximum similarity is greater than a preset similarity threshold, it is determined that there is a target similarity that meets the similarity condition, and then it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0029] The acquisition module is specifically used to extract features of the clothing image to be identified based on the first branch network of the neural network model to obtain a first depth feature vector; and to extract features of the clothing image to be identified based on the second branch network of the neural network model to obtain a first color feature vector;

[0030] The first determination module is specifically used to determine, for each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library; and perform weighted addition of the depth similarity and color similarity to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

[0031] The device also includes:

[0032] A training module is used to input the sample images in the training set into the neural network model, extract features of the sample images based on the first branch network of the neural network model to obtain a sample depth feature vector; extract features of the sample images based on the second branch network of the neural network model to obtain a sample color feature vector; obtain sample clothing category information corresponding to the sample image based on the sample depth feature vector and the sample color feature vector; and train the first branch network and the second branch network in the neural network model according to the sample clothing category information, the labeled clothing category information corresponding to the sample image, and the loss function.

[0033] In another aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0034] Memory, used to store computer programs;

[0035] The processor is used to implement any of the method steps described above when executing the program stored in the memory.

[0036] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the above are implemented.

[0037] An embodiment of the present invention provides a clothing recognition method, device, electronic device and storage medium, the method comprising: obtaining a clothing image to be recognized, performing feature extraction on the clothing image to be recognized based on a neural network model to obtain a first feature vector; determining the similarity between the first feature vector and a second feature vector in each pre-saved feature vector library; wherein the second feature vector in each feature vector library is obtained by performing feature extraction on each type of target clothing image of the same target attribute based on the neural network model; if there is a target similarity that satisfies the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be recognized is the same as the target attribute.

[0038] The above technical solution has the following advantages or beneficial effects:

[0039] In the embodiment of the present invention, the feature extraction of the clothing image to be identified is performed based on the neural network model to obtain the first feature vector, and the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library is determined; if there is a target similarity that satisfies the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute. For the chef uniform recognition scenario, the second feature vector in each feature vector library is obtained by extracting features from the target clothing images of various types of chef uniforms based on the neural network model. The embodiment of the present invention performs clothing recognition based on the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library. Each time a new chef uniform is added, it only needs to be added to the feature vector library, and there is no need to retrain the model. Therefore, there will be no problem of sample imbalance. Therefore, the accuracy of clothing recognition in the embodiment of the present invention is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic diagram of a clothing recognition process provided in Example 1 of the present invention;

[0042] Figure 2 A schematic diagram of the structure of a neural network model provided in Example 4 of the present invention;

[0043] Figure 3 A schematic diagram of a clothing recognition process provided in Embodiment 4 of the present invention;

[0044] Figure 4A schematic diagram of a process for calculating a fixed-length feature vector using a deep neural network model provided in Example 4 of the present invention;

[0045] Figure 5 A schematic diagram of a process for determining similarity between feature vectors provided in Embodiment 4 of the present invention;

[0046] Figure 6 A schematic diagram of the structure of a clothing recognition device provided in Embodiment 5 of the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Example 6 of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Embodiment 1:

[0050] Figure 1 A schematic diagram of a clothing recognition process provided by an embodiment of the present invention, the process includes the following steps:

[0051] S101: Acquire a clothing image to be identified, and extract features of the clothing image to be identified based on a neural network model to obtain a first feature vector.

[0052] S102: Determine the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model.

[0053] S103: If there is a target similarity satisfying the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0054] The clothing recognition method provided in the embodiment of the present invention is applied to an electronic device, which can be a PC, a tablet computer or other device, or an image acquisition device with image acquisition and analysis capabilities. If the electronic device is an image acquisition device, after the image acquisition device acquires the image, it obtains the clothing image to be identified, and extracts features of the clothing image to be identified based on the neural network model to obtain a first feature vector, and then performs a subsequent clothing recognition process. If the electronic device is a PC, a tablet computer or other device, after the image acquisition device acquires the image, it sends the acquired image to the electronic device, and the electronic device acquires the clothing image to be identified, and extracts features of the clothing image to be identified based on the neural network model to obtain a first feature vector, and then performs a subsequent clothing recognition process.

[0055] The electronic device stores a pre-trained neural network model, which is used to extract features from the clothing image to be identified to obtain a first feature vector. In addition, each feature vector library is stored in the electronic device, and the feature vector in the feature vector library is called a second feature vector. Based on the neural network model, the electronic device extracts features from each type of target clothing image of the same target attribute to obtain a second feature vector in each feature vector library. Among them, one type of target clothing image corresponds to one feature vector library. If one type of target clothing image is multiple images, then the corresponding second feature vector in one feature vector library is also multiple. Taking the same target attribute as chef uniforms as an example, there are many types of chef uniforms. The chef uniforms for Chinese and Western restaurants are different, or the chef uniforms for different restaurants are also different.

[0056] Determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library. If the feature vector library includes multiple second feature vectors, calculate the similarity between the first feature vector and each second feature vector in the feature vector library. In this way, multiple similarities can be obtained, wherein each second feature vector corresponds to one similarity. Determine whether there is a target similarity that meets the similarity condition in each similarity. Among them, the electronic device can save a preset similarity threshold, determine whether there is a target similarity greater than the preset similarity threshold in each similarity, and if so, determine that there is a target similarity that meets the similarity condition. At this time, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute, for example, if the target attribute is a chef's uniform, then it is determined that the clothing in the clothing image to be identified is a chef's uniform. In addition, the embodiment of the present invention can also determine that the clothing category corresponding to the target similarity is the clothing category in the clothing image to be identified. If there are multiple target similarities greater than the preset similarity threshold, it is determined that the clothing category corresponding to the maximum target similarity is the clothing category in the clothing image to be identified.

[0057] Preferably, if there is a target similarity satisfying the similarity condition in each similarity, determining that the clothing attribute in the clothing image to be identified is the same as the target attribute comprises:

[0058] The maximum similarity is selected. If the maximum similarity is greater than a preset similarity threshold, it is determined that there is a target similarity that satisfies the similarity condition, and it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0059] In the embodiment of the present invention, after each similarity is determined, the maximum similarity is selected, and then it is determined whether the maximum similarity is greater than a preset similarity threshold. If so, it is determined that there is a target similarity that meets the similarity condition, and it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0060] In the embodiment of the present invention, the feature extraction of the clothing image to be identified is performed based on the neural network model to obtain the first feature vector, and the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library is determined; if there is a target similarity that satisfies the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute. For the chef uniform recognition scenario, the second feature vector in each feature vector library is obtained by extracting features from the target clothing images of various types of chef uniforms based on the neural network model. The embodiment of the present invention performs clothing recognition based on the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library. Each time a new chef uniform is added, it only needs to be added to the feature vector library, and there is no need to retrain the model. Therefore, there will be no problem of sample imbalance. Therefore, the accuracy of clothing recognition in the embodiment of the present invention is relatively high.

[0061] Embodiment 2:

[0062] In order to make the acquired clothing image to be identified more accurate, and thus make clothing identification more accurate, based on the above embodiment, in an embodiment of the present invention, the acquiring of the clothing image to be identified includes:

[0063] The collected image is input into a target detection model, a clothing recognition area in the image is determined based on the target detection model, and an image corresponding to the clothing recognition area is used as a clothing image to be recognized.

[0064] The image captured by the image acquisition device generally includes the whole body of the human body and the background. The background will interfere with the extraction of feature vectors and clothing recognition. In order to solve the above problems, in an embodiment of the present invention, a pre-trained target detection model is stored in the electronic device, and the captured image is input into the target detection model. The clothing recognition area in the image is determined based on the target detection model, and the image corresponding to the clothing recognition area is used as the clothing image to be recognized. Among them, the clothing recognition area can be the clothing area of ​​the entire human body, preferably, it can be the upper part of the human body. If the clothing recognition area is the upper part, then when the target detection model is trained, for the sample detection image, the location information of the upper part in the sample detection image is annotated, and then the sample detection image and the annotated upper part are input into the target detection model to train the target detection model.

[0065] In the embodiment of the present invention, the collected image is input into the target detection model, the clothing recognition area in the image is determined based on the target detection model, and the image corresponding to the clothing recognition area is used as the clothing image to be recognized. The clothing image to be recognized obtained in the embodiment of the present invention does not have background interference, so that the clothing recognition result based on the clothing image to be recognized is more accurate.

[0066] Embodiment 3:

[0067] In order to make the determination of the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library more accurate, based on the above embodiments, in an embodiment of the present invention, the determination of the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library includes:

[0068] For each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library is determined respectively, and a preset number of cosine distances are selected in order of cosine distances from small to large, and the average value of the selected cosine distances is determined, and the corresponding similarity is determined according to the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

[0069] In an embodiment of the present invention, for each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library is determined respectively, and the smaller the cosine distance, the higher the similarity. Then, in the order of cosine distance from small to large, a preset number of cosine distances are selected, and the average value of the selected cosine distances is determined. Then, the corresponding similarity is determined according to the average value of the cosine distance; wherein, the smaller the average value of the cosine distance, the greater the similarity. In an embodiment of the present invention, the difference between 1 and the average value of the cosine distance can be used as the corresponding similarity, or a corresponding relationship between the average value of the cosine distance and the similarity is established according to the principle that the smaller the average value of the cosine distance, the greater the similarity, and then after determining the average value of the cosine distance, the corresponding similarity is determined according to the corresponding relationship.

[0070] Embodiment 4:

[0071] In order to extract features from the clothing image to be identified based on the neural network model and obtain the first feature vector more accurately, based on the above embodiments, in an embodiment of the present invention, extracting features from the clothing image to be identified based on the neural network model and obtaining the first feature vector includes:

[0072] The first branch network of the neural network model extracts features of the clothing image to be identified to obtain a first depth feature vector; the second branch network of the neural network model extracts features of the clothing image to be identified to obtain a first color feature vector;

[0073] Determining the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library includes:

[0074] For each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library is determined, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library is determined; the depth similarity and the color similarity are weightedly added to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

[0075] In an embodiment of the present invention, the neural network model includes a first branch network and a second branch network. The first branch network of the neural network model performs feature extraction on the clothing image to be identified to obtain a first depth feature vector; the second branch network of the neural network model performs feature extraction on the clothing image to be identified to obtain a first color feature vector. When determining the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library, respectively, for each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library is determined, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library is determined; the depth similarity and the color similarity are weightedly added to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

[0076] When respectively determining the weighted weights corresponding to the depth similarity and the color similarity, different weights can be assigned to the depth similarity and the color similarity according to needs, or the weighted weights corresponding to the depth similarity and the color similarity can be trained using a neural network model.

[0077] The training process of the neural network model includes:

[0078] The sample images in the training set are input into the neural network model, and the features of the sample images are extracted based on the first branch network of the neural network model to obtain a sample depth feature vector; the features of the sample images are extracted based on the second branch network of the neural network model to obtain a sample color feature vector; the sample clothing category information corresponding to the sample image is obtained based on the sample depth feature vector and the sample color feature vector; the first branch network and the second branch network in the neural network model are trained according to the sample clothing category information, the labeled clothing category information corresponding to the sample image and the loss function.

[0079] like Figure 2 As shown, the first branch network includes a backbone network, a first average pooling layer, a first fully connected layer and a second fully connected layer connected in sequence; the second branch network includes a second average pooling layer and a third fully connected layer connected in sequence.

[0080] A training set is stored in the electronic device, and the training of the neural network model is completed based on the training set. Specifically, the sample image in the training set is input into the neural network model, and the sample image is feature extracted based on the first branch network of the neural network model to obtain a sample depth feature vector; the sample image is feature extracted based on the second branch network of the neural network model to obtain a sample color feature vector; the sample clothing category information corresponding to the sample image is obtained based on the sample depth feature vector and the sample color feature vector; the first branch network and the second branch network in the neural network model are trained according to the sample clothing category information, the labeled clothing category information corresponding to the sample image and the loss function. That is, the parameters of the backbone network, the first average pooling layer, the first fully connected layer and the second fully connected layer of the first branch network are trained; the parameters of the second average pooling layer and the third fully connected layer of the second branch network are trained.

[0081] The neural network model can be pre-trained based on a large number of common clothing images, and then the pre-training results of the neural network model can be fine-tuned using some chef uniforms. The loss function used to train the neural network model includes but is not limited to the Triplet loss function.

[0082] The clothing recognition process provided by the embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0083] like Figure 3 As shown, an embodiment of the present invention provides a chef uniform recognition method based on target detection and clothing retrieval, comprising the following steps:

[0084] S201: Extracting images of all top areas in the clothing image to be retrieved using a self-trained yolov5 model.

[0085] S202: For each image of the top area, a fixed-length feature vector is calculated using a deep neural network model.

[0086] S203: Using cosine distance, the image feature vector obtained in the feature vector mapping step is used to calculate the similarity between the image feature vector and each feature vector library, and the similarity is normalized to [0, 1]. The smaller the value, the higher the similarity.

[0087] S204: Sort all cosine distances of each chef uniform category from low to high, take the first k, and average them to obtain the average distance of each chef uniform category, and take the smallest one as Dmin. Compare Dmin with the cosine distance threshold. If it is less than the cosine distance threshold, the top area is a chef uniform, otherwise it is not a chef uniform.

[0088] like Figure 4As shown, the above step S202 specifically includes:

[0089] S2021: The deep neural network includes a main branch and a color branch.

[0090] S2022: The main branch is composed of a backbone network connected to an average pooling layer and two fully connected layers.

[0091] S2023: The color branch is composed of an average pooling layer and a fully connected layer.

[0092] S2024: The feature vector is composed of a depth feature obtained through the main branch and a color feature obtained through the color branch.

[0093] S2025: The deep neural network model uses the deepfashion2 model with 801,000 pieces of clothing to pre-train the retrieval model, and then uses part of the kitchen data to fine-tune it.

[0094] S2026: The deep neural network model is trained using a Triplet loss function.

[0095] S2027: When calculating the similarity, the depth feature similarity and the color feature similarity are calculated separately, and then weighted addition is performed.

[0096] like Figure 5 As shown, the above step S203 specifically includes:

[0097] S2031: The feature vector library is a library composed of feature vectors generated by passing an image of the chef uniform category marked in advance through the deep neural network model, and includes a deep feature vector library, a color feature vector library and a category file.

[0098] S2032: When calculating the similarity, the depth feature similarity and the color feature similarity are calculated separately, and then weighted addition is performed.

[0099] In the embodiment of the present invention, the yolov5 model is used for target detection. The algorithm is the latest algorithm of the yolo series, which greatly improves the accuracy of top area detection. The cosine distance is used to calculate the feature similarity, rather than the commonly used Euclidean distance. It measures the distance by calculating the cosine value of the vector directly, and mainly analyzes the vector difference from the direction of the numerical space. Since it is insensitive to the absolute value, it solves the problem of inconsistent metric standards. The deepfashion2 model with 801,000 pieces of clothing is used to pre-train the retrieval model, and then some kitchens are used to do data fine-tuning training on it, which not only ensures the richness and diversity of the data, but also ensures the specificity for the kitchen scene, so that the model is more robust. Due to the difference in the category of chef uniforms, it is largely related to its color, so a new color branch is added to avoid excessive interference from the original model by the texture details of the clothes, thereby improving the accuracy.

[0100] Embodiment 5:

[0101] Figure 6 A schematic diagram of the structure of a clothing recognition device provided by an embodiment of the present invention, the device comprises:

[0102] An acquisition module 51 is used to acquire a clothing image to be identified, and perform feature extraction on the clothing image to be identified based on a neural network model to obtain a first feature vector;

[0103] A first determination module 52 is used to determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model;

[0104] The second determination module 53 is configured to determine that the clothing attribute in the clothing image to be identified is the same as the target attribute if there is a target similarity satisfying the similarity condition in each similarity.

[0105] The acquisition module 51 is specifically used to input the collected image into the target detection model, determine the clothing recognition area in the image based on the target detection model, and use the image corresponding to the clothing recognition area as the clothing image to be recognized.

[0106] The first determination module 52 is specifically used to determine, for each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library, select a preset number of cosine distances in order from small to large, determine the average value of the selected cosine distances, and determine the corresponding similarity based on the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

[0107] The second determination module 53 is specifically used to select the maximum similarity. If the maximum similarity is greater than a preset similarity threshold, it is determined that there is a target similarity that meets the similarity condition, and then it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0108] The acquisition module 51 is specifically used to extract features of the clothing image to be identified based on the first branch network of the neural network model to obtain a first depth feature vector; and to extract features of the clothing image to be identified based on the second branch network of the neural network model to obtain a first color feature vector;

[0109] The first determination module 52 is specifically used to determine, for each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library; and perform weighted addition of the depth similarity and color similarity to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

[0110] The device also includes:

[0111] The training module 54 is used to input the sample images in the training set into the neural network model, perform feature extraction on the sample images based on the first branch network of the neural network model to obtain a sample depth feature vector; perform feature extraction on the sample images based on the second branch network of the neural network model to obtain a sample color feature vector; obtain sample clothing category information corresponding to the sample image based on the sample depth feature vector and the sample color feature vector; and train the first branch network and the second branch network in the neural network model according to the sample clothing category information, the labeled clothing category information corresponding to the sample image and the loss function.

[0112] Embodiment 6:

[0113] Based on the above embodiments, an electronic device is also provided in an embodiment of the present invention, such as Figure 7 As shown, it includes: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304;

[0114] The memory 303 stores a computer program. When the program is executed by the processor 301, the processor 301 performs the following steps:

[0115] Acquire a clothing image to be identified, and extract features of the clothing image to be identified based on a neural network model to obtain a first feature vector;

[0116] Determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model;

[0117] If there is a target similarity satisfying the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0118] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present invention. Since the principle of solving the problem by the above electronic device is similar to that of the clothing recognition method, the implementation of the above electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0119] The electronic device provided in the embodiment of the present invention may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a network side device, etc.

[0120] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0121] The communication interface 302 is used for communication between the electronic device and other devices.

[0122] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0123] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0124] In the embodiment of the present invention, when the processor executes the program stored in the memory, it obtains the clothing image to be identified, extracts features of the clothing image to be identified based on the neural network model, and obtains a first feature vector; determines the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image of the same target attribute based on the neural network model; if there is a target similarity that meets the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0125] In the embodiment of the present invention, the feature extraction of the clothing image to be identified is performed based on the neural network model to obtain the first feature vector, and the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library is determined; if there is a target similarity that satisfies the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute. For the chef uniform recognition scenario, the second feature vector in each feature vector library is obtained by extracting features from the target clothing images of various types of chef uniforms based on the neural network model. The embodiment of the present invention performs clothing recognition based on the similarity between the first feature vector and the second feature vector in each pre-saved feature vector library. Each time a new chef uniform is added, it only needs to be added to the feature vector library, and there is no need to retrain the model. Therefore, there will be no problem of sample imbalance. Therefore, the accuracy of clothing recognition in the embodiment of the present invention is relatively high.

[0126] Embodiment 7:

[0127] On the basis of the above embodiments, an embodiment of the present invention further provides a computer storage readable storage medium, wherein the computer readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:

[0128] Acquire a clothing image to be identified, and extract features of the clothing image to be identified based on a neural network model to obtain a first feature vector;

[0129] Determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model;

[0130] If there is a target similarity satisfying the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

[0131] Based on the same inventive concept, a computer-readable storage medium is also provided in an embodiment of the present invention. Since the principle of solving the problem when the processor executes the computer program stored on the above-mentioned computer-readable storage medium is similar to that of the clothing recognition method, the implementation of the processor when executing the computer program stored on the above-mentioned computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.

[0132] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in the electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc., optical storage such as CD, DVD, BD, HVD, etc., and semiconductor storage such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD), etc.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0136] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A clothing recognition method, characterized in that: The method comprises: Acquire a clothing image to be identified, and extract features of the clothing image to be identified based on a neural network model to obtain a first feature vector; Determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model; If there is a target similarity satisfying the similarity condition in each similarity, it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute; Determining the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library includes: For each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library is determined respectively, and a preset number of cosine distances are selected in order of cosine distances from small to large, and the average value of the selected cosine distances is determined, and the corresponding similarity is determined according to the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

2. The method according to claim 1, characterized in that The obtaining of the clothing image to be identified comprises: The collected image is input into a target detection model, a clothing recognition area in the image is determined based on the target detection model, and an image corresponding to the clothing recognition area is used as a clothing image to be recognized.

3. The method according to claim 1, characterized in that If there is a target similarity satisfying the similarity condition in each similarity, determining that the clothing attribute in the clothing image to be identified is the same as the target attribute includes: The maximum similarity is selected. If the maximum similarity is greater than a preset similarity threshold, it is determined that there is a target similarity that satisfies the similarity condition, and it is determined that the clothing attribute in the clothing image to be identified is the same as the target attribute.

4. The method according to claim 1, characterized in that The feature extraction of the clothing image to be identified based on the neural network model to obtain the first feature vector includes: The first branch network of the neural network model extracts features of the clothing image to be identified to obtain a first depth feature vector; the second branch network of the neural network model extracts features of the clothing image to be identified to obtain a first color feature vector; Determining the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library includes: For each feature vector library, the depth similarity between the first depth feature vector and the second depth feature vector in the feature vector library is determined, and the color similarity between the first color feature vector and the second color feature vector in the feature vector library is determined; the depth similarity and the color similarity are weightedly added to obtain the similarity between the first feature vector and the second feature vector in the feature vector library.

5. The method according to claim 4, characterized in that The training process of the neural network model includes: The sample images in the training set are input into the neural network model, and the features of the sample images are extracted based on the first branch network of the neural network model to obtain a sample depth feature vector; the features of the sample images are extracted based on the second branch network of the neural network model to obtain a sample color feature vector; the sample clothing category information corresponding to the sample image is obtained based on the sample depth feature vector and the sample color feature vector; the first branch network and the second branch network in the neural network model are trained according to the sample clothing category information, the labeled clothing category information corresponding to the sample image and the loss function.

6. The method according to claim 5, characterized in that The first branch network includes a backbone network, a first average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence; the second branch network includes a second average pooling layer and a third fully connected layer connected in sequence.

7. A clothing recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a clothing image to be identified, performing feature extraction on the clothing image to be identified based on a neural network model, and obtaining a first feature vector; A first determination module is used to determine the similarity between the first feature vector and the second feature vector in each pre-stored feature vector library; wherein the second feature vector in each feature vector library is obtained by extracting features of each type of target clothing image with the same target attribute based on the neural network model; A second determination module is used to determine that the clothing attribute in the clothing image to be identified is the same as the target attribute if there is a target similarity satisfying the similarity condition in each similarity; The first determination module is specifically used to determine, for each feature vector library, the cosine distance between the first feature vector and each second feature vector in the feature vector library, select a preset number of cosine distances in order from small to large, determine the average value of the selected cosine distances, and determine the corresponding similarity based on the average value of the cosine distances; wherein, the smaller the average value of the cosine distances, the greater the similarity.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

Images