Model training method, clothing recognition processing method, device and medium

Abstract

The present invention discloses a model training method, a clothing recognition processing method, a device, and a medium, belonging to the field of clothing recognition. During model training, sample images containing different character information are input, and then the clothing region images are classified according to the characteristics of the clothing region images in the sample images to obtain classification results. Clothing region images with the same character in any classification in the classification results are grouped as a cluster to obtain a clustering result. Then, it is determined whether the character information of the attached region images in each cluster in the clustering result is the same. If the character information is the same, training is completed; if it is not the same, the character information is re-acquired for training adjustment. Because the classification results are clustered according to the character information during model training, the character information in the final clusters is the same, which solves the problem of inaccurate classification in existing clothing clustering models. When the model is used to identify clothing information, it can be guaranteed that the clothing information obtained by the model is the same as the actual information.

Description

Technical Field

[0001] The present invention relates to the field of clothing recognition, and in particular to a model training method, a clothing recognition processing method, a device and a medium. Background Art

[0002] Films and TV series often feature a large number of attractive clothing combinations, providing viewers with a better viewing experience. Some of these combinations may trigger viewers' desire to learn more about the specific clothing brands. However, due to their limited knowledge of clothing brands and product appearance, manual identification is difficult.

[0003] This requirement can be addressed through AI algorithms. The current approach involves continuously collecting clothing images from various brands, extracting clothing feature sets, and then clustering them based on their similarity to create a clothing clustering model. Each category in the clothing clustering model corresponds to a piece of clothing information. To learn about a particular piece of clothing information, simply input the item into the clothing clustering model to identify its category, and then use the clothing information corresponding to that category as the clothing information for that item.

[0004] Existing clustering methods generate clothing clustering models, resulting in very high similarity between clothing items within each category. However, in reality, different people may wear very similar clothing items. Existing clothing clustering models ultimately classify these highly similar items as the same category, effectively including two categories. When a user uses this clothing clustering model, the clothing information obtained for one category based on the clothing clustering model will inevitably differ from the actual clothing information. Therefore, the clothing clustering model generated by existing clustering methods is inaccurate, and the clothing information obtained based on this clothing clustering model does not match the actual clothing information. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides a method, device and storage medium for clustering clothing in a video to solve the problem that the clothing clustering model obtained by the existing clustering method is inaccurately classified and the clothing information obtained according to the clothing clustering model is inconsistent with the actual clothing information.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] Firstly,

[0008] A model training method includes the following steps:

[0009] Inputting a sample image set, wherein the sample image set includes sample images with different character information;

[0010] Acquiring a clothing region image of the sample image; extracting features of the clothing region image, and classifying the clothing region image according to the features to obtain a classification result;

[0011] Obtaining character information corresponding to the clothing region image, and grouping clothing region images with the same character information in any category of the classification result as a cluster to obtain a clustering result of the clothing image region;

[0012] determining, based on the role labeling information of the sample image, whether the role information of the clothing region images in any cluster in the clustering result is the same;

[0013] If so, the model training is completed; if not, the step of obtaining the character information corresponding to the clothing area image and its subsequent steps are re-executed.

[0014] Furthermore, the obtaining of character information corresponding to the clothing area image includes:

[0015] Acquire a human body image region in the sample image and character information corresponding to the human body image region;

[0016] Calculating the degree of overlap between the clothing image area and the human body image area;

[0017] When the degree of overlap is greater than a preset degree of overlap, the character information corresponding to the human body image area is used as the character information of the clothing area image.

[0018] Furthermore, obtaining the character information corresponding to the human body image area includes:

[0019] Extracting human body feature information of the human body image area;

[0020] Calculate the similarity between human feature information in each human image area;

[0021] The human image regions with similarity greater than or equal to a preset similarity are regarded as the same character information.

[0022] Furthermore, the step of re-executing the step of obtaining the character information corresponding to the clothing area image and subsequent steps include:

[0023] The preset similarity value is adjusted, and the step of obtaining the character information corresponding to the clothing area image and subsequent steps are re-executed.

[0024] Furthermore, extracting features of the clothing region image and classifying the clothing region image according to the features to obtain a classification result includes:

[0025] Taking any clothing region image as a vertex, the similarity of features of different clothing region images is used to determine whether any vertex is connected to other vertices, so as to construct an association graph containing all vertices and their connection relationships;

[0026] Inputting the association graph into a trained first graph convolutional neural network to obtain first confidences of all vertices in the association graph, wherein the first graph convolutional neural network is obtained by training a training sample including a first training association graph using any graph convolutional neural network model and then modifying parameters of the graph convolutional neural network model, wherein any two vertices in the first training association graph are not connected to the same vertex;

[0027] Acquire all first target vertices whose first confidence is greater than a first preset confidence;

[0028] Constructing a candidate set subgraph of any first target vertex, wherein the candidate set subgraph of any first target vertex includes the any first target vertex and first target vertices adjacent to the first target vertex;

[0029] Inputting any candidate set subgraph into a trained second graph convolutional neural network to obtain a second confidence of a first target vertex in any candidate set subgraph, wherein the second graph convolutional neural network is obtained by training any graph convolutional neural network model on a training sample including only the second training association graph, and then modifying the parameters of the graph convolutional neural network model, wherein in any second training association graph, there is a vertex connected to all other vertices;

[0030] Acquire all second target vertices whose second confidence is greater than a second preset confidence;

[0031] Any second target vertex and its adjacent second target vertices are taken as the same cluster to obtain a first classification result.

[0032] Furthermore, the extracting features of the clothing region image and classifying the clothing region image according to the features to obtain a classification result further includes:

[0033] Perform at least one iterative merging on each cluster in the first classification result to obtain a second classification result; wherein any one iterative merging includes:

[0034] Obtaining the cluster center of each cluster in the first clustering result, wherein the first clustering result is the clustering result before any iterative merging;

[0035] The distance between any two cluster centers is calculated. If the distance is not greater than a preset threshold distance, the two clusters corresponding to the any two cluster centers are merged.

[0036] Furthermore, the sample image set is obtained by extracting video frames from a video.

[0037] Secondly,

[0038] A model training device, comprising:

[0039] A sample set input module, configured to input a sample image set, wherein the sample image set includes sample images with different character information;

[0040] a clothing region image classification module, configured to obtain a clothing region image of the sample image; extract features of the clothing region image; and classify the clothing region image according to the features to obtain a classification result;

[0041] a classification result clustering module for obtaining character information corresponding to the clothing region image, and grouping clothing region images with the same character information in any category of the classification result as a cluster to obtain a clustering result of the clothing image region;

[0042] A clustering result verification module, configured to determine whether the character information of the clothing region images in any cluster in the clustering result is the same based on the character labeling information of the sample image;

[0043] The adjustment determination module is used to determine whether the model training is completed if yes; if not, re-execute the step of obtaining the character information corresponding to the clothing area image and its subsequent steps.

[0044] Thirdly,

[0045] A clothing recognition processing method is applied to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by any one of the methods described in the first aspect of the technical solution, and the method comprises:

[0046] When receiving a clothing information query request sent by a clothing information query request sending end, inputting the clothing image carried in the clothing information query request into the model to determine a corresponding clothing image set through the model, obtaining a reference image corresponding to the determined clothing image set, and obtaining clothing information corresponding to the reference image;

[0047] The clothing information corresponding to the reference image is sent back to the clothing information query request sending end as clothing identification information, so that the clothing identification information is displayed on the clothing information query request sending end.

[0048] Fourthly,

[0049] A clothing recognition processing device, the device being applied to a clothing information query request receiving end, the clothing information query request receiving end being deployed with a model trained by any one of the methods described in the first aspect of the technical solution, the device comprising:

[0050] a reference image clothing information obtaining module, configured to, upon receiving a clothing information query request sent by a clothing information query request sending terminal, input a clothing image carried in the clothing information query request into the model, determine a corresponding clothing image set through the model, obtain a reference image corresponding to the determined clothing image set, and obtain clothing information corresponding to the reference image;

[0051] The clothing identification information sending module is used to send the clothing information corresponding to the reference image as clothing identification information back to the clothing information query request sending end, so that the clothing identification information is displayed on the clothing information query request sending end.

[0052] Fifthly,

[0053] A clothing identification and processing method is applied to a clothing information query request sending end, and the method comprises the following steps:

[0054] In response to the interaction instruction, obtaining a clothing image obtained according to the interaction instruction;

[0055] generating a clothing information query request based on the clothing image;

[0056] Sending the clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by any one of the methods described in the first aspect of the technical solution, and the clothing information query request receiving terminal is used to implement the clothing recognition processing method described in the third aspect of the technical solution;

[0057] Receive and display the clothing identification information returned by the clothing information query request receiving end.

[0058] Furthermore, in response to the interaction instruction, obtaining the clothing image obtained according to the interaction instruction includes:

[0059] In response to the interactive instruction, a screenshot of the frame of the currently playing video is taken, and the clothing image is obtained according to the screenshot.

[0060] Sixth aspect,

[0061] A clothing identification and processing device, the device being applied to a clothing information query request sending end, the device comprising:

[0062] A clothing image acquisition module, configured to respond to an interaction instruction and acquire a clothing image obtained according to the interaction instruction;

[0063] a request generation module, configured to generate a clothing information query request based on the clothing image;

[0064] a request sending module, configured to send the clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by any one of the methods described in the first aspect of the technical solution, and the clothing information query request receiving terminal is configured to implement the clothing recognition processing method described in the third aspect of the technical solution;

[0065] The clothing identification information receiving and displaying module is used to receive and display the clothing identification information returned by the clothing information query request receiving end.

[0066] Seventh aspect,

[0067] A clothing information query request receiving terminal, comprising:

[0068] One or more memories storing executable programs;

[0069] One or more processors are used to execute the executable program in the memory to implement the steps of the method described in any one of the technical solutions of the first aspect, and / or to implement the steps of the method described in the technical solution of the third aspect.

[0070] Eighth aspect,

[0071] A clothing information query request sending terminal, comprising:

[0072] One or more memories storing executable programs;

[0073] One or more processors, used to execute the executable program in the memory to implement the steps of the method described in any one of the technical solutions of the fifth aspect.

[0074] Ninth aspect,

[0075] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method described in any one of the technical solutions of the first aspect and / or implements the steps of the method described in the technical solution of the third aspect.

[0076] Tenth aspect,

[0077] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the technical solutions of the fifth aspect are performed.

[0078] Beneficial effects:

[0079] The technical solution of the present application provides a model training method, clothing recognition processing method, device, and medium. During model training, sample images including different character information are input, and then the clothing area images are classified according to the characteristics of the clothing area images in the sample images to obtain classification results. The clothing area images with the same character in any classification in the classification results are grouped as a cluster to obtain a clustering result. Then, it is determined whether the character information of the attached area images in each cluster in the clustering result is the same. If they are the same, the training is completed. If they are not the same, the character information is re-acquired for training adjustment. Because the classification results are clustered according to the character information during model training, the character information in the final clusters is the same, which solves the problem of inaccurate classification of existing clothing clustering models. In this way, when the model is used to identify clothing information, it can be guaranteed that the clothing information obtained by the model is the same as the actual information. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0081] Figure 1 This is a flow chart of a model training method provided by an embodiment of the present invention;

[0082] Figure 2 This is a flow chart of a specific model training method provided by an embodiment of the present invention;

[0083] Figure 3 This is a schematic diagram of the structure of a model training device provided by an embodiment of the present invention;

[0084] Figure 4 This is a flow chart of a clothing identification processing method applied to a clothing information query request receiving end provided by an embodiment of the present invention;

[0085] Figure 5 This is a schematic structural diagram of a clothing identification processing device applied to a clothing information query request receiving end provided by an embodiment of the present invention;

[0086] Figure 6 This is a flow chart of a clothing identification processing method applied to a clothing information query request sending end provided by an embodiment of the present invention;

[0087] Figure 7The present invention provides a schematic structural diagram of a clothing identification processing device applied to a clothing information query request sending end. DETAILED DESCRIPTION

[0088] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of the present invention are described in detail below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only some of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other implementation methods obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0089] The first embodiment, as Figure 1 As shown, the present invention provides a model training method, comprising the following steps:

[0090] S11: input a sample image set, where the sample image set includes sample images with different character information;

[0091] S12: Obtaining a clothing region image of the sample image; extracting features of the clothing region image, and classifying the clothing region image according to the features to obtain a classification result;

[0092] S13: Obtaining character information corresponding to the clothing region image, and taking clothing region images with the same character information in any category of the classification result as a cluster, thereby obtaining a clustering result of the clothing image region;

[0093] S14: judging whether the character information of the clothing region images in any cluster in the clustering result is the same based on the character labeling information of the sample image;

[0094] If yes, go to step S15; if no, return to step S13.

[0095] S15: Model training is performed after the judgment is completed;

[0096] An embodiment of the present invention provides a model training method. During model training, sample images containing different character information are input. The clothing region images in the sample images are then classified based on their characteristics to obtain classification results. Clothing region images with the same character in any classification in the classification results are grouped as a cluster to obtain clustering results. A determination is then made as to whether the character information of the attached region images in each cluster in the clustering results is the same. If they are the same, training is completed; if they are not, the character information is reacquired and training adjustments are made. Because the classification results are clustered based on the character information during model training, the character information in the resulting clusters is the same, thus resolving the inaccurate classification problem of existing clothing clustering models. Thus, when the model is used to identify clothing information, the clothing information obtained by the model is guaranteed to be consistent with the actual information.

[0097] In the second embodiment, in order to further supplement and illustrate the first embodiment, this embodiment provides a specific model training method, such as Figure 2 As shown, the following steps are included:

[0098] S21: Input a sample image set, which includes sample images with different character information; the sample image set is obtained by extracting video frames from a video. In actual use, the sample image set comes from a video (such as a movie or a TV series), and the video is divided into frames to obtain the sample images. Using a video frame as a sample for training enables users to more accurately identify the clothing information of a particular item of clothing when watching the video and want to identify it.

[0099] S22: Obtaining a clothing region image of the sample image, extracting features of the clothing region image, and classifying the clothing region image according to the features to obtain a classification result;

[0100] The steps of obtaining a clothing region image of a sample image and extracting features of the clothing region image include the following steps:

[0101] Clothing region position detection is performed on each video frame to obtain a clothing region image for each piece of clothing. A convolutional neural network is then used to extract features from each clothing region image to obtain clothing features for each piece of clothing. For example, clothing region position detection is performed on each frame of the video to obtain a clothing frame. The clothing region image data is then captured and subjected to convolutional neural network (CNN) feature extraction to obtain clothing features. Using CNN for feature extraction is effective and accurate.

[0102] As an optional implementation of the embodiment of the present invention, the clothing region image is classified according to the features to obtain a classification result, and the clothing region image can be classified using a graph convolutional neural network (GCN) clustering method;

[0103] The specific implementation steps of GCN are as follows: any clothing region image is regarded as a vertex, and the similarity of the features of different clothing region images is used to determine whether any vertex is connected to other vertices, so as to construct an association graph containing all vertices and their connection relationships;

[0104] Input the association graph into a trained first graph convolutional neural network to obtain first confidences of all vertices in the association graph, wherein the first graph convolutional neural network is obtained by training a training sample including the first training association graph using any graph convolutional neural network model and then modifying the parameters of the graph convolutional neural network model, wherein any two vertices in the first training association graph are not connected to the same vertex;

[0105] Acquire all first target vertices whose first confidence is greater than a first preset confidence;

[0106] Constructing a candidate set subgraph of any first target vertex, where the candidate set subgraph of any first target vertex includes any first target vertex and first target vertices adjacent to the first target vertex;

[0107] Input any candidate set subgraph into a trained second graph convolutional neural network to obtain a second confidence of the first target vertex in any candidate set subgraph, wherein the second graph convolutional neural network is obtained by training any graph convolutional neural network model on training samples including only the second training association graph, and then modifying the parameters of the graph convolutional neural network model. In any second training association graph, there is a vertex connected to all other vertices.

[0108] Acquire all second target vertices whose second confidence is greater than a second preset confidence;

[0109] Any second target vertex and its adjacent second target vertices are taken as the same cluster to obtain a first classification result.

[0110] It should be noted that in the association graph, for vertices with high confidence, their adjacent vertices often belong to the same class, while vertices with low confidence are usually adjacent to vertices of other classes. Therefore, all second target vertices that are finally connected to the same second target vertex belong to the same cluster cluster. In addition, the method based on graph convolutional neural network adopted in this embodiment is clustered, but in actual clustering, any method for clustering graphs can be used, such as GNN (Graph Neural Network), DeepWalk, node2vec, etc., which is not limited in this application. It should be noted that compared with clustering methods such as GNN, DeepWalk, node2vec, etc., GCN clustering training is simple and the results obtained are accurate.

[0111] It should be noted that when the first classification result is obtained, the first classification result can be used as the target clothing set. Due to the high accuracy of GCN, the similarity of clothing in each cluster is very high. However, in the actual process, the similarity of the same clothing at different angles is not high, so it will be divided into two categories in the first classification result. Therefore, there are many clusters in the first classification result. If clustering is performed according to the role information at this time, there will be more clusters in the final clustering result, which is not in line with practical applications. Moreover, since clustering is performed according to the role information in each category, calculations need to be performed for each category in the first classification result. The amount of calculation is large, the calculation cost is high, and the clustering effect is low. Therefore, after obtaining the first classification result, the embodiment of the present invention uses a hierarchical clustering method to cluster the first classification result again to reduce the clusters in the final clustering result and reduce the amount of calculation when clustering the role information.

[0112] The hierarchical clustering method includes performing at least one iterative merging of each cluster in the first classification result to obtain a second classification result; wherein any one iterative merging includes:

[0113] Obtaining the cluster center of each cluster in the first clustering result, wherein the first clustering result is the clustering result before any iterative merging;

[0114] The distance between any two cluster centers is calculated. If the distance is not greater than the preset threshold distance, the two clusters corresponding to the two cluster centers are merged.

[0115] For example, the weighted average of the features of each clothing item in the same category in the first classification result is taken; and the average distance between any two clusters is calculated using the following method:

[0116]

[0117] Where d() is a function for finding the average distance, u and v are any two clusters, u[i] is any element in cluster u, v[j] is any element in cluster v, |u| is the length of cluster u, and |v| is the length of cluster v. Clusters with an average distance less than the threshold are merged into one cluster in the second classification result. It should be noted that d(u[i], v[j]) is the average distance between any element in cluster u and any element in cluster v. Dividing it by (|u| * |v|) is for normalization to eliminate the influence of the cluster length itself. Preferably, a threshold of 0.5 is used in this embodiment of the present invention. Hierarchical clustering can obtain multi-level clustering structures of different granularities by setting different relevant parameter values. In terms of cluster shape, hierarchical clustering is applicable to clusters of any shape and is insensitive to the input order of samples. Therefore, after hierarchical clustering, the same clothing items in the first classification result can be further clustered, resulting in a second classification result with fewer clusters that is more consistent with the actual classification.

[0118] S23: Obtaining character information corresponding to the clothing region image, and taking clothing region images with the same character information in any category of the classification result as a cluster, thereby obtaining a clustering result of the clothing image region;

[0119] The character information corresponding to the clothing region image is obtained, and the clothing region images with the same character information in any category of the classification result are taken as a cluster, including:

[0120] Obtain the human body image area in the sample image and the character information corresponding to the human body image area; specifically, detect the human body area in the sample image to obtain the human body image area, extract the human body feature information of the human body image area, the human body feature information includes but is not limited to: face, eyes, ears, nose, mouth, hands, body shape and leg shape; calculate the similarity between the human body feature information in each human body image area; and regard the human body image areas with similarity greater than or equal to the preset similarity as the same character information. Through the above steps, the human body image area in the sample image can be divided into different character information according to the human body feature information. It should be noted that in this application, it is not necessary to know who the person in the human body image area is, and it is only necessary to know whether the people in the two human body image areas are the same person. Therefore, the character information does not need to know the role or name played in the video.

[0121] Calculate the degree of overlap between the clothing image area and the human body image area; the calculation formula for the overlap is: overlap = overlapping area / (human body image area + clothing image area - overlapping area);

[0122] When the degree of overlap is greater than a preset degree of overlap, the character information corresponding to the human body image region is used as the character information of the clothing region image.

[0123] S24: judging whether the character information of the clothing region images in any cluster in the clustering result is the same based on the character labeling information of the sample image;

[0124] If so, execute step S25; if not, adjust the preset similarity value and return to step S23.

[0125] S25: If the judgment is completed, the model training is carried out.

[0126] A specific model training method provided by an embodiment of the present invention comprises: dividing a video into frames to obtain a sample image set; extracting features of clothing region images in the sample images using a convolutional neural network; clustering the features using a graph convolutional neural network to obtain a first classification result; clustering the first classification result using hierarchical clustering to obtain a second classification result; and then splitting any category in the second classification result based on character information to obtain a final clustering result. Finally, the character information of the clothing region images in any cluster cluster in the clustering result is verified based on character annotation information to determine whether they are the same. If they are the same, model training is completed; if they are different, a preset similarity value is adjusted, the character information of the sample images is reacquired, and training is repeated. The model training method provided by an embodiment of the present invention clusters the classification results based on character information and then verifies the clustering results using character annotation information. This ensures that the character information in each cluster output by the trained model is the same, preventing the appearance of clothing worn by different characters in the same cluster. Thus, the clothing information output by the trained model is consistent with the actual clothing information, greatly improving the accuracy of model clustering.

[0127] In a third embodiment, the present invention provides a model training device, such as Figure 3 Shown, including:

[0128] The sample set input module 31 is used to input a sample image set, which includes sample images with different character information. As an optional implementation of the embodiment of the present invention, the sample image set is obtained by extracting video frames of a video.

[0129] The clothing region image classification module 32 is used to obtain the clothing region image of the sample image; extract the features of the clothing region image, and classify the clothing region image according to the features to obtain a classification result;

[0130] Specifically, the clothing area image classification module 32 takes any clothing area image as a vertex, and determines whether any vertex is connected to other vertices based on the similarity of features of different clothing area images, so as to construct an association graph containing all vertices and their connection relationships; the association graph is input into the trained first graph convolutional neural network to obtain the first confidence of all vertices in the association graph. The first graph convolutional neural network is: using any graph convolutional neural network model, training samples containing the first training association graph, and then modifying the parameters of the graph convolutional neural network model to obtain that there are any two vertices in the first training association graph that are not connected to the same vertex; obtaining all first target vertices whose first confidence is greater than the first preset confidence; constructing the first target vertex A candidate set subgraph, the candidate set subgraph of any first target vertex includes any first target vertex and the first target vertex adjacent to it; any candidate set subgraph is input into the trained second graph convolutional neural network to obtain the second confidence of the first target vertex in any candidate set subgraph, and the second graph convolutional neural network is: using any graph convolutional neural network model, for training samples that only include the second training association graph, the parameters of the graph convolutional neural network model are modified after training, and there is a vertex in any second training association graph connected to all other vertices; all second target vertices whose second confidence is greater than the second preset confidence are obtained; any second target vertex and the second target vertex adjacent to it are taken as the same cluster cluster to obtain the first classification result.

[0131] Furthermore, the clothing region image classification module 32 performs at least one iterative merging on each cluster in the first classification result to obtain a second classification result; wherein any one iterative merging includes:

[0132] Obtaining the cluster center of each cluster in the first clustering result, wherein the first clustering result is the clustering result before any iterative merging;

[0133] The distance between any two cluster centers is calculated. If the distance is not greater than the preset threshold distance, the two clusters corresponding to the two cluster centers are merged.

[0134] For example, hierarchical clustering is used to cluster the first classification results to obtain at least one target clothing set. The features of the clothing image area in the first-level classification results are weighted averaged; and the average distance between any two clusters is calculated using the following method:

[0135]

[0136] Where d() is a function for finding the average distance, u and v are any two clusters, u[i] is any element in cluster u, v[j] is any element in cluster v, |u| is the length of cluster u, and |v| is the length of cluster v. Clusters with an average distance less than the threshold are merged into one cluster for the second classification result.

[0137] The classification result clustering module 33 is configured to obtain character information corresponding to the clothing region images and group clothing region images with the same character information within any category of the classification results into a cluster, thereby obtaining a clustering result for the clothing image regions. Specifically, the classification result clustering module 33 obtains the human image region in the sample image and the character information corresponding to the human image region; calculates the degree of overlap between the clothing image region and the human image region; and when the overlap is greater than a preset degree, the character information corresponding to the human image region is used as the character information for the clothing region images.

[0138] Among them, obtaining the role information corresponding to the human body image area includes: extracting human body feature information of the human body image area; calculating the similarity between the human body feature information in each human body image area; and treating the human body image areas with similarity greater than or equal to a preset similarity as the same role information.

[0139] The clustering result verification module 34 is used to determine whether the role information of the clothing area images in any cluster in the clustering result is the same based on the role labeling information of the sample image.

[0140] Adjustment and determination module 35 is configured to determine if the model is complete and then perform model training; if not, re-execute the step of obtaining character information corresponding to the clothing region image and its subsequent steps. If the character information of the clothing region images in any cluster in the clustering result is different, adjustment and determination module 35 adjusts the preset similarity value, and classification result clustering module 33 re-executes the step of obtaining character information corresponding to the clothing region image and its subsequent steps to obtain the clustering result, which is then re-verified by clustering result verification module 34.

[0141] An embodiment of the present invention provides a model training device, comprising a sample set input module that inputs a sample image set, the sample image set including sample images with different character information; a clothing region image classification module for obtaining clothing region images of the sample images; extracting features of the clothing region images and classifying the clothing region images according to the features to obtain classification results; a classification result clustering module for obtaining character information corresponding to the clothing region images, and grouping clothing region images with the same character information in any classification of the classification results as a cluster to obtain clustering results of the clothing image regions; a clustering result verification module for determining whether the character information of the clothing region images in any clustering cluster in the clustering results is the same based on the character annotation information of the sample images; and an adjustment determination module for determining whether model training is completed if so; and re-executing the step of obtaining the character information corresponding to the clothing region images and subsequent steps if not. The model training device of the present application can verify the character information output by the model based on the character annotation information, ensuring that each clothing region image in any clustering cluster in the clustering results output by the model corresponds to the same character information, thereby improving the accuracy of the model clustering and ensuring that the clothing information output by the trained model is the same as the actual clothing information.

[0142] In a fourth embodiment, the present invention provides a clothing recognition processing method, which is applied to a clothing information query request receiving terminal, which is deployed with a model trained by the method provided by the first embodiment or the second embodiment, such as Figure 4 As shown, the method includes:

[0143] S41: Upon receiving a clothing information query request from a clothing information query request sender, the model is fed with a clothing image included in the clothing information query request. The model then determines a corresponding clothing image set, obtains a reference image corresponding to the determined clothing image set, and obtains clothing information corresponding to the reference image. Exemplarily, the reference image is a frontal image of a piece of clothing output by the model, and the clothing information obtained by the model by recognizing the clothing image included in the clothing information query request is the same as the clothing information in the frontal image.

[0144] S42: Sending the clothing information corresponding to the reference image as clothing identification information back to the clothing information query request sending end, so that the clothing identification information is displayed on the clothing information query request sending end.

[0145] Preferably, the clothing information query request sending end is a playback terminal, and the clothing information query request receiving end is a server.

[0146] An embodiment of the present invention provides a clothing recognition processing method. When a clothing information query request is received, a clothing information query request receiving end inputs the clothing image carried in the clothing information query request into a model to obtain a corresponding clothing image set. The receiving end then determines a reference image based on the clothing image set, ultimately obtaining clothing information corresponding to the reference image. The clothing information is then sent to a clothing information query request sending end, so that the clothing identification information is displayed on the clothing information query request sending end. The embodiment of the present invention provides a clothing recognition processing method that can identify the clothing information of the clothing image carried in the clothing information query request at the clothing information query request receiving end and send it to the clothing information query request sending end, thereby facilitating the user to obtain the clothing information of the clothing image.

[0147] In a fifth embodiment, the present invention provides a clothing recognition processing device, which is applied to a clothing information query request receiving end, and the clothing information query request receiving end is deployed with a model trained by the method provided in the first embodiment or the second embodiment, such as Figure 5 As shown, the device includes:

[0148] The reference image clothing information obtaining module 51 is configured to, upon receiving a clothing information query request sent by the clothing information query request sending terminal, input the clothing image carried in the clothing information query request into the model, determine a corresponding clothing image set through the model, obtain a reference image corresponding to the determined clothing image set, and obtain clothing information corresponding to the reference image;

[0149] The clothing identification information sending module 52 is configured to send the clothing information corresponding to the reference image as clothing identification information back to the clothing information query request sending end, so that the clothing identification information is displayed on the clothing information query request sending end.

[0150] Preferably, the clothing information query request sending end is a playback terminal, and the clothing information query request receiving end is a server.

[0151] In the clothing recognition and processing device provided by an embodiment of the present invention, the reference image clothing information obtaining module can obtain the clothing information of the clothing image carried in the clothing information query request after receiving the clothing information query request, and send it to the clothing information query request sending end through the clothing recognition information sending module, so as to facilitate the user to obtain the clothing information of the clothing image.

[0152] In a sixth embodiment, the present invention provides a clothing identification processing method, which is applied to a clothing information query request sending end, such as Figure 6 As shown, the method includes the following steps:

[0153] S61: Responding to the interaction instruction, obtaining a clothing image obtained according to the interaction instruction; optionally, responding to the interaction instruction, taking a screenshot of the frame of the currently playing video, and obtaining the clothing image based on the screenshot. Exemplarily, the entire frame is screenshoted and then the clothing image is extracted from the entire frame, or the user directly selects the clothing image when taking the screenshot.

[0154] S62: Generate a clothing information query request based on the clothing image.

[0155] S63: Sending the clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by the method provided by the first embodiment or the second embodiment, and the clothing information query request receiving terminal is used to implement the clothing recognition processing method provided by the fourth embodiment;

[0156] S64: Receive and display the clothing identification information returned by the clothing information query request receiving end.

[0157] In an embodiment of the present invention, a clothing identification processing method is provided. A clothing information query request sending end responds to an interactive instruction, obtains a clothing image, and sends the clothing image and clothing information query request to a clothing information query request receiving end. The clothing information query request receiving end then receives the clothing information and sends it to the clothing information query request sending end. Finally, the clothing information query request sending end receives and displays the clothing identification information returned by the clothing information query request receiving end. This clothing identification processing method allows users to obtain clothing information associated with clothing images, improving their user experience.

[0158] In a seventh embodiment, the present invention provides a clothing identification processing device, which is applied to a clothing information query request sending end, such as Figure 7 As shown, the device includes:

[0159] The clothing image acquisition module 71 is used to respond to the interaction instruction and acquire the clothing image obtained according to the interaction instruction; specifically, the clothing image acquisition module 71 responds to the interaction instruction to take a screenshot of the frame of the currently playing video and obtain the clothing image according to the screenshot.

[0160] a request generation module 72 for generating a clothing information query request based on the clothing image;

[0161] a request sending module 73 for sending a clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained using the method provided by the first embodiment or the second embodiment, and the clothing information query request receiving terminal is used to implement the clothing recognition processing method provided by the fourth embodiment;

[0162] The clothing identification information receiving and displaying module 74 is used to receive and display the clothing identification information returned by the clothing information query request receiving end.

[0163] The clothing identification and processing device provided in an embodiment of the present invention is applied to a clothing information query request sending end. A clothing image acquisition module responds to an interaction instruction and acquires a clothing image obtained according to the interaction instruction. A request generation module generates a clothing information query request based on the clothing image. A request transmission module transmits the clothing information query request to a clothing information query request receiving end. A clothing identification information receiving and displaying module receives and displays the clothing identification information returned by the clothing information query request receiving end. This device allows users to obtain clothing information associated with clothing images, providing convenient user experience.

[0164] In an eighth embodiment, the present invention provides a clothing information query request receiving terminal, comprising:

[0165] One or more memories storing executable programs;

[0166] One or more processors are used to execute the executable program in the memory to implement the steps of the method provided by the first embodiment or the second embodiment, and / or implement the steps of the method provided by the fourth embodiment.

[0167] The clothing information query request receiving end provided by the embodiment of the present invention has a memory storing executable instructions. After the processor executes the executable instructions, it can identify the clothing information of the clothing image and send it to the clothing information query request sending end, so that the user can obtain the clothing information of the clothing image and improve user experience.

[0168] In a ninth embodiment, the present invention provides a clothing information query request sending terminal, comprising:

[0169] One or more memories storing executable programs;

[0170] One or more processors are configured to execute the executable program in the memory to implement the steps of the method provided in the sixth embodiment of claim 1.

[0171] The clothing information query request sending end provided by the embodiment of the present invention has a memory storing executable instructions. After the processor executes the executable instructions, it can send a clothing image to the clothing information query request receiving end. After receiving the clothing information sent by the clothing information query request receiving end, the clothing information is displayed on the clothing information query request sending end, so that the user can obtain the clothing information of the clothing image and improve user experience.

[0172] In a tenth embodiment, the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the steps of the method provided in the first embodiment or the second embodiment, and / or implements the steps of the method provided in the fourth embodiment of claim 1.

[0173] A readable storage medium provided by an embodiment of the present invention stores a computer program. When executed by a processor, the computer program achieves the following effects: upon receiving a clothing information query request, a clothing information query request receiving end inputs the clothing image carried in the clothing information query request into a model to obtain a corresponding clothing image set, then determines a reference image based on the clothing image set, ultimately obtaining clothing information corresponding to the reference image, and then transmits the clothing information to a clothing information query request transmitting end, so that clothing identification information is displayed on the clothing information query request transmitting end. An embodiment of the present invention provides a clothing identification processing method that can identify clothing information of a clothing image carried in a clothing information query request at a clothing information query request receiving end and transmit the information to the clothing information query request transmitting end, thereby facilitating a user to obtain clothing information associated with the clothing image.

[0174] In an eleventh embodiment, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the sixth embodiment of claim 1 are implemented.

[0175] A readable storage medium provided by an embodiment of the present invention stores a computer program. When executed by a processor, the computer program achieves the following effects: a clothing information query request sending end responds to an interactive instruction, obtains a clothing image, and sends the clothing image and clothing information query request to a clothing information query request receiving end. The clothing information query request receiving end then receives the clothing information and sends it to the clothing information query request sending end. Finally, the clothing information query request sending end receives and displays the clothing identification information returned by the clothing information query request receiving end. According to this clothing identification processing method, users can obtain clothing information associated with clothing images, thereby improving the user experience.

[0176] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0177] It should be noted that, in the description of this application, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0178] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0179] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0180] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0181] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0182] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0183] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0184] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A model training method, characterized in that: The following steps are involved: Inputting a sample image set, wherein the sample image set includes sample images with different character information; Acquire a clothing region image of the sample image; Extracting features of the clothing region image, and classifying the clothing region image according to the features to obtain a classification result, including: taking any clothing region image as a vertex, judging whether any vertex is connected to other vertices based on the similarity of features of different clothing region images, so as to construct an association graph containing all vertices and their connection relationships; inputting the association graph into a trained first graph convolutional neural network to obtain first confidences of all vertices in the association graph; obtaining all first target vertices whose first confidences are greater than a first preset confidence; constructing a candidate set subgraph of any first target vertex; inputting any candidate set subgraph into a trained second graph convolutional neural network to obtain a second confidence of the first target vertex in any candidate set subgraph; obtaining all second target vertices whose second confidences are greater than a second preset confidence; and treating any second target vertex and its adjacent second target vertices as the same cluster to obtain a first classification result; obtaining character information corresponding to the clothing region image, and grouping clothing region images with the same character information in any category of the classification result as a cluster to obtain a clustering result of the clothing region image; determining, based on the role labeling information of the sample image, whether the role information of the clothing region images in any cluster in the clustering result is the same; If so, the model training is completed; if not, the step of obtaining the character information corresponding to the clothing area image and its subsequent steps are re-executed.

2. The method according to claim 1, wherein: The obtaining of character information corresponding to the clothing area image includes: Acquire a human body image region in the sample image and character information corresponding to the human body image region; Calculating the degree of overlap between the clothing area image and the human body image area; When the degree of overlap is greater than a preset degree of overlap, the character information corresponding to the human body image area is used as the character information of the clothing area image.

3. The method according to claim 2, wherein: The obtaining of character information corresponding to the human body image area includes: Extracting human body feature information of the human body image area; Calculate the similarity between human feature information in each human image area; The human image regions with similarity greater than or equal to a preset similarity are regarded as the same character information.

4. The method according to claim 3, wherein: The step of re-executing the step of obtaining the character information corresponding to the clothing area image and subsequent steps include: The preset similarity value is adjusted, and the step of obtaining the character information corresponding to the clothing area image and subsequent steps are re-executed.

5. The method according to claim 1, wherein: The first graph convolutional neural network is obtained by training a training sample including a first training association graph using any graph convolutional neural network model and then modifying the parameters of the graph convolutional neural network model, wherein any two vertices in the first training association graph are not connected to the same vertex; The candidate set subgraph of any first target vertex includes any first target vertex and first target vertices adjacent to the first target vertex; The second graph convolutional neural network is obtained by using any graph convolutional neural network model, training samples that only include the second training association graph, and then modifying the parameters of the graph convolutional neural network model. In any second training association graph, there is a vertex connected to all other vertices.

6. The method according to claim 5, characterized in that: The extracting features of the clothing region image and classifying the clothing region image according to the features to obtain a classification result further includes: Perform at least one iterative merging on each cluster in the first classification result to obtain a second classification result; wherein any one iterative merging includes: Obtaining the cluster center of each cluster in the first clustering result, wherein the first clustering result is the clustering result before any iterative merging; The distance between any two cluster centers is calculated. If the distance is not greater than a preset threshold distance, the two clusters corresponding to the any two cluster centers are merged.

7. The method according to claim 1, characterized in that The sample image set is obtained by extracting video frames from a video.

8. A model training device, characterized in that: include: A sample set input module, configured to input a sample image set, wherein the sample image set includes sample images with different character information; a clothing area image classification module, configured to obtain a clothing area image of the sample image; Extracting features of the clothing region image, and classifying the clothing region image according to the features to obtain a classification result, including: taking any clothing region image as a vertex, judging whether any vertex is connected to other vertices based on the similarity of features of different clothing region images, so as to construct an association graph containing all vertices and their connection relationships; inputting the association graph into a trained first graph convolutional neural network to obtain first confidences of all vertices in the association graph; obtaining all first target vertices whose first confidences are greater than a first preset confidence; constructing a candidate set subgraph of any first target vertex; inputting any candidate set subgraph into a trained second graph convolutional neural network to obtain a second confidence of the first target vertex in any candidate set subgraph; obtaining all second target vertices whose second confidences are greater than a second preset confidence; and treating any second target vertex and its adjacent second target vertices as the same cluster to obtain a first classification result; a classification result clustering module for obtaining character information corresponding to the clothing region images, and grouping clothing region images with the same character information in any category of the classification results as a cluster to obtain a clustering result of the clothing region images; A clustering result verification module, configured to determine whether the character information of the clothing region images in any cluster in the clustering result is the same based on the character labeling information of the sample image; The adjustment determination module is used to determine whether the model training is completed if yes; if not, re-execute the step of obtaining the character information corresponding to the clothing area image and its subsequent steps.

9. A clothing recognition and processing method, characterized by: The method is applied to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by the method according to any one of claims 1 to 7. The method comprises: When receiving a clothing information query request sent by a clothing information query request sending end, inputting the clothing image carried in the clothing information query request into the model to determine a corresponding clothing image set through the model, obtaining a reference image corresponding to the determined clothing image set, and obtaining clothing information corresponding to the reference image; The clothing information corresponding to the reference image is sent back to the clothing information query request sending end as clothing identification information, so that the clothing identification information is displayed on the clothing information query request sending end.

10. A clothing recognition and processing device, characterized in that: The device is applied to a clothing information query request receiving end, and the clothing information query request receiving end is deployed with a model trained by the method according to any one of claims 1 to 7. The device includes: a reference image clothing information obtaining module, configured to, upon receiving a clothing information query request sent by a clothing information query request sending terminal, input a clothing image carried in the clothing information query request into the model, determine a corresponding clothing image set through the model, obtain a reference image corresponding to the determined clothing image set, and obtain clothing information corresponding to the reference image; The clothing identification information sending module is used to send the clothing information corresponding to the reference image as clothing identification information back to the clothing information query request sending end, so that the clothing identification information is displayed on the clothing information query request sending end.

11. A clothing recognition processing method, characterized in that: The method is applied to a sending end of a clothing information query request, and comprises the following steps: In response to the interaction instruction, obtaining a clothing image obtained according to the interaction instruction; generating a clothing information query request based on the clothing image; Sending the clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by the method according to any one of claims 1 to 7, and the clothing information query request receiving terminal is used to implement the clothing recognition processing method according to claim 9; Receive and display the clothing identification information returned by the clothing information query request receiving end.

12. The method according to claim 11, characterized in that The step of obtaining, in response to the interaction instruction, a clothing image obtained according to the interaction instruction includes: In response to the interactive instruction, a screenshot of the frame of the currently playing video is taken, and the clothing image is obtained according to the screenshot.

13. A clothing recognition and processing device, characterized in that: The device is applied to a clothing information query request sending end, and the device includes: A clothing image acquisition module, configured to respond to an interaction instruction and acquire a clothing image obtained according to the interaction instruction; a request generation module, configured to generate a clothing information query request based on the clothing image; a request sending module, configured to send the clothing information query request to a clothing information query request receiving terminal, wherein the clothing information query request receiving terminal is deployed with a model trained by the method according to any one of claims 1 to 7, and the clothing information query request receiving terminal is configured to implement the clothing recognition processing method according to claim 9; The clothing identification information receiving and displaying module is used to receive and display the clothing identification information returned by the clothing information query request receiving end.

14. A clothing information query request receiving terminal, characterized in that: include: One or more memories storing executable programs; One or more processors, configured to execute the executable program in the memory to implement the steps of the method according to any one of claims 1 to 7, and / or to implement the steps of the method according to claim 9.

15. A clothing information query request sending terminal, characterized in that: include: One or more memories storing executable programs; One or more processors, configured to execute the executable program in the memory to implement the steps of the method according to claim 11 or 12.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7, and / or implements the steps of the method according to claim 9.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to claim 11 or 12 are implemented.

Images