Model training method, clothing retrieval method, and related apparatus

Abstract

The application provides a model training method, a garment retrieval method and related devices. The model training method comprises: obtaining a training data set; inputting each training image into a garment detection model to obtain corresponding predicted detection information of each training image; inputting each training image into a garment segmentation model to obtain corresponding predicted segmentation information of each training image; inputting each training image and the corresponding predicted detection information and predicted segmentation information into a preset retrieval network to obtain corresponding predicted retrieval information of each training image; and training the preset retrieval network by using the corresponding labeled retrieval information of each training image and the corresponding predicted retrieval information of each training image to obtain a garment retrieval model. In the application, the preset retrieval network is trained by using the corresponding predicted retrieval information of each training image to obtain the garment retrieval model. When the garment retrieval model is applied to garment retrieval, the recognition efficiency and accuracy are high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and computer vision, in particular to a model training method, a clothing retrieval method and related devices. BACKGROUND

[0002] Clothing is the first in the commonly said "food, clothing, shelter and travel" in China, has become an indispensable thing for people, according to the image information to be identified clothing retrieval has a very important application value, for example, merchants or individuals can retrieve according to the obtained image information, facilitate the selection and collocation of clothing for merchants or customers, how to efficiently and accurately retrieve clothing according to the image information to be identified becomes a key problem.

[0003] With the popularization of computer recognition technology, using image recognition technology to retrieve clothing information in the image information to be identified, the technical scheme of the final retrieval result is developed. Deep learning is a method of representing learning based on data in machine learning. The advantage of deep learning is to replace manual feature extraction with non-supervised or semi-supervised feature learning and hierarchical feature extraction algorithm. Deep learning is a new field of machine learning research, its motivation is to establish and simulate the neural network of human brain analysis and learning, to simulate the mechanism of human brain to explain data such as image, sound and text. With the growing maturity of deep learning technology, the feature extraction technology and pattern recognition technology based on deep neural convolutional network model have been greatly improved compared with traditional statistical machine learning method, so that the accuracy and real-time performance of the current clothing style feature description and classification method have been further improved.

[0004] In a prior art, input multi-target clothing image and auxiliary segmentation multi-target small image set, by extracting multi-target clothing image and auxiliary segmentation significant feature information, get the segmented significant region and clothing significant feature set; based on the extracted segmented significant region, segmentation is carried out to get the preliminary segmentation result of multi-target clothing image significant region; then, the significant value of clothing image significant region is calculated by calculating the significant value of clothing image significant region of the preliminary segmentation result and the clothing significant feature set of multi-target small image set containing the same object, and the significant value of clothing image significant region is obtained; according to the obtained significant value of clothing image significant region, the similarity between multi-region clothing features is calculated, and the image segmentation result is output, which can effectively segment the multi-target clothing image.

[0005] In another prior art, by labeling and classifying the original clothing image, the attributes of the clothing are obtained, such as sleeves, collars, etc., and then the clothing picture is preprocessed by flipping, shifting, etc., and then a target detection algorithm based on deep learning is used to identify and detect the clothing attributes. The preprocessing of the clothing picture refers to labeling and classifying the position of the clothing attribute of the picture, and then using traditional image algorithms to flip and shift the picture for data augmentation. The clothing attribute recognition and detection method based on the deep learning target detection algorithm is to first use a deep convolutional neural network to fully extract the clothing attribute features, then use a feature pyramid to fuse the multi-layer features, and finally use a fully convolutional neural network to recognize and detect the clothing attributes.

[0006] In the current prior art, the accuracy and efficiency of retrieving clothes using a large number of commercial images are both low, and there is still much room for improvement. SUMMARY

[0007] The purpose of the present application is to provide a model training method, a clothing retrieval method and related devices, to solve the problem that in the prior art, the accuracy and efficiency of retrieving clothing using a large number of commercial images are both low, and there is still much room for improvement.

[0008] The purpose of the present application is achieved by adopting the following technical solutions:

[0009] In a first aspect, the present application provides a model training method, characterized in that the model training method comprises: obtaining a training data set, each training data in the training data set comprising a training image and annotation information corresponding to the training image, the annotation information corresponding to the training image comprising annotation retrieval information corresponding to the training image, the annotation retrieval information corresponding to the training image being used to indicate a clothing in a clothing database corresponding to a clothing in the training image; inputting each training image into a clothing detection model to obtain prediction detection information corresponding to each training image; inputting each training image into a clothing segmentation model to obtain prediction segmentation information corresponding to each training image; inputting each training image and its corresponding prediction detection information and prediction segmentation information into a preset retrieval network to obtain prediction retrieval information corresponding to each training image; training the preset retrieval network using the annotation retrieval information corresponding to each training image and the prediction retrieval information corresponding to each training image to obtain a clothing retrieval model.

[0010] The beneficial effects of the technical scheme are that, based on the training images in the training data set and the annotation information corresponding to the training images, the prediction detection information and the prediction segmentation information corresponding to each training image are obtained, based on each training image, the prediction detection information and the prediction segmentation information corresponding to each training image, the prediction search information corresponding to each training image is obtained, the preset search network is trained by using the prediction search information corresponding to each training image, and the clothing search model is obtained; when the clothing search model is applied to clothing search, the recognition efficiency and accuracy are high.

[0011] In some optional embodiments, the annotation information corresponding to the training image further includes annotation detection information corresponding to the training image, the annotation detection information corresponding to the training image includes annotation classification information and annotation bounding box regression information corresponding to the training image, and the method further includes: training a preset detection network by using each training image and the annotation detection information corresponding thereto to obtain a clothing detection model, the preset detection network including a detection RPN network, a ResNeXt50 network, and a first detection head structure to a third detection head structure; wherein the training of the preset detection network by using each training image and the annotation detection information corresponding thereto to obtain the clothing detection model includes: for each training image, performing the following processing: inputting the training image into the detection RPN network to obtain ROI information corresponding to the training image; inputting the training image into the ResNeXt50 network to obtain a plurality of feature maps corresponding to the training image; constructing a feature pyramid by using the plurality of feature maps corresponding to the training image to obtain a plurality of feature maps corresponding to the training image arranged in order of feature map size; inputting the ROI information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into the first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image; inputting the first bounding box regression information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into a second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image; inputting the second bounding box regression information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into the third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image; and training the preset detection network by using the annotation classification information, the annotation bounding box regression information, the third classification information, and the third bounding box regression information corresponding to each training image to obtain the clothing detection model.

[0012] The technical scheme has the beneficial effects that: the training image is input into the detection RPN network and the ResNeXt50 network to obtain ROI information corresponding to the training image and a plurality of feature maps, the plurality of feature maps are sorted into a feature pyramid, the ROI information corresponding to the training image and the plurality of feature maps sorted are input into a first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image, the plurality of feature maps sorted and the first bounding box regression information corresponding to the training image are input into a second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image, the plurality of feature maps sorted and the second bounding box regression information corresponding to the training image are input into a third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image, and the preset detection network is trained by using the labeled classification information, the labeled bounding box regression information, the third classification information and the third bounding box regression information corresponding to each training image, so as to obtain the clothing detection model; the clothing detection model is obtained by training the information in the training data set, the clothing detection model and the clothing retrieval model can be used together for clothing retrieval, and the accuracy of clothing retrieval is further improved.

[0013] In some optional embodiments, the first detection head structure to the third detection head structure are detection head structures with the same structure; each detection head structure includes a detection ROI Align layer, a detection classification branch and a detection regression branch, the detection classification branch includes two fully connected layers, and the detection regression branch includes two convolutional layers and one fully connected layer. The technical scheme has the beneficial effects that: the head structures with the same structure make the clothing detection model more efficient when detecting clothing images, and the structure design of each detection head structure makes the detection result more accurate.

[0014] In some optional embodiments, the annotation information corresponding to the training image further includes annotation segmentation information corresponding to the training image, and the annotation segmentation information corresponding to the training image includes annotation segmentation region information corresponding to the training image. The method further includes training a preset segmentation network by using each training image and the annotation segmentation information corresponding thereto to obtain a garment segmentation model, wherein the preset segmentation network includes a feature extraction network, a segmentation RPN network, and a first segmentation head structure to a third segmentation head structure. Training the preset segmentation network by using each training image and the annotation segmentation information corresponding thereto to obtain the garment segmentation model includes, for each training image, performing the following processing: inputting the training image into the feature extraction network to obtain feature extraction information corresponding to the training image; inputting the feature extraction information corresponding to the training image into the segmentation RPN network to obtain RPN positive and negative sample classification scores and RPN coordinate frame regression information corresponding to the training image; inputting the feature extraction information corresponding to the training image, the RPN positive and negative sample classification scores, and the RPN coordinate frame regression information into the first segmentation head structure to obtain first positive and negative sample classification scores and first coordinate frame regression information corresponding to the training image, first segmentation region information, and first scores of each object category; multiplying the first positive and negative sample classification scores corresponding to the training image and the first scores of each object category, respectively, to obtain first confidence scores of each object category corresponding to the training image; inputting the feature extraction information corresponding to the training image, the first positive and negative sample classification scores, and the first coordinate frame regression information into the second segmentation head structure to obtain second positive and negative sample classification scores and second coordinate frame regression information corresponding to the training image, second segmentation region information, and second scores of each object category; multiplying the second positive and negative sample classification scores corresponding to the training image and the second scores of each object category, respectively, to obtain second confidence scores of each object category corresponding to the training image; inputting the feature extraction information corresponding to the training image, the second positive and negative sample classification scores, and the second coordinate frame regression information into the third segmentation head structure to obtain third positive and negative sample classification scores and third coordinate frame regression information corresponding to the training image, third segmentation region information, and third scores of each object category; multiplying the third positive and negative sample classification scores corresponding to the training image and the third scores of each object category, respectively, to obtain third confidence scores of each object category corresponding to the training image; and training the preset segmentation network by using the annotation segmentation region information corresponding to each training image, the third segmentation region information, and the first confidence scores to the third confidence scores of each object category corresponding to the training image to obtain the garment segmentation model.

[0015] The beneficial effects of the technical scheme are that the training image is input into a feature extraction network to obtain feature extraction information corresponding to the training image, the feature extraction information corresponding to the training image is input into a segmentation RPN network to obtain RPN positive and negative sample classification scores and RPN coordinate frame regression information corresponding to the training image; the feature extraction information corresponding to the training image, the RPN positive and negative sample classification scores and the RPN coordinate frame regression information are input into a first segmentation head structure to obtain first positive and negative sample classification scores and first coordinate frame regression information corresponding to the training image, first segmentation region information and first scores of each object category; the first positive and negative sample classification scores corresponding to the training image and the first scores of each object category are multiplied respectively to obtain first confidence scores of each object category corresponding to the training image; the feature extraction information corresponding to the training image, the first positive and negative sample classification scores and the first coordinate frame regression information are input into a second segmentation head structure to obtain second positive and negative sample classification scores and second coordinate frame regression information corresponding to the training image, second segmentation region information and second scores of each object category; third confidence scores of each object category corresponding to the training image are obtained by using the same method; the preset segmentation network is trained by using the labeled segmentation region information corresponding to each training image, the third segmentation region information and the first confidence scores to the third confidence scores of each object category corresponding to the training image, and a garment segmentation model is obtained; the garment segmentation model is trained by using information in a training data set, the garment segmentation model and the garment retrieval model can be used together for garment retrieval, and the accuracy of garment retrieval is further improved.

[0016] In some optional embodiments, the first segmentation head structure to the third segmentation head structure are segmentation head structures with the same structure; each segmentation head structure includes an RCNN Head, a Mask Head and a MaskIoU Head, the RCNN Head includes one ROI Align layer and two fully connected layers, the Mask Head includes one ROI Align layer and four convolution layers, and the MaskIoU Head includes four convolution layers and three fully connected layers. The beneficial effects of the technical scheme are that the head structures with the same structure make the garment segmentation model more efficient when performing garment image segmentation, and the structure design of each segmentation head structure makes the segmentation result more accurate.

[0017] In some optional embodiments, the preset retrieval network comprises a ResNet50 network and a BNNeck network; the inputting of each of the training images, the corresponding predicted detection information and the predicted segmentation information into the preset retrieval network to obtain the corresponding predicted retrieval information of each of the training images comprises: for each of the training images, performing the following processing: inputting the training image into the ResNet50 network to obtain the graph residual feature corresponding to the training image; inputting the graph residual feature corresponding to the training image into a pooling layer to obtain the pooling feature corresponding to the training image; inputting the pooling feature corresponding to the training image into the BNNeck network to obtain the normalized feature corresponding to the training image; and performing retrieval by using the normalized feature corresponding to the training image, the predicted detection information and the predicted segmentation information to retrieve the corresponding clothes of the training image from the clothes database as the predicted retrieval information corresponding to the training image. The technical scheme has the beneficial effects that: the training image is input into the ResNet50 network to obtain the graph residual feature corresponding to the training image; the graph residual feature corresponding to the training image is input into the pooling layer to obtain the pooling feature corresponding to the training image; the pooling feature corresponding to the training image is input into the BNNeck network to obtain the normalized feature corresponding to the training image, and the normalized feature corresponding to the training image, the predicted detection information and the predicted segmentation information are used for retrieval to retrieve the corresponding clothes of the training image from the clothes database as the predicted retrieval information corresponding to the training image; the predicted retrieval information corresponding to the training image obtained by the above method is more accurate, and the training of the clothes retrieval model by using the predicted retrieval information corresponding to the training image makes the accuracy higher when the clothes retrieval model is used for clothes image retrieval.

[0018] In some optional embodiments, the BNNeck network comprises a normalization layer and a fully connected layer; the inputting of the pooling feature corresponding to the training image into the BNNeck network to obtain the normalized feature corresponding to the training image comprises: inputting the pooling feature corresponding to the training image into the normalization layer to obtain the normalized feature corresponding to the training image, the pooling feature corresponding to the training image being used for calculating the first loss value and the second loss value corresponding to the training image; and inputting the normalized feature corresponding to the training image into the fully connected layer to obtain the fully connected feature corresponding to the training image, the fully connected feature corresponding to the training image being used for calculating the third loss value corresponding to the training image; wherein the first loss value, the second loss value and the third loss value corresponding to each of the training images are used for training the preset retrieval network. The technical scheme has the beneficial effects that: the first to third loss values are obtained by using the normalization layer and the fully connected layer, the preset retrieval network is trained by using the first to third loss values to obtain the clothes retrieval model, and the retrieval result is more accurate when the clothes retrieval model is used for clothes image retrieval.

[0019] In some optional embodiments, the full connection features corresponding to the training images are also used to calculate label smoothness values corresponding to the training images; and the first loss values, the second loss values, the third loss values, and the label smoothness values corresponding to each of the training images are used to train the preset retrieval network. The technical solution has the beneficial effect that the full connection features corresponding to the training images are used to calculate the label smoothness values corresponding to the training images, the label smoothness values corresponding to the training images are used to train the preset retrieval network, and a clothing retrieval model is obtained, which increases the generalization ability of the clothing retrieval model and prevents overfitting of the clothing retrieval model in the training process.

[0020] In some optional embodiments, the model training method further includes: for at least one training image, performing random erasing on the training image to obtain an augmented image corresponding to the training image as a new training image and store the new training image to the training data set. The technical solution has the beneficial effect that data augmentation of the training image is performed by the method of random erasing, which can reduce the risk of overfitting of the clothing retrieval model in the training process, and at the same time, has a certain robustness to occlusion of a to-be-identified clothing image when the clothing retrieval model is used to identify the clothing image.

[0021] In some optional embodiments, the training of the preset retrieval network using the annotated retrieval information corresponding to each of the training images and the predicted retrieval information corresponding to each of the training images to obtain a clothing retrieval model includes: training the preset retrieval network using the annotated retrieval information corresponding to each of the training images and the predicted retrieval information corresponding to each of the training images using a preset learning rate strategy to obtain a clothing retrieval model; the preset learning rate strategy is used to indicate a learning rate corresponding to each epoch, and the learning rate corresponding to a first epoch to an N1th epoch gradually increases to a first preset learning rate, the learning rate corresponding to an N1+1th epoch to an N2th epoch is a second preset learning rate, the learning rate corresponding to an N2+1th epoch to an N3th epoch is a third preset learning rate, and the learning rate corresponding to an N3+1th epoch to an N4th epoch is a fourth preset learning rate, wherein the first preset learning rate is less than the second preset learning rate, the second preset learning rate is greater than the third preset learning rate, and the third preset learning rate is greater than the fourth preset learning rate. The technical solution has the beneficial effect that the preset learning rate strategy is used to train the preset retrieval network, which helps to slow down the early overfitting phenomenon that occurs in the initial stage when the clothing retrieval model is trained, and maintains the stability of the deep layers of the clothing retrieval model.

[0022] In a second aspect, the present application provides a garment retrieval method, which comprises: acquiring a to-be-detected image; inputting the to-be-detected image into a garment detection model to obtain predicted detection information corresponding to the to-be-detected image; inputting the to-be-detected image into a garment segmentation model to obtain predicted segmentation information corresponding to the to-be-detected image; inputting the to-be-detected image and the corresponding detection information and segmentation information into a garment retrieval model to obtain predicted retrieval information corresponding to the to-be-detected image; wherein the garment retrieval model is obtained by using the above model training method. The technical scheme has the beneficial effects that the garment retrieval model obtained by using the above model training method is used for garment image retrieval, so that the result of garment retrieval is more accurate, and the efficiency of garment retrieval is higher.

[0023] In a third aspect, the present application provides a model training device, which comprises: a data acquisition module configured to acquire a training data set, each training data in the training data set comprising a training image and annotation information corresponding to the training image, the annotation information corresponding to the training image comprising annotated retrieval information corresponding to the training image, the annotated retrieval information corresponding to the training image being used to indicate a garment in a garment database corresponding to a garment in the training image; a predicted detection module configured to input each training image into a garment detection model to obtain predicted detection information corresponding to each training image; a predicted segmentation module configured to input each training image into a garment segmentation model to obtain predicted segmentation information corresponding to each training image; a predicted retrieval module configured to input each training image and the corresponding predicted detection information and predicted segmentation information into a preset retrieval network to obtain predicted retrieval information corresponding to each training image; and a retrieval training module configured to train the preset retrieval network by using the annotated retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image to obtain a garment retrieval model.

[0024] In some optional embodiments, the annotation information corresponding to the training image further comprises annotated detection information corresponding to the training image, the annotated detection information corresponding to the training image comprising annotated classification information and annotated detection frame regression information corresponding to the training image, and the model training device further comprises:

[0025] a detection training module configured to train a preset detection network by using each training image and the corresponding annotated detection information to obtain a garment detection model, the preset detection network comprising a detection RPN network, a ResNeXt50 network, and a first detection head structure to a third detection head structure;

[0026] The detection training module comprises:

[0027] An ROI information unit is configured to input the training image into the detection RPN network to obtain ROI information corresponding to the training image.

[0028] A feature map acquisition unit is configured to input the training image into the ResNeXt50 network to obtain a plurality of feature maps corresponding to the training image.

[0029] A feature map arrangement unit is configured to construct a feature pyramid by using the plurality of feature maps corresponding to the training image to obtain a plurality of feature maps arranged in order of feature map size corresponding to the training image.

[0030] A first detection head unit is configured to input the ROI information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image.

[0031] A second detection head unit is configured to input the first bounding box regression information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image.

[0032] A third detection head unit is configured to input the second bounding box regression information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image.

[0033] A first training unit is configured to train the preset detection network by using the labeled classification information, the labeled bounding box regression information, the third classification information, and the third bounding box regression information corresponding to each of the training images to obtain the clothing detection model.

[0034] In some optional embodiments, the first detection head structure to the third detection head structure are detection head structures of the same structure; each detection head structure includes a detection ROI Align layer, a detection classification branch, and a detection regression branch, the detection classification branch includes two fully connected layers, and the detection regression branch includes two convolutional layers and one fully connected layer.

[0035] In some optional embodiments, the labeled information corresponding to the training image further includes labeled segmentation information corresponding to the training image, the labeled segmentation information corresponding to the training image includes labeled segmentation region information corresponding to the training image, and the model training apparatus further includes:

[0036] The segmentation training module is configured to train a preset segmentation network by using each of the training images and the corresponding annotation segmentation information, to obtain a garment segmentation model, wherein the preset segmentation network comprises a feature extraction network, a segmentation RPN network, and a first segmentation head structure to a third segmentation head structure.

[0037] The segmentation training module comprises:

[0038] The feature extraction unit is configured to input the training image into the feature extraction network, to obtain corresponding feature extraction information of the training image.

[0039] The segmentation RPN unit is configured to input the corresponding feature extraction information of the training image into the segmentation RPN network, to obtain RPN positive and negative sample classification scores and RPN coordinate frame regression information corresponding to the training image.

[0040] The first segmentation head unit is configured to input the corresponding feature extraction information, the RPN positive and negative sample classification scores, and the RPN coordinate frame regression information of the training image into the first segmentation head structure, to obtain first positive and negative sample classification scores and first coordinate frame regression information corresponding to the training image, first segmentation region information, and first scores of each object category; and multiply the first positive and negative sample classification scores and the first scores of each object category corresponding to the training image respectively, to obtain first confidence scores of each object category corresponding to the training image.

[0041] The second segmentation head unit is configured to input the corresponding feature extraction information, the first positive and negative sample classification scores, and the first coordinate frame regression information of the training image into the second segmentation head structure, to obtain second positive and negative sample classification scores and second coordinate frame regression information corresponding to the training image, second segmentation region information, and second scores of each object category; and multiply the second positive and negative sample classification scores and the second scores of each object category corresponding to the training image respectively, to obtain second confidence scores of each object category corresponding to the training image.

[0042] The third segmentation head unit is configured to input the corresponding feature extraction information, the second positive and negative sample classification scores, and the second coordinate frame regression information of the training image into the third segmentation head structure, to obtain third positive and negative sample classification scores and third coordinate frame regression information corresponding to the training image, third segmentation region information, and third scores of each object category; and multiply the third positive and negative sample classification scores and the third scores of each object category corresponding to the training image respectively, to obtain third confidence scores of each object category corresponding to the training image.

[0043] The second training unit is configured to train the preset segmentation network by using the annotation segmentation region information corresponding to each of the training images, the third segmentation region information, and the first confidence score to the third confidence score of each object category corresponding to the training image, to obtain the clothing segmentation model.

[0044] In some optional embodiments, the first segmentation head structure to the third segmentation head structure are segmentation head structures of the same structure; each segmentation head structure includes an RCNN Head, a Mask Head, and a MaskIoU Head, the RCNN Head includes one ROI Align layer and two fully connected layers, the Mask Head includes one ROI Align layer and four convolution layers, and the MaskIoU Head includes four convolution layers and three fully connected layers.

[0045] In some optional embodiments, the preset retrieval network includes a ResNet50 network and a BNNeck network; and the prediction retrieval module includes:

[0046] a picture residual feature unit configured to input the training image into the ResNet50 network to obtain picture residual features corresponding to the training image;

[0047] a pooling feature unit configured to input the picture residual features corresponding to the training image into a pooling layer to obtain pooling features corresponding to the training image;

[0048] a normalization feature unit configured to input the pooling features corresponding to the training image into the BNNeck network to obtain normalization features corresponding to the training image;

[0049] a first retrieval unit configured to perform retrieval by using the normalization features corresponding to the training image, prediction detection information, and prediction segmentation information, to retrieve a garment corresponding to the training image from the garment database as prediction retrieval information corresponding to the training image.

[0050] In some optional embodiments, the BNNeck network includes a normalization layer and a fully connected layer; and the normalization feature unit includes:

[0051] a first loss sub-unit configured to input the pooling features corresponding to the training image into the normalization layer to obtain normalization features corresponding to the training image, the pooling features corresponding to the training image being used to calculate a first loss value and a second loss value corresponding to the training image;

[0052] a second loss sub-unit configured to input the normalization features corresponding to the training image into the fully connected layer to obtain fully connected features corresponding to the training image, the fully connected features corresponding to the training image being used to calculate a third loss value corresponding to the training image.

[0053] The first loss value, the second loss value, and the third loss value corresponding to each of the training images are used to train the preset retrieval network.

[0054] In some optional embodiments, the full-connection feature corresponding to each of the training images is also used to calculate a label smoothness value corresponding to each of the training images; and the first loss value, the second loss value, the third loss value, and the label smoothness value corresponding to each of the training images are used to train the preset retrieval network.

[0055] In some optional embodiments, the model training apparatus further comprises:

[0056] The random erasing module is configured to, for at least one training image, perform random erasing on the training image to obtain an augmented image corresponding to the training image as a new training image and store the new training image to the training data set.

[0057] In some optional embodiments, the retrieval training module comprises:

[0058] The learning rate strategy unit is configured to use a preset learning rate strategy to train the preset retrieval network by using the labeled retrieval information corresponding to each of the training images and the predicted retrieval information corresponding to each of the training images, to obtain a garment retrieval model; the preset learning rate strategy is used to indicate a learning rate corresponding to each epoch, and the learning rate corresponding to a first epoch to an N1th epoch gradually increases to a first preset learning rate, the learning rate corresponding to an N1+1th epoch to an N2th epoch is a second preset learning rate, the learning rate corresponding to an N2+1th epoch to an N3th epoch is a third preset learning rate, and the learning rate corresponding to an N3+1th epoch to an N4th epoch is a fourth preset learning rate, wherein the first preset learning rate is less than the second preset learning rate, the second preset learning rate is greater than the third preset learning rate, and the third preset learning rate is greater than the fourth preset learning rate.

[0059] In a fourth aspect, the present application provides a garment retrieval apparatus, which comprises:

[0060] The image acquisition module is configured to acquire a to-be-detected image.

[0061] The detection information module is configured to input the to-be-detected image into the garment detection model to obtain detection information corresponding to the to-be-detected image.

[0062] The segmentation information module is configured to input the to-be-detected image into the garment segmentation model to obtain segmentation information corresponding to the to-be-detected image.

[0063] The retrieval result module is configured to input the image to be detected and corresponding detection information and segmentation information thereof into a clothing retrieval model to obtain a retrieval result corresponding to the image to be detected, wherein the clothing retrieval model is obtained by using the model training method.

[0064] In a fifth aspect, the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the model training method or the steps of the clothing retrieval method when executing the computer program.

[0065] In a sixth aspect, the present application provides a computer readable storage medium, which stores a computer program or a clothing retrieval model; the computer program is executed by a processor to implement the steps of the model training method or the steps of the clothing retrieval method; and the clothing retrieval model is obtained by using the model training method. BRIEF DESCRIPTION OF DRAWINGS

[0066] The present application will be further described below in combination with the drawings and examples.

[0067] Figure 1 is a flowchart of a model training method provided by an embodiment of the present application;

[0068] Figure 2 is a schematic diagram of the principle of a model training method provided by an embodiment of the present application;

[0069] Figure 3 is a partial flowchart of another model training method provided by an embodiment of the present application;

[0070] Figure 4 is a flowchart of obtaining a clothing detection model provided by an embodiment of the present application;

[0071] Figure 5 is a structural diagram of a ResNext50 network provided by an embodiment of the present application;

[0072] Figure 6 is a partial flowchart of still another model training method provided by an embodiment of the present application;

[0073] Figure 7 is a flowchart of obtaining a clothing segmentation model provided by an embodiment of the present application;

[0074] Figure 8 is a structural diagram of a preset segmentation network provided by an embodiment of the present application;

[0075] Figure 9is a flowchart of a process for obtaining predicted search information provided by an embodiment of the present application;

[0076] Figure 10 is a flowchart of a process for obtaining full connection features provided by an embodiment of the present application;

[0077] Figure 11 is a structural diagram of a preset search network provided by an embodiment of the present application;

[0078] Figure 12 is a partial flowchart of another model training method provided by an embodiment of the present application;

[0079] Figure 13 is a flowchart of a clothing search method provided by an embodiment of the present application;

[0080] Figure 14 is a structural diagram of a model training device provided by an embodiment of the present application;

[0081] Figure 15 is a structural diagram of another model training device provided by an embodiment of the present application;

[0082] Figure 16 is a structural diagram of a detection training module provided by an embodiment of the present application;

[0083] Figure 17 is a structural diagram of another model training device provided by an embodiment of the present application;

[0084] Figure 18 is a structural diagram of a segmentation training module provided by an embodiment of the present application;

[0085] Figure 19 is a structural diagram of a prediction search module provided by an embodiment of the present application;

[0086] Figure 20 is a structural diagram of a normalization feature unit provided by an embodiment of the present application;

[0087] Figure 21 is a structural diagram of another model training device provided by an embodiment of the present application;

[0088] Figure 22 is a structural diagram of a clothing search device provided by an embodiment of the present application;

[0089] Figure 23 is a structural diagram of an electronic device provided by an embodiment of the present application;

[0090] Figure 24FIG. 1 is a structural schematic diagram of a program product for implementing the model training method provided in the embodiments of the present application or the garment retrieval method provided in the embodiments of the present application. DETAILED DESCRIPTION

[0091] The present application will be further described below in conjunction with the drawings and the specific embodiments. It should be noted that the embodiments described below or the technical features thereof can be combined with each other to form new embodiments without conflict.

[0092] Referring to Figure 1 and Figure 2 The embodiments of the present application provide a model training method, which comprises steps S101-S105.

[0093] Step S101: Obtain a training data set, each training data in the training data set comprising a training image and annotation information corresponding to the training image, the annotation information corresponding to the training image comprising annotation retrieval information corresponding to the training image, the annotation retrieval information corresponding to the training image being used to indicate a garment in a garment database corresponding to a garment in the training image.

[0094] The training data set can be obtained from a training database pre-stored in a storage medium, manually input by a person, or retrieved in a cloud server. The training database is obtained from a commercial garment database, for example. The commercial garment database stores a large number of garment pictures and annotation information corresponding to the garment pictures. The commercial garment database is, for example, a large benchmark data set such as DeepFashion1 or DeepFashion2.

[0095] Step S102: Input each training image into a garment detection model to obtain prediction detection information corresponding to each training image. The garment detection model can be Cascade RCNN.

[0096] The prediction detection information can comprise the category of the garment in the training image and the detection box coordinates of the target garment image.

[0097] Step S103: Input each training image into a garment segmentation model to obtain prediction segmentation information corresponding to each training image. The garment detection model can be Hybrid Task Cascade.

[0098] The prediction segmentation information can comprise the category of the garment in the training image, the detection box coordinates of the target garment image, and the contour line information (or segmentation region information) of the target garment image.

[0099] Step S104: input each of the training images and the corresponding predicted detection information and predicted segmentation information into the preset retrieval network to obtain the predicted retrieval information corresponding to each of the training images. The clothing detection model can adopt ReIDModel.

[0100] The predicted retrieval information can be used to indicate the clothing in the clothing database corresponding to the clothing in the training image.

[0101] Step S105: training the preset retrieval network by using the labeled retrieval information corresponding to each of the training images and the predicted retrieval information corresponding to each of the training images to obtain the clothing retrieval model.

[0102] Therefore, based on the training images in the training data set and the labeled information corresponding to the training images, the predicted detection information and the predicted segmentation information corresponding to each of the training images are obtained, the predicted retrieval information corresponding to each of the training images is obtained based on each of the training images, the predicted detection information and the predicted segmentation information corresponding to each of the training images, and the clothing retrieval model is obtained by training the preset retrieval network by using the predicted retrieval information corresponding to each of the training images. When the above clothing retrieval model is applied to clothing retrieval, the recognition efficiency and accuracy are high.

[0103] In a specific application scenario, the training data set for training the clothing detection model is obtained from the DeepFashion2 data set. The DeepFashion2 data set is a large benchmark data set with comprehensive tasks and annotations for fashion image understanding, with 801K clothing items, each item having rich annotations such as style, proportion, viewpoint, occlusion, bounding box, dense landmarks and mask, and also having 873K commercial clothing pairs. It is the most comprehensive benchmark data set in the same category so far. The training data set includes a plurality of training images and labeled information corresponding to the training images, which indicates the corresponding images in the DeepFashion2 data set corresponding to the training images. Each of the training images is input into the clothing detection model and the clothing segmentation model to obtain the predicted detection information and the predicted segmentation information corresponding to each of the training images. Each of the training images and the corresponding predicted detection information and predicted segmentation information is input into the preset retrieval network to obtain the predicted retrieval information corresponding to each of the training images. The preset retrieval network is trained by using the labeled retrieval information corresponding to each of the training images and the predicted retrieval information corresponding to each of the training images to obtain the clothing retrieval model. The clothing retrieval model can be used to retrieve in the database for the user-acquired clothing picture to be identified.

[0104] Reference Figure 3In some embodiments, the annotation information corresponding to the training image can further include annotation detection information corresponding to the training image, the annotation detection information corresponding to the training image including annotation classification information and annotation bounding box regression information corresponding to the training image, and the model training method can further include step S106.

[0105] Step S106: training a preset detection network using each of the training images and the annotation detection information corresponding thereto to obtain a clothing detection model, the preset detection network including a detection RPN network, a ResNeXt50 network, and a first detection head structure to a third detection head structure.

[0106] Referring to Figure 4 , the step S106 can include steps S201-S207.

[0107] For each of the training images, the following processing is performed:

[0108] Step S201: inputting the training image into the detection RPN network to obtain ROI information corresponding to the training image;

[0109] Step S202: inputting the training image into the ResNeXt50 network to obtain a plurality of feature maps corresponding to the training image;

[0110] Step S203: constructing a feature pyramid using the plurality of feature maps corresponding to the training image to obtain a plurality of feature maps corresponding to the training image arranged in order of feature map size;

[0111] Step S204: inputting the ROI information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into the first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image;

[0112] Step S205: inputting the first bounding box regression information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into the second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image;

[0113] Step S206: inputting the second bounding box regression information corresponding to the training image and the plurality of feature maps corresponding to the training image arranged in order of feature map size into the third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image;

[0114] Step S207: training the preset detection network by using the labeled classification information, the labeled bounding box regression information, the third classification information and the third bounding box regression information corresponding to each training image, to obtain the clothing detection model.

[0115] Thus, the training image is input into the detection RPN network and the ResNeXt50 network to obtain the ROI information corresponding to the training image and a plurality of feature maps, the plurality of feature maps are sorted into a feature pyramid, the ROI information corresponding to the training image and the plurality of feature maps sorted are input into the first detection head structure to obtain the first classification information and the first bounding box regression information corresponding to the training image, the plurality of feature maps sorted and the first bounding box regression information corresponding to the training image are input into the second detection head structure to obtain the second classification information and the second bounding box regression information corresponding to the training image, the plurality of feature maps sorted and the second bounding box regression information corresponding to the training image are input into the third detection head structure to obtain the third classification information and the third bounding box regression information corresponding to the training image, the preset detection network is trained by using the labeled classification information, the labeled bounding box regression information, the third classification information and the third bounding box regression information corresponding to each training image, to obtain the clothing detection model; the clothing detection model is obtained by training the information in the training data set, the clothing detection model and the clothing retrieval model can be used together for clothing retrieval, and the accuracy of clothing retrieval is further improved.

[0116] Referring to Figure 5 In some embodiments, the residual unit branch in ResNext50 can be composed of three consecutive convolution layers, the first layer has an input channel of 256 and an output channel of 4, and a convolution kernel size of 1x1; the second layer has an input channel of 4, a convolution kernel size of 3x3, and an output channel of 4; the third layer has an input channel of 4, a convolution kernel size of 1x1, and an output channel of 256. Each residual unit is composed of 32 parallel residual unit branch structures, and finally an additive operation is performed on the channels.

[0117] In some embodiments, the first detection head structure to the third detection head structure can be detection head structures of the same structure; each detection head structure includes a detection ROI Align layer, a detection classification branch and a detection regression branch, the detection classification branch includes two fully connected layers, and the detection regression branch includes two convolution layers and one fully connected layer.

[0118] Thus, the head structures of the same structure make the clothing detection model more efficient when detecting clothing images, and the structure design of each detection head structure described above makes the detection result more accurate.

[0119] Referring to Figure 6In some embodiments, the annotation information corresponding to the training image can further include annotation segmentation information corresponding to the training image, and the annotation segmentation information corresponding to the training image includes annotation segmentation region information corresponding to the training image. The model training method can further include step S107.

[0120] Step S107: training a preset segmentation network by using each of the training images and the annotation segmentation information corresponding thereto to obtain a garment segmentation model, wherein the preset segmentation network includes a feature extraction network, a segmentation RPN network, and a first segmentation head structure to a third segmentation head structure.

[0121] Referring to Figure 7 and Figure 8 , the step S107 can include steps S301-S306.

[0122] For each of the training images, the following processing is performed:

[0123] Step S301: inputting the training image into the feature extraction network to obtain feature extraction information corresponding to the training image;

[0124] Step S302: inputting the feature extraction information corresponding to the training image into the segmentation RPN network to obtain RPN positive and negative sample classification scores and RPN coordinate frame regression information corresponding to the training image;

[0125] Step S303: inputting the feature extraction information, the RPN positive and negative sample classification scores, and the RPN coordinate frame regression information corresponding to the training image into the first segmentation head structure to obtain first positive and negative sample classification scores and first coordinate frame regression information corresponding to the training image, first segmentation region information, and first scores of each object category; multiplying the first positive and negative sample classification scores corresponding to the training image and the first scores of each object category, respectively, to obtain first confidence scores of each object category corresponding to the training image;

[0126] Step S304: inputting the feature extraction information, the first positive and negative sample classification scores, and the first coordinate frame regression information corresponding to the training image into the second segmentation head structure to obtain second positive and negative sample classification scores and second coordinate frame regression information corresponding to the training image, second segmentation region information, and second scores of each object category; multiplying the second positive and negative sample classification scores corresponding to the training image and the second scores of each object category, respectively, to obtain second confidence scores of each object category corresponding to the training image;

[0127] Step S305: input the feature extraction information corresponding to the training image, the second positive and negative sample classification score and the second coordinate frame regression information into the third segmentation head structure, to obtain the third positive and negative sample classification score and the third coordinate frame regression information corresponding to the training image, the third segmentation region information, and the third score of each object category; multiply the third positive and negative sample classification score corresponding to the training image and the third score of each object category respectively, to obtain the third confidence score of each object category corresponding to the training image;

[0128] Step S306: train the preset segmentation network by using the labeled segmentation region information corresponding to each training image, the third segmentation region information, and the first confidence score to the third confidence score of each object category corresponding to the training image, to obtain the garment segmentation model.

[0129] Thus, the training image is input into the feature extraction network to obtain the feature extraction information corresponding to the training image, the feature extraction information corresponding to the training image is input into the segmentation RPN network to obtain the RPN positive and negative sample classification score and the RPN coordinate frame regression information corresponding to the training image; the feature extraction information corresponding to the training image, the RPN positive and negative sample classification score and the RPN coordinate frame regression information are input into the first segmentation head structure to obtain the first positive and negative sample classification score and the first coordinate frame regression information corresponding to the training image, the first segmentation region information, and the first score of each object category; the first positive and negative sample classification score corresponding to the training image and the first score of each object category are multiplied respectively to obtain the first confidence score of each object category corresponding to the training image; the feature extraction information corresponding to the training image, the first positive and negative sample classification score and the first coordinate frame regression information are input into the second segmentation head structure to obtain the second positive and negative sample classification score and the second coordinate frame regression information corresponding to the training image, the second segmentation region information, and the second score of each object category; the third confidence score of each object category corresponding to the training image is obtained by using the same method; the preset segmentation network is trained by using the labeled segmentation region information corresponding to each training image, the third segmentation region information, and the first confidence score to the third confidence score of each object category corresponding to the training image, to obtain the garment segmentation model; the garment segmentation model is trained by using the information in the training data set, and the garment segmentation model and the garment retrieval model can be used together for garment retrieval, further improving the accuracy of garment retrieval.

[0130] In some embodiments, the first segmentation head structure to the third segmentation head structure can be the same segmentation head structure; each segmentation head structure respectively comprises an RCNN Head, a Mask Head and a MaskIoU Head, the RCNN Head comprises one ROI Align layer and two fully connected layers, the Mask Head comprises one ROI Align layer and four convolution layers, and the MaskIoU Head comprises four convolution layers and three fully connected layers.

[0131] Thus, the same head structure makes the clothing segmentation model more efficient when performing clothing image segmentation, and the structure design of each segmentation head structure makes the segmentation result more accurate.

[0132] In some embodiments, the preset segmentation network can further comprise a semantic segmentation branch, the semantic segmentation branch comprises a semantic segmentation pyramid, the feature extraction information corresponding to the training image is input into the semantic segmentation pyramid to obtain the predicted semantic segmentation feature corresponding to the training image and a semantic segmentation prediction result, the predicted semantic segmentation feature corresponding to the training image is input into the ROI Align layer corresponding to each segmentation head structure, and the first to third coordinate frame regression information and the first to third segmentation region information corresponding to the training image are multiplied by the same element to perform data fusion; and the semantic segmentation prediction result corresponding to the training image is used to obtain the retrieval result corresponding to the training image.

[0133] Thus, by obtaining the predicted semantic segmentation feature corresponding to the training image through the semantic segmentation branch and performing data fusion with the first to third coordinate frame regression information and the first to third segmentation region information corresponding to the training image, the ability of the clothing retrieval model to distinguish image targets is enhanced.

[0134] Referring to Figure 9 In some embodiments, the preset retrieval network can comprise a ResNet50 network and a BNNeck network.

[0135] The step S104 comprises steps S401-S404.

[0136] For each training image, the following processing is performed:

[0137] Step S401: inputting the training image into the ResNet50 network to obtain image residual feature corresponding to the training image;

[0138] Step S402: inputting the image residual feature corresponding to the training image into a pooling layer to obtain the pooling feature corresponding to the training image;

[0139] Step S403: input the pooling feature corresponding to the training image into the BNNeck network to obtain normalized feature corresponding to the training image.

[0140] Step S404: search the garment database by using the normalized feature corresponding to the training image, the predicted detection information and the predicted segmentation information to obtain the garment corresponding to the training image as the predicted search information corresponding to the training image.

[0141] Thus, the training image is input into the ResNet50 network to obtain the image residual feature corresponding to the training image; the image residual feature corresponding to the training image is input into the pooling layer to obtain the pooling feature corresponding to the training image; the pooling feature corresponding to the training image is input into the BNNeck network to obtain the normalized feature corresponding to the training image, the garment corresponding to the training image is obtained from the garment database by searching by using the normalized feature corresponding to the training image, the predicted detection information and the predicted segmentation information as the predicted search information corresponding to the training image; the predicted search information corresponding to the training image obtained by the above method is more accurate, and the garment search model is trained by using the predicted search information corresponding to the training image, so that the accuracy is higher when the garment search model is used for garment image search.

[0142] In some embodiments, the ResNet50 network can include 4 stages, and the value of the stride of the fourth stage passed by the training image is set to 1.

[0143] Referring to Figure 10 In some embodiments, the BNNeck network can include a normalization layer and a fully connected layer; and the step S403 includes steps S501-S502.

[0144] Step S501: input the pooling feature corresponding to the training image into the normalization layer to obtain the normalized feature corresponding to the training image, and the pooling feature corresponding to the training image is used to calculate the first loss value and the second loss value corresponding to the training image.

[0145] Step S502: input the normalized feature corresponding to the training image into the fully connected layer to obtain the fully connected feature corresponding to the training image, and the fully connected feature corresponding to the training image is used to calculate the third loss value corresponding to the training image; wherein the first loss value, the second loss value and the third loss value corresponding to each training image are used to train the preset search network.

[0146] Thus, the first to third loss values are obtained by using the normalization layer and the fully connected layer, the preset search network is trained by using the first to third loss values to obtain the garment search model, so that the search result is more accurate when the garment search model is used for garment image search.

[0147] Referring to Figure 11 In a specific application scenario, the first loss value includes a triplet loss value calculated according to a triplet loss function, the second loss value can include a center loss value calculated according to a center loss function, and the third loss value can include an ID loss value calculated according to an ID loss function; wherein the center loss value is calculated by the following formula:

[0148]

[0149] In the formula, τ c represents the center loss value, B represents the number of samples (training data) in the training data set, f tj represents the pooling feature corresponding to the jth training data, c yj represents the average feature of all training data features corresponding to the class of the jth training data.

[0150] In some embodiments, the full connection feature corresponding to the training image can also be used to calculate the label smoothing value corresponding to the training image; the first loss value, the second loss value, the third loss value and the label smoothing value corresponding to each training image are used to train the preset retrieval network.

[0151] Therefore, the full connection feature corresponding to the training image is used to calculate the label smoothing value corresponding to the training image, and the label smoothing value corresponding to the training image is used to train the preset retrieval network to obtain the garment retrieval model, which increases the generalization ability of the garment retrieval model and prevents overfitting of the garment retrieval model in the training process.

[0152] In a specific application scenario, the label smoothing value is calculated by the following formula:

[0153]

[0154] Wherein, i represents the predicted label category of the garment retrieval model (i.e. the predicted retrieval information), P i represents the final result after label smoothing processing (i.e. the probability distribution of each predicted label category), ε is a small hyperparameter, K represents the total number of garment categories in the training data set, and y represents the real label category (i.e. the labeled retrieval information).

[0155] Referring to Figure 12 In some embodiments, the model training method can further include step S108.

[0156] Step S108: For at least one training image, randomly erase the training image to obtain an augmented image corresponding to the training image as a new training image and store it in the training dataset. Random erasing can be used.

[0157] Therefore, data augmentation of training images by random erasing can reduce the risk of overfitting of the clothing retrieval model during training, and at the same time make the clothing retrieval model robust to occlusion of the clothing image when performing clothing image recognition.

[0158] In some embodiments, the preset retrieval network may be trained using a semi-supervised learning method or a fully supervised learning method; when a fully supervised learning method is used, the model training method may further include step S109: based on the annotation information corresponding to the training image, obtaining the annotation information corresponding to the new training image.

[0159] In a specific application scenario, a rectangular box is randomly selected in the training image, and random values ​​are used at random positions to erase the original pixels of the training image. The specific method is as follows: input the training image I, the probability of erasure p, and the ratio of the erased area ranges from s l to s h , and the aspect ratio probability ranges from r1 to r2. First, determine whether a picture needs to be erased based on the probability p. If p1>p, the picture will not be processed. Otherwise, it needs to be erased. Based on the input training image I, the length, width, W, and H of the training image can be obtained, and then the area S can be obtained. According to Rand(s l , s h )*S to get the erased area S e , the length of the erased area (H e ), the width of the erased area (W e ) is obtained according to the following formula:

[0160]

[0161] Among them, r e Represents the aspect ratio of the erased area. The meanings of the remaining characters have been described in the above application scenarios and will not be repeated here.

[0162] This method can add different degrees of occlusion to the training images. Through such training images, the risk of overfitting of the clothing retrieval model to be trained can be reduced, and at the same time, it has a certain robustness to the occlusion of the image to be recognized.

[0163] In some implementations, step S105 may include step S601:

[0164] Step S601: training the preset retrieval network using a preset learning rate strategy by using the label retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image, to obtain a clothing retrieval model; the preset learning rate strategy is used to indicate a learning rate corresponding to each epoch, and the learning rate corresponding to a first epoch to an N1th epoch gradually increases to a first preset learning rate, the learning rate corresponding to an N1+1th epoch to an N2th epoch is a second preset learning rate, the learning rate corresponding to an N2+1th epoch to an N3th epoch is a third preset learning rate, and the learning rate corresponding to an N3+1th epoch to an N4th epoch is a fourth preset learning rate, where the first preset learning rate is less than the second preset learning rate, the second preset learning rate is greater than the third preset learning rate, and the third preset learning rate is greater than the fourth preset learning rate.

[0165] In this way, the preset learning rate strategy is used to train the preset retrieval network, which helps to slow down the early overfitting phenomenon occurring in the initial stage when the clothing retrieval model is trained, and maintains the stability of the deep layers of the clothing retrieval model.

[0166] In a specific application scenario, the learning rate corresponding to each epoch indicated by the preset learning rate strategy is calculated according to the following formula:

[0167]

[0168] Where t represents the serial number of the epoch, and lr(t) represents the learning rate corresponding to the tth epoch.

[0169] Referring to Figure 13 The embodiments of the present application also provide a clothing retrieval method, which comprises steps S701-S704.

[0170] Step S701: obtaining a to-be-detected image.

[0171] Step S702: inputting the to-be-detected image into the clothing detection model to obtain predicted detection information corresponding to the to-be-detected image.

[0172] Step S703: inputting the to-be-detected image into the clothing segmentation model to obtain predicted segmentation information corresponding to the to-be-detected image.

[0173] Step S704: inputting the to-be-detected image and the detection information and the segmentation information corresponding thereto into the clothing retrieval model to obtain predicted retrieval information corresponding to the to-be-detected image.

[0174] The clothing retrieval model is obtained by training the model training method provided by the embodiments of the present application.

[0175] Therefore, the clothing retrieval model trained by the above-mentioned model training method provided in the embodiment of the present application is used to perform clothing image retrieval, so that the clothing retrieval results are more accurate and the clothing retrieval efficiency is higher.

[0176] See also Figure 14 The embodiment of the present application also provides a model training device, the specific implementation method of which is consistent with the implementation method and technical effects recorded in the embodiment of the above-mentioned model training method, and some contents will not be repeated here.

[0177] The model training device includes: a data acquisition module 101, used to acquire a training data set, each training data in the training data set includes a training image and annotation information corresponding to the training image, the annotation information corresponding to the training image includes annotation retrieval information corresponding to the training image, and the annotation retrieval information corresponding to the training image is used to indicate the clothing in the clothing database corresponding to the clothing in the training image; a prediction detection module 102, used to input each of the training images into a clothing detection model to obtain the prediction detection information corresponding to each of the training images; a prediction segmentation module 103, used to input each of the training images into a clothing segmentation model to obtain the prediction segmentation information corresponding to each of the training images; a prediction retrieval module 104, used to input each of the training images and their corresponding prediction detection information and prediction segmentation information into a preset retrieval network to obtain the prediction retrieval information corresponding to each of the training images; a retrieval training module 105, used to train the preset retrieval network using the annotation retrieval information corresponding to each of the training images and the prediction retrieval information corresponding to each of the training images to obtain a clothing retrieval model.

[0178] See also Figure 15 In some embodiments, the annotation information corresponding to the training image may further include the annotation detection information corresponding to the training image, and the annotation detection information corresponding to the training image includes the annotation classification information and the annotation detection frame regression information corresponding to the training image. The model training device may further include: a detection training module 106, which is used to train a preset detection network using each of the training images and their corresponding annotation detection information to obtain a clothing detection model, wherein the preset detection network includes a detection RPN network, a ResNeXt50 network, and a first detection head structure to a third detection head structure; see Figure 16The detection training module 106 comprises: an ROI information unit 201, configured to input the training image into the detection RPN network to obtain ROI information corresponding to the training image; a feature map acquisition unit 202, configured to input the training image into the ResNeXt50 network to obtain a plurality of feature maps corresponding to the training image; a feature map arrangement unit 203, configured to construct a feature pyramid by using the plurality of feature maps corresponding to the training image to obtain a plurality of feature maps arranged in order of feature map size corresponding to the training image; a first detection head unit 204, configured to input the ROI information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image; a second detection head unit 205, configured to input the first bounding box regression information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image; a third detection head unit 206, configured to input the second bounding box regression information corresponding to the training image and the plurality of feature maps arranged in order of feature map size corresponding to the training image into the third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image; and a first training unit 207, configured to train the preset detection network by using the labeled classification information, the labeled bounding box regression information, the third classification information and the third bounding box regression information corresponding to each training image to obtain the clothing detection model.

[0179] In some embodiments, the first detection head structure to the third detection head structure can be detection head structures of the same structure; each detection head structure comprises a detection ROI Align layer, a detection classification branch and a detection regression branch, the detection classification branch comprises two fully connected layers, and the detection regression branch comprises two convolutional layers and one fully connected layer.

[0180] Referring to Figure 17 In some embodiments, the labeled information corresponding to the training image can further comprise labeled segmentation information corresponding to the training image, the labeled segmentation information corresponding to the training image comprises labeled segmentation region information corresponding to the training image, and the model training apparatus can further comprise a segmentation training module 107 configured to train a preset segmentation network by using each training image and the labeled segmentation information corresponding thereto to obtain a clothing segmentation model, the preset segmentation network comprising a feature extraction network, a segmentation RPN network and a first segmentation head structure to a third segmentation head structure; referring to Figure 18The segmentation training module 107 comprises: a feature extraction unit 301, configured to input the training image into the feature extraction network to obtain feature extraction information corresponding to the training image; a segmentation RPN unit 302, configured to input the feature extraction information corresponding to the training image into the segmentation RPN network to obtain RPN positive and negative sample classification scores and RPN coordinate frame regression information corresponding to the training image; a first segmentation head unit 303, configured to input the feature extraction information, the RPN positive and negative sample classification scores and the RPN coordinate frame regression information corresponding to the training image into the first segmentation head structure to obtain first positive and negative sample classification scores and first coordinate frame regression information, first segmentation region information and first scores of each object category corresponding to the training image; multiply the first positive and negative sample classification scores and the first scores of each object category corresponding to the training image respectively to obtain first confidence scores of each object category corresponding to the training image; a second segmentation head unit 304, configured to input the feature extraction information, the first positive and negative sample classification scores and the first coordinate frame regression information corresponding to the training image into the second segmentation head structure to obtain second positive and negative sample classification scores and second coordinate frame regression information, second segmentation region information and second scores of each object category corresponding to the training image; multiply the second positive and negative sample classification scores and the second scores of each object category corresponding to the training image respectively to obtain second confidence scores of each object category corresponding to the training image; a third segmentation head unit 305, configured to input the feature extraction information, the second positive and negative sample classification scores and the second coordinate frame regression information corresponding to the training image into the third segmentation head structure to obtain third positive and negative sample classification scores and third coordinate frame regression information, third segmentation region information and third scores of each object category corresponding to the training image; multiply the third positive and negative sample classification scores and the third scores of each object category corresponding to the training image respectively to obtain third confidence scores of each object category corresponding to the training image; and a second training unit 306, configured to train the preset segmentation network by using the first confidence scores to the third confidence scores of each object category corresponding to the training image, the labeled segmentation region information and the third segmentation region information corresponding to each of the training images to obtain the garment segmentation model.

[0181] In some embodiments, the first segmentation head structure to the third segmentation head structure can be segmentation head structures of the same structure; each segmentation head structure comprises an RCNN Head, a Mask Head and a MaskIoU Head, the RCNN Head comprises one ROI Align layer and two fully connected layers, the Mask Head comprises one ROI Align layer and four convolution layers, and the MaskIoU Head comprises four convolution layers and three fully connected layers.

[0182] Referring toFigure 19 In some embodiments, the preset retrieval network can include a ResNet50 network and a BNNeck network; the prediction retrieval module 104 includes: a graph residual feature unit 401, configured to input the training image into the ResNet50 network to obtain a graph residual feature corresponding to the training image; a pooling feature unit 402, configured to input the graph residual feature corresponding to the training image into a pooling layer to obtain a pooling feature corresponding to the training image; a normalized feature unit 403, configured to input the pooling feature corresponding to the training image into the BNNeck network to obtain a normalized feature corresponding to the training image; and a first retrieval unit 404, configured to perform retrieval by using the normalized feature corresponding to the training image, prediction detection information and prediction segmentation information to retrieve a garment corresponding to the training image from the garment database as prediction retrieval information corresponding to the training image.

[0183] Referring to Figure 20 In some embodiments, the BNNeck network can include a normalization layer and a fully connected layer; the normalized feature unit 403 includes: a first loss sub-unit 501, configured to input the pooling feature corresponding to the training image into the normalization layer to obtain a normalized feature corresponding to the training image, the pooling feature corresponding to the training image being used to calculate a first loss value and a second loss value corresponding to the training image; and a second loss sub-unit 502, configured to input the normalized feature corresponding to the training image into the fully connected layer to obtain a fully connected feature corresponding to the training image, the fully connected feature corresponding to the training image being used to calculate a third loss value corresponding to the training image; wherein the first loss value, the second loss value and the third loss value corresponding to each training image are used to train the preset retrieval network.

[0184] In some embodiments, the fully connected feature corresponding to the training image can also be used to calculate a label smoothness value corresponding to the training image; the first loss value, the second loss value, the third loss value and the label smoothness value corresponding to each training image are used to train the preset retrieval network.

[0185] Referring to Figure 21 In some embodiments, the model training apparatus can further include a random erasing module 108, configured to perform random erasing on at least one training image to obtain an augmented image corresponding to the training image as a new training image and store the new training image into the training dataset.

[0186] In some embodiments, the model training apparatus can further include an acquisition labeling module 109, configured to acquire labeling information corresponding to the new training image based on labeling information corresponding to the training image.

[0187] In some embodiments, the search training module 105 can include a learning rate strategy unit 601 configured to train the preset search network using a preset learning rate strategy by using the labeled search information corresponding to each of the training images and the predicted search information corresponding to each of the training images, to obtain a clothing search model; the preset learning rate strategy is used to indicate a learning rate corresponding to each epoch, and the learning rate corresponding to a first epoch to an N1th epoch gradually increases to a first preset learning rate, the learning rate corresponding to an N1+1th epoch to an N2th epoch is a second preset learning rate, the learning rate corresponding to an N2+1th epoch to an N3th epoch is a third preset learning rate, and the learning rate corresponding to an N3+1th epoch to an N4th epoch is a fourth preset learning rate, wherein the first preset learning rate is less than the second preset learning rate, the second preset learning rate is greater than the third preset learning rate, and the third preset learning rate is greater than the fourth preset learning rate.

[0188] Referring to Figure 22 The embodiments of the present application also provide a clothing search device, and the specific implementation manners and the achieved technical effects of the clothing search device are the same as those of the embodiments of the clothing search method, and some contents will not be described herein again.

[0189] The clothing search device includes an image acquisition module 701 configured to acquire a to-be-detected image, a detection information module 702 configured to input the to-be-detected image into a clothing detection model to obtain detection information corresponding to the to-be-detected image, a segmentation information module 703 configured to input the to-be-detected image into a clothing segmentation model to obtain segmentation information corresponding to the to-be-detected image, and a search result module 704 configured to input the to-be-detected image and the detection information and the segmentation information corresponding to the to-be-detected image into a clothing search model to obtain a search result corresponding to the to-be-detected image, wherein the clothing search model is obtained by using the model training method.

[0190] Referring to Figure 23 The embodiments of the present application also provide an electronic device 200, which includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0191] The memory 210 can include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 211 and / or a cache memory 212, and can further include a read-only memory (ROM) 213.

[0192] The memory 210 can further include a computer program stored therein, which can be executed by the processor 220 to enable the processor 220 to perform the steps of the model training method or the steps of the garment retrieval method in the embodiments of the present application. The specific implementation manners and the achieved technical effects are the same as those described in the embodiments of the model training method or the garment retrieval method, and some of the contents will not be described herein.

[0193] The memory 210 can further include a utility 214 having at least one program module 215, and the program module 215 includes but is not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof can include implementation of a network environment.

[0194] Correspondingly, the processor 220 can execute the above computer program and can execute the utility 214.

[0195] The bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus architectures.

[0196] The electronic device 200 can also communicate with one or more external devices 240, such as a keyboard or a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices that enable interaction with the electronic device 200, and / or with any devices (such as routers, modems, etc.) that enable the electronic device 200 to communicate with one or more other computing devices. Such communication can be carried out through the input / output interface 250. In addition, the electronic device 200 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the electronic device 200 through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 200, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0197] The embodiments of the present application also provide a computer readable storage medium for storing a computer program or a garment retrieval model; the computer program is executed to implement the steps of the model training method or the garment retrieval method in the embodiments of the present application; the garment retrieval model is trained by using any of the above model training methods; the specific implementation manners and the achieved technical effects are the same as those described in the embodiments of the model training method or the garment retrieval method, and some of the contents will not be described herein.

[0198] Figure 24 A program product 300 for implementing the above model training or garment retrieval method provided by the embodiment is shown, which can adopt a portable compact disc read-only memory (CD-ROM) and include program codes, and can run on a terminal device, such as a personal computer. However, the program product 300 of the present application is not limited to this, and in the present application, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or apparatus. The program product 300 can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0199] The computer readable storage medium can include a data signal carried in a baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium can also be any readable medium that can send, propagate or transmit the program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the readable storage medium can be transmitted in any suitable medium, including but not limited to wireless, wired, optical, cable, RF, or any suitable combination of the above. The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, and the like, and conventional procedural programming languages such as the C programming language or similar programming languages. The program code can be executed entirely on the user computing device, partially on an associated device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0200] The application is described from the use purpose, efficiency, progress and novelty, which meets the function improvement and use requirements emphasized by the patent law. The above description and drawings are only the preferred embodiments of the application, and are not limited to the application. Therefore, all similar, similar, equivalent replacement or modification, etc. made within the scope of the patent application of the application, shall be within the scope of the patent application protection of the application.

Claims

1. A model training method, characterized in that, The model training method comprises: obtaining a training data set, each training data in the training data set comprising a training image and corresponding annotation information of the training image, the corresponding annotation information of the training image comprising corresponding annotation retrieval information of the training image, the corresponding annotation retrieval information of the training image being used to indicate clothes in a clothes database corresponding to clothes in the training image; inputting each training image into a clothes detection model to obtain corresponding predicted detection information of each training image; inputting each training image into a clothes segmentation model to obtain corresponding predicted segmentation information of each training image; inputting each training image, corresponding predicted detection information and predicted segmentation information thereof into a preset retrieval network to obtain corresponding predicted retrieval information of each training image; training the preset retrieval network by using corresponding annotation retrieval information of each training image and corresponding predicted retrieval information of each training image to obtain a clothes retrieval model; the preset retrieval network comprises a ResNet50 network and a BNNeck network; the inputting each training image, corresponding predicted detection information and predicted segmentation information thereof into the preset retrieval network to obtain corresponding predicted retrieval information of each training image comprises: for each training image, the following processing is performed: inputting the training image into the ResNet50 network to obtain corresponding image residual feature of the training image; inputting the corresponding image residual feature of the training image into a pooling layer to obtain corresponding pooling feature of the training image; inputting the corresponding pooling feature of the training image into the BNNeck network to obtain corresponding normalized feature of the training image; performing retrieval by using the corresponding normalized feature, predicted detection information and predicted segmentation information of the training image to retrieve corresponding clothes of the training image from the clothes database as the corresponding predicted retrieval information of the training image.

2. The model training method of claim 1, wherein, The corresponding annotation information of the training image further comprises corresponding annotation detection information of the training image, the corresponding annotation detection information of the training image comprising corresponding annotation classification information and annotation detection box regression information of the training image, and the model training method further comprises: training a preset detection network by using each training image and corresponding annotation detection information thereof to obtain a clothes detection model, the preset detection network comprising a detection RPN network, a ResNeXt50 network and first to third detection head structures; wherein the training the preset detection network by using each training image and corresponding annotation detection information thereof to obtain the clothes detection model comprises: for each training image, the following processing is performed: inputting the training image into the detection RPN network to obtain corresponding ROI information of the training image; inputting the training image into the ResNeXt50 network to obtain a plurality of feature maps corresponding to the training image; constructing a feature pyramid by using the plurality of feature maps corresponding to the training image to obtain a plurality of feature maps corresponding to the training image arranged in order of feature map size; inputting the ROI information corresponding to the training image and the multiple feature maps corresponding to the training image and arranged in order of feature map size into the first detection head structure to obtain first classification information and first bounding box regression information corresponding to the training image; inputting the first bounding box regression information corresponding to the training image and the multiple feature maps corresponding to the training image and arranged in order of feature map size into a second detection head structure to obtain second classification information and second bounding box regression information corresponding to the training image; inputting the second bounding box regression information corresponding to the training image and the multiple feature maps corresponding to the training image and arranged in order of feature map size into the third detection head structure to obtain third classification information and third bounding box regression information corresponding to the training image; training the preset detection network by using the labeled classification information, the labeled bounding box regression information, the third classification information and the third bounding box regression information corresponding to each of the training images to obtain the clothing detection model.

3. The model training method of claim 2, wherein, The first detection head structure to the third detection head structure are detection head structures of the same structure. Each detection head structure includes a detection ROI Align layer, a detection classification branch and a detection regression branch, the detection classification branch includes two fully connected layers, and the detection regression branch includes two convolutional layers and one fully connected layer.

4. The model training method of claim 1, wherein, The labeled information corresponding to the training image further includes labeled segmentation information corresponding to the training image, the labeled segmentation information corresponding to the training image includes labeled segmentation region information corresponding to the training image, and the model training method further includes: training a preset segmentation network by using each of the training images and the labeled segmentation information corresponding thereto to obtain a clothing segmentation model, the preset segmentation network including a feature extraction network, a segmentation RPN network and a first segmentation head structure to a third segmentation head structure; The training of the preset segmentation network by using each of the training images and the labeled segmentation information corresponding thereto to obtain the clothing segmentation model includes: for each of the training images, the following processing is performed: inputting the training image into the feature extraction network to obtain feature extraction information corresponding to the training image; inputting the feature extraction information corresponding to the training image into the segmentation RPN network to obtain RPN positive and negative sample classification scores and RPN coordinate box regression information corresponding to the training image; inputting the feature extraction information, the RPN positive and negative sample classification scores and the RPN coordinate box regression information corresponding to the training image into the first segmentation head structure to obtain first positive and negative sample classification scores and first coordinate box regression information corresponding to the training image, first segmentation region information and first scores of each object category; multiplying the first positive and negative sample classification scores corresponding to the training image and the first scores of each object category, respectively, to obtain first confidence scores of each object category corresponding to the training image; inputting the feature extraction information corresponding to the training image, the first positive and negative sample classification scores, and the first coordinate frame regression information into a second segmentation head structure to obtain second positive and negative sample classification scores and second coordinate frame regression information corresponding to the training image, second segmentation region information, and second scores of each object category; and multiplying the second positive and negative sample classification scores corresponding to the training image and the second scores of each object category to obtain second confidence scores of each object category corresponding to the training image; inputting the feature extraction information corresponding to the training image, the second positive and negative sample classification scores, and the second coordinate frame regression information into the third segmentation head structure to obtain third positive and negative sample classification scores and third coordinate frame regression information corresponding to the training image, third segmentation region information, and third scores of each object category; and multiplying the third positive and negative sample classification scores corresponding to the training image and the third scores of each object category to obtain third confidence scores of each object category corresponding to the training image; training the preset segmentation network by using the labeled segmentation region information corresponding to each training image, the third segmentation region information, and the first confidence score to the third confidence score of each object category corresponding to the training image to obtain the clothing segmentation model.

5. The model training method of claim 4, wherein, The first segmentation head structure to the third segmentation head structure are segmentation head structures of the same structure. Each segmentation head structure includes an RCNN Head, a Mask Head, and a MaskIoU Head, the RCNN Head includes one ROI Align layer and two fully connected layers, the Mask Head includes one ROI Align layer and four convolution layers, and the MaskIoU Head includes four convolution layers and three fully connected layers.

6. The model training method of claim 1, wherein, The BNNeck network includes a normalization layer and a fully connected layer. The inputting of the pooling feature corresponding to the training image into the BNNeck network to obtain the normalized feature corresponding to the training image includes: inputting the pooling feature corresponding to the training image into the normalization layer to obtain the normalized feature corresponding to the training image, wherein the pooling feature corresponding to the training image is used to calculate the first loss value and the second loss value corresponding to the training image; inputting the normalized feature corresponding to the training image into the fully connected layer to obtain the fully connected feature corresponding to the training image, wherein the fully connected feature corresponding to the training image is used to calculate the third loss value corresponding to the training image. The first loss value, the second loss value, and the third loss value corresponding to each training image are used to train the preset retrieval network.

7. The model training method of claim 6, wherein, The fully connected feature corresponding to the training image is also used to calculate a label smoothness value corresponding to the training image. The first loss value, the second loss value, the third loss value, and the label smoothness value corresponding to each training image are used to train the preset retrieval network.

8. The model training method of claim 1, wherein, The model training method further includes: performing random erasing on at least one training image to obtain an augmented image corresponding to the training image as a new training image and store the new training image into the training data set. 9.The method of claim 1, wherein, The training of the preset retrieval network by using the labeled retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image obtains a clothing retrieval model, and the clothing retrieval model comprises the following steps: The preset learning rate strategy is used to train the preset retrieval network by using the labeled retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image, and a clothing retrieval model is obtained; the preset learning rate strategy is used to indicate the learning rate corresponding to each epoch, and the learning rate corresponding to the first epoch to the N1th epoch gradually increases to a first preset learning rate, the learning rate corresponding to the N1+1th epoch to the N2th epoch is a second preset learning rate, the learning rate corresponding to the N2+1th epoch to the N3th epoch is a third preset learning rate, and the learning rate corresponding to the N3+1th epoch to the N4th epoch is a fourth preset learning rate, wherein the first preset learning rate is less than the second preset learning rate, the second preset learning rate is greater than the third preset learning rate, and the third preset learning rate is greater than the fourth preset learning rate.

10. A garment retrieval method characterized by, The clothing retrieval method comprises the following steps: An image to be detected is obtained; The image to be detected is input into the clothing detection model to obtain predicted detection information corresponding to the image to be detected; The image to be detected is input into the clothing segmentation model to obtain predicted segmentation information corresponding to the image to be detected; The image to be detected and the corresponding detection information and segmentation information are input into the clothing retrieval model to obtain predicted retrieval information corresponding to the image to be detected; The clothing retrieval model is obtained by training the model training method according to any one of claims 1-9.

11. A model training apparatus, comprising: The model training device comprises: A data acquisition module is configured to acquire a training data set, each training data in the training data set comprising a training image and labeled information corresponding to the training image, the labeled information corresponding to the training image comprising labeled retrieval information corresponding to the training image, the labeled retrieval information corresponding to the training image being used to indicate a clothing in a clothing database corresponding to a clothing in the training image; A prediction detection module is configured to input each training image into a clothing detection model to obtain predicted detection information corresponding to each training image; A prediction segmentation module is configured to input each training image into a clothing segmentation model to obtain predicted segmentation information corresponding to each training image; A prediction retrieval module is configured to input each training image and the corresponding predicted detection information and predicted segmentation information into a preset retrieval network to obtain predicted retrieval information corresponding to each training image; A retrieval training module is configured to train the preset retrieval network by using the labeled retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image to obtain a clothing retrieval model; The preset retrieval network comprises a ResNet50 network and a BNNeck network; The training of the preset retrieval network by using the labeled retrieval information corresponding to each training image and the predicted retrieval information corresponding to each training image obtains a clothing retrieval model, and the clothing retrieval model comprises the following steps: For each of the training images, the following processing is performed: inputting the training image into the ResNet50 network to obtain a graph residual feature corresponding to the training image; inputting the graph residual feature corresponding to the training image into a pooling layer to obtain a pooling feature corresponding to the training image; inputting the pooling feature corresponding to the training image into the BNNeck network to obtain a normalized feature corresponding to the training image; performing retrieval using the normalized feature corresponding to the training image, predicted detection information and predicted segmentation information to retrieve a garment corresponding to the training image from the garment database as predicted retrieval information corresponding to the training image.

12. A garment retrieval device, characterized by, The garment retrieval device comprises: an image acquisition module configured to acquire a to-be-detected image; a detection information module configured to input the to-be-detected image into a garment detection model to obtain detection information corresponding to the to-be-detected image; a segmentation information module configured to input the to-be-detected image into a garment segmentation model to obtain segmentation information corresponding to the to-be-detected image; a retrieval result module configured to input the to-be-detected image and the detection information and the segmentation information corresponding thereto into a garment retrieval model to obtain a retrieval result corresponding to the to-be-detected image, wherein the garment retrieval model is trained using the model training method of any one of claims 1-9.

13. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the model training method of any one of claims 1-9 or the steps of the garment retrieval method of claim 10 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program or a garment retrieval model; The computer program is executed by the processor to implement the steps of the model training method of any one of claims 1-9 or the steps of the garment retrieval method of claim 10; The garment retrieval model is trained using the model training method of any one of claims 1-9.

Images