A clothing identification method, device, terminal and storage medium

By acquiring multiple images of different clothing perspectives, performing perspective recognition and feature extraction, and integrating multi-view characteristics, the problem of low accuracy of clothing recognition in the prior art is solved, and higher recognition accuracy is achieved.

CN114863405BActive Publication Date: 2024-11-26BEIJING IQIYI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210356711.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-06
Publication Date
2024-11-26
Estimated Expiration
2042-04-06

AI Technical Summary

Technical Problem

In the prior art, the accuracy of clothing recognition is low, and it is impossible to effectively identify clothing information in the image.

Method used

By obtaining multiple clothing images from different clothing perspectives, clothing perspective recognition and feature extraction are carried out, and multi-view clothing features are integrated to improve recognition accuracy.

Benefits of technology

It improves the accuracy of clothing recognition and can more accurately identify the style and information of the target clothing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114863405B_ABST
    Figure CN114863405B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a clothing recognition method, device, terminal and storage medium, wherein the method includes: obtaining multiple clothing images containing different clothing perspectives; the target clothing to be identified in the multiple clothing images is the same clothing style; performing clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image; performing feature extraction on each clothing image to obtain a first clothing feature; fusing the first clothing features and clothing perspectives corresponding to the multiple clothing images to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features. The embodiment of the present invention identifies the target clothing to be identified by using the multi-perspective clothing features as a whole, which can improve the accuracy of clothing recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a clothing recognition method, device, terminal and storage medium. Background Art

[0002] Clothing is an indispensable item in people's daily life. With the emergence of online shopping platforms, people often buy clothing on online shopping platforms, which greatly facilitates people's lives. In order to further provide convenience for people, clothing recognition technology can be used to identify clothing in images provided by people, and provide people with relevant clothing information. However, in the prior art, the clothing in the image is identified to provide relevant clothing information, and the information of the relevant clothing can only be determined by a single image recognition method, and the accuracy rate is low. Summary of the invention

[0003] The present invention provides a clothing recognition method, device, terminal and computer-readable storage medium, so as to solve the problem of low clothing recognition accuracy in the prior art to a certain extent.

[0004] According to a first aspect of the present invention, a clothing recognition method is provided, the method comprising:

[0005] Acquire multiple clothing images containing different clothing perspectives; the target clothing to be identified in the multiple clothing images is the same clothing style;

[0006] Performing clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image;

[0007] Extracting features from each clothing image to obtain a first clothing feature;

[0008] The first clothing features and clothing perspectives corresponding to the plurality of clothing images are fused to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain a recognition result of the target clothing to be identified based on the multi-perspective clothing features.

[0009] Optionally, extracting features from the clothing image to obtain a first clothing feature includes:

[0010] Performing human body key point detection and clothing key point detection on the clothing image to obtain human body key points and clothing key points corresponding to the clothing image;

[0011] Extracting global features from the clothing image to obtain global features corresponding to the clothing image; the global features are used to characterize the style of the target clothing to be identified in the clothing image;

[0012] The global feature is processed according to the human body key points and the clothing key points to obtain a first clothing feature of the clothing image.

[0013] Optionally, the fusing of the first clothing features and clothing perspectives corresponding to the plurality of clothing images to obtain multi-perspective clothing features corresponding to the target clothing to be identified includes:

[0014] A feature sequence composed of the first clothing features corresponding to the multiple clothing images and a perspective coding sequence composed of the perspective coding corresponding to the clothing perspectives of the multiple clothing images are input into a pre-trained transformer structure, and the transformer structure fuses the feature sequence and the perspective coding sequence to output multi-perspective clothing features corresponding to the target clothing to be identified.

[0015] Optionally, performing clothing perspective recognition on the clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image includes:

[0016] Detecting posture information of a target garment to be identified in the garment image;

[0017] The viewing angle corresponding to the posture information is determined as the clothing viewing angle corresponding to the target clothing to be identified in the clothing image.

[0018] Optionally, performing clothing perspective recognition on the clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image includes:

[0019] The clothing image is input into a perspective recognition model, and the perspective recognition model outputs a clothing perspective corresponding to a target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with a clothing perspective of the first training clothing.

[0020] Optionally, acquiring a plurality of clothing images from different clothing perspectives includes:

[0021] Extracting multiple frame images to be identified at different moments from the video to be identified;

[0022] Detecting a clothing area in the frame image to be identified, and extracting image features of the clothing area;

[0023] Clustering the plurality of frame images to be identified based on image features corresponding to each of the frame images to be identified;

[0024] A plurality of frame images to be identified that are obtained after clustering and belong to the same category are used as the clothing images.

[0025] Optionally, obtaining the recognition result of the target clothing to be recognized based on the multi-view clothing features includes:

[0026] Calculating the similarity between the multi-view clothing feature and each preset multi-view clothing feature in the database;

[0027] A target preset multi-view clothing feature corresponding to a similarity that meets a preset requirement is determined, and clothing information corresponding to the target preset multi-view clothing feature is used as a recognition result of the target clothing to be recognized.

[0028] According to a second aspect of the present invention, there is provided a clothing recognition device, the device comprising:

[0029] The first acquisition module is used to acquire a plurality of clothing images including different clothing perspectives; the target clothing to be identified in the plurality of clothing images is of the same clothing style;

[0030] A view angle recognition module, used to perform clothing view angle recognition on each clothing image, and determine the clothing view angle corresponding to the target clothing to be recognized in the clothing image;

[0031] A feature extraction module, used for extracting features from each clothing image to obtain a first clothing feature;

[0032] The multi-view recognition module is used to fuse the first clothing features and clothing perspectives corresponding to the multiple clothing images to obtain the multi-view clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-view clothing features.

[0033] Optionally, the feature extraction module includes:

[0034] A key point detection module, used to perform human key point detection and clothing key point detection on the clothing image to obtain human key points and clothing key points corresponding to the clothing image;

[0035] A global feature extraction module, used to extract global features from the clothing image to obtain global features corresponding to the clothing image; the global features are used to characterize the style of the target clothing to be identified in the clothing image;

[0036] The first clothing feature determination module is used to process the global feature according to the human body key points and clothing key points to obtain the first clothing feature of the clothing image.

[0037] Optionally, the multi-perspective recognition module is used to input a feature sequence composed of first clothing features corresponding to multiple clothing images and a perspective code sequence composed of perspective codes corresponding to clothing perspectives of multiple clothing images into a pre-trained transformer structure, and the transformer structure fuses the feature sequence and the perspective code sequence to output multi-perspective clothing features corresponding to the target clothing to be identified.

[0038] Optionally, the viewing angle recognition module includes:

[0039] A posture information determination module, used to detect the posture information of the target clothing to be identified in the clothing image;

[0040] The module for determining a viewing angle based on the posture information is used to determine the viewing angle corresponding to the posture information as the clothing viewing angle corresponding to the target clothing to be identified in the clothing image.

[0041] Optionally, the viewing angle recognition module includes:

[0042] A model-based perspective determination module is used to input the clothing image into a perspective recognition model, and the perspective recognition model outputs a clothing perspective corresponding to a target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with a clothing perspective of the first training clothing.

[0043] Optionally, the first acquisition module includes:

[0044] The module for extracting frames of images to be identified is used to extract multiple frames of images to be identified at different moments from the video to be identified;

[0045] An image feature extraction module, used to detect the clothing area in the frame image to be identified and extract the image features of the clothing area;

[0046] An image clustering module, used for clustering a plurality of the frame images to be identified based on image features corresponding to each of the frame images to be identified;

[0047] A clustering result-based determination module is used to use a plurality of to-be-identified frame images belonging to the same category obtained after clustering as the clothing images.

[0048] Optionally, the multi-view recognition module includes:

[0049] A similarity calculation module, used to calculate the similarity between the multi-view clothing feature and each preset multi-view clothing feature in the database;

[0050] The recognition result determination module is used to determine the target preset multi-view clothing features corresponding to the similarity that meets the preset requirements, and use the clothing information corresponding to the target preset multi-view clothing features as the recognition result of the target clothing to be identified.

[0051] According to a third aspect of the present invention, a terminal is provided, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, any of the above-mentioned clothing recognition methods is implemented.

[0052] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any of the above-mentioned clothing recognition methods is implemented.

[0053] Compared with the background technology, the embodiments of the present invention have the following advantages:

[0054] In the embodiment of the present invention, multiple clothing images containing different clothing perspectives are obtained; the target clothing to be identified in the multiple clothing images is of the same clothing style; clothing perspective identification is performed on each clothing image to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image; feature extraction is performed on each clothing image to obtain a first clothing feature; the first clothing features and clothing perspectives corresponding to the multiple clothing images are integrated to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features. In the process of clothing recognition on the clothing images, the embodiment of the present invention combines the first clothing features and clothing perspectives of each clothing image to determine the multi-perspective clothing features corresponding to the target clothing to be identified in the clothing image, and then identifies the target clothing to be identified based on the multi-perspective clothing features. Compared with the prior art that can only perform clothing recognition on clothing features of a single perspective, the embodiment of the present invention identifies the target clothing to be identified as a whole using multi-perspective clothing features, which can improve the accuracy of clothing recognition.

[0055] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0057] Figure 1 is a schematic flow chart of steps of a clothing recognition method provided by an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of a model structure corresponding to an exemplary clothing recognition method in an embodiment of the present invention;

[0059] Figure 3 It is a schematic structural block diagram of a clothing recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0061] In view of the problem of low accuracy in clothing recognition in the prior art, an embodiment of the present invention provides a clothing recognition method, which obtains multiple clothing images of target clothing of the same clothing style to be processed from different clothing perspectives; performs clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image; performs feature extraction on each clothing image to obtain the corresponding first clothing feature; and fuses the first clothing features and clothing perspective corresponding to the multiple clothing images to obtain multi-perspective clothing features corresponding to the target clothing to be recognized, so as to obtain the recognition result of the target clothing to be recognized based on the multi-perspective clothing features. The embodiment of the present invention recognizes the target clothing to be recognized by using the multi-perspective clothing features as a whole, which can improve the accuracy of clothing recognition.

[0062] Figure 1 is a flowchart of an exemplary clothing recognition method provided by an embodiment of the present invention. The method can be applied to electronic devices to accurately recognize target clothing to be recognized. The electronic device can be hardware or software. When the electronic device is hardware, it can be implemented as a cluster consisting of multiple servers or terminal devices, or a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. Figure 1 As shown, the clothing recognition method provided in this embodiment may include the following steps:

[0063] Step 101, obtaining a plurality of clothing images including different clothing perspectives; the target clothing to be identified in the plurality of clothing images is of the same clothing style.

[0064] The acquired clothing image is the clothing image to be identified. The clothing style can be used as the identity of the clothing, that is, the clothing of the same clothing style has the same appearance, and at most there is a difference in size. The clothing perspective refers to the perspective of the camera shooting the clothing. For example, the perspective obtained from the front of the clothing is the front clothing perspective, and the perspective obtained from the left side of the clothing is the left clothing perspective.

[0065] In this embodiment, the acquired multiple clothing images need to include different clothing perspectives. It can be understood that the acquired multiple clothing images involve at least two clothing perspectives of the target clothing to be identified. For example, the clothing perspective corresponding to the target clothing to be identified in at least one clothing image is the front clothing perspective, and at the same time, the clothing perspective corresponding to the target clothing to be identified in at least one clothing image is the left clothing perspective.

[0066] The process of acquiring multiple clothing images may include extracting multiple frame images to be identified at different moments from the video as clothing images to be identified, or receiving multiple clothing images sent by the user as clothing images to be identified, or acquiring multiple clothing images to be identified from a specified image library, etc. The source of the clothing images to be identified is not specifically limited. The target clothing to be identified in the multiple clothing images to be identified should belong to the same clothing style. It can be understood that when the target clothing corresponding to the acquired clothing images involves different clothing styles, the acquired clothing images need to be clustered to obtain multiple clothing images of the same clothing style as the clothing images to be identified, and the multiple clothing images to be identified should correspond to different clothing perspectives of the target clothing to be identified.

[0067] The target clothing can be considered as the main clothing in the clothing image to be identified. Generally, when the clothing image to be identified contains only one type of clothing, the target clothing to be identified is the clothing contained in the clothing image; when the clothing image to be identified contains multiple types of clothing, the target clothing to be identified can be the clothing that occupies the most pixels in the clothing image, or it can be the clothing with the highest clarity in the image, or it can be the clothing located in the foreground area of ​​the clothing image, etc., without specific limitation.

[0068] Step 102 , performing clothing perspective recognition on each clothing image, and determining the clothing perspective corresponding to the target clothing to be recognized in the clothing image.

[0069] After determining a plurality of clothing images to be identified, it is necessary to perform clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image.

[0070] Step 103: extract features from each clothing image to obtain a first clothing feature.

[0071] After determining the multiple clothing images to be identified, it is necessary to extract features from each clothing image. Since the target clothing to be identified in the clothing image has a corresponding clothing perspective, and the clothing perspective is related to the posture information of the target clothing to be identified, the first clothing feature obtained by feature extraction of the clothing image can be understood as the clothing feature enhanced by the posture information of the target clothing. For example, if the target clothing to be identified is a top, and the corresponding clothing perspective is the left clothing perspective, the first clothing feature extracted is more of a feature of the sleeve part of the top, which is a local feature related to the clothing perspective.

[0072] Step 104 , fusing the first clothing features and the viewing angles corresponding to the plurality of clothing images to obtain multi-view clothing features corresponding to the target clothing to be identified, so as to obtain the identification result of the target clothing to be identified based on the multi-view clothing features.

[0073] Through the above steps 102 and 103, the first clothing features and clothing perspectives corresponding to each clothing image can be obtained. By fusing the first clothing features and clothing perspectives corresponding to all clothing images containing the target clothing to be identified, the multi-perspective clothing features corresponding to the target clothing to be identified can be obtained. The multi-perspective clothing features can be understood as the fusion of clothing features of the target clothing to be identified under multiple clothing perspectives, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features; that is, based on the multi-perspective clothing features, the clothing information of the same or similar style as the target clothing to be identified is determined.

[0074] The embodiment of the present invention obtains multiple clothing images of different clothing perspectives; the target clothing to be identified in the multiple clothing images is of the same clothing style; the clothing perspective is identified for each clothing image to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image; the feature extraction is performed on each clothing image to obtain a first clothing feature; the first clothing features and clothing perspectives corresponding to the multiple clothing images are integrated to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features. In the process of clothing recognition for the clothing images to be identified, the embodiment of the present invention combines the first clothing features and clothing perspectives of each clothing image to determine the multi-perspective clothing features corresponding to the target clothing to be identified in the clothing image, and then identifies the target clothing to be identified based on the multi-perspective clothing features. Compared with the prior art that can only perform clothing recognition on clothing features of a single perspective, the embodiment of the present invention identifies the target clothing to be identified by using multi-perspective clothing features as a whole, which can improve the accuracy of clothing recognition.

[0075] Next, the clothing recognition method in this exemplary embodiment will be further described.

[0076] In step 101, a plurality of clothing images including different clothing perspectives are obtained; the target clothing to be identified in the plurality of clothing images is of the same clothing style.

[0077] Among them, the process of obtaining multiple clothing images to be identified may include extracting multiple frame images to be identified at different moments from the video as clothing images to be identified, or receiving multiple clothing images sent by the user as clothing images to be identified, or obtaining multiple clothing images to be identified from a specified image library, etc. The target clothing to be identified in the multiple clothing images to be identified should belong to the same clothing style, and the multiple clothing images to be identified should involve multiple clothing perspectives of the target clothing to be identified. It can be understood that there are at least two clothing perspectives of the target clothing to be identified in the multiple clothing images to be identified, but it does not exclude the situation that two or more clothing images to be identified correspond to the same clothing perspective of the target clothing to be identified. For example, when there are five clothing images to be identified (such as clothing image 1, clothing image 2, clothing image 3, clothing image 4, and clothing image 5), the clothing perspectives corresponding to the target clothing in the five clothing images may be different, that is, the five clothing images correspond to five different clothing perspectives (such as clothing image 1 corresponds to clothing perspective 1, clothing image 2 corresponds to clothing perspective 2, clothing image 3 corresponds to clothing perspective 3, clothing image 4 corresponds to clothing perspective 4, and clothing image 5 corresponds to clothing perspective 5); the clothing perspectives corresponding to the target clothing in the five clothing images may be partially the same, that is, the five clothing images correspond to less than five different clothing perspectives (such as clothing image 1 corresponds to clothing perspective 1, clothing image 2 corresponds to clothing perspective 2, clothing image 3 corresponds to clothing perspective 3, and clothing image 4 and clothing image 5 both correspond to clothing perspective 4).

[0078] When there are multiple clothing images to be identified corresponding to the target clothing to be identified for the same clothing perspective, one of the clothing images to be identified can be randomly selected to represent the clothing image to be identified from the clothing perspective, or one of the clothing images to be identified can be selected based on specified rules to represent the clothing image to be identified from the clothing perspective, wherein the specified rules can be the highest clarity, the largest number of pixels corresponding to the target clothing to be identified, etc.; the clothing image obtained by merging multiple clothing images to be identified from the same clothing perspective can also be used as the clothing image to be identified representing the perspective, and so on.

[0079] In one example, the process of acquiring multiple clothing images including different clothing perspectives may include:

[0080] Extracting multiple frame images to be identified at different moments from the video to be identified;

[0081] Detecting the clothing area in the frame image to be identified and extracting the image features of the clothing area;

[0082] Clustering the multiple frame images to be identified based on the image features corresponding to each frame image to be identified;

[0083] The multiple frame images to be identified that belong to the same category and are obtained after clustering are used as clothing images to be identified.

[0084] Among them, the video to be identified is a video for clothing recognition. Exemplarily, the video to be identified can be a video of a fashion show, a TV series, a movie, a variety show, etc. There should be people and / or clothing in the video so that the clothing recognition method provided by the embodiment of the present invention can be used to identify the clothing appearing in the video, and then provide the user with relevant information about the clothing. The relevant information may include other display pictures related to the clothing, size information, matching recommendations, purchase links, etc. of the clothing.

[0085] Extract multiple frame images to be identified at different moments from the video to be identified, that is, extract multiple frame images at different moments from the video to be identified according to a set rule. Specifically, multiple frame images to be identified can be extracted from the video to be identified at a preset time interval, or multiple frame images to be identified corresponding to the lens contained in the video to be identified can be extracted from the video to be identified according to the lens contained in the video to be identified, and so on. Exemplarily, multiple frame images to be identified can be extracted from the video to be identified at a preset time interval during the playback of the video to be identified, and the frame images to be identified can be video frames that are being played or have been played.

[0086] After extracting the frame image to be identified from the video to be identified, each frame image to be identified can be subjected to image processing, including detecting the clothing area in the frame image to be identified and extracting the image features of the clothing area. The image features can be understood as the global features of the target clothing to be identified in the frame image to be identified, and can represent the clothing style of the target clothing to be identified. Generally, when the frame image to be identified contains only one type of clothing, the target clothing to be identified is the clothing contained in the frame image to be identified; when the frame image to be identified contains multiple types of clothing, the target clothing to be identified can be the clothing that occupies the most pixels in the frame image to be identified, or it can also be the clothing with the highest definition in the frame image to be identified, or it can also be the clothing located in the foreground area of ​​the frame image to be identified, etc.

[0087] Clustering is the process of dividing a collection of physical or abstract objects into multiple classes consisting of similar objects. It can be understood that the process of clustering multiple frames to be identified based on the image features corresponding to each frame image to be identified is to classify the frames to be identified containing the same clothing style into the same class.

[0088] After clustering, one or more types of frame images to be identified can be obtained. The frame images to be identified belonging to the same type are used as clothing images to be identified.

[0089] For example, when extracting a frame image to be identified from a video to be identified, each time a frame image to be identified is extracted, the clothing area in the frame image to be identified is detected, and the image features of the clothing area are extracted to determine the category to which the frame image to be identified belongs. Alternatively, at a certain time interval, the clothing areas of all frame images to be identified extracted within the time interval are detected, and the image features of the clothing area are extracted, and the frame images to be identified extracted within the same time interval are uniformly clustered to determine the category to which each frame image to be identified belongs.

[0090] In another example, the process of acquiring multiple clothing images to be identified may include:

[0091] Receive multiple images input by the user;

[0092] Detect clothing areas in the image and extract image features of the clothing areas;

[0093] Clustering multiple images based on image features corresponding to each image;

[0094] The multiple images belonging to the same category obtained after clustering are used as clothing images to be identified.

[0095] In this example, an input interface of clothing images to be identified can be displayed to the user, and the user selects the image to be identified and inputs it into the input interface. After detecting multiple images input by the user, or after detecting an instruction to start identification, the clothing area of ​​the multiple images input in the input interface is detected, and the image features of the clothing area are extracted. If there are multiple types of image features corresponding to the multiple images input in the input interface, that is, the styles of the target clothing to be identified in each input image are different, that is, the image features of the multiple images are inconsistent, then the images are clustered to obtain images of the same category (i.e., the same clothing style), and the images belonging to the same category are used as clothing images to be identified, and the subsequent clothing identification steps are performed on the images of the same category. Among them, the clustering process can be referred to the description in the previous text, and will not be repeated here.

[0096] Optionally, the input interface may further include a camera start control. When the user triggers the camera start control, the camera may be started and the image captured by the camera may be input into the input interface. When the camera captures a video, multiple images to be identified may be determined from the captured video in a manner similar to extracting multiple frames of images to be identified at different moments from the video to be identified.

[0097] In order to improve the accuracy of clothing recognition, in an embodiment of the present invention, the clothing images to be recognized should include images of the same clothing style from different clothing perspectives. Taking the example that the clothing images to be recognized are obtained by processing the images input by the user, the input interface may also include prompt information for prompting the user to input images of the same clothing style from different clothing perspectives. Alternatively, after clustering processing, it is also possible to check whether each type of image includes images from different clothing perspectives. If not, prompt information may be displayed in the interface to prompt the user to supplement the input of images from other perspectives.

[0098] It should be noted that the above-mentioned process of extracting the image features of the clothing area can also be performed by taking the frame image to be identified as the input of the clothing global feature extraction model, and the clothing global feature extraction model processes the frame image to be identified and outputs the global features corresponding to the frame image to be identified, which are the image features of the clothing area. The training process of the clothing global feature extraction model can be found in the following description.

[0099] In step 102, clothing perspective recognition is performed on each clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image.

[0100] In this embodiment, clothing perspective recognition may be performed for each clothing image to be identified, so as to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image.

[0101] Optionally, in an example, the process of performing clothing perspective recognition on each clothing image and determining the clothing perspective corresponding to the target clothing to be recognized in the clothing image may include:

[0102] Detecting the posture information of the target clothing to be identified in the clothing image;

[0103] The viewing angle corresponding to the posture information is determined as the clothing viewing angle corresponding to the target clothing to be identified in the clothing image.

[0104] Among them, detecting the posture information of the target clothing to be identified is to detect the shooting angle of the target clothing to be identified in the clothing image, and the posture information of the target clothing to be identified can be determined by detecting the key points of the target clothing to be identified. For example, when the detected key points of the target clothing to be identified are consistent with the key points corresponding to the front posture information, it means that the posture information of the target clothing to be identified is the front posture information. The posture information of the target clothing to be identified can also be determined by detecting the posture information of the human body associated with the target clothing to be identified. The posture information of the human body can be determined by detecting the key points of the human body area. After obtaining the posture information, the corresponding angle of view can be determined according to the posture information, that is, the clothing angle of view corresponding to the target clothing to be identified.

[0105] Exemplarily, the correspondence between the key points of the human body area or the key points of the clothing area and the clothing posture information, as well as the correspondence between the clothing posture information and the viewing angle can be determined in advance; after detecting the key points of the human body area or the key points of the target clothing to be identified, the posture information of the target clothing to be identified can be determined based on the correspondence between the predetermined key points and the posture information, and then the clothing perspective of the target clothing to be identified can be determined based on the correspondence between the predetermined clothing posture information and the viewing angle.

[0106] For example, it is predetermined that the key points of the back area of ​​the human body correspond to the clothing posture information as the back side, and the perspective corresponding to the clothing posture information as the back side is the rear perspective. When the key points of the human body area are detected to be the key points of the back area of ​​the human body, the clothing posture information can be determined to be the back side, and then the perspective corresponding to the clothing posture information can be determined to be the rear perspective, and finally the clothing perspective of the target clothing is the rear clothing perspective.

[0107] In another example, the process of performing clothing perspective recognition on a clothing image and determining the perspective corresponding to a target clothing to be recognized in the clothing image may include:

[0108] The clothing image is input into the perspective recognition model, and the perspective recognition model outputs the clothing perspective corresponding to the target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with the clothing perspective of the first training clothing.

[0109] In this embodiment, a perspective recognition model can be pre-constructed. Specifically, the initial perspective recognition model can be trained using the first training sample to obtain a perspective recognition model for identifying the clothing perspective of the target clothing to be identified in the clothing image. The first training sample is an image annotated with the clothing perspective containing the first training clothing. That is, the first training sample can include multiple first training images, the main clothing in each first training image is the first training clothing, and each first training image is annotated with the clothing perspective of the corresponding first training clothing.

[0110] After the perspective recognition model is obtained, each clothing image is used as the input of the perspective recognition model, and the perspective recognition model outputs the clothing perspective corresponding to the clothing image.

[0111] In step 103, feature extraction is performed on each clothing image to obtain a first clothing feature.

[0112] After determining multiple clothing images to be identified, it is necessary to perform feature extraction on each clothing image, that is, extract clothing features enhanced by the posture information of the target clothing to be identified in the clothing image, and determine the extracted features as the first clothing features.

[0113] In this embodiment, the first clothing feature is a clothing feature enhanced with posture information. It can be understood that the first clothing feature is a clothing feature obtained after multimodal information fusion based on the human body key points corresponding to the target clothing to be identified, the target clothing key points to be identified and the global features of the clothing image.

[0114] Exemplarily, the process of extracting features from the clothing image to obtain the first clothing feature may include:

[0115] Performing human body key point detection and clothing key point detection on the clothing image to obtain human body key points and clothing key points corresponding to the clothing image;

[0116] Extract global features of clothing images to obtain global features corresponding to the clothing images;

[0117] The global features are processed according to the key points of the human body and the key points of the clothing to obtain the first clothing features of the clothing image.

[0118] In this example, human body area detection and clothing area detection can be performed on each clothing image to be identified to determine the human body area and clothing area in the clothing image, and then human body key point detection is performed on the human body area, and clothing key point detection is performed on the clothing area to obtain the human body key points and clothing key points corresponding to the clothing image.

[0119] Among them, the process of extracting global features from clothing images to obtain global features corresponding to the clothing images may include: using the clothing images as inputs of clothing global feature extraction models, processing the clothing images by the clothing global feature extraction models, and outputting global features corresponding to the clothing images. Specifically, by collecting clothing data of different styles, using a deep convolutional network as the structure of the clothing global feature extraction model, and using an image classification loss function as a metric, a clothing global feature extraction model can be trained, and then global features of clothing images are extracted based on the trained clothing global feature extraction model to obtain implicit spatial feature vectors, that is, global features of the target clothing to be identified.

[0120] After obtaining the human body key points, clothing key points and global features of the clothing image, a fully connected layer can be used to map the feature vectors corresponding to the human body key points, the feature vectors corresponding to the clothing key points and the feature vectors corresponding to the global features to the vector space of the same dimension, and the vectors mapped to the vector space of the same dimension are added to achieve multimodal information fusion to obtain the clothing features enhanced with posture information, that is, the first clothing features.

[0121] Among them, in the process of mapping human key points, clothing key points and global features to the vector space of the same dimension, for example, the coordinates of the invisible key points in the human key points and clothing key points can be marked as [-1, -1], and the typical feature dimension after mapping can be 512 dimensions.

[0122] In step 104, the first clothing features and clothing perspectives corresponding to the plurality of clothing images are fused to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain a recognition result of the target clothing to be identified based on the multi-perspective clothing features.

[0123] After obtaining the first clothing feature and clothing perspective corresponding to each clothing image, the first clothing features and clothing perspective corresponding to all clothing images to be identified are fused, so that the multi-perspective clothing features corresponding to the target clothing to be identified can be obtained. The multi-perspective clothing features can be understood as the fusion of clothing features of the target clothing to be identified under multiple perspectives, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features; that is, based on the multi-perspective clothing features, the clothing information of the same or similar style as the target clothing to be identified is determined.

[0124] Exemplarily, the process of fusing the first clothing features and perspectives corresponding to the plurality of clothing images to obtain the multi-perspective clothing features corresponding to the target clothing to be identified may include:

[0125] The feature sequence composed of the first clothing features corresponding to multiple clothing images and the perspective coding sequence composed of the perspective coding corresponding to the clothing perspectives of multiple clothing images are input into a pre-trained transformer structure (Transformer). The transformer structure fuses the feature sequence and the perspective coding sequence to output the multi-perspective clothing features corresponding to the target clothing to be identified.

[0126] In this example, the first clothing features corresponding to multiple clothing images are used as the input feature sequence of the transformer structure, and the perspective coding sequence composed of the perspective coding corresponding to the clothing perspectives of multiple clothing images is used as the input attention mask sequence of the transformer structure; the corresponding feature vector is output by the transformer structure, and the multi-perspective clothing features corresponding to the target clothing to be identified are determined according to the feature vector. Among them, the perspective coding corresponding to different clothing perspectives is different, and the perspective coding corresponding to the same clothing perspective is the same.

[0127] The transformer structure in this embodiment may include multiple layers of residual attention coding modules, for example, 4 layers of residual attention coding modules, and the number of taps of each layer of residual attention coding modules may be 8, and the maximum length of the multi-view sequence is 55. When training the transformer structure, the parameters of the transformer structure are trained using a clothing data set, wherein the clothing data set contains clothing images of multiple different perspectives, and each clothing image has corresponding clothing style information (such as clothing style identification), and a classification loss function can be applied to constrain the multi-view clothing features, and the gradient back propagation method is used to optimize the parameters. Exemplarily, when training the transformer structure, considering that the perspective estimation, human body key point estimation, and clothing key point estimation belong to independent calculation modules, the modules used to realize the perspective estimation, human body key point estimation, and clothing key point estimation do not participate in the parameter iteration optimization during the training process. The clothing global feature extraction model can be determined in combination with the computing resources and the clothing global feature performance, and the parameters can be frozen or released for model adjustment.

[0128] In this embodiment, the perspective encodings corresponding to the perspectives of multiple clothing images are used as the input attention mask sequence of the transformer structure, so that the transformer structure outputs multi-perspective clothing features. The obtained multi-perspective clothing features can ensure the accuracy of the recognition results of the target clothing to be identified based on the multi-perspective clothing features.

[0129] Figure 2 FIG. 1 is a schematic diagram of a model structure corresponding to an exemplary clothing recognition method in an embodiment of the present invention. Figure 2 As shown, the clothing recognition method in this embodiment mainly includes extracting clothing features with enhanced posture information for each clothing image to be recognized (i.e., extracting the first clothing feature of each clothing image to be recognized), estimating the perspective of each clothing image to be recognized (i.e., determining the clothing perspective corresponding to each clothing image), and using a clothing feature sequence composed of clothing features with enhanced posture information extracted and a perspective attention mask sequence composed of multi-perspective encodings corresponding to the perspectives of each clothing image as inputs of a transformer structure, and the transformer structure processes the input clothing feature sequence and the perspective attention mask sequence to obtain multi-perspective clothing features corresponding to the target clothing to be recognized in the clothing image to be recognized.

[0130] Specifically, when extracting clothing features enhanced by posture information of clothing images, it is necessary to estimate the human body key points and clothing key points of clothing images, as well as extract clothing global features of clothing images, and then use the obtained human body key points, clothing key points and clothing global features as the input of the corresponding fully connected layer, and map them to the vector space of the same dimension by the fully connected layer to obtain clothing features enhanced by posture information. It should be noted that the input of the fully connected layer includes human body key points and clothing key points, and both human body key points and clothing key points are related to clothing perspective; when the target clothing in the clothing image is not worn on the person, that is, the human body key point result is empty, the clothing features enhanced by posture information can still be obtained, so it can also realize feature fusion and recognition of clothing images that are not worn on people.

[0131] For example, taking the target garment as a white V-neck, bubble sleeves, and back opening top as an example, it can be understood that the global features of the target garment are white, V-neck, bubble sleeves, and back opening. The garment images to be identified may include garment image 1, garment image 2, and garment image 3; wherein garment image 1 is a front view of the target garment, garment image 2 is a back view of the garment image, and garment image 3 is a left view of the garment image. The process of processing using the garment recognition method provided by the present invention is as follows:

[0132] By estimating the perspective of each clothing image, it can be obtained that the clothing perspective of clothing image 1 is the front clothing perspective, the clothing perspective of clothing image 2 is the back clothing perspective, and the clothing perspective of clothing image 3 is the left clothing perspective;

[0133] Extract the first clothing feature of each clothing image, that is, extract the first clothing feature 1 (i.e., V-neck feature) associated with clothing image 1 and the front clothing perspective, the first clothing feature 2 (i.e., back opening feature) associated with clothing image 2 and the back clothing perspective, and the first clothing feature 3 (i.e., bubble sleeve feature) associated with clothing image 3 and the left clothing perspective;

[0134] The first clothing features and the corresponding clothing perspectives of the three clothing images are integrated to obtain the multi-perspective clothing features of the target clothing, that is, a clothing feature sequence is obtained according to feature vector 1 corresponding to the first clothing feature 1 of clothing image 1, feature vector 2 corresponding to the first clothing feature 2 of clothing image 2, and feature vector 3 corresponding to the first clothing feature 3 of clothing image 3, which can be expressed as {feature vector 1, feature vector 2, feature vector 3}; a perspective attention mask sequence is obtained according to perspective code 1 corresponding to the front clothing perspective of clothing image 1, perspective code 2 corresponding to the back clothing perspective of clothing image 2, and perspective code 3 corresponding to the left clothing perspective of clothing image 3, which can be expressed as {perspective code 1, perspective code 2, perspective code 3}; the clothing feature sequence and the perspective attention mask sequence are input into the transformer structure, and the transformer structure outputs the multi-perspective clothing features of the target clothing, which are used to represent the clothing features of the target clothing at different clothing perspectives, including V-neck features at the front clothing perspective, back opening features at the back clothing perspective, and bubble sleeve features at the left clothing perspective. For subsequent identification, it is necessary to identify clothing information including V-neck features from the front clothing perspective, back opening features from the back clothing perspective, and bubble sleeve features from the left clothing perspective. The clothing information can be understood as display pictures, size information, matching recommendations, purchase links, etc. related to the target clothing.

[0135] Furthermore, in an optional embodiment of the present invention, the clothing recognition method may further include:

[0136] Calculating the similarity between the multi-view clothing features and each preset multi-view clothing features in the database;

[0137] The target preset multi-view clothing features corresponding to the similarity that meets the preset requirements are determined, and the clothing information corresponding to the target preset multi-view clothing features is used as the recognition result of the target clothing to be recognized.

[0138] In this embodiment, a plurality of preset multi-perspective clothing features are pre-stored in the database, wherein the preset multi-perspective clothing features can be obtained by processing the preset clothing images in the manner of steps 101-104 above, or can be obtained by receiving multi-perspective clothing features corresponding to the preset clothing images input by relevant personnel; and the preset clothing images can be understood as predetermined clothing images. The preset clothing images can be determined according to the usage scenario. Exemplarily, when the usage scenario is to recommend to the user the commodity information corresponding to the target clothing to be identified, the preset clothing images can be images of each commodity in the commodity library at different perspectives; when the usage scenario is to recommend to the user the matching recommendation information corresponding to the target clothing to be identified, the preset clothing images can be clothing images that appear in matching recommendation related information such as matching recommendation programs or platforms.

[0139] After obtaining the multi-view clothing features of the target clothing to be identified, a similarity calculation is performed between the multi-view clothing features and each preset multi-view clothing features in the database, that is, the similarity between the multi-view clothing features of the target clothing to be identified and each preset multi-view clothing features is calculated; then the preset multi-view clothing features whose similarity meets the preset requirements are determined as the target preset multi-view clothing features, and finally, the target preset clothing corresponding to the target preset multi-view clothing features is used as the recognition result of the target clothing to be identified, that is, it is determined that the target clothing to be identified is of the same style as the target preset clothing, and relevant information of the target preset clothing is displayed to the user, and the relevant information may include other display pictures related to the clothing, size information, matching recommendations, purchase links, etc. of the clothing.

[0140] Exemplarily, during the playback of a video to be identified, when a user pause operation is received, an image containing clothing displayed in the pause interface is determined as a target clothing image; or, during the playback of a video to be identified, an image containing clothing displayed in real time on a page can be determined as a target clothing image; from a plurality of extracted frame images to be identified, a plurality of target frame images to be identified that belong to the same category as the target clothing image are determined, the plurality of target frame images to be identified and the target clothing image are used as clothing images to be identified, and the clothing contained in the target clothing image is determined as the target clothing to be identified; by performing clothing perspective recognition on each clothing image to be identified, the clothing perspective of each clothing image to be identified is determined. The method comprises the following steps: determining a clothing perspective corresponding to a target clothing to be identified in each clothing image to be identified; and performing feature extraction on each clothing image to be identified to obtain a first clothing feature with posture information enhanced. Finally, according to the perspectives corresponding to all clothing images to be identified and the first clothing feature, determining the multi-perspective clothing feature corresponding to the target clothing to be identified, so as to obtain a recognition result of the target clothing to be identified based on the multi-perspective clothing feature. The recognition result includes displaying relevant information about the target clothing to be identified to the user, and specifically may include displaying product information, matching information, size information, purchase links, and other clothing images containing the target clothing to be identified, etc. of the target clothing to be identified.

[0141] The embodiment of the present invention obtains multiple clothing images containing different clothing perspectives; the target clothing to be identified in the multiple clothing images is the same clothing style; the clothing perspective is identified for each clothing image to determine the clothing perspective corresponding to the target clothing to be identified in the clothing image; the feature extraction is performed on each clothing image to obtain a first clothing feature; the first clothing features and clothing perspectives corresponding to the multiple clothing images are integrated to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-perspective clothing features. In the process of clothing recognition for the clothing images to be identified, the embodiment of the present invention combines the first clothing features and perspectives of each clothing image to determine the multi-perspective clothing features corresponding to the target clothing to be identified in the clothing image, and then identifies the target clothing to be identified based on the multi-perspective clothing features. Compared with the prior art that can only perform clothing recognition on clothing features of a single perspective, the embodiment of the present invention identifies the target clothing to be identified by using multi-perspective clothing features as a whole, which can improve the accuracy of clothing recognition.

[0142] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0143] Reference Figure 3 , shows a schematic structural block diagram of a clothing recognition device according to an embodiment of the present invention. Corresponding to the above clothing recognition method embodiment, the clothing recognition device may specifically include the following modules:

[0144] The first acquisition module 301 is used to acquire multiple clothing images including different clothing perspectives; the target clothing to be identified in the multiple clothing images is the same clothing style;

[0145] A view angle recognition module 302 is used to perform clothing view angle recognition on each clothing image, and determine the clothing view angle corresponding to the target clothing to be recognized in the clothing image;

[0146] A feature extraction module 303 is used to extract features from each clothing image to obtain a first clothing feature;

[0147] The multi-view recognition module 304 is used to fuse the first clothing features and clothing perspectives corresponding to multiple clothing images to obtain multi-view clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-view clothing features.

[0148] In an optional embodiment of the present invention, the feature extraction module 303 includes:

[0149] A key point detection module is used to perform human key point detection and clothing key point detection on clothing images to obtain human key points and clothing key points corresponding to the clothing images;

[0150] A global feature extraction module is used to extract global features of clothing images to obtain global features corresponding to the clothing images; the global features are used to characterize the styles of target clothing to be identified in the clothing images;

[0151] The first clothing feature determination module is used to process the global features according to the key points of the human body and the key points of clothing to obtain the first clothing feature of the clothing image.

[0152] In an optional embodiment of the present invention, the multi-perspective recognition module 304 is used to input a feature sequence composed of first clothing features corresponding to multiple clothing images, and a perspective coding sequence composed of perspective coding corresponding to clothing perspectives of multiple clothing images, into a pre-trained transformer structure, and the transformer structure fuses the feature sequence and the perspective coding sequence to output multi-perspective clothing features corresponding to the target clothing to be identified.

[0153] In an optional embodiment of the present invention, the viewing angle identification module 302 includes:

[0154] A posture information determination module, used to detect the posture information of the target clothing to be identified in the clothing image;

[0155] The module for determining a viewing angle based on the posture information is used to determine the viewing angle corresponding to the posture information as the clothing viewing angle corresponding to the target clothing to be identified in the clothing image.

[0156] In an optional embodiment of the present invention, the viewing angle identification module 302 includes:

[0157] A model-based perspective determination module is used to input a clothing image into a perspective recognition model, and the perspective recognition model outputs a clothing perspective corresponding to a target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with a clothing perspective of the first training clothing.

[0158] In an optional embodiment of the present invention, the first acquisition module 301 includes:

[0159] The module for extracting frames of images to be identified is used to extract multiple frames of images to be identified at different moments from the video to be identified;

[0160] An image feature extraction module is used to detect the clothing area in the frame image to be identified and extract the image features of the clothing area;

[0161] An image clustering module, used for clustering a plurality of frame images to be identified based on image features corresponding to each frame image to be identified;

[0162] The module is determined based on the clustering result and is used to use a plurality of frame images to be identified that belong to the same category and are obtained after clustering as clothing images.

[0163] In an optional embodiment of the present invention, the multi-view recognition module 304 includes:

[0164] A similarity calculation module, used to calculate the similarity between the multi-view clothing features and each preset multi-view clothing features in the database;

[0165] The recognition result determination module is used to determine the target preset multi-view clothing features corresponding to the similarity that meets the preset requirements, and use the clothing information corresponding to the target preset multi-view clothing features as the recognition result of the target clothing to be identified.

[0166] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] An embodiment of the present invention also provides a terminal, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the clothing recognition method embodiment described above are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0168] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the clothing recognition method embodiment described above is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0170] It is easy for a person skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an implementation scheme of the present invention. However, due to space limitations, this specification will not describe them in detail here.

[0171] The clothing identification method provided herein is not inherently related to any particular computer, virtual system or other device. Various general systems can also be used with the teachings based on this. Based on the above description, it is obvious that the structure required to construct a system with the scheme of the present invention is obvious. In addition, the present invention is not directed to any specific programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is to disclose the best implementation of the present invention.

[0172] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0173] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0174] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0175] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0176] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the clothing recognition method according to an embodiment of the present invention. The present invention may also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0177] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A clothing recognition method, characterized in that: The method comprises: Acquire multiple clothing images containing different clothing perspectives; the target clothing to be identified in the multiple clothing images is the same clothing style; Performing clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image; Extracting features from each clothing image to obtain a first clothing feature; fusing the first clothing features and clothing perspectives corresponding to the plurality of clothing images to obtain multi-perspective clothing features corresponding to the target clothing to be identified, so as to obtain a recognition result of the target clothing to be identified based on the multi-perspective clothing features; The step of performing clothing perspective recognition on each clothing image to determine the clothing perspective corresponding to the target clothing to be recognized in the clothing image includes: The clothing image is input into a perspective recognition model, and the perspective recognition model outputs a clothing perspective corresponding to a target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with a clothing perspective of the first training clothing.

2. The method according to claim 1, characterized in that: The step of extracting features from each clothing image to obtain a first clothing feature includes: Performing human body key point detection and clothing key point detection on the clothing image to obtain human body key points and clothing key points corresponding to the clothing image; Extracting global features from the clothing image to obtain global features corresponding to the clothing image; the global features are used to characterize the style of the target clothing to be identified in the clothing image; The global feature is processed according to the human body key points and the clothing key points to obtain a first clothing feature of the clothing image.

3. The method according to claim 1, characterized in that: The fusing of the first clothing features and clothing perspectives corresponding to the plurality of clothing images to obtain the multi-perspective clothing features corresponding to the target clothing to be identified includes: A feature sequence composed of the first clothing features corresponding to the multiple clothing images and a perspective coding sequence composed of the perspective coding corresponding to the clothing perspectives of the multiple clothing images are input into a pre-trained transformer structure, and the transformer structure fuses the feature sequence and the perspective coding sequence to output multi-perspective clothing features corresponding to the target clothing to be identified.

4. The method according to claim 1, characterized in that: The acquiring of multiple clothing images from different clothing perspectives includes: Extracting multiple frame images to be identified at different moments from the video to be identified; Detecting a clothing area in the frame image to be identified, and extracting image features of the clothing area; Clustering the plurality of frame images to be identified based on image features corresponding to each of the frame images to be identified; A plurality of frame images to be identified that are obtained after clustering and belong to the same category are used as the clothing images.

5. The method according to claim 1, characterized in that The obtaining of the recognition result of the target clothing to be recognized based on the multi-view clothing features includes: Calculating the similarity between the multi-view clothing feature and each preset multi-view clothing feature in the database; A target preset multi-view clothing feature corresponding to a similarity that meets a preset requirement is determined, and clothing information corresponding to the target preset multi-view clothing feature is used as a recognition result of the target clothing to be recognized.

6. A clothing recognition device, characterized in that: The device comprises: The first acquisition module is used to acquire a plurality of clothing images including different clothing perspectives; the target clothing to be identified in the plurality of clothing images is of the same clothing style; A view angle recognition module, used to perform clothing view angle recognition on each clothing image, and determine the clothing view angle corresponding to the target clothing to be recognized in the clothing image; A feature extraction module, used for extracting features from each clothing image to obtain a first clothing feature; A multi-view recognition module, used for fusing the first clothing features and clothing perspectives corresponding to the plurality of clothing images to obtain the multi-view clothing features corresponding to the target clothing to be identified, so as to obtain the recognition result of the target clothing to be identified based on the multi-view clothing features; The perspective recognition module is specifically used to input the clothing image into a perspective recognition model, and the perspective recognition model outputs the clothing perspective corresponding to the target clothing to be identified in the clothing image; wherein the perspective recognition model is obtained by training an initial perspective recognition model using a first training sample, and the first training sample is an image marked with the clothing perspective of the first training clothing included.

7. A terminal, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the clothing recognition method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the clothing recognition method according to any one of claims 1 to 5 is implemented.