Model training method and device based on clothing transfer, electronic equipment and medium
By generating clothing variation features, the clothing variation of pedestrians in the second type of sample is enriched, which solves the problem of clothing imbalance in the gait recognition model, improves recognition accuracy and reduces training cost.
Patent Information
- Application Number
- CN202310546572.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2025-10-21
- Estimated Expiration
- 2043-05-16
AI Technical Summary
During the training of gait recognition models, the recognition accuracy decreases due to the uneven clothing of pedestrians in the sample, and the actual cost of collecting training data is high.
By generating clothing variation features, the clothing variations of the second type of pedestrians are enriched, balancing the clothing imbalance problem in the training data. The gait recognition model is trained using the gait features of the first and second samples plus the gait features of the third sample.
It improved the recognition accuracy of the gait recognition model, alleviated the problem of clothing imbalance, and reduced training costs.
Smart Images
Figure CN116452921B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of gait recognition, and in particular to a model training method, device, electronic device and medium based on clothing migration. Background Art
[0002] Gait recognition is a method of identifying pedestrians by identifying their walking patterns. Currently, in the process of collecting training data for training gait recognition models, the gait sequence of each sample pedestrian can be collected through a camera. In the actual collection process, some sample pedestrians may change clothes, and the gait sequence of the sample pedestrian is collected separately when the sample pedestrian wears each set of clothes. Some sample pedestrians may only have one set of clothes, so the gait sequence of the sample pedestrian can only be collected when the sample pedestrian wears one set of clothes. This will cause the problem of unbalanced clothing for each sample pedestrian in the training data, which will affect the gait recognition accuracy of the gait recognition model. In addition, if each sample pedestrian is required to change clothes during the actual collection process, the cost of training samples will be high. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide a model training method, device, electronic device and medium based on clothing transfer to alleviate the problem of clothing imbalance in training data, thereby improving the gait recognition accuracy of the gait recognition model.
[0004] In a first aspect, an embodiment of the present application provides a model training method based on clothing transfer, wherein an original training set for training a gait recognition model includes a plurality of first sample gait sequences corresponding to each set of clothing when a first type of sample pedestrians wears different sets of clothing, and a plurality of second sample gait sequences when a second type of sample pedestrians wears the same set of clothing; the method comprises:
[0005] For each of the first-category sample pedestrians, clustering multiple first-category gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category, thereby generating a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing; the first-category gait features are obtained based on the first-category gait sequence;
[0006] After determining the average sample gait feature of each of the sample gait feature sets, for each of the first-category sample pedestrians, calculating the feature difference between any two of the average sample gait features corresponding to the first-category sample pedestrians, to obtain a plurality of clothing change features corresponding to the first-category sample pedestrians;
[0007] For each second sample gait feature of each second-category sample pedestrian, adding the selected preset number of clothing change features to the second sample gait feature to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian; the second sample gait feature is obtained based on the second sample gait sequence;
[0008] The gait recognition model is trained using the first sample gait feature, the second sample gait feature, and the third sample gait feature.
[0009] In combination with the first aspect, an embodiment of the present application provides a first possible implementation of the first aspect, wherein, for each of the first-category sample pedestrians, before clustering multiple first-category gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category and obtaining a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing, the method further includes:
[0010] Each of the first sample gait sequences and the second sample gait sequences contained in the original training set is input into the gait recognition model, and the first sample gait features of each of the first sample gait sequences and the second sample gait features of each of the second sample gait sequences are extracted through the gait recognition model.
[0011] In combination with the first aspect, the embodiment of the present application provides a second possible implementation of the first aspect, wherein, for each second sample gait feature of each second-category sample pedestrian, a preset number of selected clothing change features are added to the second sample gait feature to generate multiple third sample gait features corresponding to the second-category sample pedestrian, including:
[0012] The third sample gait feature corresponding to the second type of sample pedestrian is generated by the following formula:
[0013]
[0014] in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, It is the third sample gait feature of the second type of sample pedestrian.
[0015] In combination with the first aspect, an embodiment of the present application provides a third possible implementation of the first aspect, wherein the original training set also includes a first identity label for each sample pedestrian of the first category and a second identity label for each sample pedestrian of the second category; and training the gait recognition model using the first sample gait features, the second sample gait features, and the third sample gait features includes:
[0016] Inputting each of the first sample gait features, the second sample gait features, and the third sample gait features into the gait recognition model, and extracting a fourth sample gait feature from each of the first sample gait features, a fifth sample gait feature from each of the second sample gait features, and a sixth sample gait feature from each of the third sample gait features through the gait recognition model; the fourth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the first sample gait feature, the fifth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the second sample gait feature, and the sixth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the third sample gait feature;
[0017] Determining the first identity tag corresponding to each of the fourth sample gait features based on the correspondence between the first sample gait features and the first type of sample pedestrians and the correspondence between the first sample gait features and the fourth sample gait features;
[0018] Determining the second identity label corresponding to each of the fifth sample gait features based on the correspondence between the second sample gait features and the second type of sample pedestrians and the correspondence between the second sample gait features and the fifth sample gait features;
[0019] Determining the second identity tag corresponding to each of the sixth sample gait features according to the correspondence between the second sample gait feature and the third sample gait feature and the correspondence between the third sample gait feature and the sixth sample gait feature;
[0020] The gait recognition model is trained using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag.
[0021] In combination with the third possible implementation of the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the using the fourth sample gait feature and its corresponding first identity label, the fifth sample gait feature and its corresponding second identity label, and the sixth sample gait feature and its corresponding second identity label to train the gait recognition model includes:
[0022] Inputting each of the fourth sample gait features, the fifth sample gait features, and the sixth sample gait features into the gait recognition model, and identifying, by the gait recognition model, a first identity recognition result of each of the fourth sample gait features, a second identity recognition result of the fifth sample gait features, and a third identity recognition result of the sixth sample gait features;
[0023] Calculating a loss value of the gait recognition model according to the first identity recognition result and its corresponding first identity tag, the second identity recognition result and its corresponding second identity tag, and the third identity recognition result and its corresponding second identity tag;
[0024] Determine whether the loss value meets the model training cutoff condition; when the loss value meets the model training cutoff condition, stop training and use the current gait recognition model as the target gait recognition model after training; when the loss value does not meet the model training cutoff condition, use the loss value to train the learnable parameters in the gait recognition model, and re-execute the steps of clustering the multiple first sample gait features corresponding to the first class of sample pedestrians when wearing the same set of clothing into one class for each of the first class of sample pedestrians, and obtaining a set of sample gait features corresponding to the first class of sample pedestrians when wearing each set of clothing.
[0025] In a second aspect, an embodiment of the present application further provides a model training device based on clothing transfer, wherein an original training set for training a gait recognition model includes a plurality of first sample gait sequences corresponding to each set of clothing when a first type of sample pedestrian wears different sets of clothing, and a plurality of second sample gait sequences when a second type of sample pedestrian wears the same set of clothing; the device comprises:
[0026] a clustering module configured to cluster, for each of the first-category sample pedestrians, a plurality of first-category sample gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category, thereby obtaining a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing; the first-category sample gait features being obtained based on the first-category sample gait sequence;
[0027] a calculation module configured to, after determining an average sample gait feature of each of the sample gait feature sets, calculate, for each of the first-category sample pedestrians, a feature difference between any two of the average sample gait features corresponding to the first-category sample pedestrians, to obtain a plurality of clothing change features corresponding to the first-category sample pedestrians;
[0028] a generating module configured to, for each second sample gait feature of each second-category sample pedestrian, add the selected preset number of clothing change features to the second sample gait feature to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian; the second sample gait feature being obtained based on the second sample gait sequence;
[0029] A training module is used to train the gait recognition model using the first sample gait feature, the second sample gait feature and the third sample gait feature.
[0030] In combination with the second aspect, the embodiment of the present application provides a first possible implementation of the second aspect, wherein the device further includes:
[0031] an extraction module, configured to input each of the first sample gait sequences and the second sample gait sequences contained in the original training set into the gait recognition model, and extract the first sample gait features of each of the first sample gait sequences and the second sample gait features of each of the second sample gait sequences through the gait recognition model.
[0032] In conjunction with the second aspect, an embodiment of the present application provides a second possible implementation of the second aspect, wherein the generation module, when used to add the selected preset number of clothing change features to each second sample gait feature of each second-category sample pedestrian to generate multiple third sample gait features corresponding to the second-category sample pedestrian, is specifically used to:
[0033] The third sample gait feature corresponding to the second type of sample pedestrian is generated by the following formula:
[0034]
[0035] in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, It is the third sample gait feature of the second type of sample pedestrian.
[0036] In combination with the second aspect, an embodiment of the present application provides a third possible implementation of the second aspect, wherein the original training set also includes a first identity label for each sample pedestrian of the first category and a second identity label for each sample pedestrian of the second category; and when the training module is used to train the gait recognition model using the first sample gait features, the second sample gait features, and the third sample gait features, it is specifically configured to:
[0037] Inputting each of the first sample gait features, the second sample gait features, and the third sample gait features into the gait recognition model, and extracting a fourth sample gait feature from each of the first sample gait features, a fifth sample gait feature from each of the second sample gait features, and a sixth sample gait feature from each of the third sample gait features through the gait recognition model; the fourth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the first sample gait feature, the fifth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the second sample gait feature, and the sixth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the third sample gait feature;
[0038] Determining the first identity tag corresponding to each of the fourth sample gait features based on the correspondence between the first sample gait features and the first type of sample pedestrians and the correspondence between the first sample gait features and the fourth sample gait features;
[0039] Determining the second identity label corresponding to each of the fifth sample gait features based on the correspondence between the second sample gait features and the second type of sample pedestrians and the correspondence between the second sample gait features and the fifth sample gait features;
[0040] Determining the second identity tag corresponding to each of the sixth sample gait features according to the correspondence between the second sample gait feature and the third sample gait feature and the correspondence between the third sample gait feature and the sixth sample gait feature;
[0041] The gait recognition model is trained using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag.
[0042] In combination with the third possible implementation of the second aspect, the embodiment of the present application provides a fourth possible implementation of the second aspect, wherein the training module, when used to train the gait recognition model using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag, is specifically used to:
[0043] Inputting each of the fourth sample gait features, the fifth sample gait features, and the sixth sample gait features into the gait recognition model, and identifying, by the gait recognition model, a first identity recognition result of each of the fourth sample gait features, a second identity recognition result of the fifth sample gait features, and a third identity recognition result of the sixth sample gait features;
[0044] Calculating a loss value of the gait recognition model according to the first identity recognition result and its corresponding first identity tag, the second identity recognition result and its corresponding second identity tag, and the third identity recognition result and its corresponding second identity tag;
[0045] Determine whether the loss value meets the model training cutoff condition; when the loss value meets the model training cutoff condition, stop training and use the current gait recognition model as the target gait recognition model after training; when the loss value does not meet the model training cutoff condition, use the loss value to train the learnable parameters in the gait recognition model, and re-execute the steps of clustering the multiple first sample gait features corresponding to the first class of sample pedestrians when wearing the same set of clothing into one class for each of the first class of sample pedestrians, and obtaining a set of sample gait features corresponding to the first class of sample pedestrians when wearing each set of clothing.
[0046] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of any possible implementation method of the first aspect above are performed.
[0047] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any possible implementation method of the first aspect are executed.
[0048] The embodiments of the present application provide a clothing transfer-based model training method, apparatus, electronic device, and medium. In this method, based on the original training set, a third sample gait feature is newly generated for a second category of sample pedestrians with less clothing variation, thereby enriching the clothing variation of the second category of sample pedestrians and balancing the clothing imbalance between the first and second category sample pedestrians in the training data. By using the first and second sample gait features in the original training set, in addition to the third sample gait features newly generated for the second category of sample pedestrians with less clothing variation, gait recognition accuracy is improved, thereby improving the gait recognition accuracy of the gait recognition model.
[0049] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.
[0051] Figure 1 A flowchart of a model training method based on clothing transfer provided in an embodiment of the present application is shown;
[0052] Figure 2 A schematic diagram showing the correspondence between the first type of sample pedestrians, the first sample gait sequence, the first sample gait feature, and the sample gait feature set provided in an embodiment of the present application;
[0053] Figure 3 A schematic structural diagram of a model training device based on clothing transfer provided in an embodiment of the present application is shown;
[0054] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.
[0056] Considering the problem of uneven clothing among the pedestrian samples in the original training set, the embodiments of the present application provide a model training method, device, electronic device, and medium based on clothing transfer to alleviate the problem of uneven clothing in the original training set, thereby improving the gait recognition accuracy of the gait recognition model. The embodiments are described below.
[0057] Example 1:
[0058] To facilitate understanding of this embodiment, we first provide a detailed introduction to the model training method based on clothing transfer disclosed in this embodiment. The original training set used to train the gait recognition model includes multiple first sample gait sequences corresponding to each set of clothing worn by a first type of sample pedestrian, and multiple second sample gait sequences corresponding to each set of clothing worn by a second type of sample pedestrian. Figure 1 A flow chart of a model training method based on clothing transfer provided by an embodiment of the present application is shown. Figure 1 As shown, the following steps are included:
[0059] S101: For each first-category sample pedestrian, multiple first-sample gait features corresponding to the first-category sample pedestrian when wearing the same set of clothing are clustered into one category, and a set of sample gait features corresponding to the first-category sample pedestrian when wearing each set of clothing is generated; the first-sample gait features are obtained based on the first-sample gait sequence.
[0060] In this embodiment, the original training set includes a plurality of first sample gait sequences corresponding to a plurality of first-category sample pedestrians, and a plurality of second sample gait sequences corresponding to a plurality of second-category sample pedestrians.
[0061] In the process of generating the original training set, each first-category sample pedestrian changes clothes. When wearing each set of clothing, at least one (or more) first-category sample gait sequence is collected for each first-category sample pedestrian wearing that set of clothing, thereby obtaining multiple first-category sample gait sequences corresponding to each first-category sample pedestrian wearing each set of clothing. Each first-category sample pedestrian corresponds to multiple different sets of clothing.
[0062] Figure 2 A schematic diagram showing the correspondence between the first type of sample pedestrians, the first sample gait sequence, the first sample gait feature, and the sample gait feature set provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, each first-category pedestrian sample corresponds to multiple sample gait feature sets. The number of sample gait feature sets corresponding to each first-category pedestrian sample is the same as the number of clothing items corresponding to the first-category pedestrian sample. Each sample gait feature set contains multiple first sample gait features. The first sample gait features contained in the same sample gait feature set are the first sample gait features of the same first-category pedestrian wearing the same set of clothing.
[0063] In the process of generating the original training set, each second-category sample pedestrian corresponds to only one set of clothing. When wearing the set of clothing, at least one (or more) second sample gait sequences of the second-category sample pedestrian can be collected when wearing the set of clothing.
[0064] In a possible implementation, before executing step S101, the following may be performed: each first sample gait sequence and each second sample gait sequence contained in the original training set is input into a gait recognition model, and the first sample gait features of each first sample gait sequence and the second sample gait features of each second sample gait sequence are extracted through the gait recognition model.
[0065] In this embodiment, each first sample gait sequence corresponds to a first sample gait feature, and each second sample gait sequence corresponds to a second sample gait feature.
[0066] S102: After determining the average sample gait feature of each sample gait feature set, for each first-category sample pedestrian, calculate the feature difference between any two average sample gait features corresponding to the first-category sample pedestrian to obtain multiple clothing change features corresponding to the first-category sample pedestrian.
[0067] In this embodiment, for each sample gait feature set, the average feature of each first sample gait feature included in the sample gait feature set is calculated based on each first sample gait feature included in the sample gait feature set to obtain the average sample gait feature of the sample gait feature set. Each sample gait feature set corresponds to an average sample gait feature.
[0068] For each first-category pedestrian sample, the feature difference between any two average sample gait features corresponding to the first-category pedestrian sample is calculated to obtain multiple clothing change features corresponding to the first-category pedestrian sample. Here, each two average sample gait features corresponding to the same first-category pedestrian sample correspond to one clothing change feature.
[0069] In this embodiment, the clothing change feature represents the change feature between two different sets of clothing worn by the same first-category sample pedestrian. Each first-category sample pedestrian corresponds to a plurality of clothing change features.
[0070] For example, Figure 2 As shown in FIG, when the first type of sample pedestrian A1 corresponds to three sample gait feature sets, it also corresponds to three average sample gait features. Any two of these three average sample gait features correspond to one clothing change feature, so the first type of sample pedestrian A1 corresponds to three clothing change features.
[0071] S103: For each second sample gait feature of each second-category sample pedestrian, a preset number of selected clothing change features are added to the second sample gait feature to generate multiple third sample gait features corresponding to the second-category sample pedestrian; the second sample gait feature is obtained based on the second sample gait sequence.
[0072] In this embodiment, for each second sample gait feature of each second category sample pedestrian, a preset number of randomly selected clothing change features are added to the second sample gait feature to obtain a third sample gait feature corresponding to the second sample gait feature.
[0073] In this embodiment, the third sample gait features are derived by adding clothing change features to the second gait sample features. Therefore, the third sample gait features and the second gait sample features corresponding to the same second-category pedestrian differ in the clothing features they include, thereby enriching the clothing of the second-category pedestrian at the feature level. This approach, without requiring the second-category pedestrian to actually change clothes, enriches the clothing of the second-category pedestrian at the gait feature level simply by adding clothing change features to the original second gait sample features. This helps alleviate the clothing imbalance between the first-category pedestrian and second-category pedestrian samples in the original training set, and improves the recognition accuracy of the gait recognition model.
[0074] In a possible implementation, the third sample gait feature corresponding to the second sample gait feature may be generated by the following formula:
[0075]
[0076] in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, is the third sample gait feature of the second type of sample pedestrian. The preset number can be a positive integer greater than or equal to 1.
[0077] After generating the third sample gait features corresponding to each second sample gait feature of the same second-category sample pedestrian, a plurality of third sample gait features corresponding to the second-category sample pedestrian are generated.
[0078] S104: Training a gait recognition model using the first sample gait feature, the second sample gait feature, and the third sample gait feature.
[0079] In a possible implementation, the original training set also includes the first identity label of each first-category pedestrian sample and the second identity label of each second-category pedestrian sample. When executing step S104, the following steps S1041-S1045 may be specifically performed:
[0080] S1041: Input each first sample gait feature, second sample gait feature and third sample gait feature into a gait recognition model, and extract the fourth sample gait feature of each first sample gait feature, the fifth sample gait feature of each second sample gait feature, and the sixth sample gait feature of each third sample gait feature through the gait recognition model.
[0081] Among them, the sample pedestrian identity differentiation degree of the fourth sample gait feature is higher than that of the first sample gait feature, the sample pedestrian identity differentiation degree of the fifth sample gait feature is higher than that of the second sample gait feature, and the sample pedestrian identity differentiation degree of the sixth sample gait feature is higher than that of the third sample gait feature.
[0082] S1042: Determine a first identity label corresponding to each fourth sample gait feature based on a correspondence between the first sample gait feature and the first type of sample pedestrians and a correspondence between the first sample gait feature and the fourth sample gait feature.
[0083] S1043: Determine a second identity label corresponding to each fifth sample gait feature based on the correspondence between the second sample gait feature and the second type of sample pedestrians and the correspondence between the second sample gait feature and the fifth sample gait feature.
[0084] S1044: Determine a second identity tag corresponding to each sixth sample gait feature according to the correspondence between the second sample gait feature and the third sample gait feature and the correspondence between the third sample gait feature and the sixth sample gait feature.
[0085] S1045: Use the fourth sample gait feature and its corresponding first identity label, the fifth sample gait feature and its corresponding second identity label, and the sixth sample gait feature and its corresponding second identity label to train a gait recognition model.
[0086] In this embodiment, the gait recognition model is trained using the fourth sample gait feature and the first identity label corresponding to the fourth sample gait feature, the fifth sample gait feature and the second identity label corresponding to the fifth sample gait feature, and the sixth sample gait feature and the second identity label corresponding to the sixth sample gait feature.
[0087] In a possible implementation, when executing step S1045, specifically:
[0088] Inputting each of the fourth sample gait features, the fifth sample gait features, and the sixth sample gait features into a gait recognition model, and identifying, by the gait recognition model, a first identity recognition result of each of the fourth sample gait features, a second identity recognition result of the fifth sample gait features, and a third identity recognition result of the sixth sample gait features;
[0089] Calculating a loss value of the gait recognition model based on the first identity recognition result and its corresponding first identity label, the second identity recognition result and its corresponding second identity label, and the third identity recognition result and its corresponding second identity label;
[0090] Determine whether the loss value meets the model training cutoff condition; when the loss value meets the model training cutoff condition, stop training and use the current gait recognition model as the target gait recognition model after training; when the loss value does not meet the model training cutoff condition, use the loss value to train the learnable parameters in the gait recognition model, and re-execute the steps for each first-category sample pedestrian, clustering multiple first-sample gait features corresponding to the first-category sample pedestrian when wearing the same set of clothing into one category, and obtaining a set of sample gait features corresponding to the first-category sample pedestrian when wearing each set of clothing.
[0091] In this embodiment, the model training cutoff condition may be that the loss value converges, that is, the loss value no longer decreases.
[0092] Example 2:
[0093] Based on the same technical concept, the present application also provides a model training device based on clothing transfer, wherein the original training set for training the gait recognition model includes a plurality of first sample gait sequences corresponding to each set of clothing when the first type of sample pedestrians are wearing different sets of clothing, and a plurality of second sample gait sequences when the second type of sample pedestrians are wearing the same set of clothing; Figure 3 FIG. 1 shows a schematic structural diagram of a model training device based on clothing migration provided by an embodiment of the present application. Figure 3 As shown, the device includes:
[0094] Clustering module 301 is configured to cluster, for each of the first-category sample pedestrians, multiple first-category sample gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category, thereby obtaining a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing; the first-category sample gait features are obtained based on the first-category sample gait sequence;
[0095] A calculation module 302 is configured to, after determining the average sample gait feature of each of the sample gait feature sets, calculate, for each of the first-category sample pedestrians, a feature difference between any two of the average sample gait features corresponding to the first-category sample pedestrians, to obtain a plurality of clothing change features corresponding to the first-category sample pedestrians;
[0096] A generating module 303 is configured to add the selected preset number of clothing change features to each second sample gait feature of each second-category sample pedestrian to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian; the second sample gait features are obtained based on the second sample gait sequence;
[0097] The training module 304 is configured to train the gait recognition model using the first sample gait feature, the second sample gait feature, and the third sample gait feature.
[0098] Optionally, the device further includes:
[0099] an extraction module, configured to input each of the first sample gait sequences and the second sample gait sequences contained in the original training set into the gait recognition model, and extract the first sample gait features of each of the first sample gait sequences and the second sample gait features of each of the second sample gait sequences through the gait recognition model.
[0100] Optionally, when the generating module 303 is used to add the selected preset number of clothing change features to each second sample gait feature of each second-category sample pedestrian to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian, the generating module 303 is specifically used to:
[0101] The third sample gait feature corresponding to the second type of sample pedestrian is generated by the following formula:
[0102]
[0103] in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, It is the third sample gait feature of the second type of sample pedestrian.
[0104] Optionally, the original training set further includes a first identity label for each sample pedestrian of the first category and a second identity label for each sample pedestrian of the second category; when the training module 304 is used to train the gait recognition model using the first sample gait feature, the second sample gait feature, and the third sample gait feature, it is specifically configured to:
[0105] Inputting each of the first sample gait features, the second sample gait features, and the third sample gait features into the gait recognition model, and extracting a fourth sample gait feature from each of the first sample gait features, a fifth sample gait feature from each of the second sample gait features, and a sixth sample gait feature from each of the third sample gait features through the gait recognition model; the fourth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the first sample gait feature, the fifth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the second sample gait feature, and the sixth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the third sample gait feature;
[0106] Determining the first identity tag corresponding to each of the fourth sample gait features based on the correspondence between the first sample gait features and the first type of sample pedestrians and the correspondence between the first sample gait features and the fourth sample gait features;
[0107] Determining the second identity label corresponding to each of the fifth sample gait features based on the correspondence between the second sample gait features and the second type of sample pedestrians and the correspondence between the second sample gait features and the fifth sample gait features;
[0108] Determining the second identity tag corresponding to each of the sixth sample gait features according to the correspondence between the second sample gait feature and the third sample gait feature and the correspondence between the third sample gait feature and the sixth sample gait feature;
[0109] The gait recognition model is trained using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag.
[0110] Optionally, when the training module 304 is used to train the gait recognition model using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag, it is specifically configured to:
[0111] Inputting each of the fourth sample gait features, the fifth sample gait features, and the sixth sample gait features into the gait recognition model, and identifying, by the gait recognition model, a first identity recognition result of each of the fourth sample gait features, a second identity recognition result of the fifth sample gait features, and a third identity recognition result of the sixth sample gait features;
[0112] Calculating a loss value of the gait recognition model according to the first identity recognition result and its corresponding first identity tag, the second identity recognition result and its corresponding second identity tag, and the third identity recognition result and its corresponding second identity tag;
[0113] Determine whether the loss value meets the model training cutoff condition; when the loss value meets the model training cutoff condition, stop training and use the current gait recognition model as the target gait recognition model after training; when the loss value does not meet the model training cutoff condition, use the loss value to train the learnable parameters in the gait recognition model, and re-execute the steps of clustering the multiple first sample gait features corresponding to the first class of sample pedestrians when wearing the same set of clothing into one class for each of the first class of sample pedestrians, and obtaining a set of sample gait features corresponding to the first class of sample pedestrians when wearing each set of clothing.
[0114] Example 3:
[0115] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application includes: a processor 401, a memory 402 and a bus 403, wherein the memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the above-mentioned information processing method, the processor 401 communicates with the memory 402 through the bus 403, and the processor 401 executes the machine-readable instructions to perform the method steps described in Example 1.
[0116] Example 4:
[0117] The fourth embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps described in the first embodiment are executed.
[0118] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, electronic devices, and computer-readable storage media can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0119] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some communication interface, device or unit, which can be electrical, mechanical or other forms.
[0120] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0122] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0123] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims.
Claims
1. A model training method based on clothing transfer, characterized in that: The original training set used to train the gait recognition model includes a plurality of first sample gait sequences corresponding to each set of clothing when the first type of sample pedestrians are respectively wearing different sets of clothing, and a plurality of second sample gait sequences when the second type of sample pedestrians are wearing the same set of clothing; the method includes: For each of the first-category sample pedestrians, clustering multiple first-category gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category, thereby generating a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing; the first-category gait features are obtained based on the first-category gait sequence; After determining the average sample gait feature of each of the sample gait feature sets, for each of the first-category sample pedestrians, calculating the feature difference between any two of the average sample gait features corresponding to the first-category sample pedestrians, to obtain a plurality of clothing change features corresponding to the first-category sample pedestrians; For each second sample gait feature of each second-category sample pedestrian, adding the selected preset number of clothing change features to the second sample gait feature to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian; the second sample gait feature is obtained based on the second sample gait sequence; The gait recognition model is trained using the first sample gait feature, the second sample gait feature, and the third sample gait feature.
2. The method according to claim 1, characterized in that Before clustering, for each of the first-category sample pedestrians, a plurality of first-category sample gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category and obtaining a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing, the method further comprises: Each of the first sample gait sequences and the second sample gait sequences contained in the original training set is input into the gait recognition model, and the first sample gait features of each of the first sample gait sequences and the second sample gait features of each of the second sample gait sequences are extracted through the gait recognition model.
3. The method according to claim 1, characterized in that For each second sample gait feature of each second-category sample pedestrian, adding the selected preset number of clothing change features to the second sample gait feature to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian, including: The third sample gait feature corresponding to the second type of sample pedestrian is generated by the following formula: in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, It is the third sample gait feature of the second type of sample pedestrian.
4. The method according to claim 1, characterized in that The original training set further includes a first identity label for each of the first-category sample pedestrians and a second identity label for each of the second-category sample pedestrians; and training the gait recognition model using the first sample gait features, the second sample gait features, and the third sample gait features includes: Inputting each of the first sample gait features, the second sample gait features, and the third sample gait features into the gait recognition model, and extracting a fourth sample gait feature from each of the first sample gait features, a fifth sample gait feature from each of the second sample gait features, and a sixth sample gait feature from each of the third sample gait features through the gait recognition model; the fourth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the first sample gait feature, the fifth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the second sample gait feature, and the sixth sample gait feature has a higher degree of distinguishing the identity of a sample pedestrian than the third sample gait feature; Determining the first identity tag corresponding to each of the fourth sample gait features based on the correspondence between the first sample gait features and the first type of sample pedestrians and the correspondence between the first sample gait features and the fourth sample gait features; Determining the second identity label corresponding to each of the fifth sample gait features based on the correspondence between the second sample gait features and the second type of sample pedestrians and the correspondence between the second sample gait features and the fifth sample gait features; Determining the second identity tag corresponding to each of the sixth sample gait features according to the correspondence between the second sample gait feature and the third sample gait feature and the correspondence between the third sample gait feature and the sixth sample gait feature; The gait recognition model is trained using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag.
5. The method according to claim 4, characterized in that: The using the fourth sample gait feature and its corresponding first identity tag, the fifth sample gait feature and its corresponding second identity tag, and the sixth sample gait feature and its corresponding second identity tag to train the gait recognition model includes: Inputting each of the fourth sample gait features, the fifth sample gait features, and the sixth sample gait features into the gait recognition model, and identifying, by the gait recognition model, a first identity recognition result of each of the fourth sample gait features, a second identity recognition result of the fifth sample gait features, and a third identity recognition result of the sixth sample gait features; Calculating a loss value of the gait recognition model according to the first identity recognition result and its corresponding first identity tag, the second identity recognition result and its corresponding second identity tag, and the third identity recognition result and its corresponding second identity tag; Determine whether the loss value meets the model training cutoff condition; when the loss value meets the model training cutoff condition, stop training and use the current gait recognition model as the target gait recognition model after training; when the loss value does not meet the model training cutoff condition, use the loss value to train the learnable parameters in the gait recognition model, and re-execute the steps of clustering the multiple first sample gait features corresponding to the first class of sample pedestrians when wearing the same set of clothing into one class for each of the first class of sample pedestrians, and obtaining a set of sample gait features corresponding to the first class of sample pedestrians when wearing each set of clothing.
6. A model training device based on clothing transfer, characterized in that: The original training set for training the gait recognition model includes a plurality of first sample gait sequences corresponding to each set of clothing when the first type of sample pedestrians are respectively wearing different sets of clothing, and a plurality of second sample gait sequences when the second type of sample pedestrians are wearing the same set of clothing; the device includes: a clustering module configured to cluster, for each of the first-category sample pedestrians, a plurality of first-category sample gait features corresponding to the first-category sample pedestrians wearing the same set of clothing into one category, thereby obtaining a set of sample gait features corresponding to the first-category sample pedestrians wearing each set of clothing; the first-category sample gait features being obtained based on the first-category sample gait sequence; a calculation module configured to, after determining an average sample gait feature of each of the sample gait feature sets, calculate, for each of the first-category sample pedestrians, a feature difference between any two of the average sample gait features corresponding to the first-category sample pedestrians, to obtain a plurality of clothing change features corresponding to the first-category sample pedestrians; a generating module configured to, for each second sample gait feature of each second-category sample pedestrian, add the selected preset number of clothing change features to the second sample gait feature to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian; the second sample gait feature being obtained based on the second sample gait sequence; A training module is used to train the gait recognition model using the first sample gait feature, the second sample gait feature and the third sample gait feature.
7. The device according to claim 6, characterized in that The device further comprises: an extraction module, configured to input each of the first sample gait sequences and the second sample gait sequences contained in the original training set into the gait recognition model, and extract the first sample gait features of each of the first sample gait sequences and the second sample gait features of each of the second sample gait sequences through the gait recognition model.
8. The device according to claim 6, characterized in that The generating module is configured to add the selected preset number of clothing change features to each second sample gait feature of each second-category sample pedestrian to generate a plurality of third sample gait features corresponding to the second-category sample pedestrian, specifically for: The third sample gait feature corresponding to the second type of sample pedestrian is generated by the following formula: in, and are the two average sample gait features corresponding to the first type of sample pedestrian u, Indicates clothing change characteristics, N is the preset number, γ is the preset coefficient, is the second sample gait feature of the second type of sample pedestrian, It is the third sample gait feature of the second type of sample pedestrian.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 5 are performed.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method according to any one of claims 1 to 5.