Model training method, device and system, and pedestrian re-identification method

Abstract

The application discloses a model training method, device and system, and a pedestrian re-identification method. The method comprises the following steps: extracting sample features of each sample in a target training data set by using a first machine learning model to obtain a first feature set; extracting sample features of each sample in the target training data set by using a second machine learning model to obtain a second feature set; determining the similarity between a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity and calculating an accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function. The application solves the technical problem that, in the related art, in order to ensure the compatibility of new features and old features, the distribution of the new features and the old features is directly constrained to be as similar as possible, thereby causing greater damage to the performance of a new model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and in particular, to a model training method and device, a system, and a pedestrian re-identification method. BACKGROUND

[0002] In a real pedestrian re-identification system, model updating iteration is a common requirement. However, the features of images already in the library are extracted by the old model, and the features of images not yet in the library are extracted by the updated new model. Because there are differences between the parameters of the new and old models, the image features extracted by the new model are not compatible with the features extracted by the old model, resulting in the image features extracted by the new and old models being unable to be searched mutually, that is, the new model cannot be backward compatible. The engineering solution is to use the new model to brush the library, that is, to use the new model to re-extract features for the stored images, but this calculation overhead is very large. The backward compatibility of the features currently proposed means that the new model can be compatible with the old model features at the algorithm level. Most of the existing solutions directly constrain the feature distribution of the new and old models, but this method greatly damages the performance of the new model.

[0003] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0004] Embodiments of the present application provide a model training method, device, system, and pedestrian re-identification method to at least solve the technical problem that related technologies directly constrain the new and old feature distributions to be as similar as possible to ensure the compatibility of the new and old features, resulting in great damage to the performance of the new model.

[0005] According to an aspect of an embodiment of the present application, a model training method is provided, including: extracting sample features of each sample in a target training data set by using a first machine learning model to obtain a first feature set; extracting sample features of each sample in the target training data set by using a second machine learning model to obtain a second feature set, wherein the model structures of the first and second machine learning models are the same, and the training data used are different; determining the similarity of a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity, and calculating an accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function.

[0006] According to another aspect of the embodiments of the present application, a pedestrian re-identification method is also provided, including: obtaining a pedestrian image in target video data; inputting the pedestrian image into a preset machine learning model for identification to obtain an identification result of the pedestrian image, wherein the preset machine learning model is trained based on sample features extracted from target training data by a first machine learning model and sample features extracted from the target training data by a second machine learning model, wherein the first machine learning model and the second machine learning model have the same model structure but different training data; and the loss function of the preset machine learning model is determined based on the following manner: after the features extracted by the first machine learning model and the features extracted by the second machine learning model are sorted according to similarity, the accuracy index of the sorting result is determined.

[0007] According to another aspect of the embodiments of the present application, a model training apparatus is also provided, including: a first extraction module configured to extract sample features of each sample in target training data by a first machine learning model to obtain a first feature set; a second extraction module configured to extract sample features of each sample in the target training data by a second machine learning model to obtain a second feature set, wherein the first machine learning model and the second machine learning model have the same model structure but different training data; a determination module configured to determine the similarity of a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; a calculation module configured to sort according to the similarity and calculate the accuracy index of the sorting result; and a training module configured to determine a loss function based on the accuracy index and train the second machine learning model based on the loss function.

[0008] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, including a stored program, wherein when the program runs, the device where the non-volatile storage medium is located performs the above-mentioned model training method.

[0009] According to another aspect of the embodiments of the present application, a computing device is also provided, comprising: a processor; and a memory connected with the processor, configured to provide the processor with instructions to process the following processing steps: extracting sample features of each sample in a target training data set by using a first machine learning model, to obtain a first feature set; extracting sample features of each sample in the target training data set by using a second machine learning model, to obtain a second feature set, wherein the first machine learning model and the second machine learning model have the same model structure, but use different training data; determining a similarity between a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity, and calculating an accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function.

[0010] According to another aspect of the embodiments of the present application, a model training system is also provided, comprising: a client device and a server, wherein: the client device is configured to provide a target object with a human-computer interaction interface, and call program instructions in the server through the human-computer interaction interface to perform the following steps: extracting sample features of each sample in a target training data set by using a first machine learning model, to obtain a first feature set; extracting sample features of each sample in the target training data set by using a second machine learning model, to obtain a second feature set, wherein the first machine learning model and the second machine learning model have the same model structure, but use different training data; determining a similarity between a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity, and calculating an accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function.

[0011] In some embodiments of the present application, a first machine learning model is used to extract sample features of each sample in the target training data set to obtain a first feature set; a second machine learning model is used to extract sample features of each sample in the target training data set to obtain a second feature set; the similarity of a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set is determined; the similarity is sorted, and the accuracy index of the sorting result is calculated; the loss function is determined based on the accuracy index, and the second machine learning model is trained based on the loss function. Since the method is to constrain the sorting of new and old features, the constraint on the distribution of new model features is weak, the influence on the performance of the new model is small, and the method directly optimizes the accuracy of the sorting of new and old features, which can greatly improve the backward compatibility of the new model. Without the need for library brushing operation, the technical problem that the related art directly constrains the similarity of the distribution of new and old features to ensure the compatibility of the new and old features, resulting in greater damage to the performance of the new model, is solved. BRIEF DESCRIPTION OF DRAWINGS

[0012] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0013] Figure 1 is a model training method according to the related art;

[0014] Figure 2 is a model training method according to an embodiment of the present application;

[0015] Figure 3 is a hardware structure block diagram of a computer terminal according to an embodiment of the present application;

[0016] Figure 4 is a flowchart of a model training method according to an embodiment of the present application;

[0017] Figure 5 is a model training method according to an embodiment of the present application;

[0018] Figure 6 is a flowchart of a pedestrian re-identification method according to an embodiment of the present application;

[0019] Figure 7 is a structure diagram of a model training device according to an embodiment of the present application;

[0020] Figure 8 is a structure diagram of a computing device according to an embodiment of the present application;

[0021] Figure 9is a structural schematic diagram of a model training system according to an embodiment of the application. DETAILED DESCRIPTION

[0022] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0024] In order to better understand the embodiments of the present application, some nouns or terms appearing in the process of describing the embodiments of the present application below are explained as follows:

[0025] Person Re-identification: also known as person re-identification, is a technology for judging whether a specific person exists in an image or a video sequence by using computer vision technology, which is widely regarded as a sub-problem of image retrieval. Given a monitoring person image, the image of the person is retrieved under the cross-device, aiming to make up for the visual limitations of fixed cameras, and can be combined with person detection / person tracking technology, and can be widely applied to intelligent video monitoring, intelligent security and other fields.

[0026] Backward Compatibility: In a computer, it refers to the condition that a document or system created by an old version program can still be normally operated or used after a program or class library is updated to a newer version, or a program developed on the basis of an old version class library can still be normally compiled and run. Backward compatibility is a performance requirement based on component standards, and is the latest standard specification of TIA. A backward compatible component not only meets the performance requirements of its own classification, but also meets all the performance requirements of lower classifications.

[0027] New Feature: refers to the feature extracted by the new model in the embodiments of the present application

[0028] Old Feature: refers to the feature extracted by the old model in the embodiments of the present application

[0029] Euclidean metric: Euclidean distance, which is a commonly used distance definition, refers to the real distance between two points in m-dimensional space, or the natural length of a vector (i.e. the distance of the point to the origin). The so-called Euclidean distance transformation is to convert the value of the pixel in the foreground of a binary image (we assume that white is the foreground color and black is the background color) into the distance of the point to the nearest background point. The Euclidean distance transformation has a wide range of applications in digital image processing, especially for skeleton extraction of images, which is a good reference.

[0030] Nearest Neighbour (NN): The principle of the nearest neighbor method is to find a predetermined number or range of points closest to the query point in distance from the training samples, and then predict the label of the query point according to these points. The number of points found from the training samples can be a user-defined constant, which is called k-nearest neighbor learning, i.e. kNN, or can be obtained by a user-defined distance radius range, which is called radius-based nearest neighbor learning, i.e. RNN.

[0031] Embodiment 1

[0032] Figure 1 is a model training method according to the related art, as shown in Figure 1 , wherein the square represents the new feature, and the circle represents the old feature. The method is based on the distillation idea, and in order to improve the compatibility of the new and old features, the new feature and the old feature distribution are directly constrained to be as similar as possible. The constraint means includes minimizing the Euclidean distance between the features and the difference between the feature distributions. As can be seen, part of the new features in the method are incorrectly identified, that is, the performance of the new model is greatly damaged. To solve this problem, the embodiments of the present application propose the idea of ordering constraint, as shown in Figure 2 , by comparing the similarity between the new and old features, the relative ordering relationship between the new and old features is directly constrained. As can be seen, the method can improve the compatibility of the new and old features while reducing the performance damage to the new model. To achieve the above purpose, the embodiments of the present application adopt the following technical solutions, which are described in detail below.

[0033] The method embodiment provided by Embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 3A hardware structure block diagram of a computer terminal (or mobile device) for implementing the model training method is shown. As shown in Figure 3 The computer terminal 30 (or mobile device 30) can include one or more processors 302 (the processor 302 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission module 306 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 3 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 30 can include more or fewer components than those shown in Figure 3 or have a different configuration than that shown in Figure 3 .

[0034] It should be noted that the one or more processors 302 and / or other data processing circuits described above can be referred to herein as "data processing circuits" in general. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any combination thereof. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 30 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit serves as a processor to control (for example, selection of a variable resistance terminal path connected to an interface).

[0035] The memory 304 can be used to store software programs and modules of application software, such as the model training method of the embodiments of the present application, and the corresponding program instructions / data storage devices. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, that is, implements the vulnerability detection method of the application program described above. The memory 304 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory 304 can further include a memory remotely disposed with respect to the processor 302, which can be connected to the computer terminal 30 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0036] The transmission module 306 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer terminal 30. In one example, the transmission module 306 includes a network interface controller (NIC) that can connect to other network devices through a base station to communicate with the Internet. In one example, the transmission module 306 can be a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

[0037] The display can be a liquid crystal display (LCD) that is touch screen-enabled, for example, which can enable a user to interact with a user interface of the computer terminal 30 (or mobile device).

[0038] Based on the sorting constraint idea, in the above running environment, a model training method is provided in the embodiment. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0039] Figure 4 is a flowchart of a model training method according to an embodiment of the present application, as shown in Figure 4 The method at least includes steps S402-S412:

[0040] S402, a first machine learning model is used to extract sample features of each sample in the target training data set, to obtain a first feature set.

[0041] S404, a second machine learning model is used to extract sample features of each sample in the target training data set, to obtain a second feature set.

[0042] In some embodiments of the present application, the model structures of the first machine learning model and the second machine learning model are the same, but the training data used are different, wherein the training data used by the first machine learning model corresponds to an earlier collection time than the training data used by the second machine learning model. That is, in some embodiments of the present application, it can be understood that the first machine model is an old model, and the sample features extracted by the first machine model are old features; the second machine model is a new model, and the sample features extracted by the second machine model are new features.

[0043] It should be noted that the training data used by the first machine learning model and the second machine learning model should be data belonging to the same target sample, and the difference is that the collection time of the training data used by the two models is different. For example, the first machine learning model extracts feature information from images of a pedestrian in a first time period, and the second machine learning model extracts feature information from images of the pedestrian in a second time period, rather than from images of other pedestrians. The minimum value of the second time period is greater than or equal to the maximum value in the first time period, that is, the time in the second time period is later than the time in the first time period. The first time period and the second time period can be consecutive time periods of the same target object during travel.

[0044] S406, determine the similarity between the specified feature in the first feature set and the corresponding feature in the second feature set. It should be noted that the specified feature can be any feature in the first feature set. In some embodiments, after determining the specified feature, the corresponding feature label of the feature is determined first, and then the corresponding feature in the second feature set is found based on the feature label, and then the similarity between the specified feature and the similar feature is calculated.

[0045] In selecting the above specified feature, the following methods can be used, for example, before determining the similarity between the specified feature in the first feature set and the corresponding feature in the second feature set, the following processing steps can also be performed: determine the class feature center of each sample subset in the first feature set, and calculate the similarity between the feature centers of each sample subset to obtain a plurality of similarities, wherein the sample subset is obtained by dividing the features in the first feature set according to the feature type; compare the sizes of the plurality of similarities; based on the comparison result, select a target sample subset from the first feature set, and select a specified feature from each sample subset. Specifically, when selecting the target sample subset, a specified number of class feature centers with the closest similarity can be selected from the first feature set, and the sample subset corresponding to the specified number of class feature centers is taken as the target sample subset.

[0046] Wherein, the above class feature center can be a cluster center in the cluster algorithm analysis process, for example, for K-Means algorithm, it is obtained by calculating the average value of all data points in the class.

[0047] In some embodiments of the present application, the determination manner of the feature type is various, for example, can be obtained by classifying the features in the feature set based on a classification model, wherein the sample subsets are divided according to the features in the first feature set based on the target objects corresponding to the features in the first feature set, and the features in the different sample subsets belong to different target objects. That is, each target object has a corresponding sample subset. For example, in the pedestrian re-identification model, all features extracted from the images of the same pedestrian are classified into a sample subset, and each pedestrian corresponds to a sample subset.

[0048] The above similarity can be calculated by the Euclidean distance, of course, other ways can also be used to calculate the similarity, for example, Mahalanobis distance, Jaccard distance and other parameters can also be used to measure.

[0049] In some embodiments of the present application, when the target sample subset is selected from the first feature set based on the plurality of similarities, the Nearest Neighbour Random Sampling method is used, which has two sampling approaches:

[0050] The first kind: according to the kNN sampling method, the similarities can be sorted in descending order, and the first number of sample subsets with high ranking are selected as the target sample subset, wherein the first number is a pre-specified number, that is, a fixed number of sample subsets with the highest similarity are selected as the target sample subset; for example, when the preset number is 10, the sample subsets with the top 10 similarities are selected as the target sample subset. Using this method, since the number of target sample subsets is pre-specified, the subsequent sorting process can be controlled, for example, the computing resources used in the calculation process can be pre-evaluated.

[0051] The second method is as follows: according to the RNN sampling method, a similarity threshold can be preset, a plurality of similarities are compared with the preset threshold, a target similarity greater than the threshold is determined from the plurality of similarities, and a sample subset corresponding to the target similarity is taken as a target sample subset. It should be noted that the target sample subset includes all sample subsets with a similarity within the threshold range, and the number of the target sample subset is not fixed. For example, when the threshold is set to 70%, there can be 20 sample subsets with a similarity greater than 70% in the feature set, or there can be only 5 sample subsets with a similarity greater than 70%. All sample subsets with a similarity greater than 70% are selected as the target sample subset. In this way, the number of the target sample subset is related to the preset threshold, and therefore, the number of the target sample subset is not fixed. Therefore, compared with the first method, the subsequent sorting process corresponding to the second method is not easy to control.

[0052] In some embodiments of the present application, the specified features can be obtained by screening according to certain rules, for example, a second number of sample features are randomly selected from the target sample subset, and the specified features are obtained, wherein the second number is a pre-specified number. As can be seen, although the specified features are randomly selected from the target sample subset, the number of the specified features is pre-determined. When the target sample subset is a plurality of target sample subsets, a second number of sample features are randomly selected from each target sample subset.

[0053] For example, the nearest neighbor random sampling selects 10 target sample subsets, and 10 sample features are randomly selected from each sample subset, so that a total of 10*10=100 sample features are selected to participate in sorting.

[0054] S408, sorting according to the similarity and calculating an accuracy index of the sorting result.

[0055] In some embodiments of the present application, the accuracy index includes an average accuracy index (Average Precision, AP for short). Before explaining the average accuracy, the related technical terms are explained as follows:

[0056] Firstly, the following definitions are made:

[0057] True Positive (TP), a positive sample predicted as positive by the model;

[0058] True Negative (TN), a negative sample predicted as negative by the model;

[0059] False Positive (FP), a negative sample predicted as positive by the model;

[0060] False Negative (FN), a positive sample predicted as negative by the model;

[0061] On this basis, the concepts of precision and recall are introduced, the precision represents the proportion of positive samples predicted as positive by the model in all samples predicted as positive by the model (which can be understood as the proportion of correct images identified by the model in all images identified by the model), and the recall represents the proportion of positive samples predicted as positive by the model in all positive samples (which can be understood as the proportion of correct images identified by the model in all correct images), wherein,

[0062]

[0063]

[0064] In order to measure the performance of model prediction, the relationship between precision and recall, that is, the function relationship p(r) between precision and recall, can be observed as the number of samples changes, and the model in the embodiments of the present application ensures the accuracy of prediction when predicting as many positive samples as possible, that is, the recall value is increased while the precision value is kept at a high level.

[0065] In order to more intuitively reflect the performance of model prediction, the average accuracy (AP) is usually used as a standard,

[0066]

[0067] Wherein, n represents the total number of samples, and k represents the kth sample. It can be seen that the average accuracy is a heaviside function (unit step function).

[0068] S410, determining a loss function based on the accuracy index.

[0069] In some embodiments of the present application, determining a loss function based on the accuracy index comprises: performing an approximation operation calculation on the average accuracy function corresponding to the average accuracy index to obtain a derivable smooth average accuracy function; and performing a minimum processing on the average accuracy function to obtain the loss function.

[0070] Wherein, the loss function

[0071]

[0072] Since the average accuracy function is a heaviside function, which is not derivable, it can be transformed into a smooth and derivable sigmoid function by approximation, so as to obtain the required smooth average accuracy function (Smooth-AP), and the minimization processing is performed on the basis, so as to obtain the minimization loss function for training the second machine learning model.

[0073] S412, training the second machine learning model based on the loss function.

[0074] For the convenience of understanding the embodiments of the present application, Figure 5 The schematic diagram of the model training method is shown in FIG. 1. Figure 5 As shown in the figure, the square (including solid and hollow square) represents the new feature, and the rest of the pattern represents the old feature. The solid pattern in the pattern corresponding to the old feature represents the selected old feature, and the hollow pattern represents the unselected old feature. N1 and N2 are the selected nearest neighbor categories, and the solid pattern represents the sample selected for ranking. Figure 5 The content can be explained as follows:

[0075] 1. Start training a new model using the collected training data set T, which contains multiple pedestrian images, and each image set of a pedestrian is called a class;

[0076] 2. Extract the old features of the data set T using the old model to obtain the set F_old, store the old feature set F_old, and calculate the feature center of each class;

[0077] 3. Extract the new features of a training sample using the new model (for example, the black solid square and the blank square in FIG. 1), and then find the corresponding old feature set (for example, the solid pattern other than the square in FIG. 1) through the sample label; Figure 5 Figure 5

[0078] 5. Find the old feature set of the nearest N1 classes of the class feature center through the class feature center of the old feature (for example, different patterns in FIG. 1); Figure 5

[0079] 5. For the old feature set of the N1 classes, randomly select N2 samples (for example, the solid pattern other than the square) of each class to join the ranking;

[0080] 6. Perform similarity ranking on the new feature and the selected N1*N2 old features, and calculate the ranking AP;

[0081] 7. Calculate the derivable Smooth-AP through the approximation operation of the non-derivable ranking AP;

[0082] ​​​8. Adding new model training by minimizing the Smooth-AP loss (1-Smooth-AP).

[0083] The embodiments of the present application are experimented on multiple public data sets, and the feature backward compatibility is greatly improved in the embodiments of the present application, and the influence on the performance of the new model is very small, that is, the embodiments of the present application successfully solve the technical problem that the cost of brushing the library is too high, and the related art directly restricts the distribution of the new feature and the old feature to be as similar as possible to ensure the compatibility of the new and old features, resulting in greater damage to the performance of the new model.

[0084] Embodiment 2

[0085] On the basis of the embodiment 1 of the present application, a pedestrian re-identification method is further provided, as shown in Figure 6 The method at least includes steps S602-S606:

[0086] S602, obtaining a pedestrian image in target video data. The pedestrian image can be an image of a pedestrian in a running state collected by a monitoring device.

[0087] S604, inputting the pedestrian image into a preset machine learning model for recognition.

[0088] In some embodiments of the present application, the pedestrian image is input into the preset machine learning model for recognition to obtain a recognition result of the pedestrian image, wherein the preset machine learning model is trained based on sample features extracted from the target training data set by the first machine learning model and sample features extracted from the target training data set by the second machine learning model, wherein the model structure of the first machine learning model and the second machine learning model is the same, but the training data used is different; and the loss function of the preset machine learning model is determined based on the following manner: the features extracted by the first machine learning model and the features extracted by the second machine learning model are sorted according to the similarity, and the accuracy index of the sorting result is determined. As can be seen, the above-mentioned preset learning model constrains the sorting of new and old features, and does not constrain the distribution of new and old features, thereby improving the compatibility of the new model to the old feature while ensuring the performance of the new model.

[0089] S606, obtaining a pedestrian image recognition result.

[0090] It is easy to note that the training data adopted by the first machine learning model and the second machine learning model should be data belonging to the same target sample, and the difference is that the collection time of the training data adopted by the two models is different. For example, the first machine learning model extracts feature information from images of a pedestrian in a first time period, and the second machine learning model extracts feature information from images of the pedestrian in a second time period, rather than from images of other pedestrians. The minimum value of the second time period is greater than or equal to the maximum value in the first time period, that is, the time in the second time period is later than the time in the first time period. The first time period and the second time period can be consecutive time periods of the same target object during travel.

[0091] In some embodiments, the specified feature can be any one of the first feature set. In some embodiments, after determining the specified feature, the feature label corresponding to the feature is determined, and then the corresponding feature is found from the second feature set based on the feature label, and then the similarity between the specified feature and the similar feature is calculated.

[0092] In selecting the specified feature, the following processing steps can be performed before determining the similarity between the specified feature in the first feature set and the corresponding feature in the second feature set, for example: determining the class feature center of each sample subset in the first feature set, and calculating the similarity between the feature centers of each sample subset to obtain a plurality of similarities, wherein the sample subsets are obtained by dividing the features in the first feature set according to the feature type; comparing the sizes of the plurality of similarities; based on the comparison result, selecting a target sample subset from the first feature set, and selecting a specified feature from each sample subset. Specifically, when selecting the target sample subset, a specified number of class feature centers with the closest similarity can be selected from the first feature set, and the sample subsets corresponding to the specified number of class feature centers are taken as the target sample subset.

[0093] Wherein, the class feature center can be a cluster center in the cluster algorithm analysis process, for example, for K-Means algorithm, it is obtained by calculating the average of all data points in the class.

[0094] In some embodiments of this application, the determination of the aforementioned feature types can be achieved in various ways. For example, it can be obtained by classifying the features in the feature set based on a classification model. The sample subset is determined by dividing the features in the first feature set according to the target object corresponding to each feature in the first feature set. The features in the first feature set are then divided into different sample subsets, where the features in different sample subsets belong to different target objects. That is, each target object has a corresponding sample subset. For example, in a pedestrian re-identification model, all features extracted from the image of the same pedestrian are assigned to one sample subset, thus each pedestrian corresponds to one sample subset.

[0095] It should be noted that the preset machine learning model in this application embodiment is trained according to the model training method provided in embodiment 1, so its specific training process will not be described in detail here.

[0096] Example 3

[0097] According to an embodiment of this application, a model training apparatus is also provided, which is used to implement... Figure 4 The method shown is as follows: Figure 7 As shown, the device includes at least a first extraction module 70, a second extraction module 72, a determination module 74, a calculation module 76, and a training module 78, wherein:

[0098] The first extraction module 70 is used to extract sample features of each sample in the target training dataset using a first machine learning model to obtain a first feature set;

[0099] The second extraction module 72 is used to extract sample features of each sample in the target training dataset using a second machine learning model to obtain a second feature set, wherein the first machine learning model and the second machine learning model have the same model structure, but use different training data.

[0100] It's easy to note that the training data used by the first and second machine learning models should belong to the same target sample. The difference lies in the timing of the data collection. For example, if the first machine learning model extracts features from images of a pedestrian within a first time period, the second machine learning model extracts features from images of that pedestrian within a second time period, rather than from images of other pedestrians. The minimum value in the second time period is greater than or equal to the maximum value in the first time period; that is, the time in the second time period is later than the time in the first time period. The first and second time periods can be consecutive time periods of the same target object during its movement.

[0101] The determining module 74 is used to determine the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature;

[0102] The calculation module 77 is used to sort the results according to the similarity and calculate the accuracy index of the sorting results.

[0103] Training module 78 is used to determine a loss function based on the accuracy metric and to train the second machine learning model based on the loss function.

[0104] It should be noted that the above modules can be stored in memory in any combination and the functions of each module can be executed by one or more processors.

[0105] Furthermore, the preset machine learning model in this application embodiment is trained according to the model training method provided in Embodiment 1, so its specific training process will not be described in detail here.

[0106] It should be noted that the preferred implementation method in this embodiment can be found in the relevant descriptions in Embodiments 1 and 2, and will not be repeated here.

[0107] Example 4

[0108] According to embodiments of this application, a computing device is also provided, such as... Figure 8 As shown, the system includes at least a processor 80 and a memory 82, wherein the memory 82 is connected to the processor 80.

[0109] The processor is configured to execute the following processing steps stored in memory: The processor provides instructions for processing the following steps: extracting sample features from each sample in a target training dataset using a first machine learning model to obtain a first feature set; extracting sample features from each sample in the target training dataset using a second machine learning model to obtain a second feature set, wherein the first and second machine learning models have the same model structure but use different training data; determining the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature; sorting the features according to the similarity and calculating an accuracy index of the sorting results; determining a loss function based on the accuracy index; and training the second machine learning model based on the loss function.

[0110] Optionally, the processor is further configured to execute program instructions stored in the memory to determine a class feature center of each sample subset in the first feature set, and calculate a similarity between the feature centers of each sample subset, to obtain a plurality of similarities, wherein the sample subsets are obtained by dividing the features in the first feature set according to feature types; compare the plurality of similarities; based on the comparison result, filter a target sample subset from the first feature set, and select a specified feature from each sample subset.

[0111] The embodiments of the present application are tested on multiple public data sets, and the feature backward compatibility is greatly improved in the embodiments of the present application, and the influence on the performance of the new model is also very small, that is, the technical problem that the cost of brushing the library is too high is successfully solved, and the related art directly restricts the new feature and the old feature distribution to be as similar as possible to ensure the compatibility of the new and old features, resulting in a larger performance damage to the new model.

[0112] It should be noted that the preferred embodiments in the present embodiment can refer to the related descriptions in embodiments 1 and 2, which will not be repeated here.

[0113] Embodiment 5

[0114] According to the embodiments of the present application, a non-volatile storage medium is also provided, which includes a stored program, wherein when the program runs, the device where the non-volatile storage medium is located is controlled to perform a model training method including the method described in embodiments 1 and 2.

[0115] Optionally, when the program runs, the device where the non-volatile storage medium is located is controlled to perform the following steps: using a first machine learning model to extract sample features of each sample in a target training data set, to obtain a first feature set; using a second machine learning model to extract sample features of each sample in the target training data set, to obtain a second feature set, wherein the model structures of the first machine learning model and the second machine learning model are the same, but the training data used are different; determining the similarity of a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity, and calculating an accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function.

[0116] Optionally, the program is further configured to control the device where the nonvolatile storage medium is located to perform the following steps: obtaining a pedestrian image in the target video data; inputting the pedestrian image into a preset machine learning model for recognition to obtain a recognition result of the pedestrian image, wherein the preset machine learning model is trained based on sample features extracted from a target training data set by a first machine learning model and sample features extracted from the target training data set by a second machine learning model, wherein the first machine learning model and the second machine learning model have the same model structure but different training data, and the loss function of the preset machine learning model is determined based on the following manner: the features extracted by the first machine learning model and the features extracted by the second machine learning model are sorted according to the similarity, and the accuracy index of the sorting result is determined.

[0117] Optionally, the memory is further configured to store program instructions for performing the following steps: determining a class feature center of each sample subset in the first feature set, and calculating the similarity between the feature centers of each sample subset to obtain a plurality of similarities, wherein the sample subsets are obtained by dividing the features in the first feature set according to the feature types; comparing the sizes of the plurality of similarities; based on the comparison result, selecting a target sample subset from the first feature set, and selecting a specified feature from each sample subset.

[0118] It should be noted that the preferred embodiments in this embodiment can refer to the related descriptions in embodiments 1 and 2, which will not be repeated here.

[0119] Embodiment 6

[0120] The embodiment of the present application also provides a model training system, as shown in Figure 9 The system includes a client device 90 and a server 92, wherein: the client device 90 is configured to provide a human-computer interaction interface to a target object, and call program instructions in the server 92 to perform Figure 4 The steps of the model training method shown in the figure: using a first machine learning model to extract sample features of each sample in a target training data set to obtain a first feature set; using a second machine learning model to extract sample features of each sample in the target training data set to obtain a second feature set, wherein the first machine learning model and the second machine learning model have the same model structure but different training data; determining the similarity of a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set; sorting according to the similarity, and calculating the accuracy index of the sorting result; determining the loss function based on the accuracy index, and training the second machine learning model based on the loss function.

[0121] Since a large amount of training data is often required when training a model, and the hardware performance also has certain requirements, in order to save costs, the service provided by a software as a service (SaaS) platform can be used to implement training of the model. For example, in some embodiments, the client device 90 includes, but is not limited to, a tenant device of a SaaS platform, and correspondingly, the server includes a server of the SaaS platform. When training the second machine learning model, the tenant device can send a request message to the server, where the request message is used to request training of the second machine learning model, and the request message can also carry information such as conditions and resources required for this training. The server provides corresponding services to the tenant device according to the request message to perform the above training process. For a specific training process, refer to the related description in Embodiments 1-2, which will not be repeated here.

[0122] The sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0123] In the above embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0124] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit embodiment described above is only illustrative, and for example, the division of units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0125] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0126] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0127] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part that essentially contributes to the related art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0128] The above only describes the preferred embodiments of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A model training method, comprising: The first machine learning model is used to extract the sample features of each sample in the target training dataset to obtain the first feature set, wherein the target training dataset is used to represent the images of the target pedestrian within a preset time period; The second machine learning model is used to extract the sample features of each sample in the target training dataset to obtain a second feature set. The first machine learning model and the second machine learning model have the same model structure. The training data used by the first machine learning model and the second machine learning model belong to the data in the same target sample. However, the collection time corresponding to the training data used by the first machine learning model is earlier than the collection time corresponding to the training data used by the second machine learning model. Determine the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature; Sort the results according to the similarity score and calculate the accuracy index of the sorting results; A loss function is determined based on the accuracy metric, and the second machine learning model is trained based on the loss function to obtain a preset machine learning model, wherein the preset machine learning model is used to identify pedestrian images in the target video data to obtain the recognition result of the pedestrian images. The accuracy metric includes an average accuracy metric; determining the loss function based on the accuracy metric includes: performing an approximation operation on the average accuracy function corresponding to the average accuracy metric to obtain a differentiable and smooth average accuracy function; and minimizing the average accuracy function to obtain the loss function.

2. The method according to claim 1, wherein, Before determining the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature, the method further includes: The category feature centers of each sample subset in the first feature set are determined, and the similarity between the feature centers of each sample subset is calculated to obtain multiple similarity scores. The sample subsets are obtained by dividing the features in the first feature set according to the feature type. Compare the magnitudes of the multiple similarities; Based on the comparison results, a subset of target samples is selected from the first feature set, and the specified feature is selected from each subset of target samples.

3. The method according to claim 2, wherein, Based on the multiple similarities, a subset of target samples is selected from the first feature set, including: The similarity scores are sorted in descending order, and the top number of sample subsets are selected as the target sample subset.

4. The method according to claim 3, wherein, The first quantity is a pre-specified quantity.

5. The method according to claim 2, wherein, Based on the multiple similarities, a subset of target samples is selected from the first feature set, including: Compare the multiple similarities with a preset threshold; determine the target similarity among the multiple similarities that is greater than the threshold, and take the sample subset corresponding to the target similarity as the target sample subset.

6. The method according to claim 2, wherein, Selecting the specified feature from each of the target sample subsets includes: The specified features are obtained by randomly selecting a second number of sample features from each of the target sample subsets, wherein the second number is a pre-specified number.

7. The method according to claim 2, wherein, The sample subset is divided according to the following method: Based on the target objects corresponding to each feature in the first feature set, the features in the first feature set are divided into different sample subsets, wherein the features in the different sample subsets belong to different target objects.

8. The method according to any one of claims 1 to 7, wherein, The training data used by the first machine learning model was collected at a time earlier than the training data used by the second machine learning model.

9. A pedestrian re-identification method, wherein, include: Acquire pedestrian images from the target video data; The pedestrian image is input into a preset machine learning model for recognition to obtain the recognition result of the pedestrian image. The preset machine learning model is trained based on sample features extracted from the target training dataset by a first machine learning model and sample features extracted from the target training dataset by a second machine learning model. The first and second machine learning models have the same model structure. The training data used by the first and second machine learning models were collected from the same target sample at different times, but the collection time corresponding to the training data used by the first machine learning model is earlier than the collection time corresponding to the training data used by the second machine learning model. The loss function of the preset machine learning model is determined based on the following method: the features extracted by the first and second machine learning models are sorted according to similarity, and the loss function is determined based on the accuracy index of the sorting result. The preset machine learning model is trained based on the model training method described in claims 1-8. The accuracy index includes an average accuracy index. The loss function is obtained by minimizing the average accuracy function, and the average accuracy function is calculated by approximating the average accuracy function corresponding to the average accuracy index.

10. A model training device, comprising: The first extraction module is used to extract sample features of each sample in the target training dataset using a first machine learning model to obtain a first feature set, wherein the target training dataset is used to represent images of target pedestrians within a preset time period; The second extraction module is used to extract sample features of each sample in the target training dataset using a second machine learning model to obtain a second feature set. The first machine learning model and the second machine learning model have the same model structure. The training data used by the first machine learning model and the second machine learning model belong to the same target sample. However, the collection time corresponding to the training data used by the first machine learning model is earlier than the collection time corresponding to the training data used by the second machine learning model. A determination module is used to determine the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature; The calculation module is used to sort the results according to the similarity and calculate the accuracy index of the sorting results. The training module is used to determine a loss function based on the accuracy index, and to train the second machine learning model based on the loss function to obtain a preset machine learning model, wherein the preset machine learning model is used to identify pedestrian images in the target video data to obtain the recognition result of the pedestrian images; The accuracy index includes an average accuracy index. The device is further configured to perform an approximate operation calculation on the average accuracy function corresponding to the average accuracy index to obtain a differentiable and smooth average accuracy function; and to minimize the average accuracy function to obtain the loss function.

11. A non-volatile storage medium, wherein, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device where the non-volatile storage medium is located to execute the model training method according to any one of claims 1 to 7.

12. A computing device, comprising: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: A first machine learning model is used to extract sample features from each sample in the target training dataset to obtain a first feature set; a second machine learning model is used to extract sample features from each sample in the target training dataset to obtain a second feature set. The target training dataset represents images of target pedestrians within a preset time period. The first and second machine learning models have the same model structure. The training data used by the first and second machine learning models belong to the same target sample, but the acquisition time corresponding to the training data used by the first machine learning model is earlier than the acquisition time corresponding to the training data used by the second machine learning model. The similarity between a specified feature in the first feature set and a feature corresponding to the specified feature in the second feature set is determined. The similarity is then sorted, and an accuracy index of the sorting result is calculated. A loss function is determined based on the accuracy index, and the second machine learning model is trained based on the loss function to obtain a preset machine learning model. The preset machine learning model is used to identify pedestrian images in the target video data to obtain the recognition result of the pedestrian images. The accuracy metric includes an average accuracy metric; determining the loss function based on the accuracy metric includes: performing an approximation operation on the average accuracy function corresponding to the average accuracy metric to obtain a differentiable and smooth average accuracy function; and minimizing the average accuracy function to obtain the loss function.

13. A model training system, comprising: Client devices and servers, of which: The client device is used to provide a human-computer interaction interface to the target object, and to execute the following steps by calling program instructions in the server through the human-computer interaction interface: using a first machine learning model to extract sample features from each sample in the target training dataset to obtain a first feature set; using a second machine learning model to extract sample features from each sample in the target training dataset to obtain a second feature set, wherein the target training dataset represents images of target pedestrians within a preset time period; the first and second machine learning models have the same model structure; the training data used by the first and second machine learning models belongs to the same target sample, but the acquisition time corresponding to the training data used by the first machine learning model is earlier than the acquisition time corresponding to the training data used by the second machine learning model; determining the similarity between a specified feature in the first feature set and a feature in the second feature set corresponding to the specified feature; sorting according to the similarity and calculating the accuracy index of the sorting result; determining a loss function based on the accuracy index, and training the second machine learning model based on the loss function to obtain a preset machine learning model, wherein the preset machine learning model is used to identify pedestrian images in the target video data to obtain the recognition result of the pedestrian images; The accuracy metric includes an average accuracy metric; determining the loss function based on the accuracy metric includes: performing an approximation operation on the average accuracy function corresponding to the average accuracy metric to obtain a differentiable and smooth average accuracy function; and minimizing the average accuracy function to obtain the loss function.

Images