A pedestrian re-identification-based metric loss framework improvement method and related device

Abstract

The application discloses a pedestrian re-identification-based metric loss framework improvement method and related equipment, and the method comprises the following steps: dimension reduction processing is performed on a covariance matrix fitting overall distribution of different categories of samples by using a diagonalization method; diagonal elements of the covariance matrix of different categories are stored by using a proxy matrix, and the uncertainty range of the sample is mapped into a feature space by the proxy matrix; singular value decomposition is performed on an output layer parameter matrix of a feature extraction network by using an SVD decomposition network; a left singular matrix is used to replace the original parameter matrix, the correlation of the Euclidean distance of the feature vector is not changed, the feature vector is projected into an orthogonal feature solution space, and the correlation decoupling of the latitude of the feature vector is completed; and the improvement of the metric loss framework is completed according to the extracted dimension correlation decoupled feature vector. The application reduces the calculation complexity and model complexity of the loss function, and improves the training efficiency and model performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a pedestrian re-identification based metric loss framework improvement method, system, terminal and computer readable storage medium. BACKGROUND

[0002] In recent years, with the surge in security needs, the total amount of video surveillance equipment installations has shown a sustained high growth trend. In the face of massive monitoring data, namely massive images and videos, how to actively analyze these data and mine useful information has become an important content in social public security prevention and control work.

[0003] Pedestrian re-identification technology has become an important part of intelligent monitoring systems because it can search for target images in cross-camera domain images by high-level external features such as pedestrian clothing, posture and contour. Its related theories and research have important scientific research value and application value. Pedestrian re-identification research based on metric learning has made considerable achievements with the development of deep learning, such as the triplet loss, which inputs a triplet containing two same-class samples and one different-class sample to the model each time, and the optimization goal is to always make the distance between the same-class sample pair less than a certain threshold relative to the distance between the different-class sample pair. For example, using triplet loss in image retrieval tasks to mine sample similarity; then, applying triplet loss to face recognition tasks and designing a multi-scale network structure to extract image features. For example, in the pedestrian re-identification task, the triplet loss is used to construct the relationship between pedestrians of different identities in the feature space. There is also a Gaussian uncertainty modeling algorithm for the distribution of samples in the triplet loss function, but the improved loss function has the problem of high computational complexity. At the same time, the existing metric learning methods do not decouple the feature latitude at the input feature vector level, resulting in high model complexity.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a pedestrian re-identification based metric loss framework improvement method, system, terminal and computer readable storage medium, which aims to solve the problem that the existing metric learning methods in the prior art do not decouple the feature latitude at the input feature vector level, resulting in high model complexity.

[0006] To achieve the above purpose, the present application provides a pedestrian re-identification based metric loss framework improvement method, which comprises the following steps:

[0007] The diagonalization method is used to reduce the dimension of the covariance matrix fitting the overall distribution of different categories of samples;

[0008] The diagonal elements of the covariance matrix of different categories are stored by using the proxy matrix, and the uncertainty range of the sample is mapped into the feature space by the proxy matrix;

[0009] The output layer parameter matrix of the feature extraction network is singular value decomposed by using the SVD decomposition network;

[0010] The left singular matrix is used to replace the original parameter matrix, the correlation of the feature vector Euclidean distance is not changed, the feature vector is projected into the orthogonal feature solution space, and the correlation decoupling of the feature vector latitude is completed;

[0011] According to the extracted dimension correlation decoupled feature vector, the improvement of the metric loss framework is completed.

[0012] In addition, in order to achieve the above purpose, the application also provides a pedestrian re-identification-based metric loss framework improvement system, wherein the pedestrian re-identification-based metric loss framework improvement system comprises:

[0013] The matrix dimension reduction module is used to reduce the dimension of the covariance matrix fitting the overall distribution of different categories of samples by using the diagonalization method;

[0014] The sample mapping module is used to store the diagonal elements of the covariance matrix of different categories by using the proxy matrix, and the uncertainty range of the sample is mapped into the feature space by the proxy matrix;

[0015] The singular value decomposition module is used to singular value decompose the output layer parameter matrix of the feature extraction network by using the SVD decomposition network;

[0016] The correlation decoupling module is used to replace the original parameter matrix with the left singular matrix, without changing the correlation of the feature vector Euclidean distance, projecting the feature vector into the orthogonal feature solution space, and completing the correlation decoupling of the feature vector latitude;

[0017] The framework improvement module is used to improve the metric loss framework according to the extracted dimension correlation decoupled feature vector.

[0018] In addition, in order to achieve the above purpose, the application also provides a terminal, wherein the terminal comprises a memory, a processor, and a pedestrian re-identification-based metric loss framework improvement program stored on the memory and executable on the processor, and the pedestrian re-identification-based metric loss framework improvement program is executed by the processor to realize the steps of the pedestrian re-identification-based metric loss framework improvement method as described above.

[0019] In addition, in order to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a pedestrian re-identification based metric loss framework improvement program, and the pedestrian re-identification based metric loss framework improvement program, when executed by a processor, implements the steps of the pedestrian re-identification based metric loss framework improvement method as described above.

[0020] The present application utilizes a diagonalization method to reduce the dimension of the covariance matrix fitting the overall distribution of samples of different categories, stores the diagonal elements of the covariance matrices of different categories by using a proxy matrix, and maps the uncertainty range of the sample into the feature space through the proxy matrix; singular value decomposition is performed on the output layer parameter matrix of the feature extraction network by using an SVD decomposition network; the left singular matrix is used to replace the original parameter matrix, without changing the Euclidean distance correlation of the feature vector, the feature vector is projected into the orthogonal feature solution space, and the correlation decoupling of the latitude of the feature vector is completed; and the improvement of the metric loss framework is completed according to the extracted dimension correlation decoupled feature vector. The present application reduces the calculation complexity and model complexity of the loss function, and improves the training efficiency and model performance. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flowchart of a preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0022] Figure 2 is a schematic diagram of utilizing a proxy matrix to control diagonalization covariance updating in the preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0023] Figure 3 is a schematic diagram of an SVD decomposition network principle in the preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0024] Figure 4 is a schematic diagram of an improved triplet loss metric loss framework in the preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0025] Figure 5 is a schematic diagram of a traditional intra-class center algorithm based on parallelization calculation;

[0026] Figure 6 is a schematic diagram of a category center algorithm based on parallelization calculation in the preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0027] Figure 7 is a schematic diagram of loss value convergence before and after adding a diagonalization dimension reduction method in the preferred embodiment of the pedestrian re-identification based metric loss framework improvement method of the present application;

[0028] Figure 8 is a flow chart of an improved Gaussian sampling uncertainty triplet classification algorithm in a preferred embodiment of the metric loss framework improvement method based on pedestrian re-identification of the present application;

[0029] Figure 9 is a schematic diagram of the actual distribution of samples in the feature space in a preferred embodiment of the metric loss framework improvement system based on pedestrian re-identification of the present application;

[0030] Figure 10 is a principle schematic diagram of a preferred embodiment of the metric loss framework improvement system based on pedestrian re-identification of the present application;

[0031] Figure 11 is a running environment schematic diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0033] The present application aims to adapt different categories of samples to the distribution range of uncertainty in the feature space, and constructs a method based on Singular Value Decomposition (SVD) network combined with covariance diagonalization matrix, which simultaneously realizes correlation on the feature latitude level of the feature vector and the covariance matrix to be fitted, so as to only need to train and learn the elements of the covariance matrix diagonalization, so as to achieve the purpose of reducing the computational complexity and model complexity.

[0034] As shown in Figure 1 the metric loss framework improvement method based on pedestrian re-identification includes the following steps:

[0035] Step S10, using a diagonalization method to reduce the dimension of the covariance matrix fitting the overall distribution of different categories of samples.

[0036] Specifically, since the distribution of different categories in the sample set is not the same, when modeling the sample distribution, the covariance matrix of different categories is learned for each category.

[0037] If a training image dataset with C categories exists where y is the category label corresponding to the picture I, and N is the number of training images.

[0038] For the ith training set picture I, the feature vector extracted by the feature extraction network θ(·) is denoted as where n is the dimension of the eigenvector space, T represents transposition, for example, the direction of a vector can be changed from horizontal to vertical; the covariance matrix Σ a of the class samples is represented as follows:

[0039]

[0040] where a nn represents the variance of the n*n-dimensional feature.

[0041] When the covariance matrix is in general form, since the feature space dimension of the eigenvector extracted by the feature extraction network θ(·) is n, there are n 2 × n elements of the covariance matrix of one class that need to be updated in the process of iterative training. This also means that, taking a sample set with C classes as an example, C×n×n element parameter matrices need to be calculated each time the iteration is updated. However, in a high-dimensional feature space, most of the multi-dimensional Gaussian distributions tend to be distributed in a few dimensions in the feature space. Based on this assumption, in order to solve the problem of quadratic growth of the covariance matrix calculation amount caused by the growth of the feature space dimension, diagonalization, block diagonalization or multi-dimensional unit matrix can be used, that is, the original covariance matrix Σ a is constrained to decouple the correlation between each dimension. Since the covariance matrix is a real symmetric matrix, it can be diagonalized, that is, decoupled. Assuming that matrix A is the covariance matrix in the text, the specific operation is as follows: P -1 AP=P -1 P∧=∧, where the diagonal elements of the diagonal matrix ∧ are the n eigenvalues of the matrix A (the n eigenvalues may be repeated), P is composed of the eigenvectors of the matrix A, and P -1 is the inverse matrix of P. Therefore, the diagonalized covariance matrix is obtained:

[0042]

[0043] Under the condition of diagonalization constraint, the number of elements to be updated of the covariance matrix of each class is reduced to n, which also means that, for a training image data set with C classes and a feature space with a dimension of n, there are only C×n elements to be calculated for the covariance matrix Σ a , and the calculation complexity is reduced from quadratic complexity O(C×n 2 ) to linear complexity O(C×n).

[0044] In step S20, the diagonal elements of the covariance matrices of different classes are stored by using a proxy matrix, and the uncertainty range of the sample is mapped into the feature space by the proxy matrix.

[0045] Specifically, as shown in Figure 2 First, a Cxn size parameter matrix is encoded for storing the diagonal elements of the covariance matrix of each category, where C represents the number of categories in the data set, n represents the dimension of the feature space of the feature extraction model, n = 256 in the present application, the parameter matrix is called the proxy matrix, in the process of each iteration, according to the label index of the anchor in the training sample triplet, the row vector where the label index is located in the proxy matrix is extracted.

[0046] The extracted row vector is used to diagonalize the covariance matrix of the anchor, the covariance matrix is mapped into the feature space of the training image data set, the uncertainty of the anchor in the feature space is Gaussian modeled through the covariance matrix of the anchor and the feature vector, and the triplet loss loss of the uncertainty modeling is calculated, and the gradient updated through back propagation is updated to the proxy matrix of the covariance matrix.

[0047] Step S30, singular value decomposition network is used to singular value decompose the output layer parameter matrix of the feature extraction network.

[0048] Specifically, for any k x j size real matrix W, where k > j, the goal of singular value decomposition is to decompose the real matrix W into the following form:

[0049] W = USV T ;

[0050] Wherein, U, V are unit orthogonal matrices, respectively called left singular matrix and right singular matrix, U ∈ R m×m , V ∈ R n×n , R m×m Indicates that m x m real matrix, R n×m Indicates that n x n real matrix, for example, U ∈ R m×m Indicates that U is an m x m real matrix;

[0051]

[0052] Generally, the form of matrix S is as follows:

[0053]

[0054] Wherein, matrix S has singular value σ i only on the main diagonal, solve singular matrices U, V, and let:

[0055] WW T = USV T VS T U T ;

[0056] Because WW T∈R m×m By performing eigenvalue decomposition, we can obtain:

[0057]

[0058] Where the characteristic matrix Q is an orthogonal matrix, λ i The eigenvalues ​​are the column vectors q. i The characteristic matrix Q is formed, and the solution method is as follows: Let A = WW T :

[0059]

[0060] At this point, the left singular matrix U = Q; similarly, the right singular matrix V can be obtained.

[0061] like Figure 3 As shown, the principle of the SVD decomposition network is to add an unbiased, fully connected (FC) layer between the feature extraction network and the output feature vector. This projects the intermediate layer's feature vector onto an orthogonal basis, thus decoupling the feature vectors from dimensionality correlation. Specifically, for a set of input images forming a triplet I... a I p I n , among which, I a Indicates anchor point sample, I p Indicates a positive sample, I n Representing negative samples, the intermediate layer feature vectors obtained after passing through the feature extraction network are f, respectively. a f p f n If the parameter matrix of the added FC-fully connected layer is W, we can obtain:

[0062]

[0063] Where, x a x n x p This is the output feature vector after mapping by the parameter matrix W.

[0064] Singular value decomposition (SVD) is performed on the parameter matrix to obtain:

[0065] W = USV T ;

[0066] Where S is a non-square matrix with values ​​only in the main character's storyline, U is a left singular matrix, and V is a right singular matrix, all of which are orthogonal identity matrices.

[0067]

[0068] Step S40, the left singular matrix is used to replace the original parameter matrix, the correlation of the characteristic vector Euclidean distance is not changed, the characteristic vector is projected into the orthogonal characteristic solution space, and the correlation decoupling of the characteristic vector latitude is completed.

[0069] Specifically, the parameter matrix W is replaced by US, and for the characteristic vector x a , the Euclidean distance of x n is:

[0070]

[0071] The Euclidean distance of the output is represented by using the intermediate layer characteristic vector f a , f p .

[0072]

[0073] Since the right singular matrix V is an orthogonal matrix, the above formula can be obtained:

[0074]

[0075] That is, when the parameter matrix W of the feature mapping of the previous layer of the output layer is replaced by its left singular matrix US, the Euclidean distance of the characteristic vector of the output layer in the characteristic space will not be changed, and only the characteristic vector is projected into the solution space composed of orthogonal basis, thereby realizing the solution correlation of the characteristic dimension.

[0076] Step S50, according to the extracted dimension correlation decoupled characteristic vector, the improvement of the metric loss framework is completed.

[0077] Specifically, as shown in Figure 4 , K categories are sampled in the training image data set, P sample pictures are sampled for each category to form an iterative batch training sample set; for the anchor point picture in the training sample set, on the one hand, the parallel intra-class center algorithm is used to calculate the class center and obtain the distance from the intra-class center; on the other hand, the diagonalized covariance matrix of the anchor point category is encoded by the proxy matrix, and the Gaussian uncertainty range is mapped into the feature space; the feature extraction backbone network combines the SVD decomposition network to extract the dimension correlation decoupled characteristic vector; the loss of the sample group is calculated by using the proposed uncertainty modeling triplet loss, and the backbone network and the covariance matrix of each category to be fitted are updated by the back propagation algorithm.

[0078] The triplet loss measurement loss framework based on pedestrian re-identification of the application improves the method by using a diagonalization method to reduce the dimension of the covariance matrix (the covariance matrix fitting the overall distribution of different category samples) ; the diagonal elements of the covariance matrix of different categories are stored by using a proxy matrix, and the uncertainty range of the sample is mapped into the feature space by using the proxy matrix; the output layer parameter matrix of the feature extraction network is singular value decomposed by using an SVD decomposition network; the left singular matrix is used to replace the original parameter matrix, so that the feature vectors are projected into the orthogonal feature solution space without changing the correlation of the Euclidean distance of the feature vectors, and the correlation decoupling of the feature vector latitude is completed; the measurement learning framework is improved according to the above method.

[0079] The application considers that different categories of samples adapt to the distribution range of uncertainty in the feature space in the original deep metric learning model, and constructs a method based on singular value decomposition network combined with covariance diagonalization matrix; the method simultaneously realizes the correlation decoupling of the feature latitude at the level of the feature vector and the covariance matrix to be fitted, so that only the diagonal elements of the covariance matrix diagonalization need to be trained and learned, so as to reduce the calculation complexity and the model complexity.

[0080] Further, as shown in Figure 5 , a traditional serial solving category center method is shown, first, the whole training batch of samples is traversed, the center coordinate vector corresponding to each category and the number of category samples are recorded by using a hash table, when a sample is traversed, the coordinate vector and the number counter of the sample category in the hash table are inquired, and the coordinate vector of the category in the hash table is accumulated, and the value of the number counter is accumulated; when the traversal loop ends, the key in the hash table is traversed, that is, the categories in the hash table are traversed, so that the feature vector sum and the number of samples of all current category samples in the current training samples can be obtained, and thus the geometric center of each category in the feature space can be obtained.

[0081] And the category center algorithm based on parallel calculation is shown in Figure 6 , first, an n*n-dimensional same category mask matrix is constructed by using an n*1-dimensional category label matrix, then, the same category mask matrix and the feature vector matrix are multiplied to obtain the accumulation sum of the same category sample feature vectors, finally, the number of times that each category sample appears in a batch of training samples is obtained in parallel by using the same category mask matrix, the Hadamard product of the number of times matrix and the feature vector sum matrix is used to obtain the matrix of each sample corresponding to the category center, and the intra-class center distance is obtained in parallel by subtracting the original feature vector matrix from the matrix.

[0082] To verify the influence of the covariance matrix diagonalization dimension reduction method on the triplet loss, the corresponding ablation experiment was done to compare the calculation efficiency and performance influence of the original triplet loss and the improved triplet loss after adding the diagonalization method. For calculation efficiency, the required time for the triplet loss to converge during the training process before and after adding the diagonalization method was compared to verify it. For performance influence, fine-grained classification experiments were conducted on three metric learning benchmark datasets to compare the experimental results. On the CUB-200-2011 dataset, the loss value convergence of the improved triplet loss method using diagonalization for covariance matrix dimension reduction is as shown in Figure 7

[0083] It can be seen that by adding the diagonalization method, the time required for the improved triplet loss loss function proposed to converge can be significantly reduced. It is shown from the side that the calculation efficiency of the proposed method has been improved after introducing the diagonalization dimension reduction method.

[0084] Table 1. Ablation experiment of improved triplet loss algorithm

[0085]

[0086] As shown in Table 1, by comparing the experimental results of the proposed benchmark method and the improved benchmark method using covariance diagonalization, it can be concluded that using diagonalization method for dimension reduction reduces the Top-1 accuracy of the Baseline method on the Stanford car, Stanford dog and CUB-200-2011 datasets by 4.5%, 2.0% and 5.2% respectively. By analyzing the principle of this improvement, it can be concluded that the reason is that diagonalizing the covariance matrix is only applied to one side of the covariance used to fit the sample distribution, that is, this method makes a strong assumption of dimension correlation decoupling for the distribution of features in each dimension direction in the feature space. However, the features extracted by the feature extraction model often have correlations in the dimension direction after being mapped by the feature extraction network parameters.

[0087] ​Subsequently, in order to verify the effectiveness of the improved metric loss framework using the SVD decomposition network and the diagonalization covariance matrix dimensionality reduction, the experiments of the present application are verified from two aspects. On the one hand, the performance of the improved metric framework before and after using the SVD decomposition network is compared through the ablation experiment on the fine-grained classification task, so as to verify the distinguishability of the proposed method in different fine-grained categories. On the other hand, the effectiveness of the learned covariance matrix of different categories is verified by comparing the learned covariance matrix of different categories with the actual distribution of each category in the feature space. The experimental results are shown in Table 2. Through the analysis of the experimental results, it can be found that the improved triplet loss does not perform better on the fine-grained classification task only by adding the SVD decomposition network. The reason is that the SVD decomposition network only projects the features into the orthogonal solution space, and this method does not change the Euclidean distance between the features. However, in order to reduce the computational complexity of the proposed method, when the diagonalization method is used to reduce the dimensionality of the covariance matrix, the accuracy of the improved triplet loss on the three fine-grained classification datasets decreases. The reason for this phenomenon is that the diagonalization method is essentially a strong hypothesis that decouples the correlation of the dimensions of the features in the feature space, and the dimensions of the actual features in the feature space are not necessarily orthogonal and decoupled. After further adding the SVD decomposition network to project the features into the orthogonal solution space, the performance loss phenomenon of the accuracy decrease caused by the diagonalization dimensionality reduction method is alleviated to a certain extent. Among them, on the Stanford car, Stanford Dog and CUB-200-2011 datasets, the method of combining diagonalization dimensionality reduction and SVD decomposition network respectively achieves Top-1 accuracy of 94.0%, 77.8% and 85.1%.

[0088] Table 2. Ablation experiment two of improved triplet loss algorithm

[0089]

[0090] As Figure 8As shown in the figure, the flow chart of the triplet classification algorithm based on Gaussian sampling uncertainty modeling is shown, first sample K classes in the training set, sample P sample pictures for each class to form an iterative batch training sample set. Then, for the anchor picture in the training sample set, on the one hand, the in-class center algorithm is used to calculate the class center and the distance from the in-class center; on the other hand, the diagonalization covariance matrix of the anchor class is encoded by the proxy matrix, and the Gaussian uncertainty range is mapped into the feature space. The feature extraction backbone network combines the SVD decomposition network to extract the dimensionally decoupled feature vector. Finally, the improved triplet loss is used to calculate the loss of the sample group, and the backbone network and the covariance matrix of each class to be fitted are updated by the back propagation algorithm.

[0091] The feature vectors of the sample images are reduced to three-dimensional space for visualization by using the t-SNE dimension reduction method. As shown in the figure, Figure 9 As shown in the figure, the first 20 classes of sample images of Stanford Dog are visualized, and it can be found that the pictures of the same class tend to be aggregated in the feature space. Therefore, it can be concluded that the proposed metric framework can make the model have better distinguishability for similar samples in the fine-grained class division task, while also taking into account certain computational efficiency.

[0092] Further, the possible design direction of the present application is: the method based on singular value decomposition network combined with covariance diagonalization matrix can be used in fields other than pedestrian re-identification, that is, SVD decomposition network can be used for decoupling as long as there is strong correlation between each feature, and the covariance matrix diagonalization method can also be used in functions containing covariance matrix.

[0093] Further, as shown in the figure, Figure 10 Based on the above pedestrian re-identification based metric loss framework improvement method, the present application also correspondingly provides a pedestrian re-identification based metric loss framework improvement system, wherein the pedestrian re-identification based metric loss framework improvement system comprises:

[0094] The matrix dimension reduction module 51 is used for dimension reduction processing of the covariance matrix fitting the overall distribution of different class samples by using the diagonalization method;

[0095] The sample mapping module 52 is used for storing the diagonal elements of the covariance matrix of different classes by using the proxy matrix, and mapping the uncertainty range of the sample into the feature space by the proxy matrix;

[0096] The singular value decomposition module 53 is used for singular value decomposition of the output layer parameter matrix of the feature extraction network by using the SVD decomposition network;

[0097] The correlation decoupling module 54 is configured to replace the original parameter matrix with a left singular matrix, without changing the Euclidean distance correlation of the eigenvectors, project the eigenvectors into an orthogonal eigenspace, and complete the correlation decoupling of the latitude of the eigenvectors.

[0098] The framework improvement module 55 is configured to improve the metric loss framework according to the dimensionally decoupled eigenvectors extracted.

[0099] Further, as shown in the figure, Figure 11 Based on the pedestrian re-identification-based metric loss framework improvement method and system, the application further provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 8 Only some components of the terminal are shown, but it should be understood that all the components shown are not required, and more or fewer components can be alternatively implemented.

[0100] The memory 20 can be an internal storage unit of the terminal in some embodiments, for example, a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is configured to store application software and various data installed on the terminal, for example, program codes of the terminal, etc. The memory 20 can also be configured to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a pedestrian re-identification-based metric loss framework improvement program 40, which can be executed by the processor 10, so as to implement the pedestrian re-identification-based metric loss framework improvement method.

[0101] The processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, and is configured to run program codes or process data stored in the memory 20, for example, execute the pedestrian re-identification-based metric loss framework improvement method, etc.

[0102] The display 30 can be, in some embodiments, an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display 30 is used to display information at the terminal and to display a visualized user interface. The components 10-30 of the terminal communicate with each other through a system bus.

[0103] In an embodiment, the steps of the pedestrian re-identification based metric loss framework improvement method are implemented when the processor 10 executes the pedestrian re-identification based metric loss framework improvement program 40 in the memory 20.

[0104] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a pedestrian re-identification based metric loss framework improvement program, and the pedestrian re-identification based metric loss framework improvement program, when executed by a processor, implements the steps of the pedestrian re-identification based metric loss framework improvement method as described above.

[0105] In summary, the present application provides a pedestrian re-identification based metric loss framework improvement method and related equipment, the method comprising: using a diagonalization method to perform dimensionality reduction processing on a covariance matrix fitting the overall distribution of samples of different categories; using a proxy matrix to store the diagonal elements of the covariance matrices of different categories, and mapping the uncertainty range of the samples to the feature space through the proxy matrix; using an SVD decomposition network to perform singular value decomposition on the output layer parameter matrix of the feature extraction network; using a left singular matrix to replace the original parameter matrix, without changing the correlation of the feature vectors in the Euclidean distance, projecting the feature vectors into the orthogonal feature solution space, and completing the correlation decoupling of the latitude of the feature vectors; and completing the improvement of the metric loss framework according to the extracted dimensionally correlated decoupled feature vectors. The present application reduces the computational complexity of the loss function and the model complexity, and improves the training efficiency and the model performance.

[0106] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, such that processes, methods, articles, or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or terminals. Without more limitations, the element defined by the statement "including a…" does not exclude the presence of another identical element in the process, method, article, or terminal including the element.

[0107] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0108] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A pedestrian re-identification based metric loss framework improvement method, characterized in that, The pedestrian re-identification-based metric loss framework improvement method comprises: a diagonalization method is used to reduce the dimension of the covariance matrix fitting the overall distribution of samples of different categories; an agent matrix is used to store the diagonal elements of the covariance matrix of different categories, and the uncertainty range of the sample is mapped into the feature space through the agent matrix; an SVD decomposition network is used to singular value decompose the output layer parameter matrix of the feature extraction network; a left singular matrix is used to replace the original parameter matrix without changing the Euclidean distance correlation of the feature vector, the feature vector is projected into the orthogonal feature solution space, and the correlation decoupling of the feature vector latitude is completed; the dimension correlation decoupled feature vector is extracted, and the improvement of the metric loss framework is completed; the diagonalization method is used to reduce the dimension of the covariance matrix fitting the overall distribution of samples of different categories, and the covariance matrix fitting the overall distribution of samples of different categories is reduced in dimension by using the diagonalization method, specifically comprising: when modeling the sample distribution, learn the covariance matrix of different categories for each category respectively; If a training image dataset exists with C classes where y is the class label corresponding to the image I, and N is the number of training images; For the ith training set picture I, the feature vector extracted by the feature extraction network θ(·) is denoted as where n is the dimension of the feature vector space, T represents transposition; the covariance matrix Σ of the category sample is denoted as a is expressed as follows: where a nn denotes the variance of the n*n dimensional feature; The original covariance matrix Σ a To enforce the constraint, we decouple the correlation between dimensions and obtain a diagonalized covariance matrix: Under the condition of diagonalization constraint, the number of elements to be updated of the covariance matrix of each class is reduced to n, in the case of a training image dataset with C classes, the dimension of the feature space is n-dimensional, for the covariance matrix a There are only C x n elements in total, and the computational complexity is reduced from quadratic complexity O(C x n 2 ) to linear complexity O(C x n).

2. The pedestrian re-identification based metric loss framework improvement method according to claim 1, characterized in that, the agent matrix is used to store the diagonal elements of the covariance matrix of different categories, and the uncertainty range of the sample is mapped into the feature space through the agent matrix, specifically comprising: a C*n size parameter matrix is used to store the diagonal elements of the covariance matrix of each category, and the parameter matrix is called an agent matrix; in each iteration process, according to the label index of the anchor point in the training sample triplet, the row vector of the label index in the agent matrix is extracted; the extracted row vector is used to diagonalize and reconstruct the covariance matrix of the anchor point, the covariance matrix is mapped into the feature space of the training image dataset, the uncertainty of the anchor point in the feature space is Gaussian modeled through the covariance matrix of the anchor point and the feature vector, the triplet loss loss of the uncertainty modeling is calculated, and the gradient updated through back propagation is updated to the agent matrix of the covariance matrix. 3.The pedestrian re-identification based metric loss framework improvement method according to claim 2, characterized in that, the SVD decomposition network is used to singular value decompose the output layer parameter matrix of the feature extraction network, specifically comprising: for any k*j size real matrix W, where k>j, the singular value decomposition target is to decompose the real matrix W into the following form: W = USV T ; wherein U, V are unitary matrices, respectively called left and right singular matrices, U ∈ R m×m , V ∈ R n×n , R m×m denotes an m x m real matrix, R n×n denotes an n x n real matrix; the form of the matrix S is as follows: where the matrix S has singular values σ only on the main diagonal i Solve the singular matrices U, V, and let WW T = USV T VS T U T ; Since WW T ∈ R m×m , performing eigen decomposition gives Wherein, the characteristic matrix Q is a unit orthogonal matrix, λ i is an eigenvalue, and the column vector q i constitutes the characteristic matrix Q, and the solving method is as follows: let A=WW T ​ at this time, the left singular matrix U=Q; For a triple I a , I p , I n , where I a represents an anchor sample, I p represents a positive sample, and I n represents a negative sample, the intermediate layer feature vectors obtained through the feature extraction network are f a , f p , and f n , respectively, if the parameter matrix of the added FC-full connection layer is W, then the following is obtained: where x a , x n , x p are the output feature vectors after mapping by the parameter matrix W, and the parameter matrix W is subjected to singular value decomposition (SVD) to obtain: W = USV T ; where S is a non-square matrix with only the main angle line having a value, U is a left singular matrix, V is a right singular matrix, and both are unit orthogonal matrices, that is:

4. The pedestrian re-identification based metric loss framework improvement method according to claim 3, characterized in that, between the feature extraction network and the output feature vector, an FC-full connection layer without a bias term is added to project the intermediate layer feature vector onto the orthogonal basis, so as to realize the decoupling of the feature vector in the dimension correlation. 5.The pedestrian re-identification based metric loss framework improvement method according to claim 3, characterized in that, the left singular matrix is used to replace the original parameter matrix without changing the Euclidean distance correlation of the feature vector, the feature vector is projected into the orthogonal feature solution space, and the correlation decoupling of the feature vector latitude is completed, specifically comprising: Replacing the parameter matrix W with US, for the eigenvector x a , the Euclidean distance of x n is: Using the intermediate layer feature vector f a , f p Representing the Euclidean distance to the output: where, since the right singular matrix V is a unit orthogonal matrix, the above formula can be obtained: 6.The pedestrian re-identification based metric loss framework improvement method according to claim 5, characterized in that, the dimension correlation decoupled feature vector is extracted, and the improvement of the metric loss framework is completed, specifically comprising: K categories are sampled in the training image dataset, and P sample pictures of each category are sampled to form an iterative batch training sample set; For anchor pictures in the training sample set, the class center is calculated by using a parallelized in-class center algorithm, and the distance from the class center to the in-class center is obtained; The diagonalized covariance matrix of the anchor class is encoded by a proxy matrix, and the Gaussian uncertainty range is mapped into the feature space; The feature extraction backbone network is combined with an SVD decomposition network to extract a feature vector with decoupled dimension correlation; The loss of the sample group is calculated by using the proposed uncertainty modeling triplet loss, and the backbone network and the covariance matrix of each class to be fitted are updated by using a back propagation algorithm.

7. A pedestrian re-identification based metric loss framework improvement system, characterized in that, The pedestrian re-identification based metric loss framework improvement system is used to perform the pedestrian re-identification based metric loss framework improvement method according to any one of claims 1-6, and the pedestrian re-identification based metric loss framework improvement system comprises: A matrix dimension reduction module is configured to perform dimension reduction processing on the covariance matrix of the sample population distribution of different classes by using a diagonalization method; A sample mapping module is configured to store diagonal elements of the covariance matrix of different classes by using a proxy matrix, and map the uncertainty range of the sample into the feature space by using the proxy matrix; An SVD decomposition module is configured to perform singular value decomposition on the output layer parameter matrix of the feature extraction network by using an SVD decomposition network; A correlation decoupling module is configured to replace the original parameter matrix with a left singular matrix, without changing the Euclidean distance correlation of the feature vector, project the feature vector into an orthogonal feature solution space, and complete correlation decoupling of the latitude of the feature vector; A framework improvement module is configured to complete improvement of the metric loss framework according to the extracted feature vector with decoupled dimension correlation.

8. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a pedestrian re-identification based metric loss framework improvement program stored on the memory and executable on the processor, and the pedestrian re-identification based metric loss framework improvement program, when executed by the processor, implements the steps of the pedestrian re-identification based metric loss framework improvement method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a pedestrian re-identification based metric loss framework improvement program, and the pedestrian re-identification based metric loss framework improvement program, when executed by the processor, implements the steps of the pedestrian re-identification based metric loss framework improvement method according to any one of claims 1-6.

Images