Pedestrian re-identification method and system based on unsupervised learning

Abstract

The embodiment of the present invention provides a pedestrian re-identification method and system based on unsupervised learning, firstly obtaining two video frames to be identified containing several pedestrians; then inputting the two video frames to be identified into the pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian. The pedestrian re-identification model adopted in the embodiment of the present invention is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, the circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, the optimized loss function is determined. The entire training process does not require any additional algorithm modules or indirect supervision signals, such as pedestrian tracking modules or clustering algorithms, etc., and pedestrian features can be directly learned from unlabeled sample video frames to realize pedestrian re-identification, which simplifies the entire training process of the pedestrian re-identification model and makes the accuracy of pedestrian re-identification higher.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a pedestrian re-identification method and system based on unsupervised learning. Background Art

[0002] At present, pedestrian re-identification is an important application in the field of computer vision. It aims to identify the identity of pedestrians through the appearance information of pedestrians captured by the camera, and can perform identity verification and retrieval. Pedestrian re-identification has very important application prospects in security monitoring, smart cities and smart retail.

[0003] Mainstream pedestrian re-identification methods are based on deep learning and deep convolutional neural network models trained with large-scale labeled data as pedestrian feature extraction models, and then judge whether they are the same identity by comparing the feature similarities between different pedestrians. This method has achieved good practical results, but the effect is often significantly reduced when the data domain of the application scenario is very different from the data domain of the training sample. For example, the pedestrian feature extraction model trained using pedestrian data collected in Beijing in winter as training samples is often not well promoted and applied to summer scenes in Shanghai. A feasible solution is to label training samples in various scenarios as much as possible, but the cost of labeling pedestrian identities is too high, making this method unable to solve the problem well.

[0004] In order to solve this problem, the prior art provides a pedestrian re-identification method based on unsupervised learning, which obtains a pedestrian feature extraction model with good generalization performance through learning a large amount of unlabeled pedestrian data for pedestrian re-identification, that is, unsupervised learning is performed on the pedestrian feature extraction model. The unsupervised learning method of the pedestrian feature extraction model specifically introduces some auxiliary information provided by a third-party algorithm module as a supervisory signal, such as using multi-target tracking: when the tracking is correct, the individuals in the same trajectory are the same person, while the individuals in different trajectories are not the same person; or using self-paced learning to perform continuous clustering-training iterations.

[0005] However, in the pedestrian re-identification methods based on unsupervised learning existing in the prior art, the pedestrian feature extraction model overly relies on the auxiliary information provided by the third-party algorithm module for auxiliary supervision, and the auxiliary supervision often makes mistakes, resulting in inaccurate pedestrian re-identification results. For example, the use of multi-target tracking methods to unsupervisedly learn the pedestrian feature extraction model strongly depends on the quality of the tracking effect. Once the tracking error occurs (such errors are easy to occur in reality), the supervision signal is wrong. For another example, the unsupervised learning of the pedestrian feature extraction model through the clustering-training iteration method requires assigning a pseudo-label to each pedestrian, but it is very likely that the assignment of the pseudo-label is unreasonable or even wrong, which will greatly affect the unsupervised learning of the pedestrian feature extraction model, resulting in the inability to guarantee the pedestrian re-identification effect. Summary of the invention

[0006] In order to overcome the above problems or at least partially solve the above problems, an embodiment of the present invention provides a pedestrian re-identification method and system based on unsupervised learning.

[0007] In a first aspect, an embodiment of the present invention provides a pedestrian re-identification method based on unsupervised learning, comprising:

[0008] Obtain two video frames containing several pedestrians to be identified;

[0009] Inputting two video frames to be identified into a person re-identification model, and determining by the person re-identification model whether the two video frames to be identified contain the same person;

[0010] Among them, the pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, an optimized loss function is determined; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames.

[0011] Preferably, when the pedestrian re-identification model is trained, it specifically includes a first training stage and a second training stage; accordingly,

[0012] The two sample video frames containing a number of pedestrians in the first training stage are specifically: two first-category video frames in a first-category video containing a number of pedestrians captured by the same camera device;

[0013] The two sample video frames containing several pedestrians in the second training stage are specifically: two video frames in a mixed video frame set; the mixed video frame set is obtained by mixing the second type of video frames corresponding to the same moment in a second type of video pair containing several pedestrians, which are captured by two cameras with the same field of view, and the first type of video frames.

[0014] Preferably, determining a cyclic allocation matrix of pedestrians between two sample video frames containing a plurality of pedestrians specifically comprises:

[0015] Based on the pedestrian re-identification model, pedestrian features of each pedestrian in two sample video frames are respectively extracted, each sample video frame corresponds to a group of pedestrian features, and a cosine similarity matrix of the two groups of pedestrian features is determined;

[0016] Applying a Softmax function to each row of the cosine similarity matrix to determine a forward allocation matrix, and applying a Softmax function to each row of the transposed matrix of the cosine similarity matrix to determine a reverse allocation matrix;

[0017] The cyclic allocation matrix is ​​determined based on the forward allocation matrix and the reverse allocation matrix.

[0018] Preferably, the determining of the cosine similarity matrix of the two groups of pedestrian features specifically includes:

[0019] Arranging the pedestrian features of each pedestrian in the first group of pedestrian features of the two groups of pedestrian features in rows, and arranging the pedestrian features of each pedestrian in the second group of pedestrian features in columns, to obtain feature matrices corresponding to the first group of pedestrian features and the second group of pedestrian features respectively;

[0020] The feature matrices corresponding to the first set of pedestrian features and the second set of pedestrian features are multiplied to obtain the cosine similarity matrix.

[0021] Preferably, if the number of pedestrians in the two sample video frames is equal, then determining the optimization loss function based on the cyclic allocation matrix specifically includes:

[0022] The optimization loss function is determined by the following formula:

[0023]

[0024] Among them, L symmetric is the optimization loss function, K is the number of pedestrians in each sample video frame, A cycle is the cyclic allocation matrix, and I is a unit matrix having the same size as the cyclic allocation matrix.

[0025] Preferably, the step of determining the optimization loss function based on the cyclic allocation matrix further comprises:

[0026] The optimization loss function is determined by the following formula:

[0027]

[0028] Among them, L asymmetricis the optimization loss function, K1 is the number of rows in the loop allocation matrix, The element at the i-th row and j-th column in the loop allocation matrix is ​​allocated to the loop, The element at the i-th row and the i-th column in the loop allocation matrix is ​​allocated to the loop, The element in the kth row and jth column of the loop allocation matrix is ​​allocated to the loop, where m is a constant.

[0029] In a second aspect, an embodiment of the present invention provides a pedestrian re-identification system based on unsupervised learning, including: an acquisition module and a re-identification module.

[0030] The acquisition module is used to acquire two video frames to be identified that contain a number of pedestrians;

[0031] The re-identification module is used to input two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian;

[0032] Among them, the pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, an optimized loss function is determined; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames.

[0033] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the pedestrian re-identification method based on unsupervised learning as described in the first aspect are implemented.

[0034] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the pedestrian re-identification method based on unsupervised learning as described in the first aspect.

[0035] The embodiment of the present invention provides a pedestrian re-identification method and system based on unsupervised learning, which first obtains two video frames to be identified containing several pedestrians; then the two video frames to be identified are input into the pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian. The pedestrian re-identification model used in the embodiment of the present invention is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, the circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, the optimization loss function is determined. The entire training process does not require any additional algorithm modules or indirect supervision signals, such as pedestrian tracking modules or clustering algorithms, etc. Pedestrian features can be directly learned from unlabeled sample video frames to achieve pedestrian re-identification, which simplifies the entire training process of the pedestrian re-identification model and makes the accuracy of pedestrian re-identification higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flowchart of a pedestrian re-identification method based on unsupervised learning provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of sources of two sample video frames in a pedestrian re-identification method based on unsupervised learning provided by an embodiment of the present invention;

[0039] Figure 3a A schematic diagram of the spatial distribution of pedestrian features of the same pedestrian in different video frames extracted by an untrained pedestrian re-recognition model in a pedestrian re-recognition method based on unsupervised learning provided by an embodiment of the present invention;

[0040] Figure 3b A schematic diagram of the spatial distribution of pedestrian features of the same pedestrian in a first type of video frames of the same camera device extracted by a pedestrian re-recognition model after a first training phase in a pedestrian re-recognition method based on unsupervised learning provided by an embodiment of the present invention;

[0041] Figure 3c A schematic diagram of the spatial distribution of pedestrian features of the same pedestrian in different video frames extracted by the pedestrian re-identification model after the second training phase in a pedestrian re-identification method based on unsupervised learning provided by an embodiment of the present invention;

[0042] Figure 4A schematic diagram of the structure of a pedestrian re-identification system based on unsupervised learning provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a pedestrian re-identification method based on unsupervised learning, comprising:

[0046] S1, obtaining two video frames containing several pedestrians to be identified;

[0047] S2, inputting the two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian;

[0048] Among them, the pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, an optimized loss function is determined; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames.

[0049] Specifically, the pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention is executed by a server, which can be a computer or a cloud server, etc., and is not specifically limited in the embodiment of the present invention.

[0050] First, execute step S1. When acquiring two video frames to be identified that contain several pedestrians, they can be acquired specifically by a camera device, which is connected to a server for communication, and the acquired two video frames to be identified are transmitted to the server for re-identification (i.e., re-identification), that is, to determine whether the two video frames to be identified contain the same pedestrian. It should be noted that the camera device can specifically be a camera, and its application scenario can be a place with dense pedestrians, such as a street, a subway station, an airport, etc. The camera device can acquire video, and the two video frames to be identified can be video frames corresponding to two different moments in the same video, or two video frames corresponding to the same moment or different moments in different videos, which is not specifically limited in the embodiments of the present invention. Each video frame to be identified includes several pedestrians, forming a pedestrian group, and the number of pedestrians in the pedestrian group can be one or more.

[0051] Then execute step S2. The pedestrian re-identification model is used to determine whether the two video frames to be identified contain the same pedestrian, that is, for example, there are n1 pedestrians in the first video frame to be identified and n2 pedestrians in the second video frame to be identified, and it is determined whether each pedestrian in the first video frame to be identified appears in the second video frame to be identified. The pedestrian re-identification model used in the embodiment of the present invention is constructed based on a deep convolutional neural network, and when training the pedestrian re-identification model, a training sample set is first constructed, and the training sample set includes multiple sample video frames, each sample video frame contains a number of pedestrians, forming a pedestrian group, and the number of pedestrians in the pedestrian group can be 1 or more; then, the cyclic allocation matrix of pedestrians between the two sample video frames is determined, and the cyclic allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames, that is, the value of each element in the cyclic allocation matrix is ​​a number between 0 and 1. Assuming that the two sample video frames are a and b, when determining the cyclic allocation matrix, the forward allocation matrix obtained by matching the pedestrian group in the sample video frame a to the pedestrian group in the sample video frame b and the reverse allocation matrix obtained by matching the pedestrian group in the sample video frame b to the pedestrian group in the sample video frame a can be determined first, and the cyclic allocation matrix can be determined by combining the forward allocation matrix and the reverse allocation matrix. It is easy to understand that the cyclic allocation matrix is ​​to allocate each pedestrian to itself after a forward and reverse cycle, so in the embodiment of the present invention, the optimization loss function applied when training the pedestrian re-identification model is determined based on the cyclic allocation matrix.

[0052] The pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention first obtains two video frames to be identified containing several pedestrians; then the two video frames to be identified are input into the pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian. The pedestrian re-identification model used in the embodiment of the present invention is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, the circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, the optimization loss function is determined. The entire training process does not require any additional algorithm modules or indirect supervision signals, such as pedestrian tracking modules or clustering algorithms, and can directly learn pedestrian features from unlabeled sample video frames to achieve pedestrian re-identification, thereby simplifying the entire training process of the pedestrian re-identification model and making pedestrian re-identification more accurate.

[0053] On the basis of the above-mentioned embodiment, the pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention specifically includes a first training stage and a second training stage when training the pedestrian re-identification model; accordingly,

[0054] The two sample video frames containing a number of pedestrians in the first training stage are specifically: two first-category video frames in a first-category video containing a number of pedestrians captured by the same camera device;

[0055] The two sample video frames containing several pedestrians in the second training stage are specifically: two video frames in a mixed video frame set; the mixed video frame set is obtained by mixing the second type of video frames corresponding to the same moment in a second type of video pair containing several pedestrians, which are captured by two cameras with the same field of view, and the first type of video frames.

[0056] Specifically, in an embodiment of the present invention, the pedestrian re-identification model is trained through two training stages, namely the first training stage and the second training stage. In the first training stage, the two sample video frames used are sampling data in the camera device, specifically referring to two first-category video frames in the first-category video containing a number of pedestrians captured by the same camera device. The two first-category video frames in the first-category video specifically correspond to two video frames at different times, which may be adjacent or non-adjacent at different times. For example, the two sample video frames may be the 1374th frame and the 1380th frame in the first-category video, etc.

[0057] In the second training stage, the two sample video frames used are two video frames selected after mixing the sampling data within the camera device and the sampling data between the camera devices, which can be specifically understood as two video frames in the mixed video frame set. Figure 2As shown, the sampling data within the camera device refers to two first-category video frames in the first-category video containing several pedestrians captured by the same camera device e, for example, the 1374th frame and the 1380th frame in the first-category video. The sampling data between camera devices refers to the second-category video frames corresponding to the same time in the second-category video pair containing several pedestrians captured by two camera devices c and d with the same field of view, for example, the 1590th frame in the second-category video captured by camera device c and the 1590th frame in the second-category video captured by camera device d. Among them, the two camera devices with the same field of view refer to being able to capture the same area at the same time, and the area often passes by pedestrians. The mixed video frame set is obtained by mixing the sampling data within the camera device and the sampling data between the camera devices. In the embodiment of the present invention, only two camera devices with the same field of view can be used to obtain sample video frames. In the first training stage, two first-category video frames in the first-category video captured by any of the camera devices are used. In the second training stage, the second-category video frames and the first-category video frames obtained by the two camera devices at the same time are used.

[0058] In an embodiment of the present invention, when training a pedestrian re-identification model, it is divided into a first training stage and a second training stage. The first training stage only uses the sampled data in the camera device for training. Since the appearance of pedestrians in the sampled data in the camera device changes little, it is easier for the pedestrian re-identification model to learn, so the optimization loss function of the first training stage is easy to converge. After the optimization loss function converges, the second training stage is started. The second training stage uses mixed training with sampled data in the camera device and between camera devices. The appearance of pedestrians in the sampled data between camera devices changes greatly, which can enable the pedestrian re-identification model to learn a feature representation that is more robust to appearance changes.

[0059] On the basis of the above-mentioned embodiment, the pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention, wherein determining the circulant allocation matrix of pedestrians between two sample video frames containing a plurality of pedestrians specifically includes:

[0060] Based on the pedestrian re-identification model, pedestrian features of each pedestrian in two sample video frames are respectively extracted, each sample video frame corresponds to a group of pedestrian features, and a cosine similarity matrix of the two groups of pedestrian features is determined;

[0061] Applying a Softmax function to each row of the cosine similarity matrix to determine a forward allocation matrix, and applying a Softmax function to each row of the transposed matrix of the cosine similarity matrix to determine a reverse allocation matrix;

[0062] The cyclic allocation matrix is ​​determined based on the forward allocation matrix and the reverse allocation matrix.

[0063] Specifically, in an embodiment of the present invention, when determining the circulant allocation matrix, the pedestrian features of each pedestrian in the two sample video frames a and b can first be extracted by the pedestrian re-identification model, and the pedestrian features of each pedestrian can be specifically represented as a high-dimensional row vector. The pedestrian features of all pedestrians in the sample video frame a can constitute a set of pedestrian features, and similarly, the pedestrian features of all pedestrians in the sample video frame b can constitute a set of pedestrian features. Then, the cosine similarity matrix of the two sets of pedestrian features is determined. The method for determining the cosine similarity matrix can be implemented according to the method provided in the prior art, and is not specifically limited in the embodiment of the present invention.

[0064] The forward assignment matrix is ​​determined by applying the Softmax function to each row of the cosine similarity matrix. The Softmax function does not change the size of the cosine similarity matrix. It only processes each element in the cosine similarity matrix to obtain the forward assignment matrix. The forward assignment matrix is ​​used to characterize the probability of each pedestrian in the sample video frame a being assigned to the sample video frame b, that is, to find each pedestrian in the sample video frame a from the pedestrian group in the sample video frame b. The size of the forward assignment matrix is ​​consistent with that of the cosine similarity matrix, and each element in the forward assignment matrix has a value between 0 and 1.

[0065] The reverse allocation matrix is ​​determined by applying the Softmax function to each row of the transposed matrix of the cosine similarity matrix. The Softmax function does not change the size of the transposed matrix, but only processes each element in the transposed matrix to obtain the reverse allocation matrix. The reverse allocation matrix is ​​used to characterize the probability of each pedestrian in the sample video frame b being assigned to the sample video frame a, that is, to find each pedestrian in the sample video frame b from the pedestrian group in the sample video frame a. The size of the reverse allocation matrix is ​​consistent with the size of the transposed matrix, and each element in the reverse allocation matrix has a value between 0 and 1.

[0066] Finally, the cyclic allocation matrix is ​​determined by combining the forward allocation matrix and the reverse allocation matrix. Specifically, the forward allocation matrix and the reverse allocation matrix can be multiplied to obtain the cyclic allocation matrix. In the embodiment of the present invention, the cyclic allocation matrix indicates that each pedestrian is allocated to itself after a forward and reverse cycle.

[0067] Figure 3a Schematic diagram of the spatial distribution of pedestrian features of the same pedestrian in different video frames extracted by an untrained pedestrian re-identification model. Figure 3a Each box in the contains pedestrian features of different pedestrians; Figure 3b This is a schematic diagram of the spatial distribution of pedestrian features in the first type of video frames of the same pedestrian under the same camera device extracted after the first training stage of the pedestrian re-identification model. Figure 3b Each box in contains the pedestrian features of the same pedestrian; Figure 3cThis is a schematic diagram of the spatial distribution of pedestrian features of the same pedestrian in different video frames extracted after the second training phase of the pedestrian re-identification model. Figure 3c Each box in contains the pedestrian features of the same pedestrian. Figure 3a , Figure 3b as well as Figure 3c It can be seen that the spatial distribution of pedestrian features extracted by the untrained pedestrian re-identification model is disordered; the pedestrian features in the first category of video frames of the same pedestrian under the same camera device extracted by the pedestrian re-identification model after the first training stage can be clustered together in space; the pedestrian features in all video frames of the same pedestrian extracted by the pedestrian re-identification model after the second training stage can be clustered together in space regardless of the camera device, indicating that the pedestrian re-identification model has better recognition ability for large changes in pedestrian appearance after the second training stage.

[0068] On the basis of the above-mentioned embodiment, the pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention, wherein the cosine similarity matrix of two groups of pedestrian features is determined, specifically includes:

[0069] Arranging the pedestrian features of each pedestrian in the first group of pedestrian features of the two groups of pedestrian features in rows, and arranging the pedestrian features of each pedestrian in the second group of pedestrian features in columns, to obtain feature matrices corresponding to the first group of pedestrian features and the second group of pedestrian features respectively;

[0070] The feature matrices corresponding to the first set of pedestrian features and the second set of pedestrian features are multiplied to obtain the cosine similarity matrix.

[0071] Specifically, in an embodiment of the present invention, when determining the cosine similarity matrix, the pedestrian features of all pedestrians in the sample video frame a can constitute a group of pedestrian features, which are used as the first group of pedestrian features, and the pedestrian features of all pedestrians in the sample video frame b can constitute a group of pedestrian features, which are used as the second group of pedestrian features. The first group of pedestrian features can be specifically represented as a corresponding feature matrix, each row of which is a pedestrian feature of a pedestrian in the sample video frame a. The second group of pedestrian features can be specifically represented as a corresponding feature matrix, each column of which is a pedestrian feature of a pedestrian in the sample video frame b.

[0072] The cosine similarity matrix can be obtained by multiplying the feature matrix corresponding to the first set of pedestrian features and the feature matrix corresponding to the second set of pedestrian features.

[0073] On the basis of the above embodiments, the pedestrian re-identification method based on unsupervised learning provided in the embodiments of the present invention can use the gap between the cyclic allocation matrix and the unit matrix as the optimization loss function, which can be specifically expressed by the following formula (1):

[0074]

[0075] Among them, L symmetric is the optimization loss function, K is the number of pedestrians in each sample video frame, A cycle is the cyclic allocation matrix, and I is a unit matrix having the same size as the cyclic allocation matrix.

[0076] The above formula (1) can be applied to the case where the number of pedestrians in the two sample video frames is equal. However, in actual data, due to occlusion or pedestrians entering and leaving the field of view, the data in the two sample video frames are often asymmetric, that is, the number of pedestrians is not necessarily the same, nor are they necessarily the same pedestrians. Only some pedestrians can find a one-to-one correspondence, while others cannot.

[0077] Therefore, based on the above embodiment, the pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention can make the diagonal elements in the circulant allocation matrix larger than the largest elements in the row and column where they are located by a fixed interval, so that the optimized loss function obtained is more reasonable. Specifically, it can be expressed by the following formula (2):

[0078]

[0079] Among them, L asymmetric is the optimization loss function, K1 is the number of rows in the loop allocation matrix, The element at the i-th row and j-th column in the loop allocation matrix is ​​allocated to the loop, The element at the i-th row and the i-th column in the loop allocation matrix is ​​allocated to the loop, The element in the kth row and jth column of the loop allocation matrix is ​​allocated to the loop, where m is a constant.

[0080] On the basis of the above embodiments, the pedestrian re-identification method based on unsupervised learning provided in the embodiments of the present invention can select sample video frames in different scenes to train the pedestrian re-identification model, so that the pedestrian recognition model can be adapted to different application scenes. The pedestrian re-identification model in the embodiments of the present invention can also be continuously improved with the continuous increase of training data, and has a wide range of application prospects.

[0081] The pedestrian re-identification method based on unsupervised learning provided in the embodiment of the present invention applies a pedestrian re-identification model to realize pedestrian re-identification. When training the pedestrian re-identification model, it does not rely on a third-party algorithm module other than pedestrian detection to provide auxiliary supervision. The training process is simple and has strong scalability. The pedestrian features obtained by training with unlabeled video data are almost equivalent to the pedestrian features obtained through supervised learning, and significantly improve the optimal effect of the current unsupervised pedestrian feature learning.

[0082] like Figure 4 As shown, based on the above embodiment, a pedestrian re-identification system based on unsupervised learning is provided in the embodiment of the present invention, including: an acquisition module 41 and a re-identification module 42. Among them,

[0083] The acquisition module 41 is used to acquire two video frames to be identified containing a number of pedestrians;

[0084] The re-identification module 42 is used to input the two video frames to be identified into the pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian;

[0085] Among them, the pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing several pedestrians is determined, and based on the circulant allocation matrix, an optimized loss function is determined; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames.

[0086] Specifically, the functions of each module in the pedestrian re-identification system based on unsupervised learning provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.

[0087] On the basis of the above-mentioned embodiment, an embodiment of the present invention provides a pedestrian re-identification system based on unsupervised learning. When the pedestrian re-identification model is trained, it specifically includes a first training stage and a second training stage. Accordingly,

[0088] The two sample video frames containing a number of pedestrians in the first training stage are specifically: two first-category video frames in a first-category video containing a number of pedestrians captured by the same camera device;

[0089] The two sample video frames containing several pedestrians in the second training stage are specifically: two video frames in a mixed video frame set; the mixed video frame set is obtained by mixing the second type of video frames corresponding to the same moment in a second type of video pair containing several pedestrians, which are captured by two cameras with the same field of view, and the first type of video frames.

[0090] Figure 5 As shown, based on the above embodiment, an electronic device is provided in an embodiment of the present invention, including: a processor (processor) 501, a memory (memory) 502, a communication interface (Communications Interface) 503 and a communication bus 504; wherein,

[0091] The processor 501, the memory 502, and the communication interface 503 communicate with each other via a communication bus 504. The memory 502 stores program instructions that can be executed by the processor 501, and the processor 501 is used to call the program instructions in the memory 502 to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining two video frames to be identified that contain a number of pedestrians; inputting the two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian.

[0092] It should be noted that the electronic device in this embodiment can be a server, a PC, or other devices in specific implementation, as long as its structure includes the following: Figure 5 The processor 501, communication interface 503, memory 502 and communication bus 504 are shown, wherein the processor 501, communication interface 503 and memory 502 communicate with each other through the communication bus 504, and the processor 501 can call the logic instructions in the memory 502 to execute the above method. This embodiment does not limit the specific implementation form of the electronic device.

[0093] The logic instructions in the memory 502 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0094] Furthermore, an embodiment of the present invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining two video frames to be identified containing several pedestrians; inputting the two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian.

[0095] On the basis of the above embodiments, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the transmission method provided by the above embodiments, for example, including: obtaining two video frames to be identified containing a number of pedestrians; inputting the two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian re-identification method based on unsupervised learning, characterized in that: include: Obtain two video frames containing several pedestrians to be identified; Inputting two video frames to be identified into a person re-identification model, and determining by the person re-identification model whether the two video frames to be identified contain the same person; The pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing a plurality of pedestrians is determined, and an optimization loss function is determined based on the circulant allocation matrix; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames; When the pedestrian re-identification model is trained, it specifically includes a first training stage and a second training stage; accordingly, The two sample video frames containing a number of pedestrians in the first training stage are specifically: two first-category video frames in a first-category video containing a number of pedestrians captured by the same camera device; The two sample video frames containing a number of pedestrians in the second training stage are specifically: two video frames in a mixed video frame set; the mixed video frame set is obtained by mixing the second type of video frames corresponding to the same time in the second type of video pairs containing a number of pedestrians captured by two camera devices with the same field of view and the first type of video frames; The determining of the pedestrian cyclic allocation matrix between two sample video frames containing a plurality of pedestrians specifically includes: Based on the pedestrian re-identification model, pedestrian features of each pedestrian in two sample video frames are respectively extracted, each sample video frame corresponds to a group of pedestrian features, and a cosine similarity matrix of the two groups of pedestrian features is determined; Applying a Softmax function to each row of the cosine similarity matrix to determine a forward allocation matrix, and applying a Softmax function to each row of the transposed matrix of the cosine similarity matrix to determine a reverse allocation matrix; The cyclic allocation matrix is ​​determined based on the forward allocation matrix and the reverse allocation matrix.

2. The pedestrian re-identification method based on unsupervised learning according to claim 1, characterized in that: The determining of the cosine similarity matrix of two groups of pedestrian features specifically includes: Arranging the pedestrian features of each pedestrian in the first group of pedestrian features of the two groups of pedestrian features in rows, and arranging the pedestrian features of each pedestrian in the second group of pedestrian features in columns, to obtain feature matrices corresponding to the first group of pedestrian features and the second group of pedestrian features respectively; The feature matrices corresponding to the first set of pedestrian features and the second set of pedestrian features are multiplied to obtain the cosine similarity matrix.

3. The pedestrian re-identification method based on unsupervised learning according to any one of claims 1-2, characterized in that: If the number of pedestrians in the two sample video frames is equal, then determining the optimization loss function based on the cyclic allocation matrix specifically includes: The optimization loss function is determined by the following formula: Among them, L symmetric is the optimization loss function, K is the number of pedestrians in each sample video frame, A cycle is the cyclic allocation matrix, and I is a unit matrix having the same size as the cyclic allocation matrix.

4. The pedestrian re-identification method based on unsupervised learning according to any one of claims 1-2, characterized in that: The step of determining an optimization loss function based on the cycle allocation matrix further includes: The optimization loss function is determined by the following formula: Among them, L asymmetric is the optimization loss function, K1 is the number of rows in the cyclic allocation matrix, The element at the i-th row and j-th column in the loop allocation matrix is ​​allocated to the loop, The element at the i-th row and the i-th column in the loop allocation matrix is ​​allocated to the loop, The element in the kth row and jth column of the loop allocation matrix is ​​allocated to the loop, where m is a constant.

5. A pedestrian re-identification system based on unsupervised learning, characterized in that: include: An acquisition module, used for acquiring two video frames to be identified containing a number of pedestrians; A re-identification module, used for inputting two video frames to be identified into a pedestrian re-identification model, and the pedestrian re-identification model determines whether the two video frames to be identified contain the same pedestrian; The pedestrian re-identification model is constructed based on a deep convolutional neural network, and when the pedestrian re-identification model is trained, a circulant allocation matrix of pedestrians between two sample video frames containing a plurality of pedestrians is determined, and an optimization loss function is determined based on the circulant allocation matrix; the circulant allocation matrix is ​​used to characterize the matching possibility of pedestrians between the two sample video frames; When the pedestrian re-identification model is trained, it specifically includes a first training stage and a second training stage; accordingly, The two sample video frames containing a number of pedestrians in the first training stage are specifically: two first-category video frames in a first-category video containing a number of pedestrians captured by the same camera device; The two sample video frames containing a number of pedestrians in the second training stage are specifically: two video frames in a mixed video frame set; the mixed video frame set is obtained by mixing the second type of video frames corresponding to the same time in the second type of video pairs containing a number of pedestrians captured by two camera devices with the same field of view and the first type of video frames; The determining of the pedestrian cyclic allocation matrix between two sample video frames containing a plurality of pedestrians specifically includes: Based on the pedestrian re-identification model, pedestrian features of each pedestrian in two sample video frames are respectively extracted, each sample video frame corresponds to a group of pedestrian features, and a cosine similarity matrix of the two groups of pedestrian features is determined; Applying a Softmax function to each row of the cosine similarity matrix to determine a forward allocation matrix, and applying a Softmax function to each row of the transposed matrix of the cosine similarity matrix to determine a reverse allocation matrix; The cyclic allocation matrix is ​​determined based on the forward allocation matrix and the reverse allocation matrix.

6. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the pedestrian re-identification method based on unsupervised learning as described in any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the pedestrian re-identification method based on unsupervised learning as described in any one of claims 1 to 4 are implemented.

Images