Method for human re-identification based on multi-branch feature representation

Abstract

The present application provides a kind of person re-identification method based on multi-branch feature representation, for the person in complex scene in small video is re-identified, it is characterized in that, including: step S1, the small video to be processed is obtained, and the data of small video is parsed into frame by frame image sequence;Step S2, image sequence is input to predetermined detection model, and the person bounding box in each frame image of small video is obtained;Step S3, the person bounding box is input to predetermined tracking model, and the person frame sequence in each shot is acquired;Step S4, the person frame sequence is input to the person re-identification model that has been completed pre-training, so that the person frame sequence is connected and inferences, to obtain the frame sequence set of each person corresponding to person ID in small video.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a person re-identification method based on multi-branch feature representation in a small video scene. BACKGROUND

[0002] With the popularity of the fourth generation mobile network, video gradually becomes the main information carrier in the Internet. The entertainment video APP mainly based on small video is widely used and becomes the most popular information interaction method at present. Small video is a video work independently shot, edited and uploaded to the relevant platform by the user, and its duration is relatively short, generally within 2 minutes. Because of the rich content and novel theme of small video, it is favored by Internet users of all ages. The development of small video brings a lot of information to the Internet, and how to make the computer use these information is a problem to be solved in the field of image analysis.

[0003] In the field of image analysis, the task related to the person has always been a very important research field. For example, person re-identification in large airport, intelligent searching for people in amusement park, crime monitoring system in security system and so on. Person re-identification can provide the required person features for the technology in the above application scenarios, and the key appearance parts of people such as clothes, backpack, hairstyle, posture and so on are extracted in the picture or video, so the person re-identification task has very important research value and practical value.

[0004] Person re-identification in small video scene refers to connecting the person frame sequence in different shots in a video through computer vision technology to obtain the frame sequence set of each person in the video through re-identification technology. The current person re-identification research is aimed at the monitoring video scene, in which the motion posture of the person is relatively single, the scene is relatively simple, and the image of the person is relatively complete. However, person re-identification in small video scene is more complex, and the incomplete entry of the person into the mirror, the obstacle occlusion and the complex posture of the person are all the challenges of person re-identification in small video scene.

[0005] In recent years, with the development of technology, especially the progress of deep learning, person re-identification has made great progress. Deep learning methods can be roughly divided into complete person re-identification, local person re-identification and occluded person re-identification. Complete person re-identification is also the earliest sub-task to introduce deep learning into person re-identification, which realizes the similarity matching of features between persons by extracting the global features of images. Because the convolutional neural network is used instead of the previous manually designed feature extraction method, the feature expression ability is improved, so that the re-identification effect is greatly improved compared with the traditional method, and the representative method is IDE Net, APR. However, the complete person re-identification technology cannot solve the problem of person re-identification in the presence of occlusion scene. The local person re-identification task mainly removes the interference and occlusion by manual cropping, and then compares the similarity between the cropped person image and the complete person image, the main methods are DSR, SFR, etc. Occluded person re-identification directly compares the similarity between two occluded person images through the additional assistance of human pose key point detection, the main methods are HONet, FPR, etc.

[0006] The scene processed by the above traditional method is relatively simple, and some even need a large amount of manual operation to do image preprocessing, which is easy to introduce a large amount of interference error, and cannot process complex occlusion, complex posture, local appearance and other problems in small video scenes. SUMMARY

[0007] In order to solve the above problems, a re-identification method with small interference error and capable of processing complex scenes is provided, and the technical scheme adopted by the present application is as follows:

[0008] The application provides a person re-identification method based on multi-branch feature representation, which is used for re-identifying persons in a small video in a complex scene, and has the characteristics that the method comprises the following steps: S1, obtaining a small video to be processed, and parsing data of the small video into a frame-by-frame image sequence; S2, inputting the image sequence into a predetermined detection model to obtain a person bounding box in each frame of the small video; S3, inputting the person bounding box into a predetermined tracking model to obtain a person frame sequence in each shot; and S4, inputting the person frame sequence into a person re-identification model which has been pre-trained, so that the person frame sequence is connected for reasoning, thereby obtaining a frame sequence set of each person corresponding to a person ID in the small video, wherein the person re-identification model comprises a backbone network, a multi-branch feature representation network and a predetermined human pose estimation network; the pre-training process of the person re-identification model comprises the following steps: A1, collecting public person images and performing data enhancement processing on the person images; A2, obtaining a key point heat map and a key point confidence corresponding to the person images by using the predetermined human pose estimation network; A3, inputting the person images into the backbone network to obtain corresponding global person features; A4, inputting the global person features and the key point heat map into the multi-branch feature representation network, and outputting feature vectors of a plurality of feature branches of the multi-branch feature representation network to form a corresponding feature set; A5, performing network training on the multi-branch feature representation network and the backbone network in a cross-entropy loss supervision manner and calculating cross-entropy losses of the three feature branches of the multi-branch feature representation network; A6, repeating step A5 until it is judged that the network training meets a predetermined termination condition; and A7, obtaining the person re-identification model which has completed the network training.

[0009] According to the person re-identification method based on multi-branch feature representation provided by the application, the multi-branch feature representation network comprises a multi-branch feature extraction network, and the multi-branch feature extraction network is composed of three feature branches, namely a pose-guided global feature branch, a region feature branch and a pose feature branch.

[0010] According to the person re-identification method based on multi-branch feature representation provided by the application, the data enhancement processing specifically comprises the following: random translation, random rotation of positive and negative 15°, color style conversion and random salt noise addition to an image.

[0011] According to the person re-identification method based on multi-branch feature representation provided by the application, the predetermined termination condition is that the network training, i.e., step A5, is run for 60 times.

[0012] The person re-identification method based on the multi-branch feature representation provided by the application can also have the following technical features: the predetermined detection model is a publicly trained detection model YOLOv3, the predetermined tracking model is a publicly trained tracking model DeepSort, and the predetermined human body posture estimation network is a publicly trained human body posture estimation model AlphaPose; the human body posture estimation model AlphaPose guides the feature set to perform similarity calculation on the person, and the total person similarity is obtained through weighting.

[0013] The person re-identification method based on the multi-branch feature representation provided by the application can also have the following technical features: the person image in the person frame sequence is a to-be-queried person image, and the inference is performed by guiding the person re-identification model to perform respective visible region matching on the feature set through the key point heat map and the key point confidence, so as to calculate the similarity distance between the to-be-queried person image and the predetermined retrieval library person image, including the following steps: step B1, calculating the distance between the global feature branch output results of the to-be-queried person image and the predetermined retrieval library person image; step B2, calculating the distance between the regional feature branch output results of the to-be-queried person image and the predetermined retrieval library person image in the public visible region; step B3, calculating the distance between the posture feature branch output results of the to-be-queried person image and the predetermined retrieval library person image in the shared visible posture set; and step B4, obtaining the final similarity distance between the to-be-queried person image and the predetermined retrieval library person image through the weighted calculation of the final similarity distance of each feature branch.

[0014] The person re-identification method based on the multi-branch feature representation provided by the application can also have the following technical features: the feature set is a respective feature set composed of person features extracted by a plurality of feature branches; the posture feature vector and the global feature vector are spliced in the channel dimension to obtain a posture-guided global feature set; the global feature is uniformly horizontally cut to obtain a regional feature set; and the corresponding posture features are generated according to the key point heat map and the global feature, and the posture feature set of the head, the hands and the legs is correspondingly composed.

[0015] The person re-identification method based on the multi-branch feature representation provided by the application can also have the following technical features: the cross-entropy loss is calculated on the output of each feature branch.

[0016] The person re-identification method based on the multi-branch feature representation provided by the application can also have the following technical features: the calculation formula of the cross-entropy loss is as follows:

[0017]

[0018] wherein, is a predicted person ID label, and y is a true person ID label of the input image.

[0019] Inventive Action and Effects

[0020] According to the person re-identification method based on multi-branch feature representation provided by the present application, the person re-identification model based on multi-branch feature representation is constructed, the person image to be queried is input into the person re-identification model, and the predicted person ID corresponding to the person frame sequence set is obtained. The person re-identification model comprises a backbone network, a multi-branch feature representation network and a predetermined human body posture estimation network. Since the method uses multiple branches to extract different levels of features of the person, and based on the characteristics of abstract segmentation and the introduction of semantic information, different features are divided into feature sets, so that the feature representation of the person in the complex scene such as small video becomes more robust. Also, the method guides each feature set to perform respective visible region matching through the human body posture estimation network, greatly reducing the interference caused by factors such as occlusion and complex posture, so as to obtain a person re-identification model with a better matching result. Further, the multi-branch feature representation network used in the method can better extract global information, local information and posture semantic information from low to high levels in the person image compared with other networks, so that the feature extraction performance of the network is better. In addition, the method judges whether the key point is in the occlusion area by the key point confidence output by the posture estimation model, reduces the calculation complexity of the visibility confidence, and further improves the efficiency of the person re-identification. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is the flowchart of the person re-identification method based on multi-branch feature representation in the embodiment of the present application;

[0022] Figure 2 is the pre-training flowchart of the person re-identification model in the embodiment of the present application;

[0023] Figure 3 is the schematic diagram of network training in the embodiment of the present application;

[0024] Figure 4 is the posture-guided global feature branch structure diagram in the embodiment of the present application;

[0025] Figure 5 is the posture feature branch structure diagram in the embodiment of the present application;

[0026] Figure 6 is the region feature branch structure diagram in the embodiment of the present application;

[0027] Figure 7Figure is a comparison chart of experimental results of an embodiment of the present application and experimental results of a method on a public Occluded-Duke dataset. DETAILED DESCRIPTION

[0028] In order to make the technical means, creative features, purposes and effects achieved by the present application easy to understand, the following will specifically describe a person re-identification method based on multi-branch feature representation of the present application in combination with embodiments and drawings.

[0029] <EMBODIMENT>

[0030] In order to reduce interference noise caused by occlusion, complex person posture and local person in the complex scene, and improve the matching accuracy of person re-identification, the present application provides a person re-identification method based on multi-branch feature representation. The specific technical scheme adopted is:

[0031] A person re-identification model based on multi-branch feature representation is constructed, which includes a backbone network, a multi-branch feature representation network and a predetermined human posture estimation network.

[0032] In the embodiment, the predetermined human posture estimation network is a publicly trained human posture estimation model AlphaPose.

[0033] The input of the person re-identification model is a person image, and the output is a predicted person ID.

[0034] Figure 1 Figure is a flowchart of the person re-identification method based on multi-branch feature representation in the embodiment of the present application.

[0035] As shown in Figure 1 The person re-identification method based on multi-branch feature representation includes the following steps:

[0036] Step S1, a small video to be processed is obtained, and the data of the small video is parsed into a frame-by-frame image sequence.

[0037] Step S2, the image sequence is input into a predetermined detection model to obtain a person bounding box in each frame image of the small video.

[0038] In the embodiment, the predetermined detection model is a publicly trained detection model YOLOv3.

[0039] Step S3, the person bounding box is input into a predetermined tracking model to obtain a person frame sequence in each shot.

[0040] In the embodiment, the predetermined tracking model is a publicly trained tracking model DeepSort.

[0041] Step S4, inputting the person frame sequence into the pre-trained person re-identification model, connecting the person frame sequence for reasoning, so as to obtain the frame sequence set of each person corresponding to the person ID in the small video.

[0042] In the embodiment, the person frame sequence input into the person re-identification model is obtained by average pooling to obtain an average feature vector of the frame sequence as the input of the person re-identification model.

[0043] Next, the person re-identification model is trained by pre-training. In the embodiment, the training step of the person re-identification model adopts a multi-feature representation branch common training manner to solve the problem of person representation in a complex scene.

[0044] Figure 2 is a pre-training flowchart of the person re-identification model in the embodiment of the application, Figure 3 is a schematic diagram of network training in the embodiment of the application.

[0045] As shown in Figure 2 and Figure 3 , the pre-training of the person re-identification model includes the following steps:

[0046] Step A1, collecting public person images and performing data enhancement processing on the person images.

[0047] In the embodiment, the data enhancement processing specifically includes random translation, random rotation of positive and negative 15°, color style conversion, and random salt noise addition to the image.

[0048] Step A2, obtaining a key point heat map and a key point confidence corresponding to the person image from a predetermined human pose estimation network.

[0049] In the embodiment, the training data of the human pose estimation model AlphaPose is the public human key point dataset COCO, and the output is an 18-key point heat map of the person image and the corresponding 18-key point confidence.

[0050] In the embodiment, the representation of the key point is as follows:

[0051]

[0052] In the formula, KP j represents the jth human key point, cx j ,cy j represents the coordinates of the key point, is the confidence of the key point, and γ is the key point confidence threshold set in the embodiment. Specifically:

[0053] The detected key points are used to generate a heat map of a two-dimensional Gaussian distribution centered on the key point coordinates. When KP j is 0, the value of the corresponding heat map is also set to 0. The generated heat map is denoted as M j , and the heat map is down-sampled to the size of the original image frame 1 / 16 using a linear interpolation method.

[0054] Step A3, inputting the person image into the backbone network to obtain the corresponding global person feature.

[0055] In this embodiment, the backbone network adopts a ResNet50 network structure, and some modifications are made on the basis thereof: the last average pooling layer and the full connection layer are removed, the parameters of the convolution layer Conv4_1 are set to 1 from the original 2, and a model trained on a public ImageNet is used to initialize the model parameters of the backbone network.

[0056] Step A4, inputting the global person feature and the key point heat map into the multi-branch feature representation network, and outputting a feature vector from each feature branch of the multi-branch feature representation network to form a corresponding feature set.

[0057] In this embodiment, the multi-branch feature representation network comprises a multi-branch feature extraction network, which comprises three feature extraction branches, namely a posture-guided global feature branch, a region feature branch and a posture feature branch.

[0058] The feature vector is obtained from the above three different feature branches and comprises a global feature vector, a posture feature vector and a region feature vector. Specifically:

[0059] Figure 4 is a posture-guided global feature branch structure diagram in the embodiment of the application.

[0060] As Figure 4 shown, in the global feature branch, first, the global feature F output by the backbone network is subjected to a down-sampling operation by an average pooling layer (i.e. Figure 4 AvgPool in g ).

[0061] Then, the key point heat map M j output by the human posture estimation model is sequentially subjected to a dot multiplication operation with f g to obtain a corresponding posture-guided feature map

[0062] Next, for each posture-guided feature map It will go through an average pooling layer to generate a 1x2048 pose feature vector, which corresponds to the pose feature information of a certain joint of the character.

[0063] Finally, by concatenating all pose feature vectors along the channel dimension, a pose feature map of size 18x2048 is obtained. The feature vector of the pose feature map after passing through an average pooling layer is then concatenated with the global features output by the backbone network to obtain f. cat After passing through a fully connected network, a global feature vector f with a size of 1x256 for pose guidance is obtained. global .

[0064] Figure 5 This is a diagram of the posture feature branch structure in an embodiment of the present invention.

[0065] like Figure 5 As shown, in the pose feature branch, the input data consists of 18 pose feature maps. Based on the human body parts corresponding to the keypoints, the 18 keypoints are divided into a head set, a two-arm set, and a two-leg set, with each set containing 6 keypoints. Specifically:

[0066] First, those belonging to the same set of key points The pieces are then joined together and passed through an average pooling layer (i.e. Figure 5 The output of AvgPool is concatenated with the global features output by the backbone network at the channel level to obtain the final result. Where i refers to the set of key points.

[0067] Then, The vectors are fed into a fully connected network, resulting in three pose feature vectors of size 1x256.

[0068] Figure 6 This is a diagram of the regional feature branch structure in an embodiment of the present invention.

[0069] like Figure 6 As shown, in the region feature branch, firstly, the global feature F output by the backbone network ResNet50 is horizontally divided into p regions, and F is used to... i A local feature map representing the divided region.

[0070] Then, these p local feature maps are passed through an average pooling layer (i.e., Figure 6 By adding an AvgPool and a 1x1 convolutional layer, a 256-dimensional region feature vector is obtained.

[0071] The feature set consists of the respective sets composed of the three feature vectors mentioned above. Specifically:

[0072] The posture feature vector and the global feature vector are spliced in the channel dimension to obtain a posture-guided global feature set;

[0073] The global features are uniformly horizontally cut to obtain a regional feature set;

[0074] According to the key point heat map and the global feature, a corresponding posture feature is generated, and a posture feature set of the head, both hands and both legs is correspondingly composed.

[0075] Step A5, the multi-branch feature representation network and the backbone network are supervised by the cross-entropy loss to perform network training and calculate the cross-entropy loss of the three feature branches of the multi-branch feature representation network;

[0076] In this embodiment, the calculation formula of the cross-entropy loss is:

[0077]

[0078] In the formula, is the predicted person ID label, and y is the true person ID label of the input image.

[0079] For the global feature branch, the loss calculation formula is:

[0080]

[0081] In the formula, is the predicted result of the global feature branch, and y is the true person ID label of the input image.

[0082] For the regional feature branch, the loss calculation formula is:

[0083]

[0084] In the formula, p represents the total number of regions divided in this paper, represents the predicted result of the regional local feature of the i-th region, and y is the true person ID label of the input image.

[0085] For the posture feature branch, the loss calculation formula is:

[0086]

[0087] In the formula, represents the predicted result of the i-th part in the posture feature branch, and y represents the true person ID label of the input image.

[0088] The total loss calculation formula is:

[0089] Loss=λ(L r +L p )+(1-λ)Lg

[0090] where λ is a hyper-parameter balancing the loss contribution of global branch and local branches (pose branch, region branch), L p represents the cross-entropy loss of the pose branch, L g represents the cross-entropy loss of the global branch.

[0091] Step A6, repeat step A5 until it is determined that the network training meets a predetermined termination condition.

[0092] In this embodiment, the predetermined termination condition is to determine whether the network training, i.e. step A5, has been run for 60 times. If it is determined that yes, the network training is ended, and the model completed the last network training is taken as the pre-trained person re-identification model. If it is determined that no, step A5 is repeatedly executed until the predetermined termination condition is met.

[0093] Step A7, obtaining the person re-identification model completed the network training.

[0094] In this embodiment, the person image in the person frame sequence is the to-be-queried person image, and the inference is to guide the above-mentioned pre-trained person re-identification model to perform respective visible region matching on the three feature sets through the key point heat map and the key point confidence, so as to calculate the similarity distance between the to-be-queried person image and the retrieval library person image,

[0095] comprising the following steps:

[0096] Step B1, calculating the distance between the global feature branch output results of the to-be-queried person image and the retrieval library person image.

[0097] In this embodiment, for the global feature branch, there is only one element in the set, and the similarity distance is directly calculated, and the global feature distance is as follows:

[0098]

[0099] where D represents a distance metric function (in this embodiment, a cosine distance is adopted), respectively represent the feature vectors of the to-be-queried person image and the retrieval library person image in the global feature branch.

[0100] Step B2, calculating the distance between the region feature branch output results of the to-be-queried person image and the retrieval library person image in the public visible region.

[0101] In this embodiment, for the pose feature branch, the key points are classified into three human body regions: head, arms, legs, and the key point serial number set belonging to a certain pose region is set as S head ,Sarms S legs , each set of posture visible label is set to The specific settings are as follows:

[0102]

[0103] In the formula, j represents the serial number of the key point, KP j represents the coordinates corresponding to the jth key point, i represents the number of the posture region, S i represents the set of key point serial numbers contained in the posture region i, and then the similarity distance between the posture regions i is represented by

[0104]

[0105] In the formula, represents the distance calculated according to the feature representation of the posture region i in the posture branch, D represents a distance measurement function (in this embodiment, a cosine distance), respectively represent the feature vectors of the query person image and the retrieval library person image in the posture region i in the posture feature branch.

[0106] Step B3, the distance between the posture feature branch output results of the query person image and the retrieval library person image in the shared visual posture set is calculated.

[0107] In this embodiment, for the region feature branch, the person image is horizontally divided into p regions, and for each region i, the region visible label corresponding to each region is represented by

[0108]

[0109] In the formula, cy j represents the vertical coordinate of the key point KP j , i represents the ith region horizontally divided, and H represents the height of the person image. The distance between the horizontal regions of the region feature branch is represented by

[0110]

[0111] In the formula, is the similarity distance calculated according to the local feature of the ith horizontal region, D is a distance measurement function (in this embodiment, a cosine distance) selected by a person, respectively represent the feature vectors of the query person image and the retrieval library person image in the ith horizontal region in the region feature branch.

[0112] ​​Step B4, the last similarity distance of each feature branch is calculated by weighting to obtain the final similarity distance between the query image and the retrieval library image.

[0113] In this embodiment, the total distance measurement formula between two images is:

[0114]

[0115] In the formula, d g represents the cosine distance calculated according to the global feature, represents the distance calculated according to the feature vector of the i-th horizontal region, represents the distance calculated according to the feature of the i-th posture region in the posture branch, and p represents the number of horizontal division regions. respectively represent the visibility labels of the query image and the retrieval library image in the i-th region, respectively represent the visibility labels of the query image and the retrieval library image in the i-th posture region. Specifically:

[0116] The greater the value of dist, the greater the similarity between the query image and the retrieval library image.

[0117] For the region feature set, only when the visibility scores of the two images in a region are both 1, the similarity of the region is considered.

[0118] Similarly, for the posture feature set, only when the visibility scores of the two images in a body part are both 1, the similarity of the body part is considered.

[0119] In this embodiment, the feature vector of the retrieval library image is obtained in advance by the human posture estimation network and the multi-branch feature representation network.

[0120] Figure 7 is a comparison chart of the experimental results of the embodiment of the present application and the experimental results of the method on the published Occluded-Duke dataset.

[0121] In this embodiment, the multi-branch feature representation-based person re-identification model is tested regularly, and the person re-identification evaluation indicators are as follows: Rank-1 is 53.1, Rank-5 is 69.0, Rank-10 is 75.1, and mAP is 40.1. Figure 7 It can be seen that when the method on the published Occluded-Duke dataset is evaluated, the evaluation effect of the multi-branch feature representation-based person re-identification method provided by the embodiment of the present application is higher than that of other conventional person re-identification methods.

[0122] It should be noted that Rank-n and mAP are commonly used evaluation indexes for person re-identification, Rank-n represents the probability that the n top (highest confidence) search results have correct results, and mAP represents the area under the PR curve (PR curve: precision and recall of all samples are plotted in a graph).

[0123] Effects of the embodiments

[0124] According to the person re-identification method based on multi-branch feature representation provided in the embodiment, a person re-identification model based on multi-branch feature representation is constructed, a person image to be queried is input into the person re-identification model, and a predicted person ID corresponding to a person frame sequence set is obtained. The person re-identification model includes a backbone network, a multi-branch feature representation network, and a predetermined human pose estimation network. Since the method uses multiple branches to extract different levels of features of a person, and based on abstract segmentation and the introduction of semantic information, different features are divided into feature sets, so that the person re-identification model in the embodiment becomes more robust in representing the features of a person in a complex scene such as a small video. Also, the method guides each feature set to perform respective visible region matching through the human pose estimation network, greatly reducing the interference caused by factors such as occlusion and complex poses, so that the person re-identification model pays more attention to the unoccluded person region, and at the same time weakens the feature misalignment problem caused by complex poses of a person, thereby obtaining a person re-identification model with a better matching result. Further, the multi-branch feature representation network used in the method can better extract global information, local information, and pose semantic information from low to high levels in a person image compared with other networks, so that the feature extraction performance of the network is better. In addition, the method determines whether a key point is in an occluded region by using the key point confidence output by the pose estimation model, reduces the computational complexity of the visibility confidence, and further improves the efficiency of person re-identification.

[0125] The above embodiments are only used to illustrate the specific implementation manners of the present application, and the present application is not limited to the description range of the above embodiments.

Claims

1. A person re-identification method based on multi-branch feature representation, used for re-identifying persons in small videos in complex scenes, characterized in that, The method comprises the following steps: Step S1, obtaining a small video to be processed, and parsing data of the small video into a sequence of images frame by frame; Step S2, inputting the image sequence into a predetermined detection model to obtain a person bounding box in each image frame of the small video; Step S3, inputting the person bounding box into a predetermined tracking model to obtain a sequence of person frames in each shot; Step S4, inputting the sequence of person frames into a pre-trained person re-identification model to connect the sequence of person frames to perform reasoning, thereby obtaining a sequence of frames of each person corresponding to a person ID in the small video, Wherein, the person re-identification model comprises a backbone network, a multi-branch feature representation network and a predetermined human pose estimation network; The pre-training process of the person re-identification model comprises the following steps: Step A1, collecting public person images, and performing data enhancement processing on the person images; Step A2, obtaining a key point heat map and a key point confidence corresponding to the person image from the predetermined human pose estimation network; Step A3, inputting the person image into the backbone network to obtain a corresponding global person feature; Step A4, inputting the global person feature and the key point heat map into the multi-branch feature representation network, and outputting a feature vector from each feature branch of the multi-branch feature representation network to form a corresponding feature set; Step A5, performing network training on the multi-branch feature representation network and the backbone network in a cross-entropy loss supervision manner, and calculating the cross-entropy loss of the three feature branches of the multi-branch feature representation network; Step A6, repeating step A5 until it is determined that the network training meets a predetermined termination condition; Step A7, obtaining the person re-identification model after the network training is completed, Wherein, the person image in the sequence of person frames is a person image to be queried, and the reasoning is to guide the person re-identification model to perform respective visible region matching on the feature set through the key point heat map and the key point confidence, thereby calculating the similarity distance between the person image to be queried and a predetermined search library person image, comprising the following steps: Step B1, calculating the distance between the global feature branch output results of the person image to be queried and the predetermined search library person image; Step B2, calculating the distance between the regional feature branch output results of the person image to be queried and the predetermined search library person image in the public visible region; Step B3, calculating the distance between the pose feature branch output results of the person image to be queried and the predetermined search library person image in the shared visible pose set; Step B4, obtaining the final similarity distance between the person image to be queried and the predetermined search library person image by weighting the final similarity distance of each feature branch, The predetermined detection model is a publicly trained detection model YOLOv3, the predetermined tracking model is a publicly trained tracking model DeepSort, and the predetermined human pose estimation network is a publicly trained human pose estimation model AlphaPose; The human pose estimation model AlphaPose guides the feature set to perform similarity calculation on a person, and obtains a total person similarity through a weighted manner, The multi-branch feature representation network comprises a multi-branch feature extraction network, and the multi-branch feature extraction network is composed of three feature branches, namely a pose-guided global feature branch, a region feature branch, and a pose feature branch.

2. The method according to claim 1, wherein the data augmentation processing comprises random translation, random rotation by plus or minus 15 degrees, color style conversion, and random salt noise addition. wherein 3. The method according to claim 1, wherein the predetermined termination condition is that the network training, i.e., the step A5, is run for 60 times.

4. The method according to claim 1, wherein the feature set is a respective feature set composed of person features extracted by the plurality of feature branches. wherein, The pose feature vector and the global feature vector are spliced in the channel dimension to obtain a global feature set guided by the pose; the global feature is uniformly horizontally cut to obtain a region feature set; the corresponding pose feature is generated according to the key point heat map and the global feature, and the pose feature set of the head, the hands and the legs is correspondingly composed.

5. The method according to claim 1, wherein the cross-entropy loss is calculated on the output of each feature branch. wherein 6. The method according to claim 1, wherein the calculation formula of the cross-entropy loss is: ​ ​ wherein, ​ ​ wherein ​ In the formula, is the predicted person ID label, and y is the true person ID label of the input image.

Images