Pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning

Abstract

The invention discloses a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, and the method comprises the steps: obtaining a pedestrian image data set 1 and a pedestrian image data set 2 through two cameras, and carrying out the processing of the data sets; dividing the processed pedestrian image data set 1 and the processed pedestrian image data set 2; constructing an end-to-end pedestrian detection and re-identification network model, wherein the model comprises an improved SSD512 network, a pedestrian feature extraction sub-network and a semi-supervised dictionary learning module; training an end-to-end pedestrian detection and re-identification network model to obtain a target model; and inputting the data into the target model to obtain a pedestrian detection and re-identification result. According to the invention, end-to-end semi-supervised pedestrian detection and re-identification are realized, the problems of appearance difference, surrounding environment difference and the like of pedestrians under different cameras due to uneven quality of detected positioning bounding boxes are solved, and the performance of the pedestrian detection and re-identification method is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian detection and pedestrian re-identification, and in particular to a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning. Background Art

[0002] In recent years, more and more surveillance cameras have been deployed in various public places, and video surveillance is playing an increasingly important role in combating crimes and maintaining social security. In the application of video surveillance, detection and re-identification of specific pedestrians is an important requirement.

[0003] Pedestrian detection technology is used to accurately locate and identify pedestrians in a single surveillance camera. Pedestrian re-identification technology is used to determine whether the pedestrians appearing in different surveillance cameras are the same person. Combining these two technologies can quickly locate specific pedestrians of interest in a video surveillance network.

[0004] Current end-to-end person detection and re-identification methods use deep networks to learn a set of shared features and use them for both person detection and re-identification. These methods focus more on the balance between detection and re-identification subtasks, while ignoring the ability to enhance the model's ability to learn person feature representations, thus limiting the overall person detection and re-identification performance of the end-to-end model.

[0005] At present, supervised pedestrian detection and re-identification methods require the use of a large amount of labeled data, resulting in high manual labeling costs; weakly supervised pedestrian detection and re-identification methods only require the labeling information of the pedestrian detection box and no longer rely on the pedestrian identity label information, but are easily affected by noise and pedestrian detection box labeling, resulting in lower performance. Summary of the invention

[0006] The purpose of the present invention is to provide a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, which can reduce the burden of manual labeling, reduce the impact of detection results on pedestrian re-identification, and improve the performance of pedestrian detection and re-identification.

[0007] The present invention adopts the following technical solution: The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning includes the following steps: S1. Obtain pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and process the datasets; divide the processed pedestrian image dataset 1 into training set 1, verification set 1 and test set 1; divide the processed pedestrian image dataset 2 into training set 2, verification set 2 and test set 2.

[0008] S2. Based on camera 1, an end-to-end pedestrian detection and re-identification network model 1 is constructed; based on camera 2, an end-to-end pedestrian detection and re-identification network model 2 is constructed. Both model 1 and model 2 include an improved SSD512 network, a pedestrian feature extraction subnetwork and a semi-supervised dictionary learning module.

[0009] S3. Use training set 1 and validation set 1 to train end-to-end pedestrian detection and re-identification network model 1 to obtain target model 1; use training set 2 and validation set 2 to train end-to-end pedestrian detection and re-identification network model 2 to obtain target model 2.

[0010] S4. Input test set 1 into target model 2 to obtain pedestrian detection and re-identification results of test set 1; input test set 2 into target model 1 to obtain pedestrian detection and re-identification results of test set 2.

[0011] Furthermore, in step S1, dividing the pedestrian image dataset includes the following contents: S101, obtaining pedestrian image dataset 1 and pedestrian image dataset 2 includes two methods: (1) Download the pedestrian image dataset 1 and pedestrian image dataset 2 corresponding to the two cameras from the open source dataset website.

[0012] (2) Self-made pedestrian image dataset 1 and pedestrian image dataset 2. The specific contents are as follows: a set number of pedestrian images are collected using two cameras in different scenes. Each pedestrian is captured with more than two images, and the pedestrians captured by the two cameras have a repetition rate of more than 50%; each image is annotated with the camera to which it belongs.

[0013] S102, maintaining the original aspect ratio of the acquired pedestrian image, scaling the largest sides of width and height to set pixels, centering the scaled image in a black background image of a set size, and performing random cropping, scaling, horizontal flipping, random change of image brightness, random change of image contrast, random change of image chromaticity, and random change of image saturation to obtain processed pedestrian image dataset 1 and processed pedestrian image dataset 2.

[0014] S103. Divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 respectively in proportion to obtain training set 1, validation set 1, test set 1 and training set 2, validation set 2, test set 2; wherein, training set 1 and training set 2 both include labeled and unlabeled pedestrian images, and validation set 1, validation set 2, test set 2 and test set 2 all include labeled pedestrian images.

[0015] Furthermore, in step S2, building an end-to-end pedestrian detection and re-identification network model includes the following contents: S201. Improve the SSD512 network to obtain an improved SSD512 network, the specific contents of which are as follows: The backbone network is replaced with ResNet50, which includes an input layer, convolution module 1, convolution module 2, convolution module 3, convolution module 4 and convolution module 5 connected in sequence.

[0016] The process of obtaining the first multi-scale output result is replaced by processing through convolution module 4 and L2 normalization in ResNet50 in sequence; a batch normalization layer is added between each convolution layer and activation layer in convolution module 6 and convolution module 7; the last four convolution modules of the multi-scale network are replaced by 3×3 convolution with a stride of 2, batch normalization, ReLU activation function, 3×3 convolution with a stride of 1, batch normalization and ReLU activation function connected in sequence.

[0017] Set the localization branch to the pedestrian localization subnetwork; set the recognition branch to the pedestrian recognition subnetwork, change the number of categories to 2, and add a batch normalization layer after each convolution layer in the pedestrian recognition subnetwork.

[0018] S202. A pedestrian feature extraction subnetwork is added between the multi-scale network and the non-maximum suppression module of the improved SSD512 network. The network is parallel to the pedestrian positioning subnetwork and the pedestrian recognition subnetwork. The pedestrian feature extraction subnetwork includes four scales and a connection layer. Each scale includes a 3×3 convolution with a step size of 1, a batch normalization layer, a ReLU activation function and a dimension transformation connected in sequence. The dimension transformations of the four scales are all connected to the connection layer.

[0019] S203, the semi-supervised dictionary learning module includes a structured dictionary of two cameras, and the structured dictionary includes c k A sub-dictionary, k =1,2, c k Indicates the camera k The number of all pedestrian categories in c k The sub-dictionaries include a shared sub-dictionary and an exclusive sub-dictionary. The shared sub-dictionaries of the two cameras are the same, while the exclusive sub-dictionaries are different.

[0020] Furthermore, in step S3, the target model obtained includes the following contents: S301. The end-to-end pedestrian detection and re-identification network model 1 is referred to as Net1, and the end-to-end pedestrian detection and re-identification network model 2 is referred to as Net2.

[0021] The end-to-end pedestrian detection and re-identification network model includes a detection part and a re-identification part, wherein the detection part includes a backbone network, a multi-scale network, a pedestrian localization subnetwork and a pedestrian recognition subnetwork, and the re-identification part includes a pedestrian feature extraction subnetwork and a pedestrian re-identification module; the detection part and the re-identification part in Net1 are referred to as DetNet1 and ReIdNet1 respectively, and the detection part and the re-identification part in Net2 are referred to as DetNet2 and ReIdNet2 respectively.

[0022] DetNet1 and DetNet2 are trained as follows: S3011. Randomly initialize DetNet1.

[0023] S3012, use training set 1 and training set 2 to train DetNet1, use verification set 1 and verification set 2 to calculate the loss function value of DetNet1, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is terminated to obtain the trained DetNet1; otherwise, the training is continued.

[0024] The expression of DetNet1 loss function is: .

[0025] in, represents the loss function of DetNet1, N Represents the total number of positioning prediction boxes that match the true box, represents the pedestrian positioning loss function, Represents the pedestrian recognition loss function.

[0026] The expression of pedestrian positioning loss function is: .

[0027] in, Pos Represents the set of all positioning prediction boxes that match the true box; xx Represents the horizontal coordinate of the center point of the positioning prediction box and the real box; cy Indicates the vertical coordinate of the center point of the positioning prediction box and the real box; w Indicates the width of the positioning prediction box and the real box; h Indicates the height of the positioning prediction box and the real box; represents the indicator function; Represents the smooth L1 loss function; Indicates i The positioning prediction box is xx The value at time; Indicates i The positioning prediction box is cy The value at time; Indicates i The positioning prediction box is w The value at time; Indicates i The positioning prediction box is h The value at time; Indicates j The ground-truth box is relative to i The positioning prediction box is xx The offset of , Indicates i The default box is xx The value of Indicates j A real frame xx The value of Indicates i The default box is w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is cy The offset of , Indicates j A real frame cy The value of Indicates i The default box is cy The value of Indicates i The default box is h The value of Indicates j The ground-truth box is relative to i The positioning prediction box is w The offset of , Indicates j A real frame w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is h The offset of , Indicates j A real frame h The value of .

[0028] and: ; ; ; .

[0029] The expression of pedestrian recognition loss function is:

[0030] .

[0031] in, Neg Represents the set of all positioning prediction boxes that do not match the true box; express Pos After normalization i The positioning prediction box belongs to p The confidence level of the class, , express Pos Middle i The positioning prediction box belongs to p Confidence of the class; express Neg The r The confidence that the positioning prediction box belongs to the background class; Represents a logarithmic function.

[0032] The stochastic gradient descent optimizer is used to update the network weights and biases of DetNet1, and the poly strategy is used to decay the learning rate. The decay formula is: ; in, lr represents the learning rate decay, lr init represents the initial learning rate, iter Indicates the current iteration number, max _ iter Indicates the maximum number of iterations.

[0033] S3013, passing the network weights and biases in the trained DetNet1 to DetNet2, and initializing the network weights and biases of DetNet2.

[0034] S302, fix the network weights and biases of DetNet1 and DetNet2, and use training set 1, validation set 1 and training set 2, validation set 2 to train ReIdNet1 and ReIdNet2 respectively. The specific contents are as follows: S3021. Randomly initialize ReIdNet1 and ReIdNet2.

[0035] S3022, using the structured dictionary of training set 1 and camera 1 D 1. Calculate the sparse coding corresponding to the pedestrian features.

[0036] S3023, using the labeled loss function to calculate the gradient of the labeled samples in training set 1, and using the stochastic gradient descent optimizer to update D 1; Use the unlabeled loss function to calculate the gradient of the unlabeled samples in training set 1, and use the stochastic gradient descent optimizer to update D 1.

[0037] Labeled loss function The expression is:

[0038] in, Indicates the camera k Middle s Pedestrian characteristics of individual pedestrian categories, Indicates the use D k Refactoring The sparse coding of D k Indicates the camera k A structured dictionary of Indicates the use Refactoring The sparse coding of Indicates the camera k No. s Sub-dictionaries for individual pedestrian categories, Indicates the camera k No. t Sub-dictionaries for individual pedestrian categories, Indicates the use Refactoring The sparse coding of A k Indicates the use D k Refactoring X k The sparse coding of X k Indicates the camera k Pedestrian characteristics of all pedestrian categories in; represents the F-norm; , Both represent weight coefficients.

[0039] Unlabeled loss function The expression is: ; in, Indicates the camera k Pedestrian features in unlabeled images, Indicates the use D k Refactoring sparse coding.

[0040] S3024, will D The shared sub-dictionary in 1 is passed to the structured dictionary of camera 2 D 2, and replace D The corresponding shared sub-dictionary in 2.

[0041] S3025, using training set 2 and D 2. Calculate the sparse coding corresponding to the pedestrian features.

[0042] S3026, using the labeled loss function to calculate the gradient of the labeled image in training set 2, and using the stochastic gradient descent optimizer to update D 2; Use the unlabeled loss function to calculate the gradient of the unlabeled samples in training set 2, and use the stochastic gradient descent optimizer to update D 2.

[0043] S3027, will D The shared sub-dictionary in 2 is passed to D 1, and replace D The shared sub-dictionary in 1.

[0044] S3028. Loop through steps S3022-S3027 until the loss function values ​​of the semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on both the training set and the validation set converge, and then terminate the training.

[0045] Semi-supervised dictionary learning loss function The expression is: ; in, Represents the weight coefficient.

[0046] S303, use training set 1 to train Net1, use verification set 1 to calculate the loss function value of Net1, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed, and the trained Net1 is obtained, that is, the target model 1; otherwise, continue training; Use training set 2 to train Net2, use validation set 2 to calculate the loss function value of Net2, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed and the trained Net2 is obtained, that is, the target model 2; otherwise, continue training.

[0047] Loss function of the end-to-end pedestrian detection and re-identification network model The expression is: .

[0048] Furthermore, in step S4, the pedestrian detection and re-identification results of the test set 1 include the following: Step 1: Input the pedestrian image samples to be identified in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, and input the features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction subnetwork respectively to obtain the first scale dimension transformation feature, the second scale dimension transformation feature, the third scale dimension transformation feature and the fourth scale dimension transformation feature, and connect the dimension transformation features of the four scales in series along dimension 1 to obtain the pedestrian features of the positioning prediction box.

[0049] Step 2: The position information of the positioning prediction box output by the pedestrian positioning subnetwork, the pedestrian confidence of the positioning prediction box output by the pedestrian recognition subnetwork, and the pedestrian features of the positioning prediction box are input into the non-maximum suppression module for non-maximum suppression operation to obtain the position information of the positioning prediction box, the pedestrian confidence, and the pedestrian features after screening.

[0050] Step 3: Input the filtered pedestrian features into the semi-supervised dictionary learning module to obtain sparse codes, and perform pedestrian recognition based on the distance between the sparse codes.

[0051] The calculation formula for sparse coding is: ; in, x k Indicates the camera k The features of a pedestrian in a pedestrian image, Indicates the use Refactoring x k The sparse coding of .

[0052] Step 4: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in training set 2, and sort them from small to large according to the distance value; where the representative code represents the mean of the sparse codes of each class of labeled samples in the training set.

[0053] Step 5: According to the result of step 4, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rank k recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted.

[0054] Step 6: Based on the results of step 5, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 1 are summarized to obtain the pedestrian detection and re-identification results of test set 1.

[0055] Furthermore, the non-maximum suppression operation includes the following: Step 1: Put all positioning prediction boxes into the candidate set.

[0056] Step 2: Sort the positioning prediction frames in the candidate set according to their pedestrian confidence, remove the positioning prediction frame with the highest pedestrian confidence from the candidate set and put it into the selected set; delete the positioning prediction frame whose overlapping area with the positioning prediction frame is greater than the threshold from the candidate set.

[0057] Step 3: Repeat step 2 until the candidate set is empty. Since the location information, pedestrian confidence and pedestrian features of the positioning prediction box correspond one to one, the location information, pedestrian confidence and pedestrian features of the filtered positioning prediction box are obtained.

[0058] Furthermore, in step S4, the pedestrian detection and re-identification results of the test set 2 include the following: Step 1: Input the pedestrian image samples to be identified in the test set 2 into the target model 1 to obtain the location information, pedestrian confidence and sparse coding of the corresponding positioning prediction box.

[0059] Step 2: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in training set 1, and sort them from small to large according to the distance value.

[0060] Step 3: According to the result of step 2, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rank k recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted.

[0061] Step 4: Based on the results of step 3, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 2 are summarized to obtain the pedestrian detection and re-identification results of test set 2.

[0062] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning are implemented.

[0063] Furthermore, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning.

[0064] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: 1. The end-to-end pedestrian detection and re-identification network model constructed by the present invention can reduce the impact of target detection results on re-identification.

[0065] 2. The present invention uses a deeper ResNet as the backbone network to effectively improve the feature expression capability; at the same time, a pedestrian feature extraction sub-network is added to effectively enhance the ability to learn discriminative pedestrian feature representation and improve the performance of pedestrian re-identification.

[0066] 3. The present invention uses semi-supervised dictionary learning in the pedestrian re-identification module, making full use of a large number of completely unlabeled images, and combines them with existing images labeled with pedestrian positioning bounding boxes and identity information for training, which can reduce the burden of manual labeling and improve the performance of pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is an overall implementation flow chart of the present invention.

[0068] Figure 2 It is a structural diagram of ResNet50 of the present invention.

[0069] Figure 3 It is a structural schematic diagram of the dimensionality reduction residual module of the present invention.

[0070] Figure 4 It is a structural schematic diagram of the residual module without dimensionality reduction of the present invention.

[0071] Figure 5 It is a schematic diagram of the improved multi-scale network structure of the present invention.

[0072] Figure 6 It is a structural diagram of the pedestrian feature extraction subnetwork of the present invention.

[0073] Figure 7 It is a structural diagram of the semi-supervised dictionary learning module of the present invention.

[0074] Figure 8 2 is a recognition result diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0076] To achieve the above objectives, the present invention proposes a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, such as Figure 1 As shown, the specific steps are as follows: S1. Obtain pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and process the datasets; divide the processed pedestrian image dataset 1 into training set 1, verification set 1 and test set 1; divide the processed pedestrian image dataset 2 into training set 2, verification set 2 and test set 2; the specific contents are as follows: S101, obtaining pedestrian image dataset 1 and pedestrian image dataset 2 includes two methods: (1) Download the pedestrian image dataset 1 and pedestrian image dataset 2 corresponding to the two cameras from the open source dataset website http: / / www.liangzheng.com.cn. Training set 1 contains 2835 video frames, validation set 1 contains 315 video frames, and test set 1 contains 890 video frames. Training set 2 contains 2299 video frames, validation set 2 contains 255 video frames, and test set contains 2255 video frames.

[0077] (2) Self-made pedestrian image dataset 1 and pedestrian image dataset 2. The specific contents are as follows: a set number of pedestrian images are collected using two cameras in different scenes. More than two images are collected for each pedestrian, and more than half of the pedestrians captured by the two cameras are the same; among the remaining pedestrians, some pedestrians are only captured by camera 1, and the other part are only captured by camera 2; each image is annotated with the camera to which it belongs.

[0078] S102. Keep the original aspect ratio of the obtained pedestrian image, use the resize function in the torchvision.transforms toolkit to scale the largest sides of width and height to 513 pixels, center the scaled image in a 513×513 black background image, and perform random cropping, scaling, horizontal flipping, random change of image brightness, random change of image contrast, random change of image chromaticity, and random change of image saturation to obtain processed pedestrian image dataset 1 and processed pedestrian image dataset 2.

[0079] S103. Divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 respectively in proportion to obtain training set 1, validation set 1, test set 1 and training set 2, validation set 2, test set 2; wherein, training set 1 and training set 2 both include labeled and unlabeled pedestrian images, and validation set 1, validation set 2, test set 2 and test set 2 all include labeled pedestrian images.

[0080] S2. Based on camera 1, an end-to-end pedestrian detection and re-identification network model 1 is constructed; based on camera 2, an end-to-end pedestrian detection and re-identification network model 2 is constructed. Both model 1 and model 2 include an improved SSD512 network, a pedestrian feature extraction subnetwork, and a semi-supervised dictionary learning module; the specific contents are as follows: S201. Improve the SSD512 network to obtain an improved SSD512 network, the specific contents of which are as follows: Replace the backbone network VGG-16 with ResNet50, such as Figure 2 As shown in , ResNet50 includes an input layer, convolution module 1, convolution module 2, convolution module 3, convolution module 4 and convolution module 5 connected in sequence; among them, convolution module 1 includes a 7×7 convolution layer and a 3×3 maximum pooling layer, the number of output channels is 64, and the output feature map size is 257×257; convolution module 2 includes the following connected in sequence Figure 3 The one shown has a dimension reduction residual module and Figure 4 The two residual modules without dimensionality reduction shown in the figure have 256 output channels and an output feature map size of 129×129; the convolution module 3 includes the following connected in sequence: Figure 3 The one shown has a dimension reduction residual module and Figure 4 The three residual modules without dimensionality reduction shown in the figure have an output channel number of 512 and an output feature map size of 65×65; the convolution module 4 includes the following connected in sequence: Figure 3 The one shown has a dimension reduction residual module and Figure 4 The 5 residual modules without dimensionality reduction shown in the figure have an output channel number of 1024 and an output feature map size of 33×33; the convolution module 5 includes the following connected in sequence: Figure 4 The three residual modules without dimensionality reduction shown have 2048 output channels and an output feature map size of 33×33.

[0081] Among them, Figure 3 The dimensionality reduction residual module shown includes two branches, and the two branches share input; branch 1 includes a 1×1 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 3×3 convolution with a stride of 2, a batch normalization layer, a ReLU activation function, a 1×1 convolution with a stride of 1, a batch normalization layer, an addition layer and a ReLU activation function, which are connected in sequence; branch 2 includes a 1×1 convolution with a stride of 2 and a batch normalization layer, and the batch normalization layer of branch 2 is connected to the addition layer of branch 1.

[0082] like Figure 4The residual module without dimensionality reduction shown includes two branches, and the two branches share the input; branch 1 includes a 1×1 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 3×3 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 1×1 convolution with a stride of 1, a batch normalization layer, an addition layer, and a ReLU activation function, which are connected in sequence; branch 2 is directly connected to the addition layer of branch 1.

[0083] The process of obtaining the first multi-scale output result is replaced by the processing of the third convolution layer, ReLU activation function and L2 normalization of the fourth convolution module in VGG-16 to the processing of convolution module 4 and L2 normalization in ResNet50, and the feature map size of the output result is 33×33.

[0084] A batch normalization layer is added between each convolution layer and activation layer in convolution modules 6 and 7, and the feature map size output by convolution module 7 is 17×17.

[0085] Replace the last four convolutional modules of the multi-scale network with Figure 5 As shown, the 3×3 convolution with a stride of 2, the batch normalization layer, the ReLU activation function, the 3×3 convolution with a stride of 1, the batch normalization layer and the ReLU activation function are connected sequentially. The number of output channels of these four convolution modules is 512, 256, 256, and 256, respectively, and the output feature map sizes are 9×9, 5×5, 3×3, and 1×1, respectively.

[0086] The localization branch is set to the pedestrian localization subnetwork to generate the pedestrian localization prediction box and output the location information of the localization prediction box. The recognition branch is set to the pedestrian recognition subnetwork, the number of categories is changed to 2, which is used to identify whether the area marked by the localization prediction box is a pedestrian, and a batch normalization layer is added after each convolutional layer in the pedestrian recognition subnetwork. The pedestrian recognition subnetwork outputs the pedestrian confidence of the localization prediction box, and the number of output channels is the number of all pedestrian localization prediction boxes × 2.

[0087] S202, add a pedestrian feature extraction sub-network between the multi-scale network and the non-maximum suppression module of the improved SSD512 network, and the network is parallel to the pedestrian positioning sub-network and the pedestrian recognition sub-network. The structure diagram of the pedestrian feature extraction sub-network is as follows Figure 6As shown, it includes four scales and a connection layer. Each scale contains a 3×3 convolution (with a stride of 1 and the number of channels being the product of the number of default boxes at each pixel at this scale and the specified feature dimension) connected in sequence, a batch normalization layer, a ReLU activation function, and a dimension transformation (a fourth-order tensor with dimensions of height × width × the number of default boxes at each pixel at this scale × the specified feature dimension is converted into a second-order tensor with dimensions of height × width × the number of default boxes × the specified feature dimension). The dimension transformations of the four scales are all connected to the connection layer (the dimension transformation features of the four scales are connected in series along dimension 1).

[0088] S203, such as Figure 7 As shown, the semi-supervised dictionary learning module includes the structured dictionary of the two cameras. The structured dictionary includes c k A sub-dictionary, k =1,2, c k Indicates the camera k The number of all pedestrian categories in c k The sub-dictionaries can be divided into shared sub-dictionaries and exclusive sub-dictionaries. The shared sub-dictionaries of the two cameras are the same, while the exclusive sub-dictionaries are different. Indicates the use Refactoring x k The sparse coding of Indicates the camera k No. c k A sub-dictionary.

[0089] S3, using training set 1 and validation set 1 to train end-to-end pedestrian detection and re-identification network model 1, to obtain target model 1; using training set 2 and validation set 2 to train end-to-end pedestrian detection and re-identification network model 2, to obtain target model 2; the specific contents are: S301. The end-to-end pedestrian detection and re-identification network model 1 is referred to as Net1, and the end-to-end pedestrian detection and re-identification network model 2 is referred to as Net2.

[0090] To facilitate training, each end-to-end pedestrian detection and re-identification network model is split into two parts for separate training, namely the detection part and the re-identification part. The detection part includes the backbone network, the multi-scale network, the pedestrian localization subnetwork and the pedestrian recognition subnetwork, and the re-identification part includes the pedestrian feature extraction subnetwork and the pedestrian re-identification module. The detection part and the re-identification part in Net1 are referred to as DetNet1 and ReIdNet1, respectively, and the detection part and the re-identification part in Net2 are referred to as DetNet2 and ReIdNet2, respectively.

[0091] DetNet1 and DetNet2 are trained as follows: S3011. Randomly initialize DetNet1, and set the initial training parameters as follows: the initial learning rate is 0.001, the optimizer is stochastic gradient descent (SGD), the learning rate decay is poly strategy, the weight decay is 0.0005, the momentum is 0.9, the batch size is 16, and the number of training epochs is 100.

[0092] S3012, use training set 1 and training set 2 to train DetNet1, use verification set 1 and verification set 2 to calculate the loss function value of DetNet1, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is terminated to obtain the trained DetNet1; otherwise, the training is continued.

[0093] In this embodiment, the threshold is set to 0.01.

[0094] During the training process, the weighted sum of pedestrian localization loss and pedestrian recognition loss is used as the loss function of DetNet1. The expression of DetNet1 loss function is: .

[0095] in, represents the loss function of DetNet1, N Represents the total number of positioning prediction boxes that match the true box, represents the pedestrian positioning loss function, Represents the pedestrian recognition loss function.

[0096] The expression of pedestrian positioning loss function is: .

[0097] in, Pos Represents the set of all positioning prediction boxes that match the true box; xx Represents the horizontal coordinate of the center point of the positioning prediction box and the real box; cy Represents the vertical coordinate of the center point of the positioning prediction box and the real box; w Indicates the width of the positioning prediction box and the real box; h Indicates the height of the positioning prediction box and the real box; represents the indicator function; Represents the smooth L1 loss function; Indicatesi The positioning prediction box is xx The value at time; Indicates i The positioning prediction box is cy The value at time; Indicates i The positioning prediction box is w The value at time; Indicates i The positioning prediction box is h The value at time; Indicates j The ground-truth box is relative to i The positioning prediction box is xx The offset of , Indicates i The default box is xx The value of Indicates j A real frame xx The value of Indicates i The default box is w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is cy The offset of , Indicates j A real frame cy The value of Indicates i The default box is cy The value of Indicates i The default box is h The value of Indicates j The ground-truth box is relative to i The positioning prediction box is w The offset of , Indicates j A real frame w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is h The offset of , Indicates j A real frame h The value of .

[0098] and: ; ; ; .

[0099] The expression of pedestrian recognition loss function is:

[0100] .

[0101] in, Neg Represents the set of all positioning prediction boxes that do not match the true box; express Pos After normalization i The positioning prediction box belongs to p The confidence level of the class, , express Pos Middle i The positioning prediction box belongs to p Confidence of the class; express Neg The r The confidence that the positioning prediction box belongs to the background class; Represents a logarithmic function.

[0102] The gradient is calculated according to the loss function, and the stochastic gradient descent optimizer is used to update the DetNet1 network weights and biases. The poly strategy is used to decay the learning rate during training, and the decay formula is as follows: ; in, lr represents the learning rate decay, lr init represents the initial learning rate, iter Indicates the current iteration number, max _ iter Indicates the maximum number of iterations.

[0103] S3013, passing the network weights and biases in the trained DetNet1 to DetNet2, and initializing the network weights and biases of DetNet2.

[0104] S302, fix the network weights and biases of DetNet1 and DetNet2, and use training set 1, validation set 1 and training set 2, validation set 2 to train ReIdNet1 and ReIdNet2 respectively. The specific contents are as follows: S3021. Randomly initialize ReIdNet1 and ReIdNet2. The initial training parameters are set as follows: the initial learning rate is 0.001, the optimizer is stochastic gradient descent (SGD), the learning rate decay is the poly strategy, the weight decay is 0.0005, the momentum is 0.9, the batch size is 16, and the number of training epochs is 100.

[0105] S3022, using the structured dictionary of training set 1 and camera 1 D 1. Calculate the sparse coding corresponding to the pedestrian features.

[0106] S3023: For the labeled samples in training set 1, the gradient is calculated using the labeled loss function, and the stochastic gradient descent optimizer is used to update D 1; For the unlabeled samples in training set 1, the gradient is calculated using the unlabeled loss function, and the stochastic gradient descent optimizer is used to update D 1.

[0107] Labeled loss function The expression is:

[0108] in, Indicates the camera k Middle s Pedestrian characteristics of individual pedestrian categories, Indicates the use D k Refactoring The sparse coding of D k Indicates the camera k A structured dictionary of Indicates the use Refactoring The sparse coding of Indicates the camera k No. s Sub-dictionaries for individual pedestrian categories, Indicates the camera k No. t Sub-dictionaries for individual pedestrian categories, Indicates the use Refactoring The sparse coding of A k Indicates the use D k Refactoring Xk The sparse coding of X k Indicates the camera k Pedestrian characteristics of all pedestrian categories in; represents the F-norm; , Both represent weight coefficients, .

[0109] Unlabeled loss function The expression is: ; in, Indicates the camera k Pedestrian features in unlabeled images, Indicates the use D k Refactoring sparse coding.

[0110] S3024, will D The shared sub-dictionary in 1 is passed to the structured dictionary of camera 2 D 2, and replace D The corresponding shared sub-dictionary in 2.

[0111] S3025, using training set 2 and D 2. Calculate the sparse coding corresponding to the pedestrian features.

[0112] S3026, using the labeled loss function to calculate the gradient of the labeled image in training set 2, and using the stochastic gradient descent optimizer to update D 2; Use the unlabeled loss function to calculate the gradient of the unlabeled samples in training set 2, and use the stochastic gradient descent optimizer to update D 2.

[0113] S3027, will D The shared sub-dictionary in 2 is passed to D 1, and replace D The shared sub-dictionary in 1.

[0114] S3028. Loop through steps S3022-S3027 until the loss function values ​​of the semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on both the training set and the validation set converge, and then terminate the training.

[0115] Semi-supervised dictionary learning loss function The expression is: ; in, represents the weight coefficient, .

[0116] S303, use training set 1 to train Net1, use verification set 1 to calculate the loss function value of Net1, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed, and the trained Net1 is obtained, that is, the target model 1; otherwise, continue training; Use training set 2 to train Net2, use validation set 2 to calculate the loss function value of Net2, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed and the trained Net2 is obtained, that is, the target model 2; otherwise, continue training; Loss function of the end-to-end pedestrian detection and re-identification network model The expression is: .

[0117] S4. Input the test set 1 into the target model 2 to obtain the pedestrian detection and re-identification results of the test set 1; the specific contents are: Step 1: Input the pedestrian image samples to be identified in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, input these features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction subnetwork respectively, obtain the first scale dimension transformation feature, the second scale dimension transformation feature, the third scale dimension transformation feature and the fourth scale dimension transformation feature, connect the dimension transformation features of the four scales in series along dimension 1, and obtain the pedestrian features of the positioning prediction box.

[0118] Step 2: The position information of the positioning prediction box output by the pedestrian positioning subnetwork, the pedestrian confidence of the positioning prediction box output by the pedestrian recognition subnetwork, and the pedestrian features of the positioning prediction box are input into the non-maximum suppression module for non-maximum suppression operation to obtain the position information of the positioning prediction box, the pedestrian confidence, and the pedestrian features after screening.

[0119] The non-maximum suppression operation includes the following: (1) Put all positioning prediction boxes into the candidate set.

[0120] (2) Sort the positioning prediction boxes in the candidate set according to their pedestrian confidence, remove the positioning prediction box with the highest pedestrian confidence from the candidate set and put it into the selected set; delete the positioning prediction box whose overlapping area with the positioning prediction box is greater than the threshold from the candidate set.

[0121] (3) Repeat step (2) until the candidate set is empty. Since the location information, pedestrian confidence and pedestrian features of the positioning prediction box correspond one to one, the location information, pedestrian confidence and pedestrian features of the filtered positioning prediction box are obtained.

[0122] Step 3: Input the filtered pedestrian features into the semi-supervised dictionary learning module to obtain sparse codes, and perform pedestrian recognition based on the distance between the sparse codes.

[0123] The calculation formula for sparse coding is: .

[0124] in, x k Indicates the camera k The features of a pedestrian in a pedestrian image, Indicates the use Refactoring x k The sparse coding of .

[0125] Step 4: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in training set 2, and sort them from small to large according to the distance value. The representative code represents the mean of the sparse code of each class of labeled samples in the training set.

[0126] Step 5: According to the result of step 4, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rank k recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted.

[0127] Step 6: Based on the results of step 5, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 1 are summarized to obtain the pedestrian detection and re-identification results of test set 1.

[0128] Input test set 2 into target model 1 to obtain the pedestrian detection and re-identification results of test set 2. The specific contents are as follows: Step 1: Input the pedestrian image samples to be identified in the test set 2 into the target model 1 to obtain the location information, pedestrian confidence and sparse coding of the corresponding positioning prediction box.

[0129] Step 2: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in training set 1, and sort them from small to large according to the distance value.

[0130] Step 3: According to the result of step 2, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rank k recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted.

[0131] Step 4: Based on the results of step 3, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 2 are summarized to obtain the pedestrian detection and re-identification results of test set 2.

[0132] Among them, the threshold It is an adjustable parameter and should be set according to the actual task.

[0133] Figure 8 The top-1 search results of the method proposed in the present invention and the currently optimal PSTR method are shown, where the images in the Query column are images of pedestrians to be identified from camera 1, and the images in the PSTR column and the method proposed in the present invention column are images of pedestrians with the highest matching degree with the Query image in camera 2. The red frame in the image represents the detected pedestrian target, and the white frame is an enlarged view of the red frame. In the results of PTSR searching for different Query column images, the first three rows searched for the wrong pedestrians, and the last two rows searched for the correct pedestrians, while the results of the method proposed in the present invention searching for different Query column images all searched for the correct pedestrians, so the method proposed in the present invention has higher accuracy in pedestrian detection and re-identification.

[0134] The embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, the specific steps of the method provided in the embodiment of the present invention are corresponding to the specific steps of the method provided in the embodiment of the present invention, and the processor has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0135] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. It should be noted that when the computer program is executed by the processor, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, characterized in that: include: S1, respectively obtaining pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras, and processing the datasets; The processed pedestrian image dataset 1 is divided into a training set 1, a validation set 1 and a test set 1; The processed pedestrian image dataset 2 is divided into a training set 2, a validation set 2 and a test set 2; S2, based on camera 1, build an end-to-end pedestrian detection and re-identification network model 1; based on camera 2, build an end-to-end pedestrian detection and re-identification network model 2. Both model 1 and model 2 include an improved SSD512 network, a pedestrian feature extraction subnetwork, and a semi-supervised dictionary learning module; S3, using training set 1 and validation set 1 to train the end-to-end pedestrian detection and re-identification network model 1, to obtain the target model 1; using training set 2 and validation set 2 to train the end-to-end pedestrian detection and re-identification network model 2, to obtain the target model 2; S4. Input test set 1 into target model 2 to obtain pedestrian detection and re-identification results of test set 1; input test set 2 into target model 1 to obtain pedestrian detection and re-identification results of test set 2.

2. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 1 is characterized in that: In step S1, dividing the pedestrian image dataset includes the following contents: S101, obtaining pedestrian image dataset 1 and pedestrian image dataset 2 includes two methods: (1) Download the pedestrian image dataset 1 and pedestrian image dataset 2 corresponding to the two cameras from the open source dataset website; (2) Self-made pedestrian image dataset 1 and pedestrian image dataset 2. The specific contents are as follows: use two cameras in different scenes to collect a set number of pedestrian images, collect more than two images for each pedestrian, and the pedestrians captured by the two cameras have a repetition rate of more than 50%; each image is annotated with the camera to which it belongs; S102, maintaining the original aspect ratio of the acquired pedestrian image, scaling the largest sides of the width and height to set pixels, centering the scaled image in a black background image of a set size, and performing random cropping, scaling, horizontal flipping, random change of image brightness, random change of image contrast, random change of image chromaticity, and random change of image saturation to obtain processed pedestrian image dataset 1 and processed pedestrian image dataset 2; S103. Divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 respectively in proportion to obtain training set 1, validation set 1, test set 1 and training set 2, validation set 2, test set 2; wherein, training set 1 and training set 2 both include labeled and unlabeled pedestrian images, and validation set 1, validation set 2, test set 2 and test set 2 all include labeled pedestrian images.

3. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 1, characterized in that: In step S2, building an end-to-end pedestrian detection and re-identification network model includes the following: S201. Improve the SSD512 network to obtain an improved SSD512 network, the specific contents of which are as follows: The backbone network is replaced with ResNet50, which includes an input layer, convolution module 1, convolution module 2, convolution module 3, convolution module 4 and convolution module 5 connected in sequence; The process of obtaining the first multi-scale output result is replaced by the process of sequentially passing through convolution module 4 and L2 normalization in ResNet50; a batch normalization layer is added between each convolution layer and activation layer in convolution module 6 and convolution module 7; the last four convolution modules of the multi-scale network are replaced by 3×3 convolution with a stride of 2, batch normalization, ReLU activation function, 3×3 convolution with a stride of 1, batch normalization, and ReLU activation function connected in sequence; Set the localization branch to the pedestrian localization subnetwork; set the recognition branch to the pedestrian recognition subnetwork, change the number of categories to 2, and add a batch normalization layer after each convolution layer in the pedestrian recognition subnetwork; S202, adding a pedestrian feature extraction subnetwork between the multi-scale network and non-maximum suppression module of the improved SSD512 network, and the network is parallel to the pedestrian positioning subnetwork and the pedestrian recognition subnetwork; The pedestrian feature extraction subnetwork consists of four scales and a connection layer. Each scale includes a 3×3 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, and a dimension transformation, which are connected in sequence. The dimension transformations of the four scales are all connected to the connection layer. S203, the semi-supervised dictionary learning module includes structured dictionaries of two cameras, and the structured dictionary includes c k A sub-dictionary, k =1,2, c k Indicates the camera k The number of all pedestrian categories in c k The sub-dictionaries include a shared sub-dictionary and an exclusive sub-dictionary. The shared sub-dictionaries of the two cameras are the same, while the exclusive sub-dictionaries are different.

4. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 1, characterized in that: In step S3, the target model is obtained including the following contents: S301, the end-to-end pedestrian detection and re-identification network model 1 is referred to as Net1, and the end-to-end pedestrian detection and re-identification network model 2 is referred to as Net2; The end-to-end pedestrian detection and re-identification network model includes a detection part and a re-identification part, wherein the detection part includes a backbone network, a multi-scale network, a pedestrian localization sub-network and a pedestrian recognition sub-network, and the re-identification part includes a pedestrian feature extraction sub-network and a pedestrian re-identification module; the detection part and the re-identification part in Net1 are referred to as DetNet1 and ReIdNet1 respectively, and the detection part and the re-identification part in Net2 are referred to as DetNet2 and ReIdNet2 respectively; DetNet1 and DetNet2 are trained as follows: S3011, randomly initialize DetNet1; S3012, using training set 1 and training set 2 to train DetNet1, using verification set 1 and verification set 2 to calculate the loss function value of DetNet1, and calculating the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is terminated to obtain the trained DetNet1; otherwise, the training is continued; The expression of DetNet1 loss function is: ； in, represents the loss function of DetNet1, N Represents the total number of positioning prediction boxes that match the true box, represents the pedestrian positioning loss function, represents the pedestrian recognition loss function; The expression of pedestrian positioning loss function is: ； in, Pos Represents the set of all positioning prediction boxes that match the true box; xx Represents the horizontal coordinate of the center point of the positioning prediction box and the real box; cy Indicates the vertical coordinate of the center point of the positioning prediction box and the real box; w Indicates the width of the positioning prediction box and the real box; h Indicates the height of the positioning prediction box and the real box; represents the indicator function; Represents the smooth L1 loss function; Indicates i The positioning prediction box is xx The value at time; Indicates i The positioning prediction box is cy The value at time; Indicates i The positioning prediction box is w The value at time; Indicates i The positioning prediction box is h The value at time; Indicates j The ground-truth box is relative to i The positioning prediction box is xx The offset of , Indicates i The default box is xx The value of Indicates j A real frame xx The value of Indicates i The default box is w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is cy The offset of , Indicates j A real frame cy The value of Indicates i The default box is cy The value of Indicates i The default box is h The value of Indicates j The ground-truth box is relative to i The positioning prediction box is w The offset of , Indicates j A real frame w The value of Indicates j The ground-truth box is relative to i The positioning prediction box is h The offset of , Indicates j A real frame h The value of and: ； ； ； ； The expression of pedestrian recognition loss function is: ； in, Neg Represents the set of all positioning prediction boxes that do not match the true box; express Pos After normalization i The positioning prediction box belongs to p The confidence level of the class, , express Pos Middle i The positioning prediction box belongs to p Confidence of the class; express Neg The r The confidence that the positioning prediction box belongs to the background class; represents the logarithmic function; The stochastic gradient descent optimizer is used to update the network weights and biases of DetNet1, and the poly strategy is used to decay the learning rate. The decay formula is: ； in, lr represents the learning rate decay, lr init represents the initial learning rate, iter Indicates the current iteration number, max _ iter Indicates the maximum number of iterations; S3013, passing the network weights and biases in the trained DetNet1 to DetNet2, and initializing the network weights and biases of DetNet2; S302, fix the network weights and biases of DetNet1 and DetNet2, and use training set 1, validation set 1 and training set 2, validation set 2 to train ReIdNet1 and ReIdNet2 respectively. The specific contents are as follows: S3021, randomly initialize ReIdNet1 and ReIdNet2; S3022, using the structured dictionary of training set 1 and camera 1 D 1. Calculate the sparse coding corresponding to the pedestrian features; S3023, using the labeled loss function to calculate the gradient of the labeled samples in training set 1, and using the stochastic gradient descent optimizer to update D 1; Use the unlabeled loss function to calculate the gradient of the unlabeled samples in training set 1, and use the stochastic gradient descent optimizer to update D 1; Labeled loss function The expression is: ； in, Indicates the camera k Middle s Pedestrian characteristics of individual pedestrian categories, Indicates the use D k Refactoring The sparse coding of D k Indicates the camera k A structured dictionary of Indicates the use Refactoring The sparse coding of Indicates the camera k No. s Sub-dictionaries for individual pedestrian categories, c k Indicates the camera k The number of all pedestrian categories in Indicates the camera k No. t Sub-dictionaries for individual pedestrian categories, Indicates the use Refactoring The sparse coding of A k Indicates the use D k Refactoring X k The sparse coding of X k Indicates the camera k Pedestrian characteristics of all pedestrian categories in; represents the F-norm; , All represent weight coefficients; Unlabeled loss function The expression is: ； in, Indicates the camera k Pedestrian features in unlabeled images, Indicates the use D k Refactoring Sparse coding; S3024, will D The shared sub-dictionary in 1 is passed to the structured dictionary of camera 2 D 2, and replace D The corresponding shared sub-dictionary in 2; S3025, using training set 2 and D 2. Calculate the sparse coding corresponding to the pedestrian features; S3026, using the labeled loss function to calculate the gradient of the labeled image in training set 2, and using the stochastic gradient descent optimizer to update D 2; Use the unlabeled loss function to calculate the gradient of the unlabeled samples in training set 2, and use the stochastic gradient descent optimizer to update D 2; S3027, will D The shared sub-dictionary in 2 is passed to D 1, and replace D The shared sub-dictionary in 1; S3028, looping through steps S3022-S3027 until the loss function values ​​of the semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on the training set and the validation set converge, then the training ends; Semi-supervised dictionary learning loss function The expression is: ； in, represents the weight coefficient; S303, use training set 1 to train Net1, use verification set 1 to calculate the loss function value of Net1, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed, and the trained Net1 is obtained, that is, the target model 1; otherwise, continue training; Use training set 2 to train Net2, use validation set 2 to calculate the loss function value of Net2, and calculate the absolute value of the difference between the loss function value and the loss function value of the previous round of training; if the absolute value is less than the threshold, the training is completed and the trained Net2 is obtained, that is, the target model 2; otherwise, continue training; Loss function of the end-to-end pedestrian detection and re-identification network model The expression is: 。 5. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 3 is characterized in that: In step S4, the pedestrian detection and re-identification results of the test set 1 include the following: Step 1: Input the pedestrian image samples to be identified in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, input the features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction subnetwork respectively, obtain the first scale dimension transformation feature, the second scale dimension transformation feature, the third scale dimension transformation feature and the fourth scale dimension transformation feature, connect the four scale dimension transformation features in series along dimension 1, and obtain the pedestrian feature of the positioning prediction box; Step 2: The location information of the positioning prediction box output by the pedestrian positioning subnetwork, the pedestrian confidence of the positioning prediction box output by the pedestrian recognition subnetwork, and the pedestrian features of the positioning prediction box are input into the non-maximum suppression module for non-maximum suppression operation to obtain the location information, pedestrian confidence, and pedestrian features of the filtered positioning prediction box; Step 3: Input the filtered pedestrian features into the semi-supervised dictionary learning module to obtain sparse codes, and perform pedestrian recognition based on the distance between the sparse codes; The calculation formula for sparse coding is: ； in, x k Indicates the camera k The features of a pedestrian in a pedestrian image, Indicates the camera k No. s Sub-dictionaries for individual pedestrian categories, Indicates the use Refactoring x k The sparse coding of ; Step 4: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in the training set 2, and sort them in ascending order of distance value; where the representative code represents the mean of the sparse code of each class of labeled samples in the training set; Step 5: According to the result of step 4, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rankk recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted; Step 6: Based on the results of step 5, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 1 are summarized to obtain the pedestrian detection and re-identification results of test set 1.

6. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 5, characterized in that: The non-maximum suppression operation includes the following: Step 1: Put all positioning prediction boxes into the candidate set; Step 2: Sort the positioning prediction frames in the candidate set according to their pedestrian confidence, remove the positioning prediction frame with the highest pedestrian confidence from the candidate set and put it into the selected set; delete the positioning prediction frame whose overlapping area with the positioning prediction frame is greater than the threshold from the candidate set; Step 3: Repeat step 2 until the candidate set is empty. Since the location information, pedestrian confidence and pedestrian features of the positioning prediction box correspond one to one, the location information, pedestrian confidence and pedestrian features of the filtered positioning prediction box are obtained.

7. The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning according to claim 5, characterized in that: In step S4, the pedestrian detection and re-identification results of test set 2 include the following: Step 1: Input the pedestrian image samples to be identified in the test set 2 into the target model 1 to obtain the location information, pedestrian confidence and sparse coding of the corresponding positioning prediction box; Step 2: Calculate the Euclidean distance between the sparse code and the representative code of each class of samples in training set 1, and sort them in ascending order of distance value; Step 3: According to the result of step 2, if the minimum distance is greater than the threshold , then the pedestrian positioning prediction box in the pedestrian image sample to be identified is marked as an unknown class; otherwise, the rankk recognition results of the pedestrian positioning prediction box in the pedestrian image sample to be identified are counted; Step 4: Based on the results of step 3, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all pedestrian image samples to be identified in test set 2 are summarized to obtain the pedestrian detection and re-identification results of test set 2.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning described in any one of claims 1 to 7 is executed.

Images