Pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning

Abstract

The application discloses a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, comprising the following steps: acquiring pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and processing the datasets; dividing the processed pedestrian image dataset 1 and pedestrian image dataset 2; constructing an end-to-end pedestrian detection and re-identification network model, wherein the model comprises an improved SSD512 network, a pedestrian feature extraction subnetwork and a semi-supervised dictionary learning module; training the end-to-end pedestrian detection and re-identification network model to obtain a target model; and inputting data into the target model to obtain pedestrian detection and re-identification results. The application realizes end-to-end semi-supervised pedestrian detection and re-identification, solves problems such as uneven quality of detected positioning bounding boxes, appearance differences and peripheral environment differences of pedestrians under different cameras, and improves the performance of the pedestrian detection and re-identification method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pedestrian detection and pedestrian re-identification, in particular to a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning. BACKGROUND

[0002] In recent years, more and more surveillance cameras are deployed in various public places, and video surveillance is playing an increasingly important role in combating illegal crimes and maintaining social security. In the application of video surveillance, the detection and re-identification of specific pedestrians is an important requirement.

[0003] Pedestrian detection technology is used to accurately locate and identify pedestrians in a single surveillance camera. Pedestrian re-identification technology is used to determine whether the pedestrians appearing under different surveillance cameras are the same person. Combining these two technologies together can quickly locate the specific pedestrian of interest in the video surveillance network.

[0004] Current end-to-end pedestrian detection and re-identification methods use deep network to learn a set of shared features, and use the features for pedestrian detection and re-identification at the same time. Such methods pay more attention to the balance between the detection and re-identification sub-tasks of the model, and ignore the enhancement of the model's ability to learn pedestrian feature representation, thereby limiting the overall pedestrian detection and re-identification performance of the end-to-end model.

[0005] Currently, supervised pedestrian detection and re-identification methods require the use of a large amount of labeled data, resulting in high labor labeling cost; weakly supervised pedestrian detection and re-identification methods only require pedestrian detection frame labeling information, and no longer rely on pedestrian identity label information, but are easily affected by noise and pedestrian detection frame labeling, resulting in lower performance. SUMMARY

[0006] The purpose of the present application is to provide a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, which can reduce the burden of manual labeling, reduce the impact of detection results on pedestrian re-identification, and improve the performance of pedestrian detection and re-identification.

[0007] The present application adopts the following technical solutions:

[0008] The pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning comprises the following steps:

[0009] S1, obtain pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and process the dataset; divide the processed pedestrian image dataset 1 to obtain training set 1, validation set 1 and test set 1; divide the processed pedestrian image dataset 2 to obtain training set 2, validation set 2 and test set 2.

[0010] S2, based on camera 1, construct an end-to-end pedestrian detection and re-identification network model 1; based on camera 2, construct an end-to-end pedestrian detection and re-identification network model 2, model 1 and model 2 both include an improved SSD512 network, a pedestrian feature extraction subnetwork and a semi-supervised dictionary learning module.

[0011] S3, train the end-to-end pedestrian detection and re-identification network model 1 using training set 1 and validation set 1 to obtain target model 1; train the end-to-end pedestrian detection and re-identification network model 2 using training set 2 and validation set 2 to obtain target model 2.

[0012] S4, input test set 1 into target model 2 to obtain pedestrian detection and re-identification results of test set 1; input test set 2 into target model 1 to obtain pedestrian detection and re-identification results of test set 2.

[0013] Further, in step S1, dividing the pedestrian image dataset includes the following contents:

[0014] S101, obtaining pedestrian image dataset 1 and pedestrian image dataset 2 includes two ways:

[0015] (1) Download the pedestrian image dataset 1 and the pedestrian image dataset 2 corresponding to the two cameras from the open source dataset website.

[0016] (2) Self-made pedestrian image dataset 1 and pedestrian image dataset 2, the specific content is: using two cameras of different scenes to collect a certain number of pedestrian images, collecting more than two images for each pedestrian, and the repetition rate of pedestrians captured by the two cameras is more than 50%; label each image with the camera it belongs to.

[0017] S102, keep the original aspect ratio of the obtained pedestrian image, and scale the largest side of the width and height to a certain pixel, center the scaled image in a black background image of a certain size, and perform random cropping, scaling, horizontal flipping, random image brightness change, random image contrast change, random image chroma change, and random image saturation change to obtain the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2.

[0018] S103, divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 by proportion to obtain training set 1, validation set 1, test set 1, and training set 2, validation set 2, test set 2; wherein, the training set 1 and the training set 2 both include labeled and unlabeled pedestrian images, and the validation set 1, the validation set 2, the test set 2 and the test set 2 all include labeled pedestrian images.

[0019] Further, in step S2, constructing an end-to-end pedestrian detection and re-identification network model includes the following contents:

[0020] S201, the SSD512 network is improved to obtain an improved SSD512 network, and the specific content is:

[0021] The backbone network is replaced by ResNet50, and the ResNet50 includes an input layer, a convolution module 1, a convolution module 2, a convolution module 3, a convolution module 4 and a convolution module 5 connected in sequence.

[0022] The process of obtaining the first multi-scale output result is replaced by a process of sequentially passing through the convolution module 4 in the ResNet50 and the L2 normalization; a batch normalization layer is added between each convolution layer and the activation layer in the convolution module 6 and the convolution module 7; the last four convolution modules of the multi-scale network are all replaced by a 3x3 convolution with a step of 2, a batch normalization, a ReLU activation function, a 3x3 convolution with a step of 1, a batch normalization and a ReLU activation function connected in sequence.

[0023] The positioning branch is set as a pedestrian positioning subnetwork; the recognition branch is set as a pedestrian recognition subnetwork, and the number of categories is changed to 2, and a batch normalization layer is added after each convolution layer in the pedestrian recognition subnetwork.

[0024] S202, a pedestrian feature extraction subnetwork is added between the multi-scale network of the improved SSD512 network and the non-maximum suppression module, and the network is parallel to the pedestrian positioning subnetwork and the pedestrian recognition subnetwork; the pedestrian feature extraction subnetwork includes four scales and a connection layer, each scale includes a 3x3 convolution with a step of 1, a batch normalization layer, a ReLU activation function and a conversion connected in sequence, and the conversion of the four scales is connected with the connection layer.

[0025] S203, the semi-supervised dictionary learning module includes a structured dictionary of two cameras, and the structured dictionary includes c k a sub-dictionary, k =1,2, c k represents the number of all pedestrian categories in the camera k c k The sub-dictionary includes a shared sub-dictionary and a unique sub-dictionary, the shared sub-dictionaries of the two cameras are the same, and the unique sub-dictionaries are different.

[0026] Further, in step S3, obtaining the target model includes the following content:

[0027] S301, an end-to-end pedestrian detection and re-identification network model 1 is abbreviated as Net1, and an end-to-end pedestrian detection and re-identification network model 2 is abbreviated as Net2.

[0028] ​The end-to-end pedestrian detection and re-identification network model comprises a detection part and a re-identification part, wherein the detection part comprises a backbone network, a multi-scale network, a pedestrian positioning sub-network and a pedestrian identification sub-network, and the re-identification part comprises a pedestrian feature extraction sub-network and a pedestrian re-identification module; the detection part and the re-identification part in Net1 are simply referred to as DetNet1 and ReIdNet1 respectively, and the detection part and the re-identification part in Net2 are simply referred to as DetNet2 and ReIdNet2 respectively.

[0029] The DetNet1 and the DetNet2 are trained, and the specific content is as follows:

[0030] S3011, randomly initializing the DetNet1.

[0031] S3012, training the DetNet1 using the training set 1 and the training set 2, calculating the loss function value of the DetNet1 using the verification set 1 and the verification set 2, and calculating the absolute value of the difference between the loss function value and the loss function value of the last round of training; if the absolute value is less than a threshold value, the training is ended, and the trained DetNet1 is obtained; otherwise, the training is continued.

[0032] The expression of the DetNet1 loss function is as follows:

[0033] .

[0034] Wherein, represents the loss function of the DetNet1, N represents the total number of positioning prediction boxes matched with the real boxes, represents the pedestrian positioning loss function, represents the pedestrian identification loss function.

[0035] The expression of the pedestrian positioning loss function is as follows:

[0036] .

[0037] Wherein, Pos represents the set of all positioning prediction boxes matched with the real boxes; cx represents the horizontal coordinate of the center point of the positioning prediction box and the real box; cy represents the vertical coordinate of the center point of the positioning prediction box and the real box; w represents the width of the positioning prediction box and the real box; h represents the height of the positioning prediction box and the real box; represents an indicator function; represents a smooth L1 loss function; represents the i th positioning prediction box in cxThe value at time; Indicates the first i A localization prediction box in cy The value at time; Indicates the first i A localization prediction box in w The value at time; Indicates the first i A localization prediction box in h The value at time; Indicates the first j The first true bounding box is relative to the first... i A localization prediction box in cx The offset, , Indicates the first i A default box in cx The value, Indicates the first j A real frame cx The value, Indicates the first i A default box in w The value; Indicates the first j The first true bounding box is relative to the first... i A localization prediction box in cy The offset, , Indicates the first j A real frame cy The value, Indicates the first i A default box in cy The value, Indicates the first i A default box in h The value; Indicates the first j The first true bounding box is relative to the first... i A localization prediction box in w The offset, , Indicates the first j A real frame w The value; Indicates the first j The first true bounding box is relative to the first... i A localization prediction box in h The offset, , Indicates the first j A real frame h The value of .

[0038] and:

[0039] ;

[0040] ;

[0041] ;

[0042] .

[0043] The expression of the pedestrian recognition loss function is:

[0044] .

[0045] wherein, Neg represents a set of all positioning prediction boxes that are not matched with the real box; represents Pos the confidence that the normalized i th positioning prediction box in p belongs to the th class, , Pos represents the confidence that the i th positioning prediction box in p belongs to the th class; Neg represents the confidence that the r th positioning prediction box in belongs to the background class;

[0046] The network weights and biases of DetNet1 are updated by using a stochastic gradient descent optimizer, and the learning rate is decayed by using a poly strategy, and the decay formula is:

[0047] ;

[0048] wherein, lr represents learning rate decay, lr init represents initial learning rate, iter represents current iteration number, max _ iter represents maximum iteration number.

[0049] S3013, the network weights and biases in the trained DetNet1 are passed to DetNet2, and the network weights and biases of DetNet2 are initialized.

[0050] S302, the network weights and biases of DetNet1 and DetNet2 are fixed, and ReIdNet1 and ReIdNet2 are trained by using training set 1, validation set 1 and training set 2, validation set 2 respectively, and the specific content is:

[0051] S3021, ReIdNet1 and ReIdNet2 are randomly initialized.

[0052] S3022, using the structured dictionary of camera 1 and training set 1 D 1. Calculate the sparse code corresponding to the pedestrian feature.

[0053] S3023, calculate the gradient of the labeled sample in training set 1 using the labeled loss function, and update using the stochastic gradient descent optimizer D 1. Calculate the gradient of the unlabeled sample in training set 1 using the unlabeled loss function, and update using the stochastic gradient descent optimizer D 1.

[0054] Labeled loss function The expression is:

[0055]

[0056] Among them, represents the pedestrian feature of the i-th pedestrian class in camera k , s represents the structured dictionary of camera , D k Reconstruct the sparse code , D k represents the structured dictionary of camera k , Reconstruct the sparse code , represents the i-th pedestrian class sub-dictionary of camera , k represents the i-th pedestrian class sub-dictionary of camera s , Reconstruct the sparse code k , t Reconstruct the sparse code , Reconstruct the sparse code , A k Reconstruct the sparse code D k , X k Reconstruct the sparse code X k represents the pedestrian feature of all pedestrian classes in camera k ; represents the F norm; , Both represent weight coefficients.

[0057] Unlabeled loss function The expression is:

[0058] ;

[0059] wherein, represents the camera k feature of the pedestrian in the unlabeled image, represents using D k reconstructs the sparse code.

[0060] S3024, pass the shared sub-dictionary in S3023 to the structured dictionary of the camera 2 in S3022, and replace the corresponding shared sub-dictionary in S3022. D 1. D D 2.

[0061] S3025, calculate the sparse code corresponding to the pedestrian feature by using the training set 2 and S3024. D 2.

[0062] S3026, calculate the gradient of the labeled image in the training set 2 by using the labeled loss function, and update S3025 by using the stochastic gradient descent optimizer; calculate the gradient of the unlabeled sample in the training set 2 by using the unlabeled loss function, and update S3025 by using the stochastic gradient descent optimizer. D 2. D 2.

[0063] S3027, pass the shared sub-dictionary in S3026 to S3021, and replace the shared sub-dictionary in S3021. D 1. D 1. D 1.

[0064] S3028, cyclically execute steps S3022-S3027 until the loss function values of the semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on the training set and the validation set are converged, and then end the training.

[0065] The expression of the semi-supervised dictionary learning loss function is as follows:

[0066] ;

[0067] wherein, represents the weight coefficient.

[0068] S303, train Net1 by using the training set 1, calculate the loss function value of Net1 by using the validation set 1, and calculate the absolute value of the difference between the loss function value and the loss function value of the last round of training; if the absolute value is less than a threshold value, the training is completed, and the trained Net1, i.e., the target model 1, is obtained; otherwise, the training is continued.

[0069] ​The Net2 is trained using the training set 2, the loss function value of the Net2 is calculated using the verification set 2, and the absolute value of the difference between the loss function value and the loss function value of the last round of training is calculated; if the absolute value is less than a threshold value, the training is completed, and the trained Net2, that is, the target model 2, is obtained; otherwise, the training is continued.

[0070] Loss function of the end-to-end pedestrian detection and re-identification network model The expression is:

[0071] .

[0072] Further, in step S4, the pedestrian detection and re-identification result of the test set 1 includes the following contents:

[0073] Step 1, input the to-be-identified pedestrian image sample in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, input the features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction sub-network respectively, obtain the first scale trans-dimensional feature, the second scale trans-dimensional feature, the third scale trans-dimensional feature and the fourth scale trans-dimensional feature, and connect the trans-dimensional features of the four scales along the dimension 1 to obtain the pedestrian feature of the positioning prediction box.

[0074] Step 2, input the position information of the positioning prediction box output by the pedestrian positioning sub-network, the pedestrian confidence of the positioning prediction box output by the pedestrian recognition sub-network and the pedestrian feature of the positioning prediction box into the non-maximum suppression module for non-maximum suppression operation to obtain the position information, the pedestrian confidence and the pedestrian feature of the screened positioning prediction box.

[0075] Step 3, input the screened pedestrian feature into the semi-supervised dictionary learning module to obtain the sparse coding, and perform pedestrian recognition according to the distance between the sparse coding.

[0076] The calculation formula of the sparse coding is:

[0077] ;

[0078] Wherein, x k represents the feature of a pedestrian in a pedestrian image of a camera k , represents the sparse coding of reconstructing x k , .

[0079] Step 4, calculate the Euclidean distance between the sparse code and the representative code of each class of samples in the training set 2, and sort the distance values from small to large; wherein the representative code represents the mean value of the sparse code of each class of labeled samples in the training set.

[0080] Step 5, according to the result of step 4, if the minimum distance is greater than the threshold , mark the positioning prediction box of the pedestrian in the to-be-identified pedestrian image sample as an unknown class; otherwise, count the rank k recognition result of the pedestrian positioning prediction box in the to-be-identified pedestrian image sample.

[0081] Step 6, according to the result of step 5, aggregate the position information and rank k recognition result of all pedestrian positioning prediction boxes of all to-be-identified pedestrian image samples in the test set 1, to obtain the pedestrian detection and re-identification result of the test set 1.

[0082] Further, the non-maximum suppression operation includes the following contents:

[0083] Step 1, put all positioning prediction boxes into the candidate set.

[0084] Step 2, sort the positioning prediction boxes in the candidate set according to the pedestrian confidence of the positioning prediction boxes, move the positioning prediction box with the highest pedestrian confidence out of the candidate set and into the selected set; delete the positioning prediction boxes with an overlap area greater than a threshold value from the candidate set.

[0085] Step 3, repeat step 2 until the candidate set is empty; since the position information, pedestrian confidence and pedestrian feature of the positioning prediction box are one-to-one corresponding, the position information, pedestrian confidence and pedestrian feature of the screened positioning prediction box are obtained.

[0086] Further, in step S4, obtaining the pedestrian detection and re-identification result of the test set 2 includes the following contents:

[0087] Step 1, input the to-be-identified pedestrian image sample in the test set 2 into the target model 1 to obtain the position information, pedestrian confidence and sparse code of the corresponding positioning prediction box.

[0088] Step 2, calculate the Euclidean distance between the sparse code and the representative code of each class of samples in the training set 1, and sort the distance values from small to large.

[0089] Step 3, according to the result of step 2, if the minimum distance is greater than the threshold , mark the positioning prediction box of the pedestrian in the to-be-identified pedestrian image sample as an unknown class; otherwise, count the rank k recognition result of the pedestrian positioning prediction box in the to-be-identified pedestrian image sample.

[0090] Step 4, according to the result of step 3, the position information and rank k identification result of all pedestrian positioning prediction boxes of all to-be-identified pedestrian image samples in test set 2 are summarized, and the pedestrian detection and re-identification result of test set 2 is obtained.

[0091] Further, the present application also proposes an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning when executing the computer program.

[0092] Further, the present application also proposes a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to perform the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning.

[0093] Compared with the prior art, the present application has the following technical effects:

[0094] 1. The end-to-end pedestrian detection and re-identification network model constructed by the present application can reduce the influence of target detection result on re-identification.

[0095] 2. The present application uses a deep ResNet as a backbone network, effectively improving the feature expression ability; at the same time, a pedestrian feature extraction sub-network is added, which effectively enhances the ability to learn discriminative pedestrian feature representation and improves the performance of pedestrian re-identification.

[0096] 3. The present application uses semi-supervised dictionary learning in the pedestrian re-identification module, fully utilizes a large number of completely unlabeled images, and combines the existing pedestrian positioning boundary box and identity information labeled images for training, which can reduce the burden of manual labeling and improve the performance of pedestrian re-identification. BRIEF DESCRIPTION OF DRAWINGS

[0097] Fig. 1 is the overall implementation flowchart of the present application.

[0098] Fig. 2 is the structure schematic diagram of ResNet50 of the present application.

[0099] Fig. 3 is the structure schematic diagram of the dimensionality reduction residual module of the present application.

[0100] Fig. 4 is the structure schematic diagram of the dimensionality reduction residual module of the present application.

[0101] Fig. 5 is the improved multi-scale network structure schematic diagram of the present application.

[0102] Fig. 6 is a structural schematic diagram of a pedestrian feature extraction subnetwork of the present application.

[0103] Fig. 7 is a structural schematic diagram of a semi-supervised dictionary learning module of the present application.

[0104] Fig. 8 is a recognition result diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0105] The present application will be further described below in conjunction with the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0106] To achieve the above-mentioned purpose, the present application proposes a pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, as shown in the following specific steps: Fig. 1

[0107] S1, obtain pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and process the dataset; divide the processed pedestrian image dataset 1 to obtain training set 1, validation set 1 and test set 1; divide the processed pedestrian image dataset 2 to obtain training set 2, validation set 2 and test set 2; the specific content is:

[0108] S101, obtaining pedestrian image dataset 1 and pedestrian image dataset 2 includes two ways:

[0109] (1) Download pedestrian image dataset 1 and pedestrian image dataset 2 corresponding to two cameras from the open source dataset website http: / / www.liangzheng.com.cn. Training set 1 contains 2835 video frames, validation set 1 contains 315 video frames, test set 1 contains 890 video frames, training set 2 contains 2299 video frames, validation set 2 contains 255 video frames, and test set contains 2255 video frames.

[0110] (2) Self-made pedestrian image dataset 1 and pedestrian image dataset 2, the specific content is: use two cameras of different scenes to collect a certain number of pedestrian images, collect more than two images for each pedestrian, and more than half of the pedestrians captured by the two cameras are the same; among the remaining pedestrians, a part of the pedestrians are only captured by camera 1, and another part of the pedestrians are only captured by camera 2; label each image with the camera it belongs to.

[0111] ​S102, keep the original aspect ratio of the acquired pedestrian image, use the resize function in the torchvision.transforms toolkit to scale the largest side of the width and height to 513 pixels, center the scaled image in a black background image of 513x513, and perform random cropping, scaling, horizontal flipping, random image brightness change, random image contrast change, random image chroma change, and random image saturation change to obtain a processed pedestrian image dataset 1 and a processed pedestrian image dataset 2.

[0112] S103, proportionally divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 to obtain training set 1, validation set 1, test set 1 and training set 2, validation set 2, test set 2; wherein the training set 1 and the training set 2 both include labeled and unlabeled pedestrian images, and the validation set 1, the validation set 2, the test set 2 and the test set 2 all include labeled pedestrian images.

[0113] S2, based on camera 1, an end-to-end pedestrian detection and re-identification network model 1 is constructed; based on camera 2, an end-to-end pedestrian detection and re-identification network model 2 is constructed, and the model 1 and the model 2 both include an improved SSD512 network, a pedestrian feature extraction subnetwork and a semi-supervised dictionary learning module; the specific content is:

[0114] S201, the SSD512 network is improved to obtain an improved SSD512 network, and the specific content is:

[0115] The backbone network VGG-16 is replaced by ResNet50, as shown in Fig. 2 The ResNet50 includes an input layer, a convolution module 1, a convolution module 2, a convolution module 3, a convolution module 4 and a convolution module 5 connected in sequence; wherein the convolution module 1 includes 1 7x7 convolution layer and 1 3x3 max pooling layer, the output channel number is 64, and the output feature map size is 257x257; the convolution module 2 includes 1 dimension reduction residual module as shown in Fig. 3 and 2 non-dimension reduction residual modules as shown in Fig. 4 connected in sequence, the output channel number is 256, and the output feature map size is 129x129; the convolution module 3 includes 1 dimension reduction residual module as shown in Fig. 3 and 3 non-dimension reduction residual modules as shown in Fig. 4 connected in sequence, the output channel number is 512, and the output feature map size is 65x65; the convolution module 4 includes 1 dimension reduction residual module as shown in Fig. 3 and 5 non-dimension reduction residual modules as shown in Fig. 4 connected in sequence, the output channel number is 1024, and the output feature map size is 33x33; the convolution module 5 includes 1 dimension reduction residual module as shown inFig. 4 The three residual modules without dimensionality reduction shown have 2048 output channels and an output feature map size of 33×33.

[0116] Among them, such as Fig. 3 The dimension reduction residual module shown includes two branches that share the same input. Branch 1 includes a 1×1 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 3×3 convolution with a stride of 2, a batch normalization layer, a ReLU activation function, a 1×1 convolution with a stride of 1, a batch normalization layer, an additive layer, and a ReLU activation function, connected in sequence. Branch 2 includes a 1×1 convolution with a stride of 2 and a batch normalization layer, with the batch normalization layer of branch 2 connected to the additive layer of branch 1.

[0117] like Fig. 4 The dimensionless residual module shown includes two branches that share the same input. Branch 1 includes a 1×1 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 3×3 convolution with a stride of 1, a batch normalization layer, a ReLU activation function, a 1×1 convolution with a stride of 1, a batch normalization layer, an additive layer, and a ReLU activation function, connected in sequence. Branch 2 is directly connected to the additive layer of branch 1.

[0118] The process of obtaining the first multi-scale output result is changed from sequentially passing through the third convolutional layer of the fourth convolutional module in VGG-16, the ReLU activation function, and L2 normalization to sequentially passing through the fourth convolutional module and L2 normalization in ResNet50. The feature map size of the output result is 33×33.

[0119] A batch normalization layer is added between each convolutional layer and activation layer in convolutional module 6 and convolutional module 7. The feature map size output by convolutional module 7 is 17×17.

[0120] Replace the last four convolutional modules of the multi-scale network with... Fig. 5 The diagram shows a 3×3 convolution with a stride of 2, a batch normalization layer, a ReLU activation function, a 3×3 convolution with a stride of 1, a batch normalization layer, and a ReLU activation function connected in sequence. The number of output channels for these four convolutional modules are 512, 256, 256, and 256, respectively, and the output feature map sizes are 9×9, 5×5, 3×3, and 1×1, respectively.

[0121] The localization branch is set as a pedestrian localization subnetwork to generate pedestrian localization prediction boxes and output the location information of the prediction boxes. The recognition branch is set as a pedestrian recognition subnetwork, and the number of categories is changed to 2 to identify whether the area marked by the localization prediction box is a pedestrian. A batch normalization layer is added after each convolutional layer in the pedestrian recognition subnetwork. The pedestrian recognition subnetwork outputs the pedestrian confidence of the localization prediction boxes, and the number of output channels is the number of all pedestrian localization prediction boxes × 2.

[0122] S202. A pedestrian feature extraction subnetwork is added between the multi-scale network and the non-maximum suppression module of the improved SSD512 network. This subnetwork runs in parallel with the pedestrian localization subnetwork and the pedestrian recognition subnetwork. The structure diagram of the pedestrian feature extraction subnetwork is shown below. Fig. 6 As shown, it includes four scales and one connection layer. Each scale contains a series of 3×3 convolutions (with a stride of 1 and the number of channels being the product of the number of default boxes at each pixel in that scale and the specified feature dimension), a batch normalization layer, a ReLU activation function, and a dimension transformation (converting a fourth-order tensor with dimensions of height × width × number of default boxes at each pixel in that scale × specified feature dimension into a second-order tensor with dimensions of height × width × number of default boxes × specified feature dimension). The dimension transformations of the four scales are all connected to the connection layer (concatenating the dimension transformation features of the four scales along dimension 1).

[0123] S203, such as Fig. 7 As shown, the semi-supervised dictionary learning module includes structured dictionaries from two cameras. The structured dictionaries include... c k Individual dictionary k =1,2, c k Indicates camera k The number of all pedestrian categories, this c k Sub-dictionaries can be further divided into shared sub-dictionaries and exclusive sub-dictionaries. The shared sub-dictionaries are the same for both cameras, while their exclusive sub-dictionaries are different for each. (See diagram) Indicates use Refactoring x k sparse coding, Indicates camera k The c k Individual dictionary.

[0124] S3. Train end-to-end pedestrian detection and re-identification network model 1 using training set 1 and validation set 1 to obtain target model 1; train end-to-end pedestrian detection and re-identification network model 2 using training set 2 and validation set 2 to obtain target model 2; the specific content is as follows:

[0125] S301. The end-to-end pedestrian detection and re-identification network model 1 is referred to as Net1, and the end-to-end pedestrian detection and re-identification network model 2 is referred to as Net2.

[0126] For the convenience of training, each end-to-end pedestrian detection and re-identification network model is split into two parts for training, i.e., a detection part and a re-identification part, wherein the detection part includes a backbone network, a multi-scale network, a pedestrian positioning sub-network and a pedestrian recognition sub-network, and the re-identification part includes a pedestrian feature extraction sub-network and a pedestrian re-identification module; the detection part and the re-identification part in Net1 are simply referred to as DetNet1 and ReIdNet1 respectively, and the detection part and the re-identification part in Net2 are simply referred to as DetNet2 and ReIdNet2 respectively.

[0127] DetNet1 and DetNet2 are trained, and the specific content is as follows:

[0128] S3011, randomly initialize DetNet1, and set the training initial parameters as follows: the initial learning rate is 0.001, the optimization is stochastic gradient descent (SGD), the learning rate decay is poly strategy, the weight decay is 0.0005, the momentum is 0.9, the batch size is 16, and the training epoch is 100.

[0129] S3012, train DetNet1 using training set 1 and training set 2, calculate the loss function value of DetNet1 using validation set 1 and validation set 2, and calculate the absolute value of the difference between the loss function value and the loss function value of the last training; if the absolute value is less than a threshold value, the training is ended, and the trained DetNet1 is obtained; otherwise, the training is continued.

[0130] In this embodiment, the threshold value is set to 0.01.

[0131] During the training process, the weighted sum of the pedestrian positioning loss and the pedestrian recognition loss is used as the loss function of DetNet1, and the expression of the DetNet1 loss function is as follows:

[0132] .

[0133] wherein, represents the loss function of DetNet1, N represents the total number of positioning prediction boxes matched with the real boxes, represents the pedestrian positioning loss function, represents the pedestrian recognition loss function.

[0134] The expression of the pedestrian positioning loss function is as follows:

[0135] .

[0136] wherein, Pos denotes a set of all localization prediction boxes matching the ground truth boxes; cx denotes the horizontal coordinate of the center point of the localization prediction box, the ground truth box; cy denotes the vertical coordinate of the center point of the localization prediction box, the ground truth box; w denotes the width of the localization prediction box, the ground truth box; h denotes the height of the localization prediction box, the ground truth box; denotes an indicator function; denotes a smooth L1 loss function; denotes the value of the i th localization prediction box at cx ; denotes the value of the i th localization prediction box at cy ; denotes the value of the i th localization prediction box at w ; denotes the value of the i th localization prediction box at h ; denotes the offset of the j th ground truth box relative to the i th localization prediction box at cx , , denotes the value of the i th default box at cx , denotes the value of the j th ground truth box at cx , denotes the value of the i th default box at w ; denotes the offset of the j th ground truth box relative to the i th localization prediction box at cy , , denotes the value of the j th ground truth box at cy , denotes the value of the i th default box at cy , denotes the value of the i th default box at h ; denotes the offset of the j th ground truth box relative to the i th localization prediction box at w , , represents the value of the first real frame in j ; w represents the offset of the first real frame relative to the first positioning prediction frame in j , i h , represents the value of the first real frame in j . h

[0137] and:

[0138] ;

[0139] ;

[0140] ;

[0141] .

[0142] The expression of the pedestrian recognition loss function is:

[0143] .

[0144] wherein, Neg represents the set of all positioning prediction frames that are not matched with real frames; represents the confidence of the first positioning prediction frame in Pos , i p , represents the confidence of the first positioning prediction frame in Pos , i p represents the confidence of the first positioning prediction frame in Neg , r represents the logarithmic function.

[0145] According to the loss function, the gradient is calculated, and the DetNet1 network weight and bias are updated by using the stochastic gradient descent optimizer; during the training process, the learning rate is decayed by using the poly strategy, and the decay formula is as follows:

[0146] ;

[0147] wherein, lr represents the learning rate decay, lr init represents the initial learning rate, iter ​​​​​​​​​Indicates the current iteration number. max _ iter This indicates the maximum number of iterations.

[0148] S3013. Pass the network weights and biases from the trained DetNet1 to DetNet2 to initialize the network weights and biases of DetNet2.

[0149] S302. Fix the network weights and biases of DetNet1 and DetNet2, and train ReIdNet1 and ReIdNet2 using training set 1, validation set 1, and training set 2, validation set 2 respectively. The specific content is as follows:

[0150] S3021. Randomly initialize ReIdNet1 and ReIdNet2, and set the initial training parameters as follows: initial learning rate of 0.001, optimizer of stochastic gradient descent (SGD), learning rate decay of poly policy, weight decay of 0.0005, momentum of 0.9, batch size of 16, and training epochs of 100.

[0151] S3022, Using the structured dictionary of training set 1 and camera 1 D 1. Calculate the sparse coding corresponding to pedestrian features.

[0152] S3023. For the labeled samples in training set 1, calculate the gradient using the labeled loss function and update the data using a stochastic gradient descent optimizer. D 1; For the unlabeled samples in training set 1, the gradient is calculated using the unlabeled loss function, and the stochastic gradient descent optimizer is used to update the data. D 1.

[0153] Labeled loss function The expression is:

[0154]

[0155] in, Indicates camera k The Middle s Other pedestrian characteristics Indicates use D k Refactoring sparse coding, D k Indicates camera k A structured dictionary Indicates use Refactoring sparse coding, Indicates camera k The s Each person's sub-dictionary is a different type of dictionary. Indicates camera k The t Each person's sub-dictionary is a different type of dictionary. Indicates use Refactoring sparse coding, A k Indicates use D k Refactoring X k sparse coding, X k Indicates camera k Pedestrian characteristics of all pedestrians in the data; Denotes the F-norm; , All represent weighting coefficients. .

[0156] Label-free loss function The expression is:

[0157] ;

[0158] in, Indicates camera k Pedestrian features in unlabeled images Indicates use D k Refactoring Sparse coding.

[0159] S3024, will D The shared sub-dictionary in camera 1 is passed to the structured dictionary in camera 2. D 2, and replace D The corresponding shared sub-dictionary in 2.

[0160] S3025, using training set 2 and D 2. Calculate the sparse coding corresponding to pedestrian features.

[0161] S3026. Calculate the gradient of the labeled images in training set 2 using the labeled loss function, and update the image using a stochastic gradient descent optimizer. D 2. Calculate the gradient of the unlabeled samples in training set 2 using the unlabeled loss function, and update the data using a stochastic gradient descent optimizer. D 2.

[0162] S3027, willD 2 is passed to D 1 and replaces D the shared sub-dictionary in 1.

[0163] S3028, steps S3022-S3027 are circularly executed until the loss function values of the semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on the training set and the validation set are converged, and then the training is ended.

[0164] The expression of the semi-supervised dictionary learning loss function is as follows:

[0165] ;

[0166] wherein, represents a weight coefficient, .

[0167] S303, the training set 1 is used to train Net1, the loss function value of Net1 is calculated using the validation set 1, and the absolute value of the difference between the loss function value and the loss function value of the last round of training is calculated; if the absolute value is less than a threshold value, the training is completed, and the trained Net1, i.e. the target model 1, is obtained; otherwise, the training is continued;

[0168] The training set 2 is used to train Net2, the loss function value of Net2 is calculated using the validation set 2, and the absolute value of the difference between the loss function value and the loss function value of the last round of training is calculated; if the absolute value is less than a threshold value, the training is completed, and the trained Net2, i.e. the target model 2, is obtained; otherwise, the training is continued;

[0169] The expression of the loss function of the end-to-end pedestrian detection and re-identification network model is as follows:

[0170] .

[0171] S4, the test set 1 is input into the target model 2 to obtain the pedestrian detection and re-identification results of the test set 1; the specific content is as follows:

[0172] Step 1, input the to-be-identified pedestrian image sample in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, input these features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction sub-network respectively to obtain the first scale trans-dimensional feature, the second scale trans-dimensional feature, the third scale trans-dimensional feature and the fourth scale trans-dimensional feature, and connect the trans-dimensional features of the four scales in series along the dimension 1 to obtain the pedestrian features of the positioning prediction frame.

[0173] Step 2, the position information of the positioning prediction frame output by the pedestrian positioning sub-network, the pedestrian confidence of the positioning prediction frame output by the pedestrian recognition sub-network and the pedestrian feature of the positioning prediction frame are jointly input into a non-maximum suppression module for non-maximum suppression operation to obtain the position information, pedestrian confidence and pedestrian feature of the screened positioning prediction frame.

[0174] The non-maximum suppression operation includes the following contents:

[0175] (1) Put all the positioning prediction frames into a candidate set.

[0176] (2) According to the pedestrian confidence of the positioning prediction frame, sort the positioning prediction frames in the candidate set, move the positioning prediction frame with the highest pedestrian confidence out of the candidate set and into the selected set; delete the positioning prediction frames in the candidate set that overlap with the positioning prediction frame by more than a threshold.

[0177] (3) Repeat step (2) until the candidate set is empty; since the position information, pedestrian confidence and pedestrian feature of the positioning prediction frame are one-to-one, the position information, pedestrian confidence and pedestrian feature of the screened positioning prediction frame are obtained.

[0178] Step 3, input the screened pedestrian feature into a semi-supervised dictionary learning module to obtain sparse coding, and perform pedestrian recognition according to the distance between the sparse coding.

[0179] The calculation formula of the sparse coding is:

[0180] .

[0181] Wherein, x k represents the feature of a pedestrian in a pedestrian image of a camera k , represents the sparse coding of reconstructing x k , .

[0182] Step 4, calculate the Euclidean distance between the sparse coding and the representative coding of each class of samples in the training set 2, and sort the distance values from small to large. Wherein, the representative coding represents the mean value of the sparse coding of the labeled samples in each class in the training set.

[0183] Step 5, according to the result of step 4, if the minimum distance is greater than a threshold , mark the positioning prediction frame of the pedestrian in the to-be-recognized pedestrian image sample as unknown class; otherwise, count the rank k recognition result of the pedestrian positioning prediction frame in the to-be-recognized pedestrian image sample.

[0184] Step 6: According to the result of step 5, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all to-be-recognized pedestrian image samples in test set 1 are summarized to obtain the pedestrian detection and re-identification results of test set 1.

[0185] Test set 2 is input into target model 1 to obtain the pedestrian detection and re-identification results of test set 2, and the specific content is as follows:

[0186] Step 1: The to-be-recognized pedestrian image samples in test set 2 are input into target model 1 to obtain the position information of the corresponding positioning prediction box, the pedestrian confidence and the sparse coding.

[0187] Step 2: The Euclidean distance between the sparse coding and the representative coding of each sample in training set 1 is calculated, and the distance values are sorted from small to large.

[0188] Step 3: According to the result of step 2, if the minimum distance is greater than the threshold value , the positioning prediction box of the pedestrian in the to-be-recognized pedestrian image sample is marked as an unknown class; otherwise, the rank k recognition result of the positioning prediction box of the pedestrian in the to-be-recognized pedestrian image sample is counted.

[0189] Step 4: According to the result of step 3, the position information and rank k recognition results of all pedestrian positioning prediction boxes of all to-be-recognized pedestrian image samples in test set 2 are summarized to obtain the pedestrian detection and re-identification results of test set 2.

[0190] Wherein, the threshold value is an adjustable parameter and is set according to the actual task.

[0191] Fig. 8 The Top-1 search results of the method proposed in the application and the current optimal PSTR method are shown, wherein the image in the Query column is a to-be-recognized pedestrian image from camera 1, the images in the PSTR column and the method column proposed in the application are pedestrian images with the highest matching degree with the image in the Query column in camera 2, the red box in the image represents a detected pedestrian target, and the white box is an enlarged view of the red box. In the search results of the PTSR for different Query column images, the first three rows search for wrong pedestrians, and the last two rows search for correct pedestrians, while the method proposed in the application searches for correct pedestrians for different Query column images, so the method proposed in the application has higher accuracy in pedestrian detection and re-identification.

[0192] The embodiment of the present application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor. It should be noted that the processor executes the computer program to correspond to the specific steps of the method provided by the embodiment of the present application, has the function modules and beneficial effects corresponding to the execution method. The technical details not described in detail in the embodiment can be referred to the method provided by the embodiment of the present application.

[0193] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. It should be noted that the computer program is run by the processor to correspond to the specific steps of the method provided by the embodiment of the present application, has the function modules and beneficial effects corresponding to the execution method. The technical details not described in detail in the embodiment can be referred to the method provided by the embodiment of the present application.

[0194] The above is only the preferred embodiment of the present application, it should be pointed out that, for the ordinary skilled in the art, without departing from the technical principles of the present application, can make a number of improvements and deformation, these improvements and deformation also should be considered as the protection scope of the present application.

Claims

1. A pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning, characterized in that, The application relates to a method for training an end-to-end pedestrian detection and re-identification network model. The method comprises the following steps: S1, acquiring pedestrian image dataset 1 and pedestrian image dataset 2 through two cameras respectively, and processing the pedestrian image dataset 1 and the pedestrian image dataset 2; dividing the processed pedestrian image dataset 1 to obtain a training set 1, a verification set 1 and a test set 1; dividing the processed pedestrian image dataset 2 to obtain a training set 2, a verification set 2 and a test set 2; S2, constructing an end-to-end pedestrian detection and re-identification network model 1 based on the camera 1, and constructing an end-to-end pedestrian detection and re-identification network model 2 based on the camera 2, wherein the model 1 and the model 2 both comprise an improved SSD512 network, a pedestrian feature extraction subnetwork and a semi-supervised dictionary learning module; specifically, S201, improving the SSD512 network to obtain an improved SSD512 network, and the specific content is as follows: replacing the backbone network with ResNet50, wherein the ResNet50 comprises an input layer, a convolution module 1, a convolution module 2, a convolution module 3, a convolution module 4 and a convolution module 5 which are sequentially connected; replacing the process of obtaining the first multi-scale output result with the process of sequentially passing through the convolution module 4 in the ResNet50 and L2 normalization; adding a batch normalization layer between each convolution layer and an activation layer in the convolution module 6 and the convolution module 7; and replacing the last four convolution modules of the multi-scale network with a 3*3 convolution with a step of 2, a batch normalization, a ReLU activation function, a 3*3 convolution with a step of 1, a batch normalization and a ReLU activation function which are sequentially connected; setting the positioning branch as a pedestrian positioning subnetwork; setting the identification branch as a pedestrian identification subnetwork, and changing the category number to 2; and adding a batch normalization layer after each convolution layer in the pedestrian identification subnetwork; S202, adding a pedestrian feature extraction subnetwork between the multi-scale network of the improved SSD512 network and a non-maximum suppression module, wherein the network is parallel to the pedestrian positioning subnetwork and the pedestrian identification subnetwork; S203, the semi-supervised dictionary learning module includes a structured dictionary from two cameras, and the structured dictionary includes c k A dictionary of sub-dictionaries, k = 1, 2, c k c represents the total number of pedestrian categories in camera k. k Each sub-dictionary includes a shared sub-dictionary and a unique sub-dictionary. The shared sub-dictionary is the same for both cameras, while the unique sub-dictionaries are different for each camera. the pedestrian feature extraction subnetwork comprises four scales and a connection layer, each scale comprises a 3*3 convolution with a step of 1, a batch normalization layer, a ReLU activation function and a conversion which are sequentially connected, and the conversion of the four scales is connected with the connection layer; S3, training the end-to-end pedestrian detection and re-identification network model 1 by using the training set 1 and the verification set 1 to obtain a target model 1; and training the end-to-end pedestrian detection and re-identification network model 2 by using the training set 2 and the verification set 2 to obtain a target model 2; 2.The method of claim 1, wherein, S4, inputting the test set 1 into the target model 2 to obtain pedestrian detection and re-identification results of the test set 1; and inputting the test set 2 into the target model 1 to obtain pedestrian detection and re-identification results of the test set 2. In step S1, the division of the pedestrian image dataset comprises the following content: S101, acquiring the pedestrian image dataset 1 and the pedestrian image dataset 2 comprises two modes: (1) downloading the pedestrian image dataset 1 and the pedestrian image dataset 2 corresponding to the two cameras from an open-source dataset website; (2) self-made pedestrian image dataset 1 and pedestrian image dataset 2, the specific content is: using two cameras in different scenes to collect a certain number of pedestrian images, collecting more than two images for each pedestrian, and the repetition rate of pedestrians captured by the two cameras is more than 50%; label the camera to which each image belongs; S102, keep the original aspect ratio of the acquired pedestrian image, and scale the largest side of the width and height to a set of pixels, center the scaled image in a black background image of a set size, and perform random cropping, scaling, horizontal flipping, random image brightness change, random image contrast change, random image chroma change, and random image saturation change processing to obtain processed pedestrian image dataset 1 and processed pedestrian image dataset 2; S103, respectively divide the processed pedestrian image dataset 1 and the processed pedestrian image dataset 2 according to the proportion to obtain the training set 1, the validation set 1, the test set 1 and the training set 2, the validation set 2, the test set 2; wherein the training set 1 and the training set 2 both include labeled and unlabeled pedestrian images, and the validation set 1, the validation set 2, the test set 2 and the test set 2 all include labeled pedestrian images. 3.The method of claim 1, wherein, In step S3, obtaining the target model includes the following content: S301, the end-to-end pedestrian detection and re-identification network model 1 is abbreviated as Net1, and the end-to-end pedestrian detection and re-identification network model 2 is abbreviated as Net2; The end-to-end pedestrian detection and re-identification network model includes a detection part and a re-identification part, wherein the detection part includes a backbone network, a multi-scale network, a pedestrian positioning sub-network and a pedestrian recognition sub-network, and the re-identification part includes a pedestrian feature extraction sub-network and a pedestrian re-identification module; the detection part and the re-identification part in Net1 are abbreviated as DetNet1 and ReIdNet1 respectively, and the detection part and the re-identification part in Net2 are abbreviated as DetNet2 and ReIdNet2 respectively; Training DetNet1 and DetNet2, the specific content is: S3011, randomly initialize DetNet1; S3012, use the training set 1 and the training set 2 to train DetNet1, use the validation set 1 and the validation set 2 to calculate the loss function value of DetNet1, and calculate the absolute value of the difference between the loss function value and the loss function value of the last round of training; if the absolute value is less than a threshold value, the training is ended, and the trained DetNet1 is obtained; otherwise, continue training; The expression of the pedestrian positioning loss function is: Wherein, L det represents the loss function of DetNet1, N represents the total number of positioning prediction boxes matched with the real box, L loc represents the pedestrian positioning loss function, L conf represents the pedestrian recognition loss function; The expression of the pedestrian positioning loss function is: wherein Pos represents a set of all positioning prediction boxes matched with real boxes; cx represents a horizontal coordinate of a center point of a positioning prediction box, a real box; cy represents a vertical coordinate of the center point of the positioning prediction box, the real box; w represents a width of the positioning prediction box, the real box; and h represents a height of the positioning prediction box, the real box; represents an indicator function; smooth L1 (·) represents a smooth L1 loss function; represents a value of the i-th positioning prediction box at cx; represents a value of the i-th positioning prediction box at cy; represents a value of the i-th positioning prediction box at w; represents a value of the i-th positioning prediction box at h; represents an offset of the j-th real box at cx relative to the i-th positioning prediction box, represents a value of the i-th default box at cx, represents a value of the j-th real box at cx, represents a value of the i-th default box at w; represents an offset of the j-th real box at cy relative to the i-th positioning prediction box, represents a value of the j-th real box at cy, represents a value of the i-th default box at cy, represents a value of the i-th default box at h; represents an offset of the j-th real box at w relative to the i-th positioning prediction box, represents a value of the j-th real box at w; represents an offset of the j-th real box at h relative to the i-th positioning prediction box, represents a value of the j-th real box at h; The expression of the pedestrian recognition loss function is: The network weight and bias of DetNet1 are updated by using the stochastic gradient descent optimizer, and the learning rate decay is performed by using the poly strategy, and the decay formula is: wherein Neg represents a set of all positioning prediction boxes that are not matched with the real box; represents a confidence that the i-th positioning prediction box in Pos belongs to the p-th class after normalization, represents a confidence that the i-th positioning prediction box in Pos belongs to the p-th class; represents a confidence that the r-th positioning prediction box in Neg belongs to the background class; log(·) represents a logarithmic function; S3013, pass the network weight and bias of the trained DetNet1 to DetNet2 to initialize the network weight and bias of DetNet2; wherein lr denotes a learning rate decay, lr init denotes an initial learning rate, iter denotes a current iteration number, and max iter denotes a maximum iteration number; ​ S302, fix the network weights and biases of DetNet1 and DetNet2, train ReIdNet1 and ReIdNet2 respectively by using training set 1, validation set 1 and training set 2, validation set 2, and the specific content is as follows: S3021, randomly initialize ReIdNet1 and ReIdNet2; S3022, calculate the sparse coding corresponding to the pedestrian feature by using the structured dictionary D1 of camera 1 and training set 1; S3023, calculate the gradient of the labeled sample in training set 1 by using the labeled loss function, and update D1 by using the stochastic gradient descent optimizer; calculate the gradient of the unlabeled sample in training set 1 by using the unlabeled loss function, and update D1 by using the stochastic gradient descent optimizer; The expression of the marked loss function L labeled is: wherein, denotes the pedestrian feature of the s-th pedestrian class in camera k, denotes the sparse coding of X using D k reconstruction denotes the sparse coding of X using D k denotes the structured dictionary of camera k, denotes the sparse coding of X using D reconstruction denotes the sparse coding of X using D denotes the sub-dictionary of the s-th pedestrian class in camera k, k denotes the number of all pedestrian classes in camera k, denotes the sub-dictionary of the t-th pedestrian class in camera k, denotes the sparse coding of X using D reconstruction denotes the sparse coding of X using D k denotes the sparse coding of X using D k reconstruction k denotes the sparse coding of X using D k denotes the pedestrian feature of all pedestrian classes in camera k; denotes the F-norm; λ1, λ2 both denote weight coefficients; The loss function L without label unlabeled The expression is: wherein, represents the pedestrian features in the camera k without the marker image, represents the sparse coding of the D k reconstruction sparse coding of S3024, pass the shared sub-dictionary in D1 to the structured dictionary D2 of camera 2, and replace the corresponding shared sub-dictionary in D2; S3025, calculate the sparse coding corresponding to the pedestrian feature by using D2 and training set 2; S3026, calculate the gradient of the labeled image in training set 2 by using the labeled loss function, and update D2 by using the stochastic gradient descent optimizer; calculate the gradient of the unlabeled sample in training set 2 by using the unlabeled loss function, and update D2 by using the stochastic gradient descent optimizer; S3027, pass the shared sub-dictionary in D2 to D1, and replace the shared sub-dictionary in D1; S3028, loop steps S3022-S3027 until the loss function value of semi-supervised dictionary learning of ReIdNet1 and ReIdNet2 on the training set and the validation set converges, and then end the training; Semi-supervised dictionary learning loss function L dic The expression is: L dic = L labeled + λ3L unlabeled Wherein, λ3 represents the weight coefficient; S303, train Net1 using training set 1, calculate the loss function value of Net1 using validation set 1, and calculate the absolute value of the difference between the loss function value and the loss function value of the last round of training; if the absolute value is less than a threshold value, the training is completed, and the trained Net1, i.e. target model 1, is obtained; otherwise, continue training; Train Net2 using training set 2, calculate the loss function value of Net2 using validation set 2, and calculate the absolute value of the difference between the loss function value and the loss function value of the last round of training; if the absolute value is less than a threshold value, the training is completed, and the trained Net2, i.e. target model 2, is obtained; otherwise, continue training; Loss function L of an end-to-end pedestrian detection and re-identification network model total The expression is: L total = L det + L dic . 4.The method of claim 1, wherein, In step S4, the pedestrian detection and re-identification results of the test set 1 include the following contents: Step 1, input the to-be-identified pedestrian image sample in the test set 1 into the target model 2, obtain the first scale feature, the second scale feature, the third scale feature and the fourth scale feature through the multi-scale network, input the features into the first scale, the second scale, the third scale and the fourth scale of the pedestrian feature extraction sub-network respectively, obtain the first scale transfinite feature, the second scale transfinite feature, the third scale transfinite feature and the fourth scale transfinite feature, and connect the transfinite features of the four scales along the dimension 1 to obtain the pedestrian features of the positioning prediction box; Step 2, the position information of the positioning prediction frame output by the pedestrian positioning sub-network, the pedestrian confidence of the positioning prediction frame output by the pedestrian recognition sub-network and the pedestrian feature of the positioning prediction frame are input into a non-maximum suppression module for non-maximum suppression operation to obtain the position information, pedestrian confidence and pedestrian feature of the screened positioning prediction frame; Step 3, the screened pedestrian feature is input into a semi-supervised dictionary learning module to obtain sparse coding, and pedestrian recognition is performed according to the distance between the sparse coding; The calculation formula of the sparse coding is as follows: where x k represents a feature of a pedestrian in a pedestrian image of camera k, represents a sub-dictionary of the s-th pedestrian class of camera k, represents a sparse coding of x reconstructing x k , s = 1, 2,..., c k ; Step 4, the Euclidean distance between the sparse coding and the representative coding of each sample in the training set 2 is calculated, and the distance values are sorted from small to large; wherein the representative coding represents the mean value of the sparse coding of each labeled sample in the training set; Step 5, according to the result of step 4, if the minimum distance is greater than the threshold value θ, the positioning prediction frame of the pedestrian in the to-be-recognized pedestrian image sample is marked as an unknown class; otherwise, the rank k recognition result of the pedestrian positioning prediction frame in the to-be-recognized pedestrian image sample is counted; Step 6, according to the result of step 5, the position information and the rank k recognition result of all pedestrian positioning prediction frames of all to-be-recognized pedestrian image samples in the test set 1 are summarized to obtain the pedestrian detection and re-identification result of the test set 1.

5. The method of claim 4, wherein the method further comprises: The non-maximum suppression operation includes the following contents: Step 1, all positioning prediction frames are put into a candidate set; Step 2, the positioning prediction frames in the candidate set are sorted according to the pedestrian confidence of the positioning prediction frames, the positioning prediction frame with the highest pedestrian confidence is removed from the candidate set and put into a selected set, and the positioning prediction frames in the candidate set that overlap with the positioning prediction frame by more than a threshold value are deleted; Step 3, repeat step 2 until the candidate set is empty; since the position information, pedestrian confidence and pedestrian feature of the positioning prediction frame are one-to-one corresponding, the position information, pedestrian confidence and pedestrian feature of the screened positioning prediction frame are obtained.

6. The method of claim 4, wherein the method further comprises: In step S4, the pedestrian detection and re-identification result of the test set 2 includes the following contents: Step 1, the to-be-recognized pedestrian image sample in the test set 2 is input into the target model 1 to obtain the position information, pedestrian confidence and sparse coding of the corresponding positioning prediction frame; Step 2, the Euclidean distance between the sparse coding and the representative coding of each sample in the training set 1 is calculated, and the distance values are sorted from small to large; Step 3, according to the result of step 2, if the minimum distance is greater than the threshold value θ, the positioning prediction frame of the pedestrian in the to-be-recognized pedestrian image sample is marked as an unknown class; otherwise, the rank k recognition result of the pedestrian positioning prediction frame in the to-be-recognized pedestrian image sample is counted; Step 4, according to the result of step 3, the position information and the rank k recognition result of all pedestrian positioning prediction frames of all to-be-recognized pedestrian image samples in the test set 2 are summarized to obtain the pedestrian detection and re-identification result of the test set 2.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the steps of the pedestrian detection and re-identification method based on SSD and semi-supervised dictionary learning in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium being characterized by, The computer program is run by a processor to perform the method for pedestrian detection and re-identification based on SSD and semi-supervised dictionary learning according to any one of claims 1 to 6.

Images