A face tracking method and tracking system for dense crowds

Abstract

A method and system for tracking faces in dense crowds, belonging to the field of face tracking technology. The present invention solves the problems of low accuracy of existing face tracking methods in dense crowd scenes and the risk of leakage of facial image information. The present invention combines an improved loss function to train the constructed target frame detection network, so that the detection results can comprehensively consider features at different levels and depths, improve the accuracy of target frame detection, and the detection results are not affected by missing facial information and image blur. The facial feature extraction network constructed using the present invention can improve the quality of feature extraction to ensure the accuracy of face tracking. Moreover, based on the training method of federated learning, each client uses local data to perform training locally, and sends the locally trained model parameters to the central server for parameter aggregation, thereby avoiding the risk of leakage that may exist during the transmission of facial images. The method of the present invention can be applied to face tracking in dense crowds.

Description

Technical Field

[0001] The present invention belongs to the technical field of face tracking, and in particular relates to a face tracking method and a tracking system for dense crowds. Background Art

[0002] Facial information processing has become one of the most popular applications in computer vision. The first steps in facial information processing are face detection and face tracking. Face detection is the process of determining the locations of all faces in an input image, while face tracking is the process of determining the correspondence between faces across frames in an image sequence, that is, determining the motion trajectory of each face. The development of face detection and tracking technology has brought about a real revolution in video surveillance systems, significantly saving manpower and material resources.

[0003] Face detection and tracking primarily rely on foreground segmentation techniques to segment the face from the image and then track the segmented face area. Traditional face detection algorithms generally rely on manual feature extraction. With technological advancements, deep learning-based face detection algorithms have demonstrated significant growth and are gradually replacing traditional face detection algorithms. Examples include the evolving R-CNN series of detection models, SSP-Net, Fast-RCNN, Faster-RCNN, and Mask-RCNN detection models. In recent years, with the emergence of the YOLO and SSD detection models, detection speed and accuracy have gradually improved, pushing object detection to new heights. However, in crowded scenes, due to mutual occlusion between people and the angle of video shooting, some facial information is inevitably missing. Existing deep learning-based detection methods do not take these factors into consideration when designing. As a result, the applicability of existing deep learning detection methods in these cases is still relatively poor. The accuracy of face detection when directly applied to crowded scenes is still low. In addition, the movement of people is inevitable during video shooting, which will also cause different degrees of blurring in the face area of ​​the image when each frame is obtained from the captured video. Image blur will also affect feature extraction, thereby affecting the accuracy of face detection, which will greatly reduce the accuracy of subsequent face tracking. Therefore, proposing a new face tracking method suitable for crowded scenes is an urgent problem that needs to be solved.

[0004] At the same time, since face detection and tracking technology is applicable to various scenarios requiring video surveillance, in order to ensure the model performance of face detection and tracking technology based on deep learning, each local client needs to obtain sufficient image samples from multiple local databases, and then use the obtained image samples to train the model locally. In this case, there is a risk of leakage of facial image information during the transmission of the obtained image samples. Therefore, avoiding the risk of leakage of facial image information during the acquisition of training data is also a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of low accuracy of existing face tracking methods in crowded scenes and the risk of leakage of facial image information, and to propose a face tracking method and tracking system for dense crowds.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] According to one aspect of the present invention, a method for tracking faces in a dense crowd is provided, the method specifically comprising the following steps:

[0008] Step 1: The video captured by the camera is frame-processed to obtain each frame image, and then the frame images are numbered in ascending order according to the order in which the frame images were captured;

[0009] Step 2: Use the built target detection network to detect the face area in the first frame image;

[0010] Step 3: The person corresponding to each detected face area is treated as a tracking object, and each tracking object is assigned a different identity ID;

[0011] Then, the facial feature extraction network is used to extract the features of each facial region image in the first frame image;

[0012] Step 4: Initialize the number of image frames k=2;

[0013] Step 5: Use the built target detection network to detect the face area in the k-th frame image, and then use the face feature extraction network to extract the features of each face area image in the k-th frame image;

[0014] Step 6: Perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image;

[0015] Step 7: Set k=k+1, and then return to step 5.

[0016] According to another aspect of the present invention, a face tracking system for dense crowds includes an image acquisition module, a target detection module, a face feature extraction module, and a face tracking module; wherein:

[0017] The image acquisition module is used to shoot video with a camera, and to perform frame processing on the shot video to obtain each frame image, and then to number each frame image in ascending order according to the order in which each frame image was shot;

[0018] The target detection module uses the target detection network to detect the face area in each frame image, and identifies the person corresponding to each face area in the first frame image as a tracking object, and assigns a different identity ID to each tracking object.

[0019] The face feature extraction module uses the face feature extraction network to extract features from each face region image in each frame image;

[0020] The face tracking module is used to perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image; specifically:

[0021] Step 1: Process each face region image in the k-th frame image using two-dimensional multi-level wavelet decomposition to obtain a clarity score for each face region image in the k-th frame image;

[0022] Number the facial region images in descending order of clarity scores;

[0023] Step 2: Set the face region image count k′ to 1, and initialize the image set K to include all face region images in the k-1th frame image;

[0024] Step 3: Calculate the feature similarity between the k′th face region image in the kth frame image and each face region image in the image set K, and record the face region image corresponding to the maximum feature similarity in the image set K as Then determine whether the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1;

[0025] If the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1, proceed to step 4;

[0026] If the calculated maximum feature similarity is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID;

[0027] Step 4: Determine the face area image Is the clarity score of less than the score threshold Q2?

[0028] (1) If the face area image If the clarity score is less than the score threshold Q2, then the image with the face area is tracked forward. Match the face region image with the smallest feature similarity, and calculate the feature similarity between the tracked face region image and the k′th face region image;

[0029] If the feature similarity between the tracked face region image and the k′th face region image is greater than or equal to the similarity threshold Q1, the k′th face region image is successfully matched, and the face region image is Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5;

[0030] If the feature similarity between the tracked face region image and the k′th face region image is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; and step 5 is executed;

[0031] (2) If the face area image If the clarity score of the face is greater than or equal to the score threshold Q2, the face region image of the k′th person is successfully matched. Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5;

[0032] Step 5: Set k′=k′+1 and return to step 3.

[0033] The beneficial effects of the present invention are:

[0034] By using the target frame detection network built by the method of the present invention and training the target frame detection network in combination with the improved loss function, the detection results can comprehensively consider the features of different levels and depths, improve the accuracy of target frame detection, and the detection results are not affected by the lack of facial information and image blur. At the same time, the facial feature extraction network built by the present invention can further improve the quality of feature extraction to ensure the accuracy of face tracking. Moreover, based on the training method of federated learning, each client can use local data to train locally, and then send the locally trained model parameters to the central server for parameter aggregation. In this way, there is no need for each client to upload local data, avoiding the risk of leakage during the transmission of facial images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1The present invention is a flow chart of a method for tracking faces in a dense crowd. DETAILED DESCRIPTION

[0036] Specific implementation method 1: Combination Figure 1 This embodiment describes a method for tracking faces in a dense crowd, and the method specifically includes the following steps:

[0037] Step 1: The video captured by the camera is framed to obtain individual frames, and then the frames are numbered from small to large according to the order in which they were captured, and the first frame is recorded as frame 1;

[0038] Step 2: Use the built target detection network to detect the face area in the first frame image;

[0039] Step 3: The person corresponding to each detected face area is treated as a tracking object, and each tracking object is assigned a different identity ID;

[0040] Then, the facial feature extraction network is used to extract the features of each facial region image in the first frame image;

[0041] Step 4: Initialize the number of image frames k=2;

[0042] Step 5: Use the built target detection network to detect the face area in the k-th frame image, and then use the face feature extraction network to extract the features of each face area image in the k-th frame image;

[0043] Step 6: Perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image;

[0044] Step 7: Set k=k+1, and then return to step 5.

[0045] The method of the present invention can be used to process the monitoring area covered by each camera separately. Taking the current camera as an example, after tracking a person in each frame image captured by the current camera, if the person disappears after moving to the image boundary covered by the current camera, the person who disappeared in the area covered by the current camera can be matched in the adjacent camera through the existing facial feature matching method. Through the tracking process within the coverage range of the adjacent cameras, the tracking trajectory can be connected.

[0046] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the target detection network includes a first module, a second module, a third module, a fourth module, an average pooling layer, a fully connected layer and a SoftMax function layer;

[0047] The first module includes the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer;

[0048] The working process of the first module is:

[0049] The image input to the target detection network is used as the input of the first convolutional layer and the third convolutional layer respectively;

[0050] The output of the first convolutional layer passes through the second convolutional layer, and the output of the third convolutional layer passes through the fourth convolutional layer;

[0051] After fusing the output of the second convolutional layer and the output of the fourth convolutional layer, the fusion result is passed through the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer in sequence;

[0052] The second module includes the eighth convolution layer, the ninth convolution layer, the tenth convolution layer, the eleventh convolution layer, the twelfth convolution layer, the thirteenth convolution layer, the fourteenth convolution layer, the fifteenth convolution layer and the second maximum pooling layer;

[0053] The working process of the second module is:

[0054] The output of the first maximum pooling layer is used as the input of the eighth and ninth convolutional layers respectively;

[0055] Then the output of the eighth convolutional layer passes through the tenth and twelfth convolutional layers in sequence; the output of the ninth convolutional layer passes through the eleventh and thirteenth convolutional layers in sequence;

[0056] After fusing the output of the twelfth convolutional layer and the output of the thirteenth convolutional layer, the fusion result is passed through the fourteenth convolutional layer, the fifteenth convolutional layer, and the second maximum pooling layer in sequence;

[0057] The third module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the twentieth convolutional layer, the twenty-first convolutional layer, the twenty-second convolutional layer, the twenty-third convolutional layer, the twenty-fourth convolutional layer and the third maximum pooling layer;

[0058] The working process of the third module is:

[0059] The output of the second maximum pooling layer is used as the input of the sixteenth and seventeenth convolutional layers respectively;

[0060] The output of the sixteenth convolutional layer then passes through the eighteenth, twentieth, and twenty-second convolutional layers in sequence; the output of the seventeenth convolutional layer passes through the nineteenth, twenty-first, and twenty-third convolutional layers in sequence;

[0061] After fusing the output of the 22nd convolutional layer and the output of the 23rd convolutional layer, the fusion result is passed through the 24th convolutional layer and the third maximum pooling layer in sequence;

[0062] The fourth module includes a twenty-fifth convolutional layer, a twenty-sixth convolutional layer, a twenty-seventh convolutional layer, a twenty-eighth convolutional layer, a twenty-ninth convolutional layer, a thirtieth convolutional layer, a thirty-first convolutional layer, a thirty-second convolutional layer, a thirty-third convolutional layer, and a thirty-fourth convolutional layer;

[0063] The working process of the fourth module is:

[0064] The output of the third maximum pooling layer is used as the input of the 25th and 26th convolutional layers respectively;

[0065] The output of the 25th convolutional layer then passes through the 27th, 29th, 31st, and 33rd convolutional layers in sequence; the output of the 26th convolutional layer passes through the 28th, 30th, 32nd, and 34th convolutional layers in sequence;

[0066] Then, the outputs of the 33rd convolutional layer and the 34th convolutional layer are fused to obtain the fusion result a;

[0067] Finally, the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer are fused to obtain the fusion result A; the fusion result A is then passed through the average pooling layer, the fully connected layer, and the SoftMax function layer, and the SoftMax function layer is used to output the target box position of the face area in the image.

[0068] Other steps and parameters are the same as those in the first embodiment.

[0069] Since the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer respectively represent features at different levels, this embodiment fuses the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer, so that the detection result of the target frame comprehensively considers features at different levels and depths, solves the problem of low target frame detection accuracy due to occlusion (i.e., missing facial information) and image blur, improves the accuracy of target frame detection, and provides a guarantee for subsequent personnel detection and tracking.

[0070] Specific embodiment three: This embodiment differs from specific embodiment one or two in that the loss function used in the target detection network training is:

[0071]

[0072] Among them, IoU is the intersection-over-union ratio of the predicted box output by the target detection network and the real box, |·| represents the absolute value, x1 is the center point of the predicted box output by the target detection network, x2 is the center point of the real box, ρ(x1,x2) is the Euclidean distance between x1 and x2, l is the diagonal length of the minimum circumscribed box between the predicted box and the real box, μ is the intermediate variable, and β is the scale variable;

[0073]

[0074] Among them, w gt is the width of the prediction box, h gt is the height of the predicted box, w is the width of the real box, and h is the height of the real box;

[0075]

[0076] Other steps and parameters are the same as those in the first or second embodiment.

[0077] The loss function of this embodiment can make the predicted box and the real box as close as possible, ensuring the training effect of the model.

[0078] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that the facial feature extraction network includes a first convolution module, a first CBAM module, a second convolution module, a second CBAM module, a third convolution module, a third CBAM module, and a fourth convolution module; and the working process of the facial feature extraction network is:

[0079] The input of the facial feature extraction network is used as the input of the first convolutional module;

[0080] The output of the first convolution module is used as the input of the first CBAM module, and the Hadamard product of the output of the first CBAM module and the output of the first convolution module is calculated, and the calculated Hadamard product is recorded as T1;

[0081] Use T1 as the input of the second convolution module, and then use the output of the second convolution module as the input of the second CBAM module;

[0082] and calculating the Hadamard product of the output of the second CBAM module and the output of the second convolution module, and recording the calculated Hadamard product as T2;

[0083] Use T2 as the input of the third convolution module, and then use the output of the third convolution module as the input of the third CBAM module;

[0084] and calculating the Hadamard product of the output of the third CBAM module and the output of the third convolution module, and recording the calculated Hadamard product as T3;

[0085] Use T3 as the input of the fourth convolution module;

[0086] The output of the second convolutional module passes through the convolutional layer C1, the global average pooling layer P1 and the fully connected layer F1 in sequence to obtain the output of the fully connected layer F1;

[0087] The output of the third convolutional module passes through the convolutional layer C2, the global average pooling layer P2, and the fully connected layer F2 in sequence to obtain the output of the fully connected layer F2;

[0088] The output of the fourth convolutional module passes through the convolutional layer C3, the global average pooling layer P3, and the fully connected layer F3 in sequence to obtain the output of the fully connected layer F3;

[0089] The output of the fully connected layer F1, the output of the fully connected layer F2, and the output of the fully connected layer F3 are spliced ​​to obtain the extracted facial features.

[0090] The other steps and parameters are the same as those in the first to third embodiments.

[0091] The network of this embodiment performs feature extraction on each face area image separately, and accurate features can be extracted. The quality of feature extraction will not be reduced due to missing face information and blurred face area images.

[0092] Specific embodiment five: This embodiment is different from any of specific embodiments one to four in that the first convolution module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer, a tenth convolution layer, an eleventh convolution layer, a twelfth convolution layer, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer, a sixteenth convolution layer, a seventeenth convolution layer, an eighteenth convolution layer, a nineteenth convolution layer, a twentieth convolution layer, a twenty-first convolution layer, a twenty-second convolution layer, a twenty-third convolution layer, a twenty-fourth convolution layer, a twenty-fifth convolution layer, a twenty-sixth convolution layer, a twenty-seventh convolution layer, a twenty-eighth convolution layer, a twenty-ninth convolution layer, a thirtieth convolution layer, a thirty-first convolution layer, a thirty-second convolution layer and a thirty-third convolution layer; wherein:

[0093] In the first convolution module, the input image is first used as the input of the first convolution layer;

[0094] The output of the first convolutional layer passes through the second and third convolutional layers in sequence, and then the output of the third convolutional layer is fused with the output of the first convolutional layer to obtain the fusion result V1;

[0095] The fusion result V1 passes through the fourth convolutional layer and the fifth convolutional layer in sequence, and then the output of the fifth convolutional layer is fused with the fusion result V1 to obtain the fusion result V2;

[0096] The fusion result V2 passes through the sixth and seventh convolutional layers in sequence, and then the output of the seventh convolutional layer is fused with the fusion result V2 to obtain the fusion result V3;

[0097] The fusion result V3 is passed through the eighth and ninth convolutional layers in sequence, and the output of the ninth convolutional layer is fused with the fusion result V3 to obtain the fusion result V4.

[0098] The fusion result V4 is passed through the tenth convolutional layer and the eleventh convolutional layer in sequence, and the output of the eleventh convolutional layer is fused with the fusion result V4 to obtain the fusion result V5;

[0099] The fusion result V5 passes through the twelfth convolutional layer and the thirteenth convolutional layer in sequence, and then the output of the thirteenth convolutional layer is fused with the fusion result V5 to obtain the fusion result V6;

[0100] The fusion result V6 passes through the fourteenth convolutional layer and the fifteenth convolutional layer in sequence, and then the output of the fifteenth convolutional layer is fused with the fusion result V6 to obtain the fusion result V7;

[0101] The fusion result V7 is passed through the sixteenth convolutional layer and the seventeenth convolutional layer in sequence, and then the output of the seventeenth convolutional layer is fused with the fusion result V7 to obtain the fusion result V8;

[0102] The fusion result V8 passes through the 18th convolutional layer and the 19th convolutional layer in sequence, and then the output of the 19th convolutional layer is fused with the fusion result V8 to obtain the fusion result V9;

[0103] The fusion result V9 passes through the 20th convolutional layer and the 21st convolutional layer in sequence, and then the output of the 21st convolutional layer is fused with the fusion result V9 to obtain the fusion result V10;

[0104] The fusion result V10 passes through the 22nd convolutional layer and the 23rd convolutional layer in sequence, and then the output of the 23rd convolutional layer is fused with the fusion result V10 to obtain the fusion result V11;

[0105] The fusion result V11 passes through the 24th convolutional layer and the 25th convolutional layer in sequence, and then the output of the 25th convolutional layer is fused with the fusion result V11 to obtain the fusion result V12;

[0106] The fusion result V12 passes through the 26th convolutional layer and the 27th convolutional layer in sequence, and then the output of the 27th convolutional layer is fused with the fusion result V12 to obtain the fusion result V13;

[0107] The fusion result V13 passes through the 28th convolutional layer and the 29th convolutional layer in sequence, and then the output of the 29th convolutional layer is fused with the fusion result V13 to obtain the fusion result V14;

[0108] The fusion result V14 passes through the 30th convolutional layer and the 31st convolutional layer in sequence, and then the output of the 31st convolutional layer is fused with the fusion result V14 to obtain the fusion result V15;

[0109] The fusion result V15 passes through the 32nd convolutional layer and the 33rd convolutional layer in sequence, and then the output of the 33rd convolutional layer is fused with the fusion result V15 to obtain the fusion result V16;

[0110] The fusion result V16 is used as the output of the first convolution module.

[0111] The other steps and parameters are the same as those in the first to fourth embodiments.

[0112] The structure and working process of the second, third, and fourth convolutional modules are the same as those of the first convolutional module. It should be noted that the numbering of each layer in the target bounding box detection network is independent of the numbering of each layer in the first convolutional module. For example, the first convolutional layer only represents the first convolutional layer in the network, not the same convolutional layer.

[0113] Specific embodiment 6: This embodiment is different from specific embodiments 1 to 5 in that the working process of the first CBAM module is:

[0114] The input of the first CBAM module is denoted as x. In the first CBAM module, the input x first passes through the channel attention layer, and then the output of the channel attention layer is concatenated with the input x to obtain the first concatenation result;

[0115] The first splicing result is used as the input of the spatial attention layer, and the output of the spatial attention layer is spliced ​​with the first splicing result for the second time, and the second splicing result is used as the output of the first CBAM module.

[0116] The other steps and parameters are the same as those in the first to fifth embodiments.

[0117] The structures and working processes of the second CBAM module and the third CBAM module are the same as those of the first CBAM module.

[0118] Specific embodiment 7: This embodiment differs from any one of specific embodiments 1 to 6 in that the specific process of step 6 is as follows:

[0119] Step 61: Process each face region image in the k-th frame image using two-dimensional multi-level wavelet decomposition to obtain a clarity score for each face region image in the k-th frame image;

[0120] It should be noted that the clarity score calculation methods that can be used in the present invention include but are not limited to two-dimensional multi-level wavelet decomposition;

[0121] Number the facial region images in descending order of clarity scores; the image with the highest clarity score is numbered 1;

[0122] It should be noted that the number here is independent of the identity ID and is only used for feature matching in the next step;

[0123] Step 62: Set the face region image count k′ to 1, and initialize the image set K to include all face region images in the k-1th frame image;

[0124] Step 6.3: Calculate the feature similarity (similarity is the Euclidean distance between features) between the k′th face region image in the kth frame image and each face region image in the image set K. The face region image corresponding to the maximum feature similarity calculated in the image set K is recorded as Then determine whether the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1;

[0125] If the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1, then continue to step 64;

[0126] If the calculated maximum feature similarity is less than the similarity threshold Q1, the k′th face region image matching fails, indicating that there is no object in the set K that matches the k′th face region image. The k′th face region image is taken as a new tracking object and assigned a new identity ID.

[0127] Step 64: Determine the face area image Is the clarity score of less than the score threshold Q2?

[0128] (1) If the face area image If the clarity score is less than the score threshold Q2, then the image with the face area is tracked forward. The face area image with the smallest feature similarity (i.e., the face area image from the 1st frame to the k-2th frame) Among all matching face images, find the image with the face area The face area image with the smallest feature similarity. It should be noted that if the face area image If the corresponding person appears for the first time in the k-1 frame image, the face area image will be directly as the tracked face region image), calculating the feature similarity between the tracked face region image and the k′th face region image;

[0129] If the feature similarity between the tracked face region image and the k′th face region image is greater than or equal to the similarity threshold Q1, the k′th face region image is successfully matched, and the face region image is Remove it from the set K to avoid the subsequent image from matching with the image that has been matched again, which affects the accuracy of tracking. At the same time, it can also control the amount of calculation. The identity ID of the k′th face area image is given as image and execute step 65;

[0130] If the feature similarity between the tracked face region image and the k′th face region image is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; and step 65 is executed;

[0131] Since the face area image The clarity score is less than the score threshold Q2, so the present invention tracks the image in the first frame to the k-2 frame and finds the image with the face area. The image with the greatest feature similarity is used to perform matching verification with the k′th face region image to avoid mismatching when face information is missing or the face is blurred, thus improving tracking accuracy.

[0132] (2) If the face area image If the clarity score of the face is greater than or equal to the score threshold Q2, the face region image of the k′th person is successfully matched. Remove from the set K and assign the identity ID of the k′th face region image to image and execute step 65;

[0133] Step 65: Let k′=k′+1, return to step 63, and stop iterating until all face area images in the k-th frame image are matched or the set K is an empty set. When all face area images in the k-th frame image are matched and the set K is not an empty set, it means that the people corresponding to the remaining images in the set K have disappeared from the current image, and the adjacent camera is started for tracking. When the set K is an empty set and all face area images in the k-th frame image are not matched, it means that the remaining unmatched people in the k-th frame image are newly appeared people. The newly appeared people may be people who have moved from the coverage area of ​​the adjacent camera. The existing method is used to match them with the people who disappeared in the adjacent coverage area at the same time to achieve continuous tracking of people.

[0134] The other steps and parameters are the same as those in the first to sixth embodiments.

[0135] The target detection network and facial feature extraction network of the present invention are trained using a federated learning method based on knowledge distillation. Based on the federated learning training method, each client can use local data for local training, and then send the locally trained model parameters to the central server for parameter aggregation. The aggregated network parameters are then deployed on each client. In this way, there is no need for each client to upload local data, avoiding the risk of leakage that may exist during the transmission of facial images, and effectively protecting information security. At the same time, based on the knowledge distillation strategy, when it is necessary to update the model parameters based on new image samples, the distillation loss is calculated based on the output results of the original sample before and after the model parameters are updated, and the calculated distillation loss is then added to the loss function of the model training to avoid the model's catastrophic forgetting of previous sample images and ensure tracking accuracy.

[0136] Specific embodiment eight: This embodiment describes a face tracking system for dense crowds, the system comprising an image acquisition module, a target detection module, a face feature extraction module, and a face tracking module; wherein:

[0137] The image acquisition module is used to shoot video with a camera, and to perform frame processing on the shot video to obtain each frame image, and then to number each frame image in ascending order according to the order in which each frame image was shot;

[0138] The target detection module uses the target detection network to detect the face area in each frame image, and identifies the person corresponding to each face area in the first frame image as a tracking object, and assigns a different identity ID to each tracking object.

[0139] The face feature extraction module uses the face feature extraction network to extract features from each face region image in each frame image;

[0140] The face tracking module is used to perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image; specifically:

[0141] Step 1: Process each face region image in the k-th frame image using two-dimensional multi-level wavelet decomposition to obtain a clarity score for each face region image in the k-th frame image;

[0142] Number the facial region images in descending order of clarity scores;

[0143] Step 2: Set the face region image count k′ to 1, and initialize the image set K to include all face region images in the k-1th frame image;

[0144] Step 3: Calculate the feature similarity between the k′th face region image in the kth frame image and each face region image in the image set K, and record the face region image corresponding to the maximum feature similarity in the image set K as Then determine whether the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1;

[0145] If the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1, proceed to step 4;

[0146] If the calculated maximum feature similarity is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID;

[0147] Step 4: Determine the face area image Is the clarity score of less than the score threshold Q2?

[0148] (1) If the face area image If the clarity score is less than the score threshold Q2, then the image with the face area is tracked forward. Match the face region image with the smallest feature similarity, and calculate the feature similarity between the tracked face region image and the k′th face region image;

[0149] If the feature similarity between the tracked face region image and the k′th face region image is greater than or equal to the similarity threshold Q1, the k′th face region image is successfully matched, and the face region image is Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5;

[0150] If the feature similarity between the tracked face region image and the k′th face region image is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; and step 5 is executed;

[0151] (2) If the face area image If the clarity score of the face is greater than or equal to the score threshold Q2, the face region image of the k′th person is successfully matched. Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5;

[0152] Step 5: Set k′=k′+1 and return to step 3.

[0153] Specific embodiment nine: This embodiment differs from specific embodiment eight in that the target detection network includes a first module, a second module, a third module, a fourth module, an average pooling layer, a fully connected layer and a SoftMax function layer;

[0154] The first module includes the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer;

[0155] The working process of the first module is:

[0156] The image input to the target detection network is used as the input of the first convolutional layer and the third convolutional layer respectively;

[0157] The output of the first convolutional layer passes through the second convolutional layer, and the output of the third convolutional layer passes through the fourth convolutional layer;

[0158] After fusing the output of the second convolutional layer and the output of the fourth convolutional layer, the fusion result is passed through the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer in sequence;

[0159] The second module includes the eighth convolution layer, the ninth convolution layer, the tenth convolution layer, the eleventh convolution layer, the twelfth convolution layer, the thirteenth convolution layer, the fourteenth convolution layer, the fifteenth convolution layer and the second maximum pooling layer;

[0160] The working process of the second module is:

[0161] The output of the first maximum pooling layer is used as the input of the eighth and ninth convolutional layers respectively;

[0162] Then the output of the eighth convolutional layer passes through the tenth and twelfth convolutional layers in sequence; the output of the ninth convolutional layer passes through the eleventh and thirteenth convolutional layers in sequence;

[0163] After fusing the output of the twelfth convolutional layer and the output of the thirteenth convolutional layer, the fusion result is passed through the fourteenth convolutional layer, the fifteenth convolutional layer, and the second maximum pooling layer in sequence;

[0164] The third module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the twentieth convolutional layer, the twenty-first convolutional layer, the twenty-second convolutional layer, the twenty-third convolutional layer, the twenty-fourth convolutional layer and the third maximum pooling layer;

[0165] The working process of the third module is:

[0166] The output of the second maximum pooling layer is used as the input of the sixteenth and seventeenth convolutional layers respectively;

[0167] The output of the sixteenth convolutional layer then passes through the eighteenth, twentieth, and twenty-second convolutional layers in sequence; the output of the seventeenth convolutional layer passes through the nineteenth, twenty-first, and twenty-third convolutional layers in sequence;

[0168] After fusing the output of the 22nd convolutional layer and the output of the 23rd convolutional layer, the fusion result is passed through the 24th convolutional layer and the third maximum pooling layer in sequence;

[0169] The fourth module includes a twenty-fifth convolutional layer, a twenty-sixth convolutional layer, a twenty-seventh convolutional layer, a twenty-eighth convolutional layer, a twenty-ninth convolutional layer, a thirtieth convolutional layer, a thirty-first convolutional layer, a thirty-second convolutional layer, a thirty-third convolutional layer, and a thirty-fourth convolutional layer;

[0170] The working process of the fourth module is:

[0171] The output of the third maximum pooling layer is used as the input of the 25th and 26th convolutional layers respectively;

[0172] The output of the 25th convolutional layer then passes through the 27th, 29th, 31st, and 33rd convolutional layers in sequence; the output of the 26th convolutional layer passes through the 28th, 30th, 32nd, and 34th convolutional layers in sequence;

[0173] Then, the outputs of the 33rd convolutional layer and the 34th convolutional layer are fused to obtain the fusion result a;

[0174] Finally, the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer are fused to obtain the fusion result A; the fusion result A is then passed through the average pooling layer, the fully connected layer, and the SoftMax function layer, and the SoftMax function layer is used to output the target box position of the face area in the image.

[0175] Specific embodiment ten: This embodiment differs from specific embodiment eight in that the facial feature extraction network includes a first convolution module, a first CBAM module, a second convolution module, a second CBAM module, a third convolution module, a third CBAM module and a fourth convolution module; and the working process of the facial feature extraction network is:

[0176] The input of the facial feature extraction network is used as the input of the first convolutional module;

[0177] The output of the first convolution module is used as the input of the first CBAM module, and the Hadamard product of the output of the first CBAM module and the output of the first convolution module is calculated, and the calculated Hadamard product is recorded as T1;

[0178] Use T1 as the input of the second convolution module, and then use the output of the second convolution module as the input of the second CBAM module;

[0179] and calculating the Hadamard product of the output of the second CBAM module and the output of the second convolution module, and recording the calculated Hadamard product as T2;

[0180] Use T2 as the input of the third convolution module, and then use the output of the third convolution module as the input of the third CBAM module;

[0181] and calculating the Hadamard product of the output of the third CBAM module and the output of the third convolution module, and recording the calculated Hadamard product as T3;

[0182] Use T3 as the input of the fourth convolution module;

[0183] The output of the second convolutional module passes through the convolutional layer C1, the global average pooling layer P1 and the fully connected layer F1 in sequence to obtain the output of the fully connected layer F1;

[0184] The output of the third convolutional module passes through the convolutional layer C2, the global average pooling layer P2, and the fully connected layer F2 in sequence to obtain the output of the fully connected layer F2;

[0185] The output of the fourth convolutional module passes through the convolutional layer C3, the global average pooling layer P3, and the fully connected layer F3 in sequence to obtain the output of the fully connected layer F3;

[0186] The output of the fully connected layer F1, the output of the fully connected layer F2, and the output of the fully connected layer F3 are concatenated to obtain the extracted facial features;

[0187] The first convolution module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer, a tenth convolution layer, an eleventh convolution layer, a twelfth convolution layer, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer, a sixteenth convolution layer, a seventeenth convolution layer, an eighteenth convolution layer, a nineteenth convolution layer, a twentieth convolution layer, a twenty-first convolution layer, a twenty-second convolution layer, a twenty-third convolution layer, a twenty-fourth convolution layer, a twenty-fifth convolution layer, a twenty-sixth convolution layer, a twenty-seventh convolution layer, a twenty-eighth convolution layer, a twenty-ninth convolution layer, a thirtieth convolution layer, a thirty-first convolution layer, a thirty-second convolution layer and a thirty-third convolution layer; wherein:

[0188] In the first convolution module, the input image is first used as the input of the first convolution layer;

[0189] The output of the first convolutional layer passes through the second and third convolutional layers in sequence, and then the output of the third convolutional layer is fused with the output of the first convolutional layer to obtain the fusion result V1;

[0190] The fusion result V1 passes through the fourth convolutional layer and the fifth convolutional layer in sequence, and then the output of the fifth convolutional layer is fused with the fusion result V1 to obtain the fusion result V2;

[0191] The fusion result V2 passes through the sixth and seventh convolutional layers in sequence, and then the output of the seventh convolutional layer is fused with the fusion result V2 to obtain the fusion result V3;

[0192] The fusion result V3 is passed through the eighth and ninth convolutional layers in sequence, and the output of the ninth convolutional layer is fused with the fusion result V3 to obtain the fusion result V4.

[0193] The fusion result V4 is passed through the tenth convolutional layer and the eleventh convolutional layer in sequence, and the output of the eleventh convolutional layer is fused with the fusion result V4 to obtain the fusion result V5;

[0194] The fusion result V5 passes through the twelfth convolutional layer and the thirteenth convolutional layer in sequence, and then the output of the thirteenth convolutional layer is fused with the fusion result V5 to obtain the fusion result V6;

[0195] The fusion result V6 passes through the fourteenth convolutional layer and the fifteenth convolutional layer in sequence, and then the output of the fifteenth convolutional layer is fused with the fusion result V6 to obtain the fusion result V7;

[0196] The fusion result V7 is passed through the sixteenth convolutional layer and the seventeenth convolutional layer in sequence, and then the output of the seventeenth convolutional layer is fused with the fusion result V7 to obtain the fusion result V8;

[0197] The fusion result V8 passes through the 18th convolutional layer and the 19th convolutional layer in sequence, and then the output of the 19th convolutional layer is fused with the fusion result V8 to obtain the fusion result V9;

[0198] The fusion result V9 passes through the 20th convolutional layer and the 21st convolutional layer in sequence, and then the output of the 21st convolutional layer is fused with the fusion result V9 to obtain the fusion result V10;

[0199] The fusion result V10 passes through the 22nd convolutional layer and the 23rd convolutional layer in sequence, and then the output of the 23rd convolutional layer is fused with the fusion result V10 to obtain the fusion result V11;

[0200] The fusion result V11 passes through the 24th convolutional layer and the 25th convolutional layer in sequence, and then the output of the 25th convolutional layer is fused with the fusion result V11 to obtain the fusion result V12;

[0201] The fusion result V12 passes through the 26th convolutional layer and the 27th convolutional layer in sequence, and then the output of the 27th convolutional layer is fused with the fusion result V12 to obtain the fusion result V13;

[0202] The fusion result V13 passes through the 28th convolutional layer and the 29th convolutional layer in sequence, and then the output of the 29th convolutional layer is fused with the fusion result V13 to obtain the fusion result V14;

[0203] The fusion result V14 passes through the 30th convolutional layer and the 31st convolutional layer in sequence, and then the output of the 31st convolutional layer is fused with the fusion result V14 to obtain the fusion result V15;

[0204] The fusion result V15 passes through the 32nd convolutional layer and the 33rd convolutional layer in sequence, and then the output of the 33rd convolutional layer is fused with the fusion result V15 to obtain the fusion result V16;

[0205] The fusion result V16 is used as the output of the first convolution module;

[0206] The working process of the first CBAM module is:

[0207] The input of the first CBAM module is denoted as x. In the first CBAM module, the input x first passes through the channel attention layer, and then the output of the channel attention layer is concatenated with the input x to obtain the first concatenation result;

[0208] The first splicing result is used as the input of the spatial attention layer, and the output of the spatial attention layer is spliced ​​with the first splicing result for the second time, and the second splicing result is used as the output of the first CBAM module.

[0209] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A face tracking method for dense crowds, characterized in that: The method specifically comprises the following steps: Step 1: The video captured by the camera is frame-processed to obtain each frame image, and then the frame images are numbered in ascending order according to the order in which the frame images were captured; Step 2: Use the built target detection network to detect the face area in the first frame image; The target detection network includes a first module, a second module, a third module, a fourth module, an average pooling layer, a fully connected layer and a SoftMax function layer; The first module includes the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer; The working process of the first module is: The image input to the target detection network is used as the input of the first convolutional layer and the third convolutional layer respectively; The output of the first convolutional layer passes through the second convolutional layer, and the output of the third convolutional layer passes through the fourth convolutional layer; After fusing the output of the second convolutional layer and the output of the fourth convolutional layer, the fusion result is passed through the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer in sequence; The second module includes the eighth convolution layer, the ninth convolution layer, the tenth convolution layer, the eleventh convolution layer, the twelfth convolution layer, the thirteenth convolution layer, the fourteenth convolution layer, the fifteenth convolution layer and the second maximum pooling layer; The working process of the second module is: The output of the first maximum pooling layer is used as the input of the eighth and ninth convolutional layers respectively; Then the output of the eighth convolutional layer passes through the tenth and twelfth convolutional layers in sequence; the output of the ninth convolutional layer passes through the eleventh and thirteenth convolutional layers in sequence; After fusing the output of the twelfth convolutional layer and the output of the thirteenth convolutional layer, the fusion result is passed through the fourteenth convolutional layer, the fifteenth convolutional layer, and the second maximum pooling layer in sequence; The third module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the twentieth convolutional layer, the twenty-first convolutional layer, the twenty-second convolutional layer, the twenty-third convolutional layer, the twenty-fourth convolutional layer and the third maximum pooling layer; The working process of the third module is: The output of the second maximum pooling layer is used as the input of the sixteenth and seventeenth convolutional layers respectively; The output of the sixteenth convolutional layer then passes through the eighteenth, twentieth, and twenty-second convolutional layers in sequence; the output of the seventeenth convolutional layer passes through the nineteenth, twenty-first, and twenty-third convolutional layers in sequence; After fusing the output of the 22nd convolutional layer and the output of the 23rd convolutional layer, the fusion result is passed through the 24th convolutional layer and the third maximum pooling layer in sequence; The fourth module includes a twenty-fifth convolutional layer, a twenty-sixth convolutional layer, a twenty-seventh convolutional layer, a twenty-eighth convolutional layer, a twenty-ninth convolutional layer, a thirtieth convolutional layer, a thirty-first convolutional layer, a thirty-second convolutional layer, a thirty-third convolutional layer, and a thirty-fourth convolutional layer; The working process of the fourth module is: The output of the third maximum pooling layer is used as the input of the 25th convolutional layer and the 26th convolutional layer respectively; The output of the 25th convolutional layer then passes through the 27th, 29th, 31st, and 33rd convolutional layers in sequence; the output of the 26th convolutional layer passes through the 28th, 30th, 32nd, and 34th convolutional layers in sequence; Then, the outputs of the 33rd convolutional layer and the 34th convolutional layer are fused to obtain the fusion result a; Finally, the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer are fused to obtain the fusion result A; the fusion result A is then passed through the average pooling layer, the fully connected layer, and the SoftMax function layer, and the SoftMax function layer is used to output the target box position of the face area in the image; The loss function used in the target detection network training is: Among them, IoU is the intersection-over-union ratio of the predicted box output by the target detection network and the real box, |·| represents the absolute value, x1 is the center point of the predicted box output by the target detection network, x2 is the center point of the real box, ρ(x1,x2) is the Euclidean distance between x1 and x2, l is the diagonal length of the minimum circumscribed box between the predicted box and the real box, μ is the intermediate variable, and β is the scale variable; Among them, w gt is the width of the prediction box, h gt is the height of the predicted box, w is the width of the real box, and h is the height of the real box; Step 3: The person corresponding to each detected face area is treated as a tracking object, and each tracking object is assigned a different identity ID; Then, the facial feature extraction network is used to extract the features of each facial region image in the first frame image; Step 4: Initialize the number of image frames k=2; Step 5: Use the built target detection network to detect the face area in the k-th frame image, and then use the face feature extraction network to extract the features of each face area image in the k-th frame image; Step 6: Perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image; The specific process of step six is ​​as follows: Step 61: Process each face region image in the k-th frame image using two-dimensional multi-level wavelet decomposition to obtain a clarity score for each face region image in the k-th frame image; Number the facial region images in descending order of clarity scores; Step 62: Set the face region image count k′ to 1, and initialize the image set K to include all face region images in the k-1th frame image; Step 6.3: Calculate the feature similarity between the k′th face region image in the kth frame image and each face region image in the image set K, and record the face region image corresponding to the maximum feature similarity calculated in the image set K as Then determine whether the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1; If the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1, then continue to step 64; If the calculated maximum feature similarity is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; Step 64: Determine the face area image Is the clarity score of less than the score threshold Q2? (1) If the face area image If the clarity score is less than the score threshold Q2, then the image with the face area is tracked forward. Match the face region image with the smallest feature similarity, and calculate the feature similarity between the tracked face region image and the k′th face region image; If the feature similarity between the tracked face region image and the k′th face region image is greater than or equal to the similarity threshold Q1, the k′th face region image is successfully matched, and the face region image is Remove from the set K and assign the identity ID of the k′th face region image to image and execute step 65; If the feature similarity between the tracked face region image and the k′th face region image is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; and step 65 is executed; (2) If the face area image If the clarity score of the face is greater than or equal to the score threshold Q2, the face region image of the k′th person is successfully matched. Remove from the set K and assign the identity ID of the k′th face region image to image and execute step 65; Step 65: Let k′=k′+1, return to step 63, and stop iterating until all face area images in the k-th frame image are matched or the set K is an empty set. When all face area images in the k-th frame image are matched and the set K is not an empty set, it means that the people corresponding to the remaining images in the set K have disappeared from the current image, and the adjacent camera is started for tracking. When the set K is an empty set and all face area images in the k-th frame image are not matched, it means that the remaining unmatched people in the k-th frame image are newly appeared people, and the newly appeared people are matched with the people who disappeared in the adjacent coverage area at the same time, so as to achieve continuous tracking of people. Step 7: Set k=k+1, and then return to step 5.

2. The method for tracking faces in a dense crowd according to claim 1, characterized in that: The facial feature extraction network includes a first convolution module, a first CBAM module, a second convolution module, a second CBAM module, a third convolution module, a third CBAM module and a fourth convolution module; and the working process of the facial feature extraction network is: The input of the facial feature extraction network is used as the input of the first convolutional module; The output of the first convolution module is used as the input of the first CBAM module, and the Hadamard product of the output of the first CBAM module and the output of the first convolution module is calculated, and the calculated Hadamard product is recorded as T1; Use T1 as the input of the second convolution module, and then use the output of the second convolution module as the input of the second CBAM module; and calculating the Hadamard product of the output of the second CBAM module and the output of the second convolution module, and recording the calculated Hadamard product as T2; Use T2 as the input of the third convolution module, and then use the output of the third convolution module as the input of the third CBAM module; and calculating the Hadamard product of the output of the third CBAM module and the output of the third convolution module, and recording the calculated Hadamard product as T3; Use T3 as the input of the fourth convolution module; The output of the second convolutional module passes through the convolutional layer C1, the global average pooling layer P1 and the fully connected layer F1 in sequence to obtain the output of the fully connected layer F1; The output of the third convolutional module passes through the convolutional layer C2, the global average pooling layer P2, and the fully connected layer F2 in sequence to obtain the output of the fully connected layer F2; The output of the fourth convolutional module passes through the convolutional layer C3, the global average pooling layer P3, and the fully connected layer F3 in sequence to obtain the output of the fully connected layer F3; The output of the fully connected layer F1, the output of the fully connected layer F2, and the output of the fully connected layer F3 are spliced ​​to obtain the extracted facial features.

3. The method for tracking faces in a dense crowd according to claim 2, characterized in that: The first convolution module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer, a tenth convolution layer, an eleventh convolution layer, a twelfth convolution layer, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer, a sixteenth convolution layer, a seventeenth convolution layer, an eighteenth convolution layer, a nineteenth convolution layer, a twentieth convolution layer, a twenty-first convolution layer, a twenty-second convolution layer, a twenty-third convolution layer, a twenty-fourth convolution layer, a twenty-fifth convolution layer, a twenty-sixth convolution layer, a twenty-seventh convolution layer, a twenty-eighth convolution layer, a twenty-ninth convolution layer, a thirtieth convolution layer, a thirty-first convolution layer, a thirty-second convolution layer and a thirty-third convolution layer; wherein: In the first convolution module, the input image is first used as the input of the first convolution layer; The output of the first convolutional layer passes through the second and third convolutional layers in sequence, and then the output of the third convolutional layer is fused with the output of the first convolutional layer to obtain the fusion result V1; The fusion result V1 passes through the fourth convolutional layer and the fifth convolutional layer in sequence, and then the output of the fifth convolutional layer is fused with the fusion result V1 to obtain the fusion result V2; The fusion result V2 passes through the sixth and seventh convolutional layers in sequence, and then the output of the seventh convolutional layer is fused with the fusion result V2 to obtain the fusion result V3; The fusion result V3 is passed through the eighth and ninth convolutional layers in sequence, and the output of the ninth convolutional layer is fused with the fusion result V3 to obtain the fusion result V4. The fusion result V4 is passed through the tenth convolutional layer and the eleventh convolutional layer in sequence, and the output of the eleventh convolutional layer is fused with the fusion result V4 to obtain the fusion result V5; The fusion result V5 passes through the twelfth convolutional layer and the thirteenth convolutional layer in sequence, and then the output of the thirteenth convolutional layer is fused with the fusion result V5 to obtain the fusion result V6; The fusion result V6 passes through the fourteenth convolutional layer and the fifteenth convolutional layer in sequence, and then the output of the fifteenth convolutional layer is fused with the fusion result V6 to obtain the fusion result V7; The fusion result V7 is passed through the sixteenth convolutional layer and the seventeenth convolutional layer in sequence, and then the output of the seventeenth convolutional layer is fused with the fusion result V7 to obtain the fusion result V8; The fusion result V8 passes through the 18th convolutional layer and the 19th convolutional layer in sequence, and then the output of the 19th convolutional layer is fused with the fusion result V8 to obtain the fusion result V9; The fusion result V9 passes through the 20th convolutional layer and the 21st convolutional layer in sequence, and then the output of the 21st convolutional layer is fused with the fusion result V9 to obtain the fusion result V10; The fusion result V10 passes through the 22nd convolutional layer and the 23rd convolutional layer in sequence, and then the output of the 23rd convolutional layer is fused with the fusion result V10 to obtain the fusion result V11; The fusion result V11 passes through the 24th convolutional layer and the 25th convolutional layer in sequence, and then the output of the 25th convolutional layer is fused with the fusion result V11 to obtain the fusion result V12; The fusion result V12 passes through the 26th convolutional layer and the 27th convolutional layer in sequence, and then the output of the 27th convolutional layer is fused with the fusion result V12 to obtain the fusion result V13; The fusion result V13 passes through the 28th convolutional layer and the 29th convolutional layer in sequence, and then the output of the 29th convolutional layer is fused with the fusion result V13 to obtain the fusion result V14; The fusion result V14 passes through the 30th convolutional layer and the 31st convolutional layer in sequence, and then the output of the 31st convolutional layer is fused with the fusion result V14 to obtain the fusion result V15; The fusion result V15 passes through the 32nd convolutional layer and the 33rd convolutional layer in sequence, and then the output of the 33rd convolutional layer is fused with the fusion result V15 to obtain the fusion result V16; The fusion result V16 is used as the output of the first convolution module.

4. The method for tracking faces in a dense crowd according to claim 3, wherein: The working process of the first CBAM module is as follows: The input of the first CBAM module is denoted as x. In the first CBAM module, the input x first passes through the channel attention layer, and then the output of the channel attention layer is concatenated with the input x to obtain the first concatenation result; The first splicing result is used as the input of the spatial attention layer, and the output of the spatial attention layer is spliced ​​with the first splicing result for the second time, and the second splicing result is used as the output of the first CBAM module.

5. A face tracking system for dense crowds, characterized in that: The system includes an image acquisition module, a target detection module, a face feature extraction module and a face tracking module; wherein: The image acquisition module is used to shoot video with a camera, and to perform frame processing on the shot video to obtain each frame image, and then to number each frame image in ascending order according to the order in which each frame image was shot; The target detection module uses the target detection network to detect the face area in each frame image, and identifies the person corresponding to each face area in the first frame image as a tracking object, and assigns a different identity ID to each tracking object. The target detection network includes a first module, a second module, a third module, a fourth module, an average pooling layer, a fully connected layer and a SoftMax function layer; The first module includes the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer; The working process of the first module is: The image input to the target detection network is used as the input of the first convolutional layer and the third convolutional layer respectively; The output of the first convolutional layer passes through the second convolutional layer, and the output of the third convolutional layer passes through the fourth convolutional layer; After fusing the output of the second convolutional layer and the output of the fourth convolutional layer, the fusion result is passed through the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the first maximum pooling layer in sequence; The second module includes the eighth convolution layer, the ninth convolution layer, the tenth convolution layer, the eleventh convolution layer, the twelfth convolution layer, the thirteenth convolution layer, the fourteenth convolution layer, the fifteenth convolution layer and the second maximum pooling layer; The working process of the second module is: The output of the first maximum pooling layer is used as the input of the eighth and ninth convolutional layers respectively; Then the output of the eighth convolutional layer passes through the tenth and twelfth convolutional layers in sequence; the output of the ninth convolutional layer passes through the eleventh and thirteenth convolutional layers in sequence; After fusing the output of the twelfth convolutional layer and the output of the thirteenth convolutional layer, the fusion result is passed through the fourteenth convolutional layer, the fifteenth convolutional layer, and the second maximum pooling layer in sequence; The third module includes the sixteenth convolutional layer, the seventeenth convolutional layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the twentieth convolutional layer, the twenty-first convolutional layer, the twenty-second convolutional layer, the twenty-third convolutional layer, the twenty-fourth convolutional layer and the third maximum pooling layer; The working process of the third module is: The output of the second maximum pooling layer is used as the input of the sixteenth and seventeenth convolutional layers respectively; The output of the sixteenth convolutional layer then passes through the eighteenth, twentieth, and twenty-second convolutional layers in sequence; the output of the seventeenth convolutional layer passes through the nineteenth, twenty-first, and twenty-third convolutional layers in sequence; After fusing the output of the 22nd convolutional layer and the output of the 23rd convolutional layer, the fusion result is passed through the 24th convolutional layer and the third maximum pooling layer in sequence; The fourth module includes a twenty-fifth convolutional layer, a twenty-sixth convolutional layer, a twenty-seventh convolutional layer, a twenty-eighth convolutional layer, a twenty-ninth convolutional layer, a thirtieth convolutional layer, a thirty-first convolutional layer, a thirty-second convolutional layer, a thirty-third convolutional layer, and a thirty-fourth convolutional layer; The working process of the fourth module is: The output of the third maximum pooling layer is used as the input of the 25th convolutional layer and the 26th convolutional layer respectively; The output of the 25th convolutional layer then passes through the 27th, 29th, 31st, and 33rd convolutional layers in sequence; the output of the 26th convolutional layer passes through the 28th, 30th, 32nd, and 34th convolutional layers in sequence; Then, the outputs of the 33rd convolutional layer and the 34th convolutional layer are fused to obtain the fusion result a; Finally, the fusion result a, the output of the seventh convolutional layer, the output of the fifteenth convolutional layer, and the output of the twenty-fourth convolutional layer are fused to obtain the fusion result A; the fusion result A is then passed through the average pooling layer, the fully connected layer, and the SoftMax function layer, and the SoftMax function layer is used to output the target box position of the face area in the image; The face feature extraction module uses the face feature extraction network to extract features from each face region image in each frame image; The face tracking module is used to perform face matching based on the features of each face region image in the k-th frame image and the features of each face region image in the k-1-th frame image; specifically: Step 1: Process each face region image in the k-th frame image using two-dimensional multi-level wavelet decomposition to obtain a clarity score for each face region image in the k-th frame image; Number the facial region images in descending order of clarity scores; Step 2: Set the face region image count k′ to 1, and initialize the image set K to include all face region images in the k-1th frame image; Step 3: Calculate the feature similarity between the k′th face region image in the kth frame image and each face region image in the image set K, and record the face region image corresponding to the maximum feature similarity in the image set K as , and then determine whether the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1; If the calculated maximum feature similarity is greater than or equal to the similarity threshold Q1, proceed to step 4; If the calculated maximum feature similarity is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; Step 4: Determine the face area image Is the clarity score of less than the score threshold Q2? (1) If the face area image If the clarity score is less than the score threshold Q2, then the image with the face area is tracked forward. Match the face region image with the smallest feature similarity, and calculate the feature similarity between the tracked face region image and the k′th face region image; If the feature similarity between the tracked face region image and the k′th face region image is greater than or equal to the similarity threshold Q1, the k′th face region image is successfully matched, and the face region image is Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5; If the feature similarity between the tracked face region image and the k′th face region image is less than the similarity threshold Q1, the k′th face region image matching fails, and the k′th face region image is taken as a new tracking object and assigned a new identity ID; and step 5 is executed; (2) If the face area image If the clarity score of the face is greater than or equal to the score threshold Q2, the face region image of the k′th person is successfully matched. Remove from the set K and assign the identity ID of the k′th face region image to image ID; and proceed to step 5; Step 5. Let k′=k′+1, return to step 3, and stop iterating until all the face area images in the k-th frame image are matched or the set K is an empty set. When all the face area images in the k-th frame image are matched and the set K is not an empty set, it means that the people corresponding to the remaining images in the set K have disappeared from the current image, and the adjacent cameras are started for tracking. When the set K is an empty set and all the face area images in the k-th frame image have not been matched, it means that the remaining unmatched people in the k-th frame image are new people. The new people are matched with the people who disappeared in the adjacent coverage area at the same time to achieve continuous tracking of people.

6. The face tracking system for dense crowds according to claim 5, characterized in that: The facial feature extraction network includes a first convolution module, a first CBAM module, a second convolution module, a second CBAM module, a third convolution module, a third CBAM module and a fourth convolution module; and the working process of the facial feature extraction network is: The input of the facial feature extraction network is used as the input of the first convolutional module; The output of the first convolution module is used as the input of the first CBAM module, and the Hadamard product of the output of the first CBAM module and the output of the first convolution module is calculated, and the calculated Hadamard product is recorded as T1; Use T1 as the input of the second convolution module, and then use the output of the second convolution module as the input of the second CBAM module; and calculating the Hadamard product of the output of the second CBAM module and the output of the second convolution module, and recording the calculated Hadamard product as T2; Use T2 as the input of the third convolution module, and then use the output of the third convolution module as the input of the third CBAM module; and calculating the Hadamard product of the output of the third CBAM module and the output of the third convolution module, and recording the calculated Hadamard product as T3; Use T3 as the input of the fourth convolution module; The output of the second convolutional module passes through the convolutional layer C1, the global average pooling layer P1 and the fully connected layer F1 in sequence to obtain the output of the fully connected layer F1; The output of the third convolutional module passes through the convolutional layer C2, the global average pooling layer P2, and the fully connected layer F2 in sequence to obtain the output of the fully connected layer F2; The output of the fourth convolutional module passes through the convolutional layer C3, the global average pooling layer P3, and the fully connected layer F3 in sequence to obtain the output of the fully connected layer F3; The output of the fully connected layer F1, the output of the fully connected layer F2, and the output of the fully connected layer F3 are concatenated to obtain the extracted facial features; The first convolution module includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a seventh convolution layer, an eighth convolution layer, a ninth convolution layer, a tenth convolution layer, an eleventh convolution layer, a twelfth convolution layer, a thirteenth convolution layer, a fourteenth convolution layer, a fifteenth convolution layer, a sixteenth convolution layer, a seventeenth convolution layer, an eighteenth convolution layer, a nineteenth convolution layer, a twentieth convolution layer, a twenty-first convolution layer, a twenty-second convolution layer, a twenty-third convolution layer, a twenty-fourth convolution layer, a twenty-fifth convolution layer, a twenty-sixth convolution layer, a twenty-seventh convolution layer, a twenty-eighth convolution layer, a twenty-ninth convolution layer, a thirtieth convolution layer, a thirty-first convolution layer, a thirty-second convolution layer and a thirty-third convolution layer; wherein: In the first convolution module, the input image is first used as the input of the first convolution layer; The output of the first convolutional layer passes through the second and third convolutional layers in sequence, and then the output of the third convolutional layer is fused with the output of the first convolutional layer to obtain the fusion result V1; The fusion result V1 passes through the fourth convolutional layer and the fifth convolutional layer in sequence, and then the output of the fifth convolutional layer is fused with the fusion result V1 to obtain the fusion result V2; The fusion result V2 passes through the sixth and seventh convolutional layers in sequence, and then the output of the seventh convolutional layer is fused with the fusion result V2 to obtain the fusion result V3; The fusion result V3 is passed through the eighth and ninth convolutional layers in sequence, and the output of the ninth convolutional layer is fused with the fusion result V3 to obtain the fusion result V4. The fusion result V4 is passed through the tenth convolutional layer and the eleventh convolutional layer in sequence, and the output of the eleventh convolutional layer is fused with the fusion result V4 to obtain the fusion result V5; The fusion result V5 passes through the twelfth convolutional layer and the thirteenth convolutional layer in sequence, and then the output of the thirteenth convolutional layer is fused with the fusion result V5 to obtain the fusion result V6; The fusion result V6 passes through the fourteenth convolutional layer and the fifteenth convolutional layer in sequence, and then the output of the fifteenth convolutional layer is fused with the fusion result V6 to obtain the fusion result V7; The fusion result V7 is passed through the sixteenth convolutional layer and the seventeenth convolutional layer in sequence, and then the output of the seventeenth convolutional layer is fused with the fusion result V7 to obtain the fusion result V8; The fusion result V8 passes through the 18th convolutional layer and the 19th convolutional layer in sequence, and then the output of the 19th convolutional layer is fused with the fusion result V8 to obtain the fusion result V9; The fusion result V9 passes through the 20th convolutional layer and the 21st convolutional layer in sequence, and then the output of the 21st convolutional layer is fused with the fusion result V9 to obtain the fusion result V10; The fusion result V10 passes through the 22nd convolutional layer and the 23rd convolutional layer in sequence, and then the output of the 23rd convolutional layer is fused with the fusion result V10 to obtain the fusion result V11; The fusion result V11 passes through the 24th convolutional layer and the 25th convolutional layer in sequence, and then the output of the 25th convolutional layer is fused with the fusion result V11 to obtain the fusion result V12; The fusion result V12 passes through the 26th convolutional layer and the 27th convolutional layer in sequence, and then the output of the 27th convolutional layer is fused with the fusion result V12 to obtain the fusion result V13; The fusion result V13 passes through the 28th convolutional layer and the 29th convolutional layer in sequence, and then the output of the 29th convolutional layer is fused with the fusion result V13 to obtain the fusion result V14; The fusion result V14 passes through the 30th convolutional layer and the 31st convolutional layer in sequence, and then the output of the 31st convolutional layer is fused with the fusion result V14 to obtain the fusion result V15; The fusion result V15 passes through the 32nd convolutional layer and the 33rd convolutional layer in sequence, and then the output of the 33rd convolutional layer is fused with the fusion result V15 to obtain the fusion result V16; The fusion result V16 is used as the output of the first convolution module; The working process of the first CBAM module is: The input of the first CBAM module is denoted as x. In the first CBAM module, the input x first passes through the channel attention layer, and then the output of the channel attention layer is concatenated with the input x to obtain the first concatenation result; The first splicing result is used as the input of the spatial attention layer, and the output of the spatial attention layer is spliced ​​with the first splicing result for the second time, and the second splicing result is used as the output of the first CBAM module.

Images