Logistics personnel identity identification method and device based on privacy protection and related medium

Abstract

The application discloses a personnel identity recognition method and device based on privacy protection and related medium, and the method comprises the following steps: firstly, obtaining a front photo of a logistics personnel and processing a side gait video of the logistics personnel; secondly, marking the personnel type and position in the desensitized photo and multiple gait mask sequences; thirdly, training the collected data through an FCOS target detection model and a gait recognition model to obtain corresponding trained models; finally, obtaining a desensitized photo and multiple gait mask sequences of a to-be-detected personnel, inputting the desensitized photo and the multiple gait mask sequences into the trained FCOS target detection model and gait recognition model respectively, obtaining a first prediction result and a second prediction result, and fusing the first prediction result and the second prediction result to obtain a final prediction result. Through the desensitization processing of the personnel photo and the gait recognition of the personnel, and the fusion of the results after the training and reasoning of the desensitized photo and the gait mask sequence, the application improves the passing efficiency of the visiting personnel and reduces the risk of the visiting personnel leaking the privacy security.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent identity recognition, and in particular relates to a logistics personnel identity recognition method and device based on privacy protection and related media. BACKGROUND

[0002] At present, the safety management of the community, the access control of non-resident personnel is an important part of maintaining community safety, and the logistics personnel (such as delivery personnel, maintenance personnel, and courier personnel) belong to a special group of non-resident personnel. The logistics personnel do not have a community access card, but have the right to enter and exit the community. Therefore, certain technical means are needed to confirm the identity of the logistics personnel. At present, most communities use manual remote video intercom to confirm the identity of the visitors, which is relatively low in efficiency, and due to manual judgment, there are problems of false rejection and false acceptance.

[0003] In the prior art, such as invention patent 1 (application number: 202010418961.X) and invention patent 2 (application number: 202010419503.8), the target detection method is mainly used to solve the problem of manual judgment. This method takes pictures of the logos, work clothes, and delivery boxes of delivery companies, maintenance companies, and express companies, and then uses a pre-trained model for fine-tuning to obtain a trained model. After that, the pictures are inferred, and finally the multiple targets are weighted to obtain the final prediction result. However, this method also has many problems, such as in the actual scene, the logistics personnel visit frequently, and the annotation amount is large; the collected pictures need to be uploaded to the server, which has the risk of leaking the privacy of the visitors. In addition, the target detection method in the prior art generally only relies on the front face photo for identification, and the identification accuracy is low. SUMMARY

[0004] The embodiments of the present application provide a logistics personnel identity recognition method and device based on privacy protection and related media, aiming to solve the problems of large picture annotation amount, low accuracy, and privacy leakage risk of visitors in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a logistics personnel identity recognition method based on privacy protection, comprising:

[0006] Obtaining a front face photo of a logistics personnel, identifying the coordinates of the face key points in the front face photo of the logistics personnel, and performing desensitization processing on the face region in the front face photo of the logistics personnel according to the coordinates of the face key points to obtain a desensitized photo of the logistics personnel;

[0007] Obtaining a side gait video of a logistics personnel, and performing inference on the side gait video of the logistics personnel according to a Mask Transfiner model to obtain a plurality of gait mask sequences of the logistics personnel;

[0008] annotating personnel types and positions in the desensitized photo and the plurality of gait mask sequences, respectively;

[0009] training the desensitized photo through a FCOS target detection model to obtain a trained FCOS target detection model, and training the plurality of gait mask sequences through a gait recognition model to obtain a trained gait recognition model;

[0010] obtaining a front photo and a side gait video of a to-be-detected personnel, and obtaining a desensitized photo and a plurality of gait mask sequences of the to-be-detected personnel, respectively, inputting the desensitized photo and the plurality of gait mask sequences of the to-be-detected personnel into the trained FCOS target detection model and the trained gait recognition model, respectively, to obtain first and second prediction results, and fusing the first and second prediction results to obtain a final prediction result.

[0011] In a second aspect, an embodiment of the present application provides a logistics personnel identity recognition device based on privacy protection, comprising:

[0012] a photo acquisition unit configured to obtain a front photo of a logistics personnel, identify coordinates of face key points in the front photo of the logistics personnel, and perform desensitization processing on a face region in the front photo of the logistics personnel according to the coordinates of the face key points to obtain a desensitized photo of the logistics personnel;

[0013] a gait acquisition unit configured to obtain a side gait video of the logistics personnel, and perform inference on the side gait video of the logistics personnel according to a Mask Transfiner model to obtain a plurality of gait mask sequences of the logistics personnel;

[0014] a data annotation unit configured to annotate personnel types and positions in the desensitized photo and the plurality of gait mask sequences, respectively;

[0015] a data training unit configured to train the desensitized photo through a FCOS target detection model to obtain a trained FCOS target detection model, and train the plurality of gait mask sequences through a gait recognition model to obtain a trained gait recognition model;

[0016] The data fusion unit is used for acquiring a front photo and a side gait video of a to-be-detected person, and acquiring a desensitized photo and a plurality of gait mask sequences of the to-be-detected person respectively, inputting the desensitized photo and the plurality of gait mask sequences of the to-be-detected person into the trained FCOS target detection model and the gait recognition model respectively, obtaining a first prediction result and a second prediction result, and fusing the first prediction result and the second prediction result to obtain a final prediction result.

[0017] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the privacy protection-based logistics personnel identity identification method of the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the privacy protection-based logistics personnel identity identification method of the first aspect.

[0019] The embodiment of the present application provides a privacy protection-based logistics personnel identity identification method, which first acquires a front photo of a logistics personnel and a side gait video of the logistics personnel for processing, labels a personnel type and a position in the desensitized photo and the plurality of gait mask sequences respectively, trains target detection of collected data through an FCOS target detection model and a gait recognition model to obtain corresponding trained models respectively, finally acquires a desensitized photo and a plurality of gait mask sequences of a to-be-detected person, and inputs the desensitized photo and the plurality of gait mask sequences into the trained FCOS target detection model and the gait recognition model respectively to obtain a first prediction result and a second prediction result, and fuses the first prediction result and the second prediction result to obtain a final prediction result. The present application fuses a result obtained by training and reasoning a desensitized photo and a gait mask sequence after desensitizing a personnel photo and recognizing a personnel gait, the scheme only needs to label a personnel type, reduces a labeling amount, improves an identification accuracy in combination with a gait recognition result, and reduces a risk of leaking privacy security of a visitor.

[0020] The embodiment of the present application also provides a privacy protection-based logistics personnel identity identification device, a computer device and a storage medium, which also have the beneficial effects described above. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0022] Figure 1 A flowchart of a logistics personnel identity recognition method based on privacy protection provided by the embodiment of the present application;

[0023] Figure 2 An algorithm principle diagram of a logistics personnel identity recognition method based on privacy protection provided by the embodiment of the present application;

[0024] Figure 3 A principle diagram of a horizontal federated learning training stage provided by the embodiment of the present application;

[0025] Figure 4 A schematic block diagram of a logistics personnel identity recognition device based on privacy protection provided by the embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort are within the protection scope of the present application.

[0027] It should be understood that when used in the specification and the appended claims, the terms "comprise" and "include" indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0029] It should be further understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0030] Please see the following Figure 1 ,Figure 1 The flowchart of a privacy-protected logistics personnel identification method provided in this embodiment of the invention specifically includes steps S101 to S105.

[0031] S101. Obtain a frontal photo of the logistics personnel, identify the coordinates of the facial key points in the frontal photo of the logistics personnel, and perform desensitization processing on the facial area in the frontal photo of the logistics personnel according to the coordinates of the facial key points to obtain a desensitized photo of the logistics personnel.

[0032] S102. Obtain the side gait video of the logistics personnel, and infer the side gait video of the logistics personnel according to the Mask Transfiner model to obtain multiple gait mask sequences of the logistics personnel.

[0033] S103. Label the personnel types and locations in the desensitized photos and the multiple gait mask sequences respectively;

[0034] S104. The desensitized photos are trained using the FCOS target detection model to obtain a trained FCOS target detection model; and the multiple gait mask sequences are trained using the gait recognition model to obtain a trained gait recognition model.

[0035] S105. Obtain a frontal photo and a side gait video of the person to be detected, and obtain an anonymized photo and multiple gait mask sequences of the person to be detected respectively. Input the anonymized photo and multiple gait mask sequences of the person to be detected into the trained FCOS target detection model and gait recognition model respectively to obtain a first prediction result and a second prediction result. Then, fuse the first prediction result and the second prediction result to obtain the final prediction result.

[0036] Combination Figure 1 and Figure 2 In step S101, a frontal photo of the logistics personnel is first obtained. Figure 2 In the data layer section, uploaded photos are used to identify the coordinates of facial key points in the frontal photo of the logistics personnel. This allows for the determination of the upper left and lower right corner coordinates of the face region. Based on the coordinates of the facial key points, the face region in the frontal photo of the logistics personnel is anonymized. The anonymization process utilizes the DAN (Deep Alignment Networks) algorithm to identify the coordinates of the facial key points. After determining the face region, the face region is zeroed out to obtain the anonymized photo of the logistics personnel.

[0037] Specifically, the zero setting processing refers to setting the pixel value of the face region to [0, 0, 0], that is, pure black after the DAN algorithm detects the face region in the photo, which can obtain a black frame picture after removing the face region in the preprocessing part Figure 2 The purpose of the black frame picture after removing the face region in the preprocessing part is to protect the face sensitive information of the visitor and prevent the leakage of the face sensitive information. Of course, the present embodiment is not limited to the above-mentioned zero setting processing mode, for example, the pixel value of the face region can be set to other values, for example, set to pure white, which can also achieve the desensitization effect.

[0038] In step S102, the side gait video of the logistics personnel is obtained Figure 2 The gait video in the data layer part), the Mask Transfiner model is used to infer the side gait video of the logistics personnel, a plurality of gait sequences in the side gait video are recognized, and a plurality of gait mask sequences of the logistics personnel are obtained; it should be noted that before using the Mask Transfiner model, the Mask Transfiner model (the Mask Transfiner model is a high-quality and efficient instance segmentation algorithm, and the Mask Transfiner model is not operated on a regular dense tensor, but the image region is decomposed and represented as a quad tree) needs to be pre-trained on the COCO dataset (the COCO dataset is a dataset labeled by Microsoft, mainly used for target detection, segmentation and image description), and the Mask Transfiner model is put into use after pre-training is completed, after obtaining the plurality of gait mask sequences of the logistics personnel, the plurality of gait mask sequences are aligned, and the accuracy of the Mask Transfiner model is improved.

[0039] In step S103, the personnel types in the desensitized photos and the multiple gait mask sequences are labeled. First, the "person" region in the desensitized photos is bounded to determine the "person" region. It should be noted that the "person" region here is not the face region, but includes the body part of the person and the face part after zeroing. Then, the bounding box of the "person" region is represented by (l, t, r, b), where l and t represent the horizontal and vertical coordinates of the upper left corner of the rectangle, respectively, and r and b represent the horizontal and vertical coordinates of the lower right corner of the rectangle, respectively. Finally, according to the identity of the person in the photo, they are labeled as category c∈{0,1}, where 1 represents logistics personnel and 0 represents non-logistics personnel. Similarly, the personnel in the gait video are classified according to the gait mask sequence in the side gait video, and are also labeled as c∈{0,1}, where 1 represents logistics personnel and 0 represents non-logistics personnel. It should be noted that the annotation of the desensitized photos and the multiple gait mask sequences is performed on the data in the training set, so that logistics personnel can be quickly identified and screened after being put into use.

[0040] In step S104, the desensitized photo is trained using the FCOS target detection model to obtain the trained FCOS target detection model. Figure 2 The prediction layer includes FCOS; and the gait recognition model is used to train the target detection on the multiple gait mask sequences to obtain a trained gait recognition model. Figure 2 (Gait recognition network in the prediction layer); After training the FCOS object detection model and gait recognition model respectively, they can be put into use without further pre-training; Of course, pre-training is done according to different communities, and different training sets can be used for training.

[0041] In one embodiment, step S104 includes:

[0042] The current node inputs the local desensitized photo into the FCOS target detection model for this round of target detection training, and inputs multiple local gait mask sequences into the gait recognition model for this round of target detection training, to obtain the model parameters for this round;

[0043] The current node encrypts the model parameters for this round and randomly sends them to the next node, so that the next node inputs the local desensitized photos into the FCOS target detection model for target detection training in this round, and inputs multiple local gait mask sequences into the gait recognition model for target detection training in this round, thus obtaining the updated model parameters for this round.

[0044] The next node continues to encrypt and randomly send the updated model parameters of the current round to another node for training until all nodes complete the model training of the current round and obtain the final model parameters of the current round.

[0045] After completing the model training of all rounds, the final global model parameters are obtained.

[0046] In combination Figure 3 As shown in the embodiment, a horizontal federated learning model is used to realize data privacy protection and decentralized training of the FCOS target detection model and the gait recognition model. Each community can communicate with each other, and each iteration is performed locally in the community using local data to update the model weight (i.e., model parameters). The updated model weight is encrypted and randomly transmitted to the next community until the model training is completed in all communities for one round, i.e., the first round of horizontal federated learning model training is completed. The specific steps are as follows:

[0047] Step 1, initialize the model weight W0 and randomly transmit it to a certain community;

[0048] Step 2, each community synchronously inputs the desensitized photo and multiple gait mask sequences into W j for training (W0 is used as the global model in the first round of training), and obtains multiple community local models S1, S2, S3, …, Si. i The parameters of the local model S1 are encrypted and randomly transmitted to the local model S2 of the next community, and a new model W is obtained by federated learning weighted average calculation. The updated model parameters are encrypted and randomly transmitted to the i-th community local model Si. i It should be noted that S1 does not specify a specific community local model, but represents a randomly selected first community local model, such as Figure 3 In the embodiment, there are four community (neighborhood) local models, i.e., S i = S4.

[0049] The following formula is used to calculate

[0050]

[0051] where W j represents the model after the j-th round, represents the updated model of the i-th community in the j-th round, represents the number of training samples of the i-th community, and N j is the total number of training samples of all communities in the j-th round.

[0052] ​In addition, if the predetermined threshold is not reached after the first round of federated learning model training ends, a second round of federated learning model training can be performed on the basis of the first round (the first round as a new global model for the next round of training), or even a third round of federated learning model training; that is, a specified number of rounds of training can be performed, and the specific formula is as follows:

[0053]

[0054] By analogy, until the FCOS target detection model and the gait recognition model undergo multiple rounds of training under the federated learning model and converge to a specified threshold.

[0055] In an embodiment, the step S104 further comprises:

[0056] The desensitization photo is sequentially subjected to downsampling operations of 8, 16 and 32 times, and a plurality of first feature maps of different sizes are obtained from bottom to top;

[0057] After the plurality of first feature maps are respectively subjected to 1x1 convolution, a plurality of second feature maps are obtained by top-down connection.

[0058] The second feature map at the top layer is sequentially subjected to two 3x3 convolution layers with a step of 2 to obtain two third feature maps.

[0059] Further, the loss function Loss is calculated according to the following formula, and the FCOS target detection model is optimized according to the loss function:

[0060]

[0061] Wherein, N pos represents the number of positive samples, L cls represents the focal loss function, L reg represents the IOU loss function, L ctrness represents the center-ness, p x,y represents the predicted score of each class at the feature map (x, y) point, represents the real class label corresponding to the feature map (x, y) point, t x,y represents the predicted target bounding box information at the feature map (x, y) point, represents the real target bounding box (i.e. real rectangular frame) information corresponding to the feature map (x, y) point, s x,y represents the predicted center-ness at the feature map (x, y) point, represents the real center-ness corresponding to the feature map (x, y) point, 1 if the feature map (x, y) point is matched as a positive sample, otherwise 0.

[0062] Further, set each feature map P i according to the following formula:

[0063] m i-1 <max(l * ,t * ,r * ,b * )<m i

[0064] Where m i represents the maximum distance of the i-th feature map regression, l * ,t * ,r * ,b * respectively represent the distance of a point on the i-th feature map to the left boundary, upper boundary, right boundary and lower boundary of the original image relative to the positive sample.

[0065] In this embodiment, the FCOS target detection model includes a backbone network, a feature pyramid network and a detector, the feature pyramid network adopts FPN structure, and the backbone network adopts ResNet-101; after the desensitized photo is input into ResNet-101, ResNet-101 performs 8, 16 and 32 times down-sampling operations on the desensitized photo in sequence, and obtains a plurality of first feature maps with different sizes from bottom to top; in this embodiment, the plurality of first feature maps are named as C3, C4 and C5; C3, C4 and C5 are respectively subjected to 1×1 convolution, and a plurality of second feature maps, i.e. P3, P4 and P5, are obtained through top-down connection; then the second feature map at the uppermost layer is sequentially subjected to two 3×3 convolution layers with a step of 2 to obtain two third feature maps, i.e. a third feature map P6 is obtained through a convolution layer with a convolution kernel size of 3×3 and a step of 2 on the basis of the second feature map P5, and finally a third feature map P7 is obtained through a convolution layer with a convolution kernel size of 3×3 and a step of 2 on the basis of P6.

[0066] In addition, the IOU loss function is calculated according to the following formula:

[0067] L GIOU =1-GIOU(truth,pred)

[0068]

[0069]

[0070] Wherein, IOU (truth, pred) represents the area intersection ratio of the predicted rectangular frame and the real rectangular frame, C represents the smallest closed convex surface that can cover the predicted rectangular frame and the real rectangular frame, \ " represents the area in C that does not cover the predicted rectangular frame and the real rectangular frame, truth represents the true value, and pred represents the predicted value.

[0071] The focal loss function is calculated according to the following formula:

[0072] FL (p, y) = -y (1-p) γ log (p) - (1-y) p γ log (1-p)

[0073] Wherein, p represents the confidence size of the predicted frame, y represents the real category, and γ represents the weight parameter.

[0074] In the embodiment, the upper and lower limit values of each feature map P i are set, and targets of different sizes are allocated to feature maps of different sizes:

[0075] m i-1 <max (l * ,t * ,r * ,b * ) <m i

[0076] Wherein, m i represents the maximum distance of the i-th feature map regression, l * ,t * ,r * ,b * respectively represent the distance of a point on the i-th feature map to the left boundary, the upper boundary, the right boundary and the lower boundary of the original picture when mapped to the original picture; m2~m7 (corresponding to P3, P4, P5, P6 and P7) are set to 0, 32, 64, 128, 256 and ∞.

[0077] In an embodiment, the step S104 further comprises:

[0078] The plurality of gait mask sequences are sequentially input to a 5x5 convolution layer and a 3x3 convolution layer for convolution, and then input to a first pooling layer for pooling to obtain a first output;

[0079] The first output is sequentially input to two 3x3 convolution layers for convolution, and then input to a second pooling layer for pooling to obtain a second output;

[0080] The second output is sequentially input to two 3x3 convolution layers for convolution, and then input to a third pooling layer for pooling to obtain a third output;

[0081] The third output is sequentially input into two 3x3 convolution layers for convolution, and then input into a multi-feature set pooling layer for multi-frame feature aggregation to obtain a fourth output;

[0082] The fourth output is input into a pyramid pooling layer for spatial feature aggregation to obtain a fifth output;

[0083] The fifth output is input into a full connection layer and normalized to obtain a final output.

[0084] In the embodiment, the processing order of the gait recognition model is as follows from top to bottom in the table:

[0085]

[0086]

[0087] wherein, Conv-LeakyReLU represents a convolution layer, MaxPool represents a pooling layer, Set Pooling represents a multi-feature set pooling layer, HPP represents a pyramid pooling layer, SeparateFCs represents a full connection layer, and SeparateBNNecks represents a normalization layer.

[0088] The activation function uses LeakyReLU, and the MaxPool layer is a pooling step (the convolution kernel size of the Pooling layer is generally even, and the convolution kernel size is generally odd); the input channel of the gait recognition model is 1, and the output channel numbers of the convolution layers are 32, 64, 128 and 256 respectively; after obtaining the third output, the extracted multi-frame features (feature maps extracted after convolution and pooling operation on each frame of gait mask) are aggregated into a vector (fourth output) by multi-feature set pooling (Set Pooling); after obtaining the fourth output, the spatial features are aggregated by pyramid pooling (Horizontal Pyramid Pooling, HPP) to obtain a fifth output; finally, the fifth output is input into a full connection layer and normalized to obtain a final output; in addition, in order to increase the training data and improve the robustness of the gait recognition model, part of the normal walking mask sequences in the CASIA-E data set are selected for testing and used for training of the gait recognition model.

[0089] In step S105, after the FCOS target detection model and the gait recognition model are trained, a frontal photograph and a side gait video of the person to be detected are acquired, and de-identified photographs and multiple gait mask sequences of the person to be detected are acquired respectively. The de-identified photographs and multiple gait mask sequences of the person to be detected are then input into the trained FCOS target detection model and the gait recognition model respectively to obtain a first prediction result. Figure 2 Prediction result 1) and second prediction result (in the prediction layer section) Figure 2 The prediction result 2 in the prediction layer is used to fuse the first and second prediction results to obtain the final prediction result. Figure 2 The results in the fusion layer are fused, and the final prediction results are used to determine whether or not to allow passage for personnel.

[0090] In one embodiment, step S105 includes:

[0091] The first and second prediction results are fused using the following formula to obtain the final prediction result:

[0092] R = a × 1 × R1 + b × 1 × R2

[0093] Where a and b represent the weights of the first prediction result and the second prediction result respectively, and a+b=1, R represents the confidence level of logistics personnel in the final prediction result, R1 represents the confidence level of logistics personnel in the first prediction result, and R2 represents the confidence level of logistics personnel in the second prediction result.

[0094] In this embodiment, the two prediction results R1 and R2 are weighted and summed to output the final prediction result R; a and b both represent weights, a+b=1; the number 1 represents logistics personnel. Since personnel have already been classified in step S103, a number of 0 represents non-logistics personnel; Figure 2 When both prediction result 1 and prediction result 2 in the prediction layer are 0, their corresponding confidence levels are 0.13 and 0.11, respectively, and no result fusion is required. When both prediction result 1 and prediction result 2 in the prediction layer are 1, their corresponding confidence levels are 0.87 and 0.89, respectively, and a confidence level of 0.88 is obtained through result fusion. When both prediction result 1 and prediction result 2 contain 0 and 1, result fusion is also performed to obtain a final confidence level. In this embodiment, the confidence level threshold is set to 0.6. When the final confidence level is greater than 0.6, the person is identified as a logistics worker, and the gate automatically allows passage. When the final confidence level is less than 0.6, passage is denied. Of course, different confidence level thresholds can be set adaptively according to actual conditions.

[0095] Combination Figure 4 As shown,Figure 4 A schematic block diagram of a logistics personnel identity recognition device based on privacy protection provided for an embodiment of the present application, the logistics personnel identity recognition device based on privacy protection 400 comprises:

[0096] A photo acquisition unit 401 is configured to acquire a front photo of a logistics personnel, identify coordinates of face key points in the front photo of the logistics personnel, and perform desensitization processing on a face region in the front photo of the logistics personnel according to the coordinates of the face key points to obtain a desensitized photo of the logistics personnel.

[0097] A gait acquisition unit 402 is configured to acquire a side gait video of the logistics personnel, and perform inference on the side gait video of the logistics personnel according to a Mask Transfiner model to obtain a plurality of gait mask sequences of the logistics personnel.

[0098] A data labeling unit 403 is configured to label a personnel type and a position in the desensitized photo and the plurality of gait mask sequences, respectively.

[0099] A data training unit 404 is configured to perform target detection training on the desensitized photo by using a FCOS target detection model to obtain a trained FCOS target detection model, and perform target detection training on the plurality of gait mask sequences by using a gait recognition model to obtain a trained gait recognition model.

[0100] A data fusion unit 405 is configured to acquire a front photo and a side gait video of a to-be-detected personnel, and acquire a desensitized photo and a plurality of gait mask sequences of the to-be-detected personnel, respectively, input the desensitized photo and the plurality of gait mask sequences of the to-be-detected personnel into the trained FCOS target detection model and the trained gait recognition model, respectively, to obtain a first prediction result and a second prediction result, and fuse the first prediction result and the second prediction result to obtain a final prediction result.

[0101] In the embodiment, the photo acquisition unit 401 acquires a front photo of a logistics personnel, identifies coordinates of a face key point in the front photo of the logistics personnel, and performs desensitization processing to obtain a desensitized photo of the logistics personnel; the gait acquisition unit 402 acquires a side gait video of the logistics personnel, performs inference on the side gait video of the logistics personnel according to a Mask Transfiner model to obtain a plurality of gait mask sequences of the logistics personnel; the data labeling unit 403 labels a personnel type and a position in the desensitized photo and the plurality of gait mask sequences respectively; the data training unit 404 performs target detection training on the desensitized photo through an FCOS target detection model to obtain a trained FCOS target detection model, and performs target detection training on the plurality of gait mask sequences through a gait recognition model to obtain a trained gait recognition model; finally, the data fusion unit 405 acquires a front photo and a side gait video of a to-be-detected personnel, and acquires a desensitized photo and a plurality of gait mask sequences of the to-be-detected personnel respectively, inputs the desensitized photo and the plurality of gait mask sequences of the to-be-detected personnel into the trained FCOS target detection model and gait recognition model respectively, obtains first and second prediction results, and fuses the first and second prediction results to obtain a final prediction result.

[0102] In an embodiment, the data training unit 404 comprises a function of:

[0103] The current node inputs the local desensitized photo into the FCOS target detection model for target detection training of the current round, and inputs the local plurality of gait mask sequences into the gait recognition model for target detection training of the current round to obtain model parameters of the current round;

[0104] The current node encrypts and randomly sends the model parameters of the current round to the next node, so that the next node inputs the local desensitized photo into the FCOS target detection model for target detection training of the current round, and inputs the local plurality of gait mask sequences into the gait recognition model for target detection training of the current round to obtain updated model parameters of the current round;

[0105] The next node continues to encrypt and randomly send the updated model parameters of the current round to another node for training until all nodes complete the model training of the current round to obtain final model parameters of the current round;

[0106] After completing the model training of all rounds, final global model parameters are obtained.

[0107] Further, the desensitized photo is sequentially subjected to 8, 16 and 32 times downsampling operations to obtain a plurality of first feature maps of different sizes from bottom to top;

[0108] After the plurality of first feature maps are respectively subjected to 1x1 convolution, a plurality of second feature maps are obtained through top-down connection respectively;

[0109] The uppermost second feature map is sequentially subjected to two 3x3 convolution layers with a step of 2 to obtain two third feature maps.

[0110] Further, a loss function Loss is calculated according to the following formula, and the FCOS target detection model is optimized according to the loss function:

[0111]

[0112] Wherein, N pos represents the number of positive samples, L cls represents the focal loss function, L reg represents the IOU loss function, L ctrness represents the center-ness, p x,y represents the predicted score of each class at the feature map (x, y) point, represents the real class label corresponding to the feature map (x, y) point, t x,y represents the predicted target bounding box information at the feature map (x, y) point, represents the real target bounding box information corresponding to the feature map (x, y) point, s x,y represents the predicted center-ness at the feature map (x, y) point, represents the real center-ness corresponding to the feature map (x, y) point, represents 1 when the feature map (x, y) point is matched as a positive sample, otherwise 0.

[0113] Further, the upper and lower limit values of each feature map P i are set according to the following formula, and different size targets are assigned to different size feature maps:

[0114] m i-1 <max(l * ,t * ,r * ,b * )<m i

[0115] Wherein, m i represents the maximum distance of the i-th feature map regression, l * , t * , r * , b * respectively represent the distance of a certain point on the i-th feature map to the left boundary, upper boundary, right boundary and lower boundary of the original image relative to the positive sample.

[0116] In an embodiment, the data training unit 404 further comprises a function of:

[0117] inputting the plurality of gait mask sequences into a 5*5 convolution layer and a 3*3 convolution layer in sequence for convolution, and then into a first pooling layer for pooling to obtain a first output;

[0118] inputting the first output into two 3*3 convolution layers in sequence for convolution, and then into a second pooling layer for pooling to obtain a second output;

[0119] inputting the second output into two 3*3 convolution layers in sequence for convolution, and then into a third pooling layer for pooling to obtain a third output;

[0120] inputting the third output into two 3*3 convolution layers in sequence for convolution, and then into a multi-feature set pooling layer for multi-frame feature aggregation to obtain a fourth output;

[0121] inputting the fourth output into a pyramid pooling layer for spatial feature aggregation to obtain a fifth output;

[0122] inputting the fifth output into a full connection layer and performing normalization processing to obtain a final output.

[0123] Further, the first prediction result and the second prediction result are fused according to the following formula to obtain a final prediction result:

[0124] R = a * 1 * R1 + b * 1 * R2

[0125] wherein a and b respectively represent the weights of the first prediction result and the second prediction result and a + b = 1, R represents the confidence of the final prediction result as a logistics personnel, R1 represents the confidence of the first prediction result as a logistics personnel, and R2 represents the confidence of the second prediction result as a logistics personnel.

[0126] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and are not described here.

[0127] The embodiments of the present application also provide a computer readable storage medium, which has a computer program stored thereon, and the computer program can implement the steps provided by the above embodiments when executed. The storage medium can include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0128] The embodiment of the present application further provides a computer device, which can comprise a memory and a processor, the memory has a computer program stored therein, and the processor can realize the steps provided by the above embodiment when calling the computer program in the memory. Of course, the computer device can further comprise various network interfaces, power supplies and other components.

[0129] The various embodiments are described in the specification by way of progressive progression, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to. For the system disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part. It should be noted that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0130] It should be further noted that, in the specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A privacy-preserving method for identifying logistics personnel, characterized in that, include: Obtain a frontal photo of a logistics personnel, identify the coordinates of facial key points in the frontal photo of the logistics personnel, and perform desensitization processing on the facial region in the frontal photo of the logistics personnel based on the coordinates of the facial key points to obtain a desensitized photo of the logistics personnel. Acquire side gait videos of logistics personnel, and infer multiple gait mask sequences of the logistics personnel based on the Mask Transfiner model. The types and locations of people in the desensitized photos and the multiple gait mask sequences are labeled respectively; The desensitized photos are trained using the FCOS target detection model to obtain a trained FCOS target detection model; and the multiple gait mask sequences are trained using the gait recognition model to obtain a trained gait recognition model. A frontal photo and a side gait video of the person to be detected are acquired, and an anonymized photo and multiple gait mask sequences of the person to be detected are acquired respectively. The anonymized photo and multiple gait mask sequences of the person to be detected are input into the trained FCOS target detection model and gait recognition model respectively to obtain a first prediction result and a second prediction result. The first prediction result and the second prediction result are then fused to obtain the final prediction result.

2. The privacy-preserving logistics personnel identification method according to claim 1, characterized in that, The desensitized photo is trained using the FCOS target detection model to obtain a trained FCOS target detection model. And by training the multiple gait mask sequences using a gait recognition model for target detection, a trained gait recognition model is obtained, including: The current node inputs the local desensitized photo into the FCOS target detection model for this round of target detection training, and inputs multiple local gait mask sequences into the gait recognition model for this round of target detection training, to obtain the model parameters for this round; The current node encrypts the model parameters for this round and randomly sends them to the next node, so that the next node inputs the local desensitized photos into the FCOS target detection model for target detection training in this round, and inputs multiple local gait mask sequences into the gait recognition model for target detection training in this round, thus obtaining the updated model parameters for this round. The next node continues to encrypt the updated model parameters for this round and randomly send them to another node for training, until all nodes have completed the model training for this round and obtained the final model parameters for this round. After completing all rounds of model training, the final global model parameters are obtained.

3. The privacy-preserving logistics personnel identification method according to claim 1, characterized in that, The step of training the desensitized photo using the FCOS target detection model to obtain a trained FCOS target detection model includes: The desensitized photos were downsampled sequentially by 8, 16, and 32 times to obtain multiple first feature maps of different sizes from bottom to top. Multiple first feature maps are convolved with 1×1 respectively, and then connected from top to bottom to obtain multiple second feature maps; The second feature map at the top layer is convolved through two 3×3 convolutional layers with a stride of 2 to obtain two third feature maps.

4. The privacy-preserving logistics personnel identification method according to claim 3, characterized in that, The step of training the desensitized photo using the FCOS target detection model to obtain the trained FCOS target detection model further includes: The loss function Loss is calculated using the following formula, and the FCOS object detection model is optimized based on the loss function: Where, N pos L represents the number of positive samples. cls L represents the focal loss function. reg L represents the IOU loss function. ctrness p represents center-ness x,y This represents the score predicted for each class at the feature map point (x, y). t represents the true class label corresponding to the point (x,y) in the feature map. x,y This represents the predicted bounding box information of the target at the feature map point (x, y). This represents the information of the true target bounding box corresponding to the point (x,y) in the feature map, s x,y This represents the predicted center-ness at the feature map point (x, y). This represents the true center-ness corresponding to the point (x,y) in the feature map. The value is 1 when the feature map point (x,y) is matched as a positive sample, and 0 otherwise.

5. The privacy-preserving logistics personnel identification method according to claim 3, characterized in that, The step of training the desensitized photo using the FCOS target detection model to obtain the trained FCOS target detection model further includes: Each feature map P is set according to the following formula. i The upper and lower limits are used to assign targets of different sizes to feature maps of different sizes: m i-1 <max(l * ,t * ,r * ,b * )<m i Where, m i Let l represent the maximum distance regressed from the i-th feature map. * ,t * ,r * ,b * These represent the distances of a point on the i-th feature map to the left, upper, right, and lower boundaries of the positive sample when mapped to the original map.

6. The privacy-preserving logistics personnel identification method according to claim 1, characterized in that, The step of training the multiple gait mask sequences using a gait recognition model to obtain a trained gait recognition model includes: Multiple gait mask sequences are sequentially input into a 5×5 convolutional layer and a 3×3 convolutional layer for convolution, and then input into the first pooling layer for pooling to obtain the first output; The first output is sequentially input into two 3×3 convolutional layers for convolution, and then input into a second pooling layer for pooling to obtain the second output; The second output is sequentially fed into two 3×3 convolutional layers for convolution, and then fed into a third pooling layer for pooling to obtain the third output; The third output is sequentially input into two 3×3 convolutional layers for convolution, and then input into a multi-feature pooling layer for multi-frame feature aggregation to obtain the fourth output; The fourth output is input into the pyramid pooling layer for spatial feature aggregation to obtain the fifth output; The fifth output is input to the fully connected layer and normalized to obtain the final output.

7. The privacy-preserving logistics personnel identification method according to claim 1, characterized in that, The process of fusing the first prediction result and the second prediction result to obtain the final prediction result includes: The first and second prediction results are fused using the following formula to obtain the final prediction result: R = a × 1 × R1 + b × 1 × R2 Where a and b represent the weights of the first prediction result and the second prediction result respectively, and a+b=1, R represents the confidence level of logistics personnel in the final prediction result, R1 represents the confidence level of logistics personnel in the first prediction result, and R2 represents the confidence level of logistics personnel in the second prediction result.

8. A privacy-protected logistics personnel identification device, characterized in that, include: The photo acquisition unit is used to acquire a frontal photo of a logistics personnel, identify the coordinates of facial key points in the frontal photo of the logistics personnel, and perform desensitization processing on the facial region in the frontal photo of the logistics personnel based on the coordinates of the facial key points to obtain a desensitized photo of the logistics personnel. The gait acquisition unit is used to acquire side gait videos of logistics personnel, and to infer multiple gait mask sequences of the logistics personnel based on the Mask Transfiner model. The data annotation unit is used to annotate the types and locations of people in the desensitized photos and the multiple gait mask sequences, respectively; The data training unit is used to train the desensitized photo to perform target detection training using the FCOS target detection model, so as to obtain the trained FCOS target detection model. And by training the multiple gait mask sequences with a gait recognition model for target detection, a trained gait recognition model is obtained; The data fusion unit is used to acquire a frontal photo and a side gait video of the person to be detected, and to acquire an anonymized photo of the person to be detected and multiple gait mask sequences, respectively. The anonymized photo of the person to be detected and the multiple gait mask sequences are input into the trained FCOS target detection model and gait recognition model to obtain a first prediction result and a second prediction result. The first prediction result and the second prediction result are then fused to obtain the final prediction result.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the privacy-preserving logistics personnel identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the privacy-preserving logistics personnel identification method as described in any one of claims 1 to 7.

Images