A field operation personnel identity feature fusion recognition method and system

Abstract

The application discloses a kind of field operation personnel identity feature fusion identification method and system, the method includes: the decoding frame extraction of monitoring video file or real-time monitoring video stream is carried out, and sequence frame is obtained;Sequence frame is obtained to pedestrian detection, pedestrian re-identification and pedestrian tracking, and the gait image sequence and gait contour sequence of the object to be identified are extracted, and the target feature set is constituted;The target feature set of the object to be identified is matched with the target feature set in the target library constructed in advance, and the identity of the object to be identified is identified;In the application, each target feature set in the target library is bound with a worker identity in advance.The application can effectively solve the problem of identity recognition of uniformed workers in the scene where face is difficult to obtain and complete gait cycle cannot be captured, realize the identity recognition of field workers covering all scenes, and effectively improve the safety control level of plant and station.

Description

Technical Field

[0001] This invention relates to a method and system for fusing and identifying the identity features of on-site workers, belonging to the field of safety management technology. Background Technology

[0002] Power system substations, including power plants and substations, are crucial infrastructure for power operation, serving as vital links in power reception and distribution, and supporting the power grid. In recent years, with the increasing prevalence of smart substations, camera-based video surveillance technology has been increasingly used in substation monitoring. However, the management and supervision of on-site personnel still relies on manual review of surveillance videos, leading to wasted human resources and insufficient control levels. Identification of substation personnel mostly relies on ID cards and fingerprint recognition, which are inconvenient to carry and require active cooperation, thus limiting control to entry and exit. With the continuous development of machine vision technology, facial recognition has also been applied to the identification of entry and exit personnel and on-site workers. However, the recognition effect is greatly affected by the environment and distance, with technical challenges such as difficulty in capturing faces at long distances, failing to meet the identification needs of the entire work process and complex scenarios. Furthermore, on-site personnel typically wear standardized uniforms, making it difficult to distinguish them when faces cannot be captured. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for fusion identification of on-site workers' identity features. Based on computer vision and video image analysis technology, it extracts gait image sequences and gait contour sequences of pedestrian targets to form a target feature set, which is matched with a pre-built target library to identify the identity of the object to be identified. It can identify the identity of on-site workers in real time, accurately and continuously.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0005] This invention provides a method for fusing and recognizing the identity features of on-site workers, comprising:

[0006] Decode and extract frames from surveillance video files or real-time surveillance video streams to obtain a sequence of frames;

[0007] Pedestrian detection, pedestrian re-identification, and pedestrian tracking are performed on the sequence frames to extract the gait image sequence and gait contour sequence of the object to be identified, thus forming a target feature set;

[0008] The target feature set of the object to be identified is matched with a pre-built target library to identify the identity of the object to be identified; the target library includes a target feature set pre-bound to the identity of the operator and a face database of the operator.

[0009] Further steps, including pre-building the target library, include:

[0010] Obtain surveillance video files or real-time surveillance video streams containing workers, decode and extract frames to obtain a sequence of frames;

[0011] Pedestrian detection, pedestrian re-identification, and pedestrian tracking are performed on the sequence frames to obtain a gait image sequence and a gait contour sequence for each target, which constitute a target feature set; one target is a worker.

[0012] Based on the operator information database, the operator's identity is bound to the target feature set of each target, resulting in a target database composed of target feature sets bound to the operator's identity.

[0013] Furthermore, pedestrian detection, pedestrian re-identification, and pedestrian tracking are performed on the sequence frames, including:

[0014] A pedestrian target detection model is used to detect pedestrians in the obtained sequence frames. The location of the pedestrian target box containing the pedestrian target is detected, and the pedestrian image is extracted from the sequence frame image based on the location of the pedestrian target box.

[0015] Extract pedestrian re-identification features from pedestrian images;

[0016] The extracted pedestrian re-identification features are matched with the pedestrian re-identification features in the previous frame to track pedestrians.

[0017] Based on pedestrian tracking, pedestrian images of a target in n consecutive frames are acquired as a gait image sequence; n is a preset threshold, n∈N+;

[0018] Human figure segmentation is performed on the pedestrian image in each frame of the gait image sequence, and the segmentation results are normalized and binarized to obtain gait contour maps. Gait contour maps of n consecutive frames are obtained as gait contour sequences.

[0019] The gait image sequence and gait contour sequence of each pedestrian target are used as the target feature set of that pedestrian target.

[0020] Furthermore, the pedestrian target detection model adopts the YOLOv5 model.

[0021] Furthermore, the step of extracting pedestrian re-identification features from pedestrian images includes:

[0022] The OSNet model was used to extract pedestrian re-identification features.

[0023] Furthermore, human figure segmentation is performed on the pedestrian images in each frame of the gait image sequence, including:

[0024] The semantic segmentation model Fast-SCNN is used to segment human figures in pedestrian images.

[0025] Furthermore, the step of binding the identity of the operator with the target feature set of each target based on the operator information database includes:

[0026] Faces are extracted from gait image sequences of each target's feature set using a face detection algorithm.

[0027] Using the face database in the worker information database as the base database, face recognition is performed on faces extracted from gait image sequences;

[0028] If the recognition is successful, the corresponding worker identity in the worker information database will be bound to the target feature set corresponding to the face, and the faces in the worker information database and the faces extracted from the gait image sequence will be added to the face database of the target database.

[0029] If recognition fails or a face cannot be detected, the identity of the worker is confirmed and bound by viewing the gait video generated from the gait image sequence of the target feature set, and the face of the worker in the worker information database is added to the face database of the target database; all target feature sets and face databases with bound identities constitute the target database.

[0030] Furthermore, the face detection algorithm employs the MTCNN face detection model.

[0031] Furthermore, the step of performing face recognition on the face images extracted from the gait image sequence includes:

[0032] The MobileFaceNet face recognition model is used to perform face recognition on face images.

[0033] Furthermore, the step of matching the target feature set of the object to be identified with a pre-built target library to identify the identity of the object includes:

[0034] Faces are extracted from gait image sequences of the target feature set of the object to be identified using a face detection algorithm;

[0035] Using the face database in the target database as the base database, calculate the similarity between the face of the object to be identified and the faces of all workers in the target database. If the maximum similarity is greater than a preset threshold Th f If so, the object to be identified will be identified as the worker with the highest similarity.

[0036] If a face cannot be extracted or the similarity calculated from the face does not exceed the threshold Thf for any worker, then the gait contour sequences in the target database are used as the base database to calculate the similarity between the gait features of the object to be identified and the gait features of all workers in the target database. If the maximum similarity exceeds the preset threshold Thf, the similarity is calculated. g If so, the object to be identified will be identified as the worker with the highest similarity.

[0037] If the identity of the subject cannot be identified through facial and gait features, then the pedestrian re-identification features extracted from the gait image sequence in the target database are used as the base database. The similarity between the pedestrian re-identification features of the subject and the pedestrian re-identification features of all workers in the target database is calculated and ranked. The top five similarity rankings are assigned different weights according to the ranking order, and the similarity of workers in the same target database is added together to obtain the final comparison result. The worker in the target database with the highest ranking is taken as the identity recognition result of the subject.

[0038] Furthermore, the GaitSet gait recognition model is used for gait feature extraction.

[0039] Furthermore, the calculation of pedestrian re-identification features of the object to be identified includes:

[0040] Head and shoulder features and global features are extracted from the object to be identified, and the two types of features are fused with different weights to serve as pedestrian re-identification features for the object to be identified.

[0041] Furthermore, the step of decoding and extracting frames from the surveillance video file or real-time surveillance video stream to obtain a sequence of frames includes:

[0042] The open-source tool OpenCV was used to extract frames from the video to obtain a sequence of frames.

[0043] A second aspect of the present invention provides a field worker identity feature fusion and recognition system for implementing the aforementioned field worker identity feature fusion and recognition method, comprising:

[0044] The video decoding unit is used to decode and extract frames from surveillance video files or real-time surveillance video streams to obtain sequence frames and buffer them.

[0045] The feature extraction unit is used to perform pedestrian detection, pedestrian re-identification, and pedestrian tracking on the cached sequence frames, and to extract the gait image sequence and gait contour sequence of the pedestrian target to form a target feature set;

[0046] The matching unit is used to match the target feature set of the object to be identified with a pre-built target library to identify the identity of the object to be identified; the target library includes a target feature set pre-bound to the identity of the operator and a face library of the operator.

[0047] Furthermore, the feature extraction unit includes:

[0048] The pedestrian detection unit is used to perform pedestrian detection on the sequence frames cached by the video decoding unit, detect the position of the pedestrian target box containing the pedestrian target, and extract and cache the pedestrian image from the sequence frame image according to the position of the pedestrian target box.

[0049] The pedestrian tracking unit is used to extract pedestrian re-identification features from the pedestrian images cached by the pedestrian detection unit, and to match them with the pedestrian re-identification features in the previous frame to track the same pedestrian target.

[0050] The gait segmentation unit is used to acquire pedestrian images of a pedestrian target in n consecutive frames as a gait image sequence based on pedestrian tracking; n is a preset threshold, n∈N+; and to perform human figure segmentation on the pedestrian image in each frame of the gait image sequence, and to perform normalization and binarization processing on the segmentation results to obtain a gait contour map, thereby obtaining a gait contour map of n consecutive frames as a gait contour sequence and caching it.

[0051] Furthermore, it also includes:

[0052] The personnel binding unit is used to bind the identities of operators in the operator information database with the target feature set of operators obtained in advance, forming a target database.

[0053] Furthermore, the matching unit includes:

[0054] Face recognition unit: Used to extract faces from gait image sequences of the target feature set of the object to be identified using a face detection algorithm; using the face database in the target library as the base database, it calculates the similarity between the face of the object to be identified and the faces of all workers in the target database. If the maximum similarity is greater than a preset threshold Th f If so, the object to be identified will be identified as the operator with the highest similarity and cached;

[0055] Gait recognition unit: Used to calculate the similarity between the gait features of the object to be identified and the gait features of all workers in the target database, using gait contour sequences in the target database as the base database. If the maximum similarity is greater than a preset threshold Th g If so, the object to be identified will be identified as the operator with the highest similarity and cached;

[0056] Pedestrian Re-identification Unit: Using pedestrian re-identification features extracted from gait image sequences in the target database as the base database, it calculates and ranks the similarity between the pedestrian re-identification features of the object to be identified and the pedestrian re-identification features of all workers in the target database. The top five similarity rankings are assigned different weights according to their ranking order, and the similarity of workers in the same target database is added together to obtain the final comparison result. The worker in the target database with the highest ranking is taken as the identity recognition result of the object to be identified and cached.

[0057] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0058] This invention provides a method for extracting identity features based on video sequences. For the decoded video sequence, pedestrian detection is performed using object detection, and pedestrian re-identification features are extracted from the pedestrian images using pedestrian re-identification to achieve pedestrian tracking in multi-target scenarios. After tracking the trajectory of the same pedestrian target for n consecutive frames, obtaining a gait image sequence of a complete cycle for the pedestrian target, the gait image sequence is processed by human figure segmentation, normalization, and binarization to obtain a gait contour map. Furthermore, face detection is performed on the gait image sequence, and facial features are extracted. This invention can extract high-quality gait sequence images and gait contour maps of a single pedestrian target for a complete cycle in multi-target and complex background video surveillance scenarios, and extract facial features, pedestrian re-identification features, and gait features from them, meeting the recognition requirements of different recognition algorithms.

[0059] This invention proposes a method for identifying on-site workers by integrating facial features, gait features, and pedestrian re-identification features. Based on video sequences, it performs facial recognition and pedestrian re-identification, effectively improving the accuracy of identification and reducing the false recognition rate. It can solve the problem of personnel identification in scenarios where workers are uniformly dressed, faces are difficult to obtain, and complete gait cycles cannot be captured. Attached Figure Description

[0060] Figure 1 This is a flowchart of a method for fusing and recognizing the identity features of on-site workers provided in Embodiment 2 of the present invention;

[0061] Figure 2 This is a flowchart of feature capture provided in Embodiment 2 of the present invention;

[0062] Figure 3 This is a flowchart of identity binding provided in Embodiment 2 of the present invention;

[0063] Figure 4 This is a flowchart of the process for identifying the object to be identified provided in Embodiment 2 of the present invention;

[0064] Figure 5This is a schematic diagram of a field worker identity feature fusion recognition system provided in Embodiment 3 of the present invention. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0066] Example 1

[0067] This invention provides a method for fusing and recognizing the identity features of on-site workers, including:

[0068] Decode and extract frames from surveillance video files or real-time surveillance video streams to obtain a sequence of frames;

[0069] Pedestrian detection, pedestrian re-identification, and pedestrian tracking are performed on the sequence frames to extract the gait image sequence and gait contour sequence of the object to be identified, thus forming a target feature set;

[0070] The target feature set of the object to be identified is matched with the target feature set in the pre-built target library to identify the identity of the object to be identified; each target feature set in the target library is pre-bound to an operator identity.

[0071] Example 2

[0072] like Figure 1 As shown, this embodiment of the invention provides a method for fusing and recognizing the identity features of on-site workers. The specific implementation process includes:

[0073] S1. Decode and extract frames from the surveillance video file or real-time surveillance video stream to obtain a sequence of frames; and perform face detection, pedestrian detection, pedestrian re-identification and pedestrian tracking on the obtained sequence of frames to obtain a target feature set for each target, which includes a gait image sequence and a gait contour sequence.

[0074] S2. Based on the operator information database, bind the operator's identity with the target feature set to obtain the target database;

[0075] S3. Match the target feature set of the object to be identified obtained from the surveillance video file or real-time surveillance video stream with the target library constructed in step S2 to determine the target personnel corresponding to the object to be identified.

[0076] For the specific implementation process of step S1 in this embodiment, please refer to [link / reference]. Figure 2 ,include:

[0077] S11. Decode and extract frames from the monitoring video file or real-time monitoring video stream to obtain a video sequence frame.

[0078] In this step, the open-source tool OpenCV is used for video frame extraction;

[0079] S12. Perform pedestrian detection on the video sequence frames.

[0080] In this step, the YOLOv5 model is used for pedestrian target detection. The location of the pedestrian target box containing the pedestrian target is detected. Based on the location of the pedestrian target box, the pedestrian image can be extracted from the original video sequence frame image and cached.

[0081] S13. Extract pedestrian re-identification features from pedestrian images and then cache them.

[0082] In this step, the OSNet (Omni-Scale Network) model is used to extract pedestrian re-identification features.

[0083] The extracted pedestrian re-identification features are matched with the pedestrian re-identification features in the previous frame to match the pedestrian target in the current sequence frame with the pedestrian target in the previous frame, thereby enabling the tracking of the same pedestrian target and ensuring pedestrian tracking under multiple targets.

[0084] S14. Since human gait characteristics are periodic, if the trajectory of the same pedestrian target is successfully tracked between n consecutive frames, the pedestrian images of the pedestrian target in the n consecutive frames are obtained as a gait image sequence.

[0085] Human figure segmentation is performed on the pedestrian image in each frame of the gait image sequence, and the segmentation results are normalized and binarized to obtain gait contour maps. Gait contour maps of n consecutive frames are obtained as gait contour sequences and cached.

[0086] Where n is a preset threshold, n∈N+; the specific value of the threshold n can be set according to the frame frequency of the monitoring video file or the real-time monitoring video stream and the pedestrian's walking habits, so as to ensure that the gait sequence map and gait contour map of the same pedestrian target in a complete cycle can be extracted.

[0087] For example, in this embodiment, n = 15;

[0088] In this step, the human figure segmentation model uses the semantic segmentation model Fast-SCNN (Fast Semantic Segmentation Network);

[0089] S15. The gait image sequence and gait contour sequence of each target are used as the target feature set for that target, and the target feature set is cached.

[0090] In step S2 of this embodiment, the person's identity is bound to the target feature set by automatic or manual means to obtain the target library, and a specific camera can be specified to capture the target features required for identity binding.

[0091] Automatic binding is suitable for situations where the existing worker information database contains a face database and the target feature set allows for the acquisition of facial features. For details on the implementation process, please refer to [link / reference needed]. Figure 3 ,include:

[0092] S21. Use a face detection algorithm to extract faces from the gait image sequence of the target feature set.

[0093] In this step, the face detection model MTCNN (Multi-task convolutional neural network) is used to extract faces from the gait image sequence of the target feature set;

[0094] S22. Using the face database in the worker information database as the base database, perform face recognition on the faces extracted from the gait image sequence.

[0095] In this step, the MobileFaceNet face recognition model is used to perform face recognition on the face image;

[0096] S23. If the recognition is successful, bind the objects in the operator information database to the target feature set, and add the faces in the operator information database and the faces extracted from the gait image sequence to the face database of the target database.

[0097] Manual binding is suitable for situations where the target feature set cannot capture a face or the face fails to match the face database. For details, please refer to [link to implementation details]. Figure 3 ,include:

[0098] S24. By manually reviewing the gait video generated from the gait image sequence, the identity information is confirmed and bound. Personnel can be selected from the operator information database or temporary identities can be assigned.

[0099] S25. If the bound object exists in the face database of the operator information database and the face database contains the object's face, then add the face to the face database of the target database.

[0100] The target database consists of all the target feature sets and face databases that have been bound to identities.

[0101] For the specific implementation process of step S3 in this embodiment, please refer to [link / reference]. Figure 4 ,include:

[0102] S31. Decode and extract frames from the monitoring video files or real-time monitoring video streams within the plant to obtain a sequence of frames. Then, perform pedestrian detection, extract pedestrian re-identification features, and obtain the gait image sequence and gait contour map of the pedestrian target in sequence to obtain the target feature set of the object to be identified.

[0103] S32. Perform face detection on the gait image sequence of the target feature set of the object to be identified to obtain the face of the object to be identified;

[0104] S33. If a clear human face can be detected in the gait image sequence, use the face database in the target database as the base database to identify the face of the object to be identified.

[0105] The specific identification method is as follows: calculate the similarity between the face of the object to be identified and the faces of all objects in the target database. If the maximum similarity is greater than a threshold Th... f If the object to be identified is the same person as the object with the highest similarity in the target database, then the object to be identified is considered to be the same person.

[0106] S34. If no clear face is detected in the gait image sequence or the similarity calculated by facial features does not exceed the threshold Th f For objects, the gait contour sequences in the target database are used as the base database to match the gait features of the object to be identified.

[0107] The specific identification method is as follows: A gait recognition algorithm is used to identify the gait contour sequence. The similarity between the gait features of the object to be identified and the gait features of all objects in the target database is calculated. If the maximum similarity is greater than a threshold Th... g If the object to be identified is the same person as the object with the highest similarity in the target database, then the object to be identified is considered to be the same person.

[0108] In this step, the GaitSet gait recognition model is used for gait feature extraction and recognition;

[0109] S35. If it is impossible or not possible to identify the pedestrian's identity through facial features and gait features, then the pedestrian re-identification features extracted from the gait image sequence in the target database shall be used as the base database for matching the pedestrian re-identification features.

[0110] To address the issue of uniform attire among on-site personnel, pedestrian re-identification features are categorized into head and shoulder features and global features. These two types of features are then fused together with different weight ratios to form the final pedestrian re-identification features.

[0111] Pedestrian re-identification features are extracted from gait image sequences. The similarity between the pedestrian re-identification features of the object to be identified and the pedestrian re-identification features of all objects in the target database is calculated and ranked.

[0112] Taking into account the top five similarity results, different weights are assigned to the results according to the ranking order. The similarity scores of objects in the same target database are then summed to obtain the final similarity comparison result with the target database objects.

[0113] The object ranked first in the target library is taken as the identity recognition result of the object to be identified.

[0114] It should be noted that when personnel at the work site are dressed uniformly, traditional pedestrian re-identification algorithms cannot meet the recognition requirements. Therefore, feature fusion is performed by combining head and shoulder features with global features, which serve as the pedestrian re-identification features for this step. In this step, head and shoulder detection uses the STN (Spatial Transformer Network) model, and head and shoulder features and global features are extracted using the ResNet-50 model.

[0115] The weights for the top 5 similarity rankings are 0.3, 0.25, 0.2, 0.15, and 0.1, respectively.

[0116] Example 3

[0117] This embodiment provides a field worker identity feature fusion and recognition system to implement the field worker identity feature fusion and recognition method of Embodiment 1 or Embodiment 2. (See also...) Figure 5 The system includes:

[0118] The video decoding unit is used to decode and extract frames from surveillance video files or real-time surveillance video streams to obtain sequence frames and buffer them.

[0119] The feature extraction unit is used to perform pedestrian detection, pedestrian re-identification, and pedestrian tracking on the cached sequence frames, and to extract the gait image sequence and gait contour sequence of the pedestrian target to form a target feature set;

[0120] The matching unit is used to match the target feature set of the object to be identified with a pre-built target library to identify the identity of the object to be identified; the target library includes a target feature set pre-bound to the identity of the operator and a face library of the operator.

[0121] Further, see Figure 5 The feature extraction unit of this system includes:

[0122] The pedestrian detection unit is used to perform pedestrian detection on the sequence frames cached by the video decoding unit, detect the position of the pedestrian target box containing the pedestrian target, and extract and cache the pedestrian image from the sequence frame image according to the position of the pedestrian target box.

[0123] The pedestrian tracking unit is used to extract pedestrian re-identification features from the pedestrian images cached by the pedestrian detection unit, and to match them with the pedestrian re-identification features in the previous frame to track the same pedestrian target.

[0124] The gait segmentation unit is used to acquire pedestrian images of a pedestrian target in n consecutive frames as a gait image sequence based on pedestrian tracking; n is a preset threshold, n∈N+; and to perform human figure segmentation on the pedestrian image in each frame of the gait image sequence, and to perform normalization and binarization processing on the segmentation results to obtain a gait contour map, thereby obtaining a gait contour map of n consecutive frames as a gait contour sequence and caching it.

[0125] Further, see Figure 5 The matching unit of this system includes:

[0126] Face recognition unit: Used to extract faces from gait image sequences of the target feature set of the object to be identified using a face detection algorithm; using the face database in the target library as the base database, it calculates the similarity between the face of the object to be identified and the faces of all workers in the target database. If the maximum similarity is greater than a preset threshold Th f If so, the object to be identified will be identified as the operator with the highest similarity and cached;

[0127] Gait recognition unit: Used to calculate the similarity between the gait features of the object to be identified and the gait features of all workers in the target database, using gait contour sequences in the target database as the base database. If the maximum similarity is greater than a preset threshold Th g If so, the object to be identified will be identified as the operator with the highest similarity and cached;

[0128] Pedestrian Re-identification Unit: Using pedestrian re-identification features extracted from gait image sequences in the target database as the base database, it calculates and ranks the similarity between the pedestrian re-identification features of the object to be identified and the pedestrian re-identification features of all workers in the target database. The top five similarity rankings are assigned different weights according to their ranking order, and the similarity of workers in the same target database is added together to obtain the final comparison result. The worker in the target database with the highest ranking is taken as the identity recognition result of the object to be identified and cached.

[0129] Further, see Figure 5 The system also includes:

[0130] The personnel binding unit is used to bind the identities of operators in the operator information database with the target feature set of operators obtained in advance, forming a target database.

[0131] In this embodiment, the specific implementation methods of each unit can be referred to the description in the above method embodiments, and will not be repeated here.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for on-site work personnel identity feature fusion recognition, characterized in that, The method comprises the following steps: decoding and frame extraction are performed on a monitoring video file or a real-time monitoring video stream to obtain a sequence frame; pedestrian detection, pedestrian re-identification and pedestrian tracking are performed on the sequence frame to extract a gait image sequence and a gait contour sequence of the object to be identified, and a target feature set is formed; the target feature set of the object to be identified is matched with a target library constructed in advance to identify the identity of the object to be identified, which comprises the following steps: a face is extracted from the gait image sequence of the target feature set of the object to be identified by using a face detection algorithm; Taking the face library in the target library as a base library, similarity between the face of the to-be-identified object and the faces of all workers in the target library is calculated, and if the maximum similarity is greater than a preset threshold Th f , the to-be-identified object is identified as the worker with the maximum similarity. If the face cannot be extracted or the similarity calculated by the face does not exist a worker greater than a preset threshold Th f , the gait profile sequence in the target library is taken as a base library, the similarity between the gait feature of the to-be-identified object and the gait features of all workers in the target library is calculated, and if the maximum similarity is greater than a preset threshold Th g , the to-be-identified object is identified as the worker with the maximum similarity. if the identity of the object to be identified is not identified by using the face and gait features, the pedestrian re-identification features extracted from the gait image sequence in the target library are used as a base library, the similarity between the pedestrian re-identification features of the object to be identified and the pedestrian re-identification features of all workers in the target library is calculated and ranked, the top five in the similarity ranking are given different weights in the ranking order, and the similarities of the same workers in the target library are added, finally, the comparison result is obtained, and the worker in the target library ranked first is taken as the identity recognition result of the object to be identified; the pedestrian re-identification features of the object to be identified comprise the following steps: head and shoulder features and global features of the object to be identified are extracted, and the two types of features are given different weights for fusion, which are taken as the pedestrian re-identification features of the object to be identified; the weights of the top five in the similarity ranking are 0.3, 0.25, 0.2, 0.15 and 0.1 respectively; the target library comprises target feature sets and a face library of workers, and the target feature sets are pre-bound with the identities of the workers.

2. The method according to claim 1, characterized in that, The steps of pre-constructing the target library comprise the following steps: a monitoring video file or a real-time monitoring video stream containing workers is obtained, and decoding and frame extraction are performed to obtain a sequence frame; pedestrian detection, pedestrian re-identification and pedestrian tracking are performed on the sequence frame to obtain a gait image sequence and a gait contour sequence of each target, and a target feature set is formed; one target is one worker; based on a worker information library, the identities of the workers are bound with the target feature sets of each target to obtain a target library comprising target feature sets bound with the identities of the workers.

3. The method according to claim 2, wherein, The pedestrian detection, pedestrian re-identification and pedestrian tracking on the sequence frame comprise the following steps: a pedestrian target detection model is used to perform pedestrian detection on the obtained sequence frame to detect the position of a pedestrian target frame containing a pedestrian target, and a pedestrian image is cut from the sequence frame image according to the position of the pedestrian target frame; pedestrian re-identification features are extracted from the pedestrian image; the extracted pedestrian re-identification features are matched with the pedestrian re-identification features in the previous frame to track the pedestrian; based on the pedestrian tracking, pedestrian images of the pedestrian target in consecutive n frames are obtained as a gait image sequence; n is a preset threshold, and n∈N+; human body segmentation is performed on the pedestrian image in each frame of the gait image sequence, and the segmentation result is normalized and binarized to obtain a gait contour graph, and gait contour graphs of consecutive n frames are obtained as a gait contour sequence; the gait image sequence and the gait contour sequence of each pedestrian target are taken as the target feature set of the pedestrian target.

4. The method according to claim 3, characterized in that, The pedestrian target detection model adopts a yolov5 model.

5. The method according to claim 3, characterized in that, The pedestrian re-identification features are extracted from the pedestrian image, which comprises the following steps: The OSNet model is used to extract pedestrian re-identification features.

6. The method according to claim 3, characterized in that, The pedestrian image in each frame of the gait image sequence is subjected to human body segmentation, including: The pedestrian image is subjected to human body segmentation by using a semantic segmentation model Fast-SCNN.

7. The method according to claim 3, characterized in that, The identity of the worker is bound to the target feature set of each target by using the worker information library, including: The face is extracted from the gait image sequence of the target feature set of each target by using a face detection algorithm. The face library in the worker information library is taken as a base library to perform face recognition on the face extracted from the gait image sequence. If the recognition is successful, the corresponding worker identity in the worker information library is bound to the target feature set corresponding to the face, and the face in the worker information library and the face extracted from the gait image sequence are added to the face library of the target library. If the recognition is not successful or the face cannot be detected, the worker identity is confirmed and bound by viewing the gait video generated by the gait image sequence of the target feature set, and the face of the worker in the worker information library is added to the face library of the target library; all the target feature sets and the face library of the bound identity constitute the target library.

8. The method according to claim 7, characterized in that, The face detection algorithm uses a face detection model MTCNN.

9. The method according to claim 7, characterized in that, The face image extracted from the gait image sequence is subjected to face recognition, including: The face image is subjected to face recognition by using a face recognition model MobileFaceNet.

10. The method according to claim 1, characterized in that, A gait recognition model GaitSet is used to extract gait features.

11. The method according to claim 1 or 2, characterized in that, The monitoring video file or real-time monitoring video stream is decoded and frame-extracted to obtain sequence frames, including: The video is frame-extracted by using an open source tool OpenCV to obtain sequence frames.

12. A field worker identity feature fusion recognition system for implementing the field worker identity feature fusion recognition method according to any one of claims 1 to 10, characterized in that, It includes: A video decoding unit is configured to decode and frame-extract the monitoring video file or real-time monitoring video stream to obtain sequence frames and cache them; A feature extraction unit is configured to perform pedestrian detection, pedestrian re-identification and pedestrian tracking on the cached sequence frames, extract gait image sequences and gait contour sequences of pedestrian targets, and construct target feature sets; A matching unit is configured to match the target feature set of the object to be identified with a target library constructed in advance to identify the identity of the object to be identified; The target library includes target feature sets and face libraries of workers bound with worker identities in advance; The matching unit includes: The face recognition unit is configured to extract a face from the gait image sequence of the target feature set of the to-be-identified object by a face detection algorithm; take the face library in the target library as a base library, calculate the similarity between the face of the to-be-identified object and the faces of all the workers in the target library, if the maximum similarity is greater than a preset threshold Th f , the to-be-identified object is identified as the worker with the maximum similarity and cached. The gait recognition unit is configured to take the gait profile sequence in the target library as a base library, calculate the similarity between the gait feature of the object to be recognized and the gait feature of all workers in the target library, and if the maximum similarity is greater than a preset threshold Th g , the object to be recognized is identified as the worker with the maximum similarity and is cached. A pedestrian re-identification unit is configured to take pedestrian re-identification features extracted from gait image sequences in the target library as a base library, calculate the similarity between the pedestrian re-identification features of the object to be identified and the pedestrian re-identification features of all workers in the target library, and sort them, assign different weights to the top five in the similarity ranking, add the similarities of the same workers in the target library, and finally obtain the comparison result, take the worker in the target library ranked first as the identity recognition result of the object to be identified and cache it; the pedestrian re-identification features of the object to be identified include: extracting head and shoulder features and global features of the object to be identified, and fusing the two types of features with different weights as the pedestrian re-identification features of the object to be identified.

13. The live operations personnel identity characteristic fusion recognition system of claim 12, wherein, The feature extraction unit includes: The pedestrian detection unit is configured to perform pedestrian detection on the sequence frames buffered by the video decoding unit, detect a position of a pedestrian target frame containing a pedestrian target, and cut a pedestrian image from the sequence frame image according to the position of the pedestrian target frame and buffer the pedestrian image. The pedestrian tracking unit is configured to extract a pedestrian re-identification feature from the pedestrian image buffered by the pedestrian detection unit, and match the pedestrian re-identification feature with a pedestrian re-identification feature in a previous frame to track the same pedestrian target. The gait segmentation unit is configured to obtain pedestrian images of the pedestrian target in consecutive n frames as a gait image sequence based on pedestrian tracking, n is a preset threshold, n∈N+, and perform human body segmentation on the pedestrian image in each frame of the gait image sequence, normalize and binarize the segmentation result to obtain a gait contour map, obtain gait contour maps of the consecutive n frames as a gait contour sequence and buffer the gait contour sequence.

14. The live operations personnel identity characteristic fusion recognition system of claim 12, wherein, Further comprising: The personnel binding unit is configured to bind the identity of the work personnel in the work personnel information library with the target feature set of the work personnel obtained in advance to form a target library.

Images