A gait recognition sequence acquisition method and system based on deep learning
Through the deep learning-based gait recognition sequence acquisition method, the problems of target detection error detection and static target capture in the prior art are solved. Through pedestrian target detection, motion target detection and quality verification, the accuracy of gait recognition is improved.
Patent Information
- Application Number
- CN202111203193.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Among the existing gait recognition technology, object detection and tracking technology are prone to false detection, static targets are frequently captured, and the output detection targets do not include or the gait information is incomplete, which affects the accuracy of gait recognition.
The method of gait recognition sequence acquisition based on deep learning includes collecting pedestrian sample data, training pedestrian object detection model, collecting data to be identified for object detection and tracking, performing motion object detection and gait sequence screening, and each frame of image is Perform pedestrian quality verification and output gait recognition sequence.
Effectively filter out invalid gait frame data and static targets, ensuring the rationality and effectiveness of gait sequences, greatly improving the accuracy of gait recognition.
Smart Images

Figure CN113963437B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning, and in particular to a method and system for acquiring a gait recognition sequence based on deep learning. Background Art
[0002] Gait is a biometric feature. Compared with other biometric features such as irises, palm prints, and veins, gait has significant advantages such as non-contact, remote detectability, non-invasiveness, and difficulty in hiding. It can be widely used in security, transportation, industry and other industries. In the existing gait recognition technology, the pedestrian target image sequence is first generated through target detection and tracking technology, and then the sequence is segmented to finally obtain the gait recognition sequence. At present, the existing detection and tracking technology is prone to target misdetection and frequent capture of static targets. When applied to gait recognition scenarios, the output detection target may not include gait information or the gait information may be incomplete, which seriously affects the accuracy of gait recognition. Summary of the invention
[0003] In view of the above problems, the present invention is proposed to provide a method and system for acquiring gait recognition sequences based on deep learning, which overcome the above problems or at least partially solve the above problems.
[0004] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:
[0005] A method for acquiring a gait recognition sequence based on deep learning, comprising:
[0006] S100. Collect pedestrian sample data suitable for gait recognition scenarios, where the pedestrian sample data includes positive samples and negative samples;
[0007] S200. Train a pedestrian target detection model;
[0008] S300. Collecting data to be identified;
[0009] S400. Performing target detection on the identified data;
[0010] S500. Tracking the detected target;
[0011] S600. Perform moving target detection on the tracking output;
[0012] S700. Screening the gait sequence to be segmented;
[0013] S800. Perform pedestrian quality verification on each frame of the segmented sequence;
[0014] S900. Output gait recognition sequence.
[0015] Furthermore, in S100, pedestrian sample data suitable for gait recognition scenarios is collected, specifically including:
[0016] S101. Collect positive sample data of pedestrian detection in video surveillance scenes and classify and label them;
[0017] S102. Collect negative sample data for human detection in video surveillance scenarios and classify and label them.
[0018] Furthermore, in S200, training a pedestrian target detection model specifically includes:
[0019] S201. The collected sample data is made into a VOC data set format;
[0020] S202. Train the pedestrian target detection model through the yolov5 framework.
[0021] Furthermore, in S300, collecting the data to be identified specifically includes:
[0022] S301. By collecting local video files as offline data to be identified;
[0023] S302. By collecting real-time video data from surveillance cameras as real-time data to be identified;
[0024] S303. Collect historical video data from surveillance cameras as video data to be identified.
[0025] Further, in S400, target detection is performed on the data to be identified, specifically including:
[0026] S401. Loading data to be identified;
[0027] S402. Load the trained pedestrian target detection algorithm model based on yolov5;
[0028] S403. Detecting the data to be identified by a pedestrian target detection algorithm;
[0029] S404. Return the test result.
[0030] Furthermore, in S500, tracking the detected target specifically includes:
[0031] S501. Input the test result output by the test link;
[0032] S502. Target tracking is performed by kcf algorithm;
[0033] S503. Return the tracking result.
[0034] Further, in S600, motion target detection is performed on the output of tracking, specifically including:
[0035] S601. Input the result returned by KCF tracking;
[0036] S602. Obtain the video frame input data information;
[0037] S603. Obtain the video foreground image;
[0038] S604. Obtain the target box data;
[0039] S605. Obtain the number of foreground points contained in the target box;
[0040] S606. Obtain the foreground percentage in the target box;
[0041] S607. Return the motion target detection result.
[0042] Further, in S700, the gait sequences to be segmented are screened, specifically including:
[0043] S701. Input the pedestrian sequence output after motion target detection, where the number of sequences is S;
[0044] S702. Set the minimum number of gait sequences N and the maximum number of gait sequences M;
[0045] S703. Determine the selected sequence index number, set the start index SI = 0, the end index EI = 0. If S > M, then SI = (S - M) / 2, EI = SI + (M / 2) - 1; if S < M, then SI = 0, EI = S - 1;
[0046] S704. Determine the initial sequence L0 to be segmented according to the selected index number;
[0047] S705. Set the aspect ratio threshold of each frame of image data as T;
[0048] S706. Calculate the aspect ratio of each frame of data in the segmentation sequence L0 in turn, compare it with the threshold T, and discard those less than T;
[0049] S707. Output the gait sequences to be segmented.
[0050] Further, in S800, pedestrian quality verification is performed on each frame of the sequence to be segmented, specifically including:
[0051] S801. Load the gait sequences to be segmented;
[0052] S802. Select a target K with the highest confidence in the frame of the sequence, and set the score as score1;
[0053] S803. Load the offline trained person re-identification ReID model;
[0054] S804. Calculate pedestrian features of target K;
[0055] S805. Calculate pedestrian features of other frame data targets in the input sequence in sequence through the loaded ReID model;
[0056] S806. Set the target comparison threshold to L;
[0057] S807. Calculate the similarity between target K and other frame data targets;
[0058] S808. Compare the calculated similarity with the threshold L, and discard the sequences that are less than L;
[0059] S809. Output the gait sequence to be recognized.
[0060] The present invention also discloses a gait recognition sequence acquisition system based on deep learning, comprising: a pedestrian target detection model training module, a moving target detection module for data to be identified, and a gait recognition sequence output module; wherein:
[0061] The pedestrian target detection model training module is used to collect pedestrian sample data suitable for gait recognition scenarios and train them through the yolov5 framework to form a pedestrian target detection model;
[0062] The moving target detection module of the data to be identified is used to collect the data to be identified; perform target detection on the data to be identified, track the detected target, and perform moving target detection on the tracked output;
[0063] The gait recognition sequence output module is used to screen the gait sequence to be segmented; to verify the pedestrian quality of each frame image of the segmented sequence; and to output the gait recognition sequence.
[0064] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:
[0065] The present invention discloses a method for acquiring a gait recognition sequence based on deep learning, comprising: collecting pedestrian sample data suitable for gait recognition scenarios, the pedestrian sample data including positive samples and negative samples; training a pedestrian target detection model; collecting data to be recognized; performing target detection on the data to be recognized; tracking the detected target; performing motion target detection on the output of the tracking; screening the gait sequence to be segmented; performing pedestrian quality verification on each frame image of the segmented sequence; and outputting a gait recognition sequence. The present invention discloses a target detection method and system suitable for gait recognition scenarios, which effectively filters out invalid gait frame data; the present invention performs motion target detection on the tracking output and filters out stationary targets; the present invention screens the gait sequence to be segmented and verifies the pedestrian quality, thereby ensuring the rationality and effectiveness of the gait sequence and greatly improving the accuracy of gait recognition.
[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0068] Figure 1 This is a flowchart of a method for acquiring a gait recognition sequence based on deep learning in Embodiment 1 of the present invention;
[0069] Figure 2 This is a flow chart of target detection model training in Example 1 of the present invention;
[0070] Figure 3 This is a flow chart of moving target detection in Embodiment 1 of the present invention;
[0071] Figure 4 This is a flowchart of pedestrian quality verification in Example 1 of the present invention. DETAILED DESCRIPTION
[0072] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0073] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a method and system for acquiring a gait recognition sequence based on deep learning.
[0074] Example 1
[0075] This embodiment discloses a method for acquiring a gait recognition sequence based on deep learning, such as Figure 1 ,include:
[0076] S100. Collecting pedestrian sample data applicable to gait recognition scenarios, where the pedestrian sample data includes positive samples and negative samples; specifically, in this embodiment S100, collecting pedestrian sample data applicable to gait recognition scenarios specifically includes:
[0077] S101. Collect positive sample data of pedestrian detection in video surveillance scenes and classify and label them;
[0078] S102. Collect negative sample data for human detection in video surveillance scenarios and classify and label them.
[0079] S200. Train a pedestrian target detection model; specifically, Figure 2 In this embodiment S200, training a pedestrian target detection model specifically includes:
[0080] S201. The collected sample data is made into a VOC data set format;
[0081] S202. Train the pedestrian target detection model through the yolov5 framework.
[0082] S300. Collecting data to be identified; Specifically, in this embodiment S300, collecting data to be identified specifically includes:
[0083] S301. By collecting local video files as offline data to be identified;
[0084] S302. By collecting real-time video data from surveillance cameras as real-time data to be identified;
[0085] S303. Collect historical video data from surveillance cameras as video data to be identified.
[0086] S400. Target detection is performed on the identified data; Specifically, in the present embodiment S400, target detection is performed on the identified data, specifically including:
[0087] S401. Loading data to be identified;
[0088] S402. Load the trained pedestrian target detection algorithm model based on yolov5;
[0089] S403. Detecting the data to be identified by a pedestrian target detection algorithm;
[0090] S404. Return the test result.
[0091] S500. Track the detected target; specifically, in this embodiment S500, tracking the detected target specifically includes:
[0092] S501. Input the detection result output by the detection link;
[0093] S502. Perform target tracking through the KCF algorithm;
[0094] S503. Return the tracking result.
[0095] S600. Perform moving target detection on the output of tracking; specifically, in this embodiment S600, performing moving target detection on the output of tracking is specifically as Figure 3 , including:
[0096] S601. Input the result returned by KCF tracking;
[0097] S602. Obtain the video frame input data information;
[0098] S603. Obtain the video foreground image;
[0099] S604. Obtain the target box data;
[0100] S605. Obtain the number of foreground points contained in the target box;
[0101] S606. Obtain the foreground percentage in the target box;
[0102] S607. Return the moving target detection result.
[0103] S700. Screen the gait sequence to be segmented; specifically, in this embodiment S700, screening the gait sequence to be segmented specifically includes:
[0104] S701. Input the pedestrian sequence output after moving target detection, where the number of sequences is S;
[0105] S702. Set the minimum number of gait sequences N and the maximum number of gait sequences M;
[0106] S703. Determine the selected sequence index number, set the start index SI = 0, the end index EI = 0. If S > M, then SI = (S - M) / 2, EI = SI + (M / 2) - 1; if S < M, then SI = 0, EI = S - 1;
[0107] S704. Determine the initial sequence L0 to be segmented according to the selected index number;
[0108] S705. Set the aspect ratio threshold of each frame of image data as T;
[0109] S706. Calculate the aspect ratio of each frame of data in the segmentation sequence L0 in turn, compare it with the threshold T, and discard those that are less than T;
[0110] S707. Output the gait sequence to be segmented.
[0111] S800. Perform pedestrian quality verification on each frame of the segmented sequence; Specifically, in the present embodiment S800, perform pedestrian quality verification on each frame of the segmented sequence, specifically as Figure 4 ,include:
[0112] S801. Loading the gait sequence to be segmented;
[0113] S802. Select a target K with the highest frame target confidence in the sequence, and set the score to score1;
[0114] S803. Load the offline trained person re-identification ReID model;
[0115] S804. Calculate pedestrian features of target K;
[0116] S805. Calculate pedestrian features of other frame data targets in the input sequence in sequence through the loaded ReID model;
[0117] S806. Set the target comparison threshold to L;
[0118] S807. Calculate the similarity between target K and other frame data targets;
[0119] S808. Compare the calculated similarity with the threshold L, and discard the sequences that are less than L;
[0120] S809. Output the gait sequence to be recognized.
[0121] S900. Output gait recognition sequence.
[0122] This embodiment also discloses a gait recognition sequence acquisition system based on deep learning, including: a pedestrian target detection model training module, a moving target detection module for to-be-recognized data, and a gait recognition sequence output module; wherein:
[0123] The pedestrian target detection model training module is used to collect pedestrian sample data suitable for gait recognition scenarios and train them through the yolov5 framework to form a pedestrian target detection model;
[0124] The moving target detection module of the data to be identified is used to collect the data to be identified; perform target detection on the data to be identified, track the detected target, and perform moving target detection on the tracked output;
[0125] The gait recognition sequence output module is used to screen the gait sequence to be segmented; to verify the pedestrian quality of each frame image of the segmented sequence; and to output the gait recognition sequence.
[0126] Among them, the specific workflows of the pedestrian target detection model training module, the moving target detection module for data to be identified, and the gait recognition sequence output module disclosed in this implementation have been described in detail in the above-mentioned deep learning-based gait recognition sequence acquisition method, and will not be repeated here.
[0127] The present invention discloses a method for acquiring a gait recognition sequence based on deep learning, comprising: collecting pedestrian sample data suitable for gait recognition scenarios, the pedestrian sample data including positive samples and negative samples; training a pedestrian target detection model; collecting data to be recognized; performing target detection on the data to be recognized; tracking the detected target; performing motion target detection on the output of the tracking; screening the gait sequence to be segmented; performing pedestrian quality verification on each frame image of the segmented sequence; and outputting a gait recognition sequence. The present invention discloses a target detection method and system suitable for gait recognition scenarios, which effectively filters out invalid gait frame data; the present invention performs motion target detection on the tracking output and filters out stationary targets; the present invention screens the gait sequence to be segmented and verifies the pedestrian quality, thereby ensuring the rationality and effectiveness of the gait sequence and greatly improving the accuracy of gait recognition.
[0128] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.
[0129] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
[0130] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein can all be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above around their functions. Whether such functions are implemented as hardware or software depends on specific applications and the design constraints imposed on the entire system. A skilled person can implement the described functions in an alternative manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of the present disclosure.
[0131] The steps of the method or algorithm described in conjunction with the embodiments herein may be directly embodied as hardware, a software module executed by a processor, or a combination thereof. The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a mobile disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and the storage medium may also be present in a user terminal as discrete components.
[0132] For software implementation, the techniques described in this application can be implemented with modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or outside the processor. In the latter case, it is coupled to the processor in a communication manner via various means, which are well known in the art.
[0133] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".
Claims
1. A method for obtaining gait recognition sequences based on deep learning, characterized in that, it includes: S100. Collect pedestrian sample data suitable for the gait recognition scenario, where the pedestrian sample data includes positive samples and negative samples; S200. Train a pedestrian target detection model; S300. Collect data to be recognized; S400. Perform target detection on the data to be recognized; S500. Track the detected targets; S600. Perform moving target detection on the output of the tracking; S700. Screen the gait sequences to be segmented; In S700, screening the gait sequences to be segmented specifically includes: S701. Input the pedestrian sequence output after moving target detection, where the number of sequences is S; S702. Set the minimum number of gait sequences N and the maximum number of gait sequences M; S703. Determine the selected sequence index number, set the start index SI = 0, the end index EI = 0. If S > M, then SI = (S - M) / 2, EI = SI + (M / 2) - 1; If S < M, then SI = 0, EI = S - 1; S704. Determine the initial sequence L0 to be segmented according to the selected index number; S705. Set the aspect ratio threshold of each frame of image data as T; S706. Calculate the aspect ratio of each frame of data in the segmentation sequence L0 in turn, and compare it with the threshold T. Those less than T are discarded; S707. Output the gait sequences to be segmented; S800. Perform pedestrian quality verification on each frame of the sequence to be segmented; S900. Output the gait recognition sequences.
2. A method for obtaining gait recognition sequences based on deep learning as described in claim 1, characterized in that, in S100, collecting pedestrian sample data suitable for the gait recognition scenario specifically includes: S101. Collect positive sample data for pedestrian detection in the video surveillance scenario and perform classification annotation; S102. Collect negative sample data for pedestrian detection in the video surveillance scenario and perform classification annotation.
3. A method for obtaining gait recognition sequences based on deep learning as described in claim 1, characterized in that, in S200, training the pedestrian target detection model specifically includes: S201. Make the collected sample data into the format of the VOC dataset; S202. Train the pedestrian target detection model through the yolov5 framework.
4. A method for obtaining gait recognition sequences based on deep learning as described in claim 1, characterized in that, in S300, collecting data to be recognized specifically includes: S301. Collect local video files as offline data to be recognized; S302. Collect real-time video data from surveillance cameras as real-time data to be recognized; S303. Collect historical video data from surveillance cameras as video data to be recognized.
5. A method for obtaining gait recognition sequences based on deep learning as described in claim 1, characterized in that, in S400, performing target detection on the data to be recognized specifically includes: S401. Load the data to be recognized; S402. Load the trained pedestrian target detection algorithm model based on yolov5; S403. Detecting the data to be identified by a pedestrian target detection algorithm; S404. Return the test result.
6. A method for acquiring a gait recognition sequence based on deep learning as claimed in claim 1, It is characterized in that In S500, the detected target is tracked, which specifically includes: S501. Input the test result output by the test link; S502. Target tracking is performed by kcf algorithm; S503. Return the tracking result.
7. The method for acquiring a gait recognition sequence based on deep learning as claimed in claim 1, It is characterized in that In S600, moving target detection is performed on the tracking output, specifically including: S601. Input the result returned by kcf tracking; S602. Obtain video frame input data information; S603. Obtaining a video foreground image; S604. Obtain target frame data; S605. Obtain the number of foreground points contained in the target frame; S606. Obtaining the foreground percentage in the target frame; S607. Return the moving target detection result.
8. The method for acquiring a gait recognition sequence based on deep learning as claimed in claim 1, It is characterized in that In S800, pedestrian quality verification is performed on each frame of the segmented sequence, specifically including: S801. Loading the gait sequence to be segmented; S802. Select a target K with the highest frame target confidence in the sequence, and set the score to score1; S803. Load the offline trained person re-identification ReID model; S804. Calculate pedestrian features of target K; S805. Calculate pedestrian features of other frame data targets in the input sequence in sequence through the loaded ReID model; S806. Set the target comparison threshold to L; S807. Calculate the similarity between target K and other frame data targets; S808. Compare the calculated similarity with the threshold L, and discard the sequences that are less than L; S809. Output the gait sequence to be recognized.
9. A gait recognition sequence acquisition system based on deep learning, It is characterized in that include: Pedestrian target detection model training module, moving target detection module for data to be identified, gait recognition sequence output module; among which: The pedestrian target detection model training module is used to collect pedestrian sample data suitable for gait recognition scenarios and train them through the yolov5 framework to form a pedestrian target detection model; The moving target detection module of the data to be identified is used to collect the data to be identified; perform target detection on the data to be identified, track the detected target, and perform moving target detection on the tracked output; The gait recognition sequence output module is used to screen the gait sequence to be segmented; perform pedestrian quality verification on each frame image of the segmented sequence; and output the gait recognition sequence; wherein the screening of the gait sequence to be segmented specifically includes: S701. Input the pedestrian sequence output after moving target detection, where the number of sequences is S; S702. Set the minimum gait sequence number N and the maximum gait sequence number M; S703. Determine the selection sequence index number, set the start index SI = 0, the end index EI = 0. If S > M, then SI = (S - M) / 2, EI = SI + (M / 2) - 1; if S < M, then SI = 0, EI = S - 1; S704. Determine the initial sequence L0 to be segmented according to the selected index number; S705. Set the aspect ratio threshold of each frame of image data as T; S706. Calculate the aspect ratio of each frame of data in the segmentation sequence L0 in turn, compare it with the threshold T, and discard those less than T; S707. Output the gait sequence to be segmented.
Citation Information
Patent Citations
Gait-based identification method, apparatus, and electronic device
CN109271888A
T gait cycle detection method based on a convolutional neural network
CN109766838A
A gait recognition method based on an extreme learning machine
CN109784206A
Gait recognition method based on hierarchical real-time memory
CN109858351A
Suspicious person cross-lens tracking method and device based on spatial constraint
CN110706259A
