A robust spatiotemporal hybrid gait feature learning method and system

Through the mixed gait feature learning method of time and space, the human body is divided into parts that are susceptible to and not easily affected by dress changes, and combined with multiple subnets to extract robust features, the problem of performance degradation of existing gait recognition algorithms under clothing changes is solved, and the performance degradation of existing gait recognition algorithms is achieved. More robustness and adaptability.

CN114882593BActive Publication Date: 2024-12-06XIAMEN MEIYA PICO INFORMATION CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210551600.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-18
Publication Date
2024-12-06
Estimated Expiration
2042-05-18

AI Technical Summary

Technical Problem

Existing gait recognition algorithms have significantly deteriorated their recognition performance under the influence of external factors (such as clothing changes), making it difficult to maintain robustness in the real world.

Method used

A robust space-time hybrid gait feature learning method is proposed, by segmenting the human body into parts that are susceptible to dress changes and parts that are not susceptible to dress changes, and using local spatial feature extraction networks, global spatial feature extraction networks and continuous Gait dynamic feature extraction network within short time intervals extracts robust space-time mixed gait features, combined with horizontal pyramid matching projection to more discriminant feature spaces.

Benefits of technology

It significantly improves the robustness and adaptability of gait recognition, can effectively suppress the impact of changes in pedestrian dress on recognition performance, and shows stronger robustness and adaptability, and the verification effect is performed on the public gait recognition dataset. Output significant gain.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114882593B_ABST
    Figure CN114882593B_ABST
Patent Text Reader

Abstract

The present invention provides a robust spatiotemporal hybrid gait feature learning method and system, including for a set of input gait frame sequences, dividing the human body in each frame into a part that is susceptible to clothing changes and a part that is not susceptible to clothing changes, the part that is not susceptible to clothing changes includes the head and the calf; for the part that is not susceptible to clothing changes, gait features are extracted respectively through three different sub-networks, and the three extracted gait features are cascaded as the gait features for the final gait recognition, the three different sub-networks include a local space feature extraction network, a global space feature extraction network and a gait dynamic feature extraction network within a continuous short time interval. Each sub-network depicts different aspects of the gait feature, and the series mixing can synthesize a more robust gait feature. The method and system have a significant gain effect when dealing with the gait recognition problem of pedestrian clothing changes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer image analysis, and in particular to a robust spatiotemporal hybrid gait feature learning method and system. Background Art

[0002] In recent years, pedestrian attributes have been a major research topic in intelligent surveillance video systems and have received widespread application attention. In recent years, the public's demand for social security has continued to grow. Since each person's biometric features (such as face, iris, fingerprint, gait, etc.) have unique characteristics, the use of biometrics for pedestrian recognition has recently become the first choice for most social security surveillance systems. However, in the real world, pedestrian images captured by surveillance cameras may be affected by factors such as low image resolution and insufficient lighting conditions. At the same time, the monitored subjects can also cover up or destroy their external biometric features by wearing masks, glasses, or gloves. Therefore, a biometric feature that can show stronger robustness and adaptability to these internal and external factors will become more popular in real-world surveillance systems; and gait is a biometric feature with strong robustness and adaptability.

[0003] Due to the unique biological characteristics of gait in the field of pedestrian recognition, gait recognition has attracted widespread attention in recent years. Although various gait recognition methods have been proposed, most methods can only achieve good recognition performance when the internal and external conditions are similar. Once external factors (such as changes in clothing) affect people's gait, the recognition performance of these methods will be significantly degraded. Summary of the invention

[0004] In order to solve the technical problem in the prior art that the recognition performance of the gait recognition algorithm will suffer significant performance degradation when external factors (such as changes in clothing) affect people's gait, the present invention proposes a robust spatiotemporal hybrid gait feature learning method and system to solve the above technical problem.

[0005] According to one aspect of the present invention, a robust spatiotemporal hybrid gait feature learning method comprises:

[0006] S1: For a set of input gait frame sequences, the human body in each frame is divided into a part that is susceptible to clothing changes and a part that is not susceptible to clothing changes;

[0007] S2: For the parts that are not easily affected by clothing changes, gait features are extracted through three different sub-networks respectively, and the three extracted gait features are cascaded as the final gait features for gait recognition.

[0008] In some specific embodiments, the parts that are not easily affected by clothing changes include the head and the calf. Compared with other body parts, the upper body of each person is more susceptible to clothing changes. Therefore, the body parts that are not easily affected by clothing changes in this application mainly refer to two parts, the head and the calf.

[0009] In some specific embodiments, the three different sub-networks include a local spatial feature extraction network, a global spatial feature extraction network, and a gait dynamic feature extraction network within continuous short time intervals.

[0010] In some specific embodiments, the convolution modules in the local spatial feature extraction network are placed in parallel to form three convolution branches and three parallel sub-networks. Information is exchanged between any two sub-networks through upsampling and downsampling, and the extracted features of each frame are mapped into features of the entire sequence through pooling operations.

[0011] In some specific embodiments, the global spatial feature extraction network includes two branch networks for processing the head and calf parts respectively, each branch network includes three convolution modules, each convolution module includes two convolution layers and one pooling layer, and after all the convolution modules, the extracted features of each frame are mapped into the features of the entire sequence through a pooling operation, and the features of the head and calf parts are cascaded as global spatial features.

[0012] In some specific embodiments, the gait dynamic feature extraction network within continuous short time intervals extracts the required gait features within continuous short time intervals from the segmented calf part, and the gait dynamic features within continuous short time intervals are extracted through a sliding window model with two window sizes on the frames of the short time intervals.

[0013] In some specific embodiments, the mapped sequence features are projected into a more discriminative feature space through horizontal pyramid matching.

[0014] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any of the above methods is implemented.

[0015] According to a third aspect of the present invention, a robust spatiotemporal hybrid gait feature learning system is proposed, the system comprising:

[0016] A gait frame sequence segmentation unit is configured to segment the human body of each frame into a part susceptible to clothing changes and a part not susceptible to clothing changes for a set of input gait frame sequences, wherein the part not susceptible to clothing changes includes a head and a calf part;

[0017] Gait feature extraction unit: It is configured to extract gait features of the parts that are not easily affected by clothing changes through three different sub-networks, and cascade the three extracted gait features as the final gait features for gait recognition. The three different sub-networks include a local space feature extraction network, a global space feature extraction network, and a gait dynamic feature extraction network within a continuous short time interval.

[0018] In some specific embodiments, the convolution modules in the local spatial feature extraction network are placed in parallel to form three convolution branches and three parallel sub-networks. Information is exchanged between any two sub-networks through upsampling and downsampling, and a pooling operation is used to map the extracted features of each frame into the features of the entire sequence.

[0019] In some specific embodiments, the global spatial feature extraction network includes two branch networks for processing the head and calf parts respectively, each branch network includes three convolution modules, each convolution module includes two convolution layers and one pooling layer, and after all the convolution modules, the extracted features of each frame are mapped into the features of the entire sequence through the pooling operation, and the features of the head and calf parts are cascaded as global spatial features.

[0020] In some specific embodiments, the gait dynamic feature extraction network within continuous short time intervals extracts the required gait features within continuous short time intervals from the segmented calf part, and the gait dynamic features within continuous short time intervals are extracted through a sliding window model with two window sizes on the frames of the short time intervals.

[0021] In some specific embodiments, the mapped sequence features are projected into a more discriminative feature space through horizontal pyramid matching.

[0022] In order to suppress the adverse effects of changes in pedestrian clothing on gait recognition, the present invention proposes a robust spatiotemporal hybrid gait feature learning method and system. First, the human body is segmented into parts that are susceptible to changes in clothing and parts that are not susceptible to changes in clothing. Then, for the segmented body parts that are not susceptible, a deep learning network is proposed to extract robust spatiotemporal hybrid gait features. This network contains three independent sub-networks, each of which describes different aspects of gait features, so their series mixing can synthesize a more robust gait feature. For example, since local features have stronger discrimination ability than global features in the field of gait recognition, a sub-network is specially designed in this application to extract gait features with local fine differences. At the same time, since dynamic information can be used to improve the performance of gait recognition, a sub-network is also specially designed in this network to extract gait dynamic features within continuous short time intervals. The effectiveness of this application has been verified on two public gait recognition data sets. Compared with other gait recognition methods, the method proposed in this application has a significant gain effect when dealing with the gait recognition problem of pedestrian clothing changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0024] Figure 1 is a flowchart of a robust spatiotemporal hybrid gait feature learning method according to an embodiment of the present application;

[0025] Figure 2 It is a framework diagram of a robust spatiotemporal hybrid gait feature learning method of a specific embodiment of the present application;

[0026] Figure 3 It is a local space feature extraction network structure diagram of a specific embodiment of the present application;

[0027] Figure 4 is a network structure diagram of gait dynamic feature extraction within a continuous short time interval in a specific embodiment of the present application;

[0028] Figure 5 is a framework diagram of a robust spatiotemporal hybrid gait feature learning system according to an embodiment of the present application;

[0029] Figure 6It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] According to a robust spatiotemporal hybrid gait feature learning method of one embodiment of the present application, Figure 1 FIG. 4 is a flowchart of a robust spatiotemporal hybrid gait feature learning method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0033] S101: For a set of input gait frame sequences, the human body in each frame is segmented into a part that is easily affected by clothing changes and a part that is not easily affected by clothing changes.

[0034] In a specific embodiment, the human body of each frame is first segmented into a part susceptible to changes in clothing and a part not susceptible to changes in clothing according to human anatomy knowledge. For a human body of height H, it can be divided into several parts according to the positions of some key points, for example, knee (0.285H), pelvis (0.48H), waist (0.535H), neck (0.87H). In addition, considering that the design and production of clothing are not always completely consistent with human anatomy knowledge, the present application relaxes the position restrictions of these key points when segmenting the human body, resulting in the body part that is not susceptible to changes in clothing obtained by the final segmentation being larger than ideally. In addition, compared to other body parts, the upper body part of each person is more susceptible to changes in clothing. Therefore, in the present application, the body part that is not susceptible to changes in clothing mainly refers to two parts, the head and the calf part.

[0035] S102: For the part that is not easily affected by the change of clothing, gait features are extracted through three different sub-networks respectively, and the three extracted gait features are cascaded as the gait features for the final gait recognition. The three different sub-networks include a local space feature extraction network, a global space feature extraction network, and a gait dynamic feature extraction network within a continuous short time interval.

[0036] In a specific embodiment, the present application proposes a deep learning network to extract robust spatiotemporal hybrid gait features for the segmented body parts that are less susceptible. This network contains three independent sub-networks, and each sub-network characterizes different aspects of the gait features. For example, since local features have stronger discrimination capabilities than global features in the field of gait recognition, the present application specifically designs a sub-network to extract and retain local gait features with obvious local fine differences. At the same time, since gait motion information can be used to improve the performance of gait recognition, a sub-network is specially designed in this network to extract gait dynamic features within continuous short time intervals. Relevant experiments show that compared with other gait recognition methods, the method proposed in this application has a significant gain effect when dealing with the gait recognition problem of pedestrian clothing changes.

[0037] In a specific embodiment, Figure 2 A framework diagram of a robust spatiotemporal hybrid gait feature learning method according to a specific embodiment of the present application is shown, as Figure 2 As shown, the overall framework includes the following process: for a set of input gait frame sequences, the present application first divides the human body in each frame into parts that are susceptible to clothing changes and parts that are not susceptible to clothing changes based on human anatomical knowledge. For the parts of the body that are not susceptible to clothing changes, the present application designs three different sub-networks to extract gait features with strong robustness. Finally, the three extracted gait-related features are cascaded as gait features for the final gait recognition.

[0038] In a specific embodiment, Figure 3 FIG. 4 shows a local spatial feature extraction network structure diagram according to a specific embodiment of the present application, such as Figure 3 As shown, unlike most gait recognition networks that simply rely on superimposed convolution modules to increase network depth to capture gait features, the convolution modules in the network of this application are placed in parallel. In addition, only gait features captured in the shallow layer will be used for subsequent gait recognition tasks. In this application, the first convolution unit is used as the first convolution branch, and more convolution branches are gradually created by adding more convolution units. Each time a convolution branch is created, a subnetwork is constructed. These subnetworks are arranged in parallel, and finally three convolution branches and three parallel subnetworks are formed. In this subnetwork, the feature scale will gradually decrease with the continuous increase of convolution branches, but these feature scales will remain unchanged in each subsequent subnetwork.

[0039] In a specific embodiment, in view of the fact that the local spatial feature extraction network structure can learn multi-scale gait recognition features, the network also establishes an information exchange mechanism to enhance the robustness of the extracted features. Information is exchanged between any two sub-networks by upsampling and downsampling. Through this information exchange mechanism, the network can integrate more robust gait features. On the one hand, by adding more convolution branches, more robust global features are formed in the network; at the same time, the use of the information exchange mechanism makes the local refined differences captured by the shallow branches more robust. On the other hand, the spatial perception clues of the local area are well transmitted in each sub-network, which can guide the deeper convolution branches to capture more effective global semantic features. Through this information exchange mechanism, the network forms a virtuous cycle of mutual utilization and mutual promotion between sub-networks, thereby also improving the robustness of the extracted gait features.

[0040] In a specific embodiment, the local spatial feature extraction network structure uses a pooling operation to map the extracted features of each frame into the features of the entire sequence. At the same time, after the horizontal pyramid matching HPM, the mapped sequence features are projected into a more discriminative feature space. Since local features have stronger discriminative power than global features in the field of gait recognition, only the two features extracted from the shallowest layer are used in the network.

[0041] In a specific embodiment, the global spatial feature network consists of two branches, which are intended to process the segmented head part and the calf part respectively. Each branch includes three convolution modules, and each convolution module consists of two convolution layers and a pooling layer. After all the convolution modules, a pooling operation is used to map the extracted features of each frame into the features of the entire sequence. At the same time, the mapped sequence features are projected into a more discriminative feature space through HPM. Finally, the extracted head and calf part features are cascaded and used as the required global spatial features.

[0042] In a specific embodiment, Figure 4 FIG. 4 shows a network structure diagram of gait dynamic feature extraction within a continuous short time interval according to a specific embodiment of the present application, such as Figure 4As shown, since the calf part can provide more gait movement clues, the present application mainly extracts the required gait dynamic features within continuous short time intervals from the segmented calf part. A continuous gait sequence is taken as input. The subnetwork first processes each frame of the input image independently. Then, a special module is designed to capture the gait dynamic features within continuous short time intervals. In this network, the gait dynamic features within continuous short time intervals are extracted by applying maximum pooling operations on frames with short time intervals. These maximum pooling operations can be regarded as a sliding window model. In addition, in order to mix multi-scale gait dynamic features, the present application adopts two window sizes. Finally, similarly, the pooling operation and HPM are used to enhance the information of the captured gait dynamic features.

[0043] Continue to refer Figure 5 , Figure 5 The framework diagram of the robust spatiotemporal hybrid gait feature learning system according to the embodiment of the present application is shown. The system specifically comprises a gait frame sequence segmentation unit 501 and a gait feature extraction unit 502 .

[0044] In a specific embodiment, the gait frame sequence segmentation unit 501 is configured to segment the human body in each frame of a set of input gait frame sequences into a part that is susceptible to clothing changes and a part that is not susceptible to clothing changes, and the part that is not susceptible to clothing changes includes the head and the calf.

[0045] In a specific embodiment, the gait feature extraction unit 502 is configured to extract gait features of the parts that are not easily affected by changes in clothing through three different sub-networks, and cascade the three extracted gait features as the final gait features for gait recognition. The three different sub-networks include a local space feature extraction network, a global space feature extraction network, and a gait dynamic feature extraction network within a continuous short time interval.

[0046] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing an electronic device of an embodiment of the present application. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0047] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0048] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0049] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0050] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0051] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0052] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0053] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: for a set of input gait frame sequences, the human body of each frame is divided into a part that is susceptible to changes in clothing and a part that is not susceptible to changes in clothing; for the part that is not susceptible to changes in clothing, gait features are extracted respectively through three different sub-networks, and the three extracted gait features are cascaded as the gait features for the final gait recognition.

[0054] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A robust spatiotemporal hybrid gait feature learning method, characterized in that: include: S1: for a set of input gait frame sequences, the human body in each frame is segmented into a part that is susceptible to clothing changes and a part that is not susceptible to clothing changes, wherein the part that is not susceptible to clothing changes includes the head and the calf; S2: For the part that is not easily affected by changes in clothing, gait features are extracted respectively through three different sub-networks, and the three extracted gait features are cascaded as the gait features for final gait recognition. The three different sub-networks include a local spatial feature extraction network, a global spatial feature extraction network, and a gait dynamic feature extraction network within a continuous short time interval. The convolution modules in the local spatial feature extraction network are placed in parallel to form three convolution branches and three parallel sub-networks. Information is exchanged between any two sub-networks through upsampling and downsampling. The extracted features of each frame are mapped into features of the entire sequence through pooling operations.

2. The robust spatiotemporal hybrid gait feature learning method according to claim 1, characterized in that: The global spatial feature extraction network includes two branch networks for processing the head and the calf respectively, each of the branch networks includes three convolution modules, each of the convolution modules includes two convolution layers and one pooling layer, after all the convolution modules, the extracted features of each frame are mapped into the features of the entire sequence through the pooling operation, and the features of the head and the calf are cascaded as global spatial features.

3. The robust spatiotemporal hybrid gait feature learning method according to claim 1, characterized in that: The gait dynamic feature extraction network within the continuous short time interval extracts the required gait features within the continuous short time interval from the segmented calf part, and the gait dynamic features within the continuous short time interval are extracted through a sliding window model with two window sizes on the frames of the short time interval.

4. The robust spatiotemporal hybrid gait feature learning method according to any one of claims 1 to 3, characterized in that: After horizontal pyramid matching, the mapped sequence features are projected into a more discriminative feature space.

5. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 4 is implemented.

6. A robust spatiotemporal hybrid gait feature learning system, characterized in that: The system comprises: A gait frame sequence segmentation unit is configured to segment the human body of each frame into a part susceptible to clothing changes and a part not susceptible to clothing changes for a set of input gait frame sequences, wherein the part not susceptible to clothing changes includes a head and a calf part; Gait feature extraction unit: configured to extract gait features of the part that is not easily affected by clothing changes through three different sub-networks, and cascade the three extracted gait features as the gait features for final gait recognition. The three different sub-networks include a local space feature extraction network, a global space feature extraction network, and a gait dynamic feature extraction network within a continuous short time interval. The convolution modules in the local space feature extraction network are placed in parallel to form three convolution branches and three parallel sub-networks. Information is exchanged between any two sub-networks through upsampling and downsampling. The extracted features of each frame are mapped into features of the entire sequence through pooling operations.

7. The robust spatiotemporal hybrid gait feature learning system according to claim 6, characterized in that: The global spatial feature extraction network includes two branch networks for processing the head and the calf respectively, each of the branch networks includes three convolution modules, each of the convolution modules includes two convolution layers and one pooling layer, after all the convolution modules, the extracted features of each frame are mapped into the features of the entire sequence through the pooling operation, and the features of the head and the calf are cascaded as global spatial features.

8. The robust spatiotemporal hybrid gait feature learning system according to claim 6, characterized in that: The gait dynamic feature extraction network within the continuous short time interval extracts the required gait features within the continuous short time interval from the segmented calf part, and the gait dynamic features within the continuous short time interval are extracted through a sliding window model with two window sizes on the frames of the short time interval.

9. The robust spatiotemporal hybrid gait feature learning system according to any one of claims 6-8, characterized in that: After horizontal pyramid matching, the mapped sequence features are projected into a more discriminative feature space.

Citation Information

Patent Citations

  • Multi-mode pedestrian identity recognition method and system based on pedestrian appearance and gait information

    CN111860291A