Pedestrian body and shadow combined gait recognition method based on fixed camera

Abstract

The invention discloses a pedestrian body and shadow combined gait recognition method based on a fixed camera. The method comprises the following steps: obtaining a stable video background scene; obtaining a pedestrian gait and a pedestrian shadow gait foreground through background subtraction; respectively obtaining a pedestrian body gait binary image sequence and a shadow gait binary image sequence in the pedestrian gait foreground and the pedestrian shadow gait foreground obtained by the fixed camera; performing centroid normalization to generate a binary image sequence of pedestrian body gaits and shadow gaits; aligning according to a gait period, and generating a pedestrian body gait energy diagram and a shadow gait energy diagram of each pedestrian sequence; two-dimensional principal component dimensionality reduction is carried out on the two gait energy diagrams, and feature vectors after dimensionality reduction are fused at a feature level to generate new gait recognition features; and carrying out comparison identification by using a nearest neighbor classifier according to the new identification features. According to the method, the pedestrian shadow gait and the pedestrian entity gait under the fixed camera are combined for recognition, and a better recognition effect is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of biometric feature recognition, and in particular relates to a method for joint gait recognition of a pedestrian body and a shadow based on a fixed camera. Background Art

[0002] Biometric identification technology is a technology that collects and processes the physiological or behavioral characteristics inherent in the human body through computers to identify individuals. Physiological characteristics mainly include facial images, irises, ears, fingerprints, palm prints, etc., among which fingerprint and facial feature recognition are more widely used. Behavioral characteristics mainly include signatures, voices, gait, etc. Physiological characteristics are mostly innate characteristics, while behavioral characteristics are mostly acquired living habits. Due to the uniqueness of the gait biological characteristics of behavioral characteristics, it has been increasingly valued by researchers in the past thirty years. Figure 1 It is the basic process of gait recognition. Compared with other biometric identity recognition technologies, gait biometrics has the following advantages:

[0003] 1. Non-invasiveness: Human gait can be detected without being noticed by the person being tested, unlike biometric technologies such as fingerprints or irises, which require certain contact and interaction with the person being tested. Therefore, it is particularly suitable for covert implementation occasions, such as crime detection in criminal investigation technology;

[0004] 2. Long-distance recognition: A person's gait can be captured at a long distance. In long-distance monitoring, although a person's face can be captured at a certain distance, it is strongly restricted by distance and angle, and has high requirements for the quality and angle of the shooting equipment. Gait recognition technology is based on the physiological fact that a person's gait is a periodic signal, and has relatively low requirements for angle and distance.

[0005] 3. Low requirements for image clarity: Unlike facial features, gait features can still be well extracted in video sequences with lower resolution;

[0006] 4. Difficult to hide: Fingerprints can be eliminated or forged by wearing gloves or stealing fingerprints, and faces can be hidden by masking, disguise, etc., but gait is difficult to hide or disguise. It must be done during living movement. The more disguised, the easier it is for people to find the flaws.

[0007] The existing urban monitoring system usually uses fixed cameras to capture pedestrian gait. In the presence of sunlight or other light sources, the captured images have shadows. Because the cross-sectional area of ​​the pedestrian is small, the features are not prominent. However, the shadow can form an enlarged "silhouette" with different angles of the light source, forming a shadow gait; how to use the pedestrian shadow gait and the pedestrian entity gait for gait recognition can be used as a new application technology.

[0008] The pedestrian body and shadow gait based on fixed cameras provide another perspective for gait recognition. This difference leads to the inconsistency in the technical route between pedestrian gait recognition based on fixed cameras and traditional gait recognition identity features. The essence of pedestrian gait recognition technology based on fixed cameras is the extension of gait recognition technology. From the perspective, it changes from a side view to a top view, so traditional gait recognition technology can be used as a reference. However, pedestrian gait recognition based on fixed cameras explains the gait recognition problem from a top view. The change in angle makes the gait form different from the normal silhouette and has its own characteristics. Pedestrian gait recognition based on fixed cameras has a person's entity gait and shadow gait in the presence of sunlight. How to integrate these two gaits to improve the recognition rate and robustness of gait identity recognition is also a new and difficult problem. Summary of the invention

[0009] The main purpose of the present invention is to provide a joint gait recognition method for pedestrian bodies and shadows based on a fixed camera. The method combines the gait of pedestrian shadows captured by a fixed camera with the gait of pedestrian entities for recognition, and utilizes the silhouette gait features formed by the shadow to achieve better gait recognition under overhead conditions.

[0010] The technical solution adopted by the present invention is:

[0011] A method for joint gait recognition of pedestrians and their shadows based on a fixed camera, comprising the following steps:

[0012] 1) Using mixed Gaussian method and other methods to obtain a stable video scene background scene in the video stream obtained by a fixed camera;

[0013] 2) Obtain the pedestrian gait and pedestrian shadow gait foreground through background subtraction;

[0014] 3) Obtain the binary image sequences of the pedestrian's gait and shadow's gait in the foreground of the pedestrian's gait and the pedestrian's shadow's gait respectively from the fixed camera;

[0015] 4) Perform morphological corrosion and dilation operations on the binary image sequences of pedestrian gait and shadow gait to eliminate holes; the corrosion and dilation operations are used to fill the holes of pedestrian gait and shadow gait, so as to obtain a good quality image sequence of pedestrian gait and shadow gait;

[0016] 5) Normalizing the centroid of the binary image sequences of the pedestrian's gait and shadow's gait to generate centroid-normalized binary image sequences of the pedestrian's gait and shadow's gait;

[0017] 6) Align the centroided binary images of the pedestrian's gait and shadow's gait according to the gait cycle, and generate the pedestrian's gait energy map and shadow's gait energy map for each pedestrian sequence;

[0018] 7) The pedestrian's main gait energy map and the shadow gait energy map are respectively reduced in dimension using a two-dimensional principal component analysis algorithm, and the reduced feature vectors are fused at the feature level to generate new gait features;

[0019] 8) Based on the newly generated gait features, the feature vector obtained after dimensionality reduction is used as the recognition feature; the recognition feature is compared and recognized with the samples in the pedestrian gait library by using the nearest neighbor classifier.

[0020] A further solution is that in step 3), the pedestrian's gait and shadow's gait binary images can be manually annotated as Groud-truth data, or other methods such as background subtraction can be used to obtain the pedestrian's gait and shadow's gait, and then image segmentation can be performed to separate the pedestrian's gait and shadow's gait.

[0021] A further solution is that in step 6), the method for generating the pedestrian's main body gait energy map and the shadow gait energy map is: after obtaining all the centroid normalized images of all pedestrians' gait cycles, they are superimposed and averaged according to the definition of the gait energy map to obtain the gait energy map.

[0022] A further solution is that in step 7), the method of fusing the pedestrian's main body gait energy map and the shadow gait energy map at the feature level is: using 2D-PCA to reduce the dimension of the pedestrian's main body gait energy map and the shadow gait energy map, and using the fused feature components after dimension reduction as gait features.

[0023] A further solution is that in step 8), the newly generated gait feature, that is, the feature vector obtained after fusion and dimensionality reduction is used as the recognition feature; the method of comparing and identifying the recognition feature with the samples in the pedestrian gait library using the nearest neighbor classifier is as follows: the feature dimension of the fused component of the human body gait energy map and the shadow gait energy map after 2D-PCA dimensionality reduction is used as the similarity measure, the Euclidean distance is used as the standard unit, the distance between the fused components is directly compared, and the closest sample in the pedestrian gait library is selected as the recognition matching pair to achieve gait recognition.

[0024] A further solution is that in step 6), the pedestrian's main body gait energy map or the shadow gait energy map is used for identification, so as to be applicable to an environment where no shadow is formed.

[0025] The technical terms used in the present invention are:

[0026] 1. Mathematical representation of gait energy graph

[0027] The gait energy imag (GEI) is the superposition mean average of a periodic sequence of binary gait images. It overcomes the disadvantage of large data volume of gait image sequence, saves time and space, reduces the influence of noise, and has high recognition rate and robustness. For a gait sequence with a gait cycle of N frames, the gait image is actually the average silhouette image of all gait images, which is similar to the concept of average face in face recognition. Its mathematical representation is as follows:

[0028]

[0029] Where (x, y) is the coordinate of the pixel in the image; B t (x, y) refers to the binary image of the tth frame; the gait energy map is ultimately obtained as a grayscale image, which represents the gait information superimposed in a cycle, saving space and improving computational efficiency.

[0030] The gait energy map has another advantage, it reflects the gait changes of a cycle, so for gait at any angle and direction, the periodic signals can be superimposed together to eliminate the influence of noise, and the application is relatively simple. The disadvantage is that it has high requirements for the pre-processing results of the image sequence, and it is necessary to obtain a high-quality gait contour binary map.

[0031] 2. Two-dimensional principal component analysis algorithm

[0032] The two-dimensional principal component analysis algorithm is abbreviated as 2D-PCA (Two dimentional principle component analysis, 2D-PCA), also known as the generalized principal component analysis algorithm. It is an algorithm developed on the basis of the 1DPCA algorithm. 1DPCA first converts the image matrix into an image vector before projection, which requires a relatively high dimension and complex calculations. However, 2D-PCA directly uses the image matrix as the analysis object. It does not need to construct the image covariance matrix after dimensionality reduction like 1DPCA. It directly uses the image matrix to construct the covariance matrix and then performs dimensionality reduction, so it has higher computational efficiency and accuracy. It is easy to implement and has been widely used. Its mathematical representation is as follows:

[0033] Y=AX (2)

[0034] Where X is the projection matrix and Y is the projection eigenvector of the image matrix A. The quality of the optimal projection axis X is measured according to the total discreteness criterion function of the eigenvector Y, which is expressed as follows:

[0035] J(X)=argmin(tr{S}) (3)

[0036]

[0037] G=E[(A-EA) T (A-EA)] (5)

[0038] Among them, S represents the covariance matrix of the projected eigenvector Y, represents the trace of S, is a non-negative positive definite matrix of size n×n, called the covariance matrix of the image, and its purpose is to maximize the sample inter-class scatter matrix of the eigenvector obtained after projecting all training samples onto the projection axis, that is, the overall dispersion of the eigenvectors obtained after the matrix is ​​projected on the axis is maximized.

[0039] 2D-PCA is generally used to reduce the dimension according to the size and distribution of the eigenvalues. In general, after the eigenvalues ​​are arranged by size, the front eigenvalues ​​have a larger total weight, and the coefficient is the proportion of the eigenvalue in the entire eigenvalue.

[0040] The two-dimensional principal component analysis algorithm is easy to implement and is a good tool for dimensionality reduction. The purpose of the present invention is to verify the feasibility of gait identification, and there will be noise in the detected shadows, so it is used as a tool to reduce the dimension of the gait energy map, so as to technically reduce the influence of noise, avoid comparative operations on the entire image, and improve calculation efficiency.

[0041] 3. Algorithm Classifier

[0042] The Nearest Neighbour Classifier (NN) is the most commonly used classifier in pattern recognition. It directly compares the distance between the feature dimensions of the unknown sample and all the training samples as the similarity standard, and attributes the closest sample to the category to which it belongs. It is the most intuitive display of the difference between classes, does not require prior knowledge, and the algorithm is simple and intuitive. It is the most commonly used method in unsupervised learning. The present invention uses the feature dimension of the component of the gait energy graph after 2D-PCA dimension reduction as the similarity measure, adopts the Euclidean distance as the standard unit, directly compares the distance between the components after the sample dimension reduction, and selects the closest sample as the identification matching pair (leave one detection method). Better classifiers have better effects.

[0043]

[0044] The beneficial effects of the present invention are:

[0045] The present invention combines the pedestrian's main gait and the pedestrian's shadow gait to perform identity recognition, thereby improving the recognition rate of gait identity recognition;

[0046] It can recognize gait, filling the gap that gait recognition can only be performed using standard silhouettes, and can effectively utilize information under shadow conditions;

[0047] The present invention extracts the gait data of a pedestrian in a gait cycle and synthesizes a gait energy graph, and uses the two-dimensional PCA dimension reduction method and the leave-one-out detection method for recognition; the experimental results show that the pedestrian's main gait tends to be stable after the 9th dimension, and the recognition rate remains at 88.24%. The pedestrian's shadow gait tends to be stable after the 12th dimension, and the gait recognition rate reaches up to 96.08%.

[0048] The present invention proposes a method for identifying pedestrian body and shadow gait by fusing fixed cameras. The fixed camera fusion of pedestrian body gait and shadow gait tends to be stable in the 7th dimension, and the recognition rate is stable at 96.08%, indicating that under certain specific conditions, shadow gait can assist pedestrian body gait recognition, expand the amount of information for gait recognition, be more robust in recognition, and obtain better results in fewer feature dimensions;

[0049] Under certain specific conditions, the shadow gait of pedestrians can provide a higher recognition rate of the pedestrian's main gait. The fusion of shadow gait and real gait at the feature level can improve the recognition effect and increase the gait recognition rate. At the same time, it can form a practical crime-solving technology in criminal investigation and improve the applicability of gait recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 It is a schematic diagram of the basic process of gait recognition;

[0052] Figure 2 This is a flow chart of the joint gait recognition method of pedestrian body and shadow based on a fixed camera;

[0053] Figure 3 This is an example of manually annotated shadows;

[0054] Figure 4 is an example of a shadow gait;

[0055] Figure 5 is an example of morphological post-processing; Figure 5 (a) is the original shadow gait; Figure 5 (b) is the shadow gait after morphological post-processing;

[0056] Figure 6 is the normalized center of mass of the shadow gait; Figure 6 (a) is the original shadow gait; Figure 6 (b) is the centroidal gait;

[0057] Figure 7 is the shadow gait graph;

[0058] Figure 8 is the relationship between the correct recognition rate and dimension of the ideal pedestrian shadow gait;

[0059] Fig. 9 is the pedestrian's gait sequence;

[0060] Fig.10 is the gait energy graph of the pedestrian entity;

[0061] Fig.11 is the correct recognition rate of pedestrian entity gait;

[0062] Fig.12 It is the correct recognition rate of the fusion of pedestrian body and shadow gait features. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0064] See also Figure 2 , a joint gait recognition method of pedestrian body and shadow based on a fixed camera, comprising the following steps:

[0065] 1) Using the mixed background method to obtain a stable video scene background in the video stream obtained by a fixed camera;

[0066] 2) Obtain the pedestrian gait and pedestrian shadow gait foreground by background subtraction in the video sequence acquired under a fixed camera;

[0067] 3) Obtain the binary images of the pedestrian's gait and shadow's gait respectively in the foreground of the pedestrian's gait and the pedestrian's shadow's gait obtained by the fixed camera; mainly manually annotate the binary images of the pedestrian's gait and shadow's gait as Groud-truth data, and other shadow detection algorithms can also be used to detect shadows, and then segment the pedestrian's gait;

[0068] 4) Perform morphological corrosion and dilation operations on the binary images of the pedestrian's gait and shadow gait to eliminate holes; the corrosion and dilation operations are used to fill the pedestrian's gait and shadow gait, and a gait shadow image with better quality can be obtained;

[0069] 5) Normalizing the centroid of the binary image sequences of the pedestrian's gait and shadow's gait to generate binary image sequences of the pedestrian's gait and shadow's gait;

[0070] 6) Centroiding the binary image sequences of the pedestrian's main gait and shadow gait and aligning them according to the gait cycle to generate the pedestrian's main gait energy map and shadow gait energy map of each pedestrian sequence; specifically: after obtaining all centroid normalized images of all gaits in one cycle, superimposing and averaging them according to the definition of the gait energy map to obtain the gait energy map;

[0071] 7) The feature vectors of the pedestrian's gait energy map and the shadow's gait energy map after dimension reduction are fused at the feature level to generate new gait features; 2D-PCA is mainly used to reduce the dimension of the pedestrian's gait energy map and the shadow's gait energy map, and the fused feature components after dimension reduction are used as gait features;

[0072] 8) The feature vector of the newly generated gait feature is used as the recognition feature; the recognition feature is compared and recognized with the samples in the pedestrian gait library by the nearest neighbor classifier, mainly including: the feature dimension of the fused component of the pedestrian's main gait energy map and the shadow gait energy map after 2D-PCA dimensionality reduction is used as the similarity measure, the Euclidean distance is used as the standard unit, the distance between the components after dimensionality reduction is directly compared, and the sample with the closest distance in the gait recognition sample library is selected as the recognition matching pair to realize gait recognition.

[0073] Specific examples:

[0074] 1. Pedestrian shadow gait recognition based on fixed camera

[0075] Pedestrian shadow gait recognition based on fixed cameras is generally divided into two parts: pedestrian shadow gait energy map acquisition and shadow gait recognition.

[0076] 1) Obtaining shadow gait energy graph

[0077] The acquisition of shadow gait energy map is divided into three parts: pedestrian shadow gait detection, pedestrian shadow gait morphological post-processing, and gait energy map generation.

[0078] (1) Shadow gait detection

[0079] For each sample gait sequence, the size is 350×700, and the gait data of a gait cycle are manually annotated with the shadows of these gaits as Groud-truth data, see Figure 3 .

[0080] (2) Post-processing of shadow gait morphology

[0081] Figure 4The shadow gait data detected by the shadow detection method proposed in this embodiment has some holes in the inner area due to the influence of different materials in the illumination and shadow area. In the field of gait recognition, these holes are generally post-processed by mathematical morphology to obtain a gait image with a full inner area and smooth outer edges. This embodiment uses two corrosion and dilation operations to fill the shadow gait and obtain a gait shadow image with better quality.

[0082] Figure 5 (a) is a frame of shadow gait detected. Figure 5 (b) is the post-processing after two corrosion and dilation operations with the structure [0 1 0; 1 1 1; 01 0]. The edges of the post-processed image are clearer, the internal holes are much reduced, and the shadow gait is significantly improved.

[0083] (3) Gait energy map generation

[0084] There are two main steps in the generation of gait energy map. First, the shadow gait binary image is normalized, and then a dynamic image sequence is generated based on the normalized shadow result. Image centroiding is the basis for the generation of gait energy map, that is, the centroids of all shadow gaits are at the same point, so that the periodic gait energy map can be accurately described. The object positioning of this embodiment is to calculate the centroid coordinates of the shadow binary image according to the following formula based on the detected gait shadow binary image.

[0085]

[0086] In formula (7), C is the shadow part of the binary image. Then the centroid of this image is translated to the center of the new image, as Figure 7 shown.

[0087] After all the centroid normalized images of all the gait cycles are obtained, they are superimposed and averaged according to the definition of the gait energy map to obtain the gait energy map. Figure 7 The figure shows the three-cycle gait sequences of 17 pedestrians used in the experiment, a group of 51 gait energy graphs, and a total of 17 sample gait shadow graphs with a size of 300×750. Figure 7 It can be seen that the shadow gait and the silhouette gait are extremely similar. All the gait details of a cycle are integrated into one image, which saves space and improves the efficiency of subsequent operations. The different forward and backward angles, heights, weights, and swing directions and amplitudes of each pedestrian when walking are all reflected in detail in the shadow gait energy map, and can therefore be used as identification features.

[0088] 2) Pedestrian shadow gait recognition

[0089] In order to verify the feasibility of pedestrian shadow gait recognition based on a fixed camera, under the premise that the shadow detection method of this embodiment detects the shadow, 2D-PCA is used to reduce the dimension of the gait energy graph, and the feature components after dimensionality reduction are used as gait features. For the 51 gait sequence samples of 17 people participating in the experiment of this embodiment, the leave-one-out cross-validation method suitable for small data samples is used to perform gait recognition test. Each time, a gait energy graph generated by a pedestrian gait sequence is taken as a test sample, and the gait energy graphs generated by the remaining fifty gait sequences are taken as training samples. Each time, the distance between the test sequence and the training sample in the dimensionality reduction feature space is compared, and the sample with the smallest distance (or several samples with the smallest distance) is taken to compare whether the test sample and a sample in the closest sample set belong to the gait sequence of the same pedestrian (the condition that the three gait sequences of the same pedestrian are of the same category is a known priori condition). Because the goal of this study is identity recognition, the correct recognition rate indicator is used to evaluate the recognition performance.

[0090] Figure 8 is the correct recognition rate relationship of the ideal pedestrian shadow gait when the gait energy graph is reduced to 1-35 dimensional components, from Figure 8 It can be seen that the gait recognition rate of the ideal pedestrian shadow gait increases with the increase of dimension, indicating that the increase of dimension has an impact on the recognition effect. However, the ideal shadow gait reaches its maximum value and tends to be stable at 12 dimensions. This is because the projection energy of the 2D-PCA method in space tends to be concentrated in the first few components, and the latter components do not contribute much to the recognition rate. The gait recognition rate of the ideal pedestrian shadow gait reaches up to 96.08%, showing that the pedestrian shadow gait recognition under the condition of fixed camera overlooking has practical discriminability.

[0091] 2. Pedestrian entity gait recognition based on fixed camera

[0092] The pedestrian entity gait based on a fixed camera is a pedestrian entity walking sequence captured in a bird's-eye view, such as Fig. 9 As shown in the figure, in essence, it is also a reflection of walking gait from a perspective, which is equivalent to the orthographic projection of pedestrians on the ground when walking, and therefore also has periodic repetitive characteristics, such as the swing of hands and feet, so it can also be used as a basis for gait recognition. However, it is equivalent to the silhouette commonly used in gait recognition research. The gait shot from a bird's-eye view has the disadvantages of small cross-sectional area, unclear hand and foot movements, and difficult to obtain features.

[0093] This embodiment directly uses the cutout software to obtain the pedestrian entity gait sequence as the basic data for pedestrian entity gait recognition, so as to verify and evaluate the feasibility of pedestrian entity gait recognition.

[0094] Similar to the gait energy graph, the pedestrian entity gait energy graph is defined as the average of the gait binary images of the pedestrians walking in one cycle. The generation method is the same as that of the shadow gait energy graph. This embodiment uses the cutout software to obtain all the pedestrian entity images in a sequence cycle. After binarization, the average gait value in one cycle is calculated after centroid calculation. The pedestrian entity gait energy graph corresponding to the shadow gait energy graph of this patent is as follows: Fig.10 As shown in the figure, compared with the shadow gait energy map, the pedestrian entity gait energy map has a smaller area, a smaller swing range of the hands and feet, and provides less information. However, due to the different walking habits of each pedestrian, there are certain differences between the gait energy maps in the overhead view, so it is subjectively feasible to use it as the basis for gait recognition.

[0095] The 51 gait energy graphs of pedestrian entities corresponding to the pedestrian shadow gait were used for experimental verification using the leave-one-out cross-validation method. Fig.11 It is the correct recognition rate curve of the pedestrian entity gait when the gait image is reduced to 1-35 dimensions.

[0096] Fig.11 The correct recognition rate of the pedestrian entity gait increases with the increase of feature dimension, but there is a disturbance between the 5th and 8th dimensions, which shows that the energy graph of the pedestrian entity gait has little difference within the class after dimensionality reduction. This is closely related to the small area of ​​the energy graph of the pedestrian entity gait and the few available features. The performance is that the size of the 5th and 8th dimension components is not much different, which affects the recognition rate. However, it tends to be stable after the 9th dimension, and the recognition rate remains at 88.24%, which is less than 96.08% of the ideal pedestrian shadow gait, indicating that the pedestrian entity energy graph is not as good as the pedestrian shadow gait at this time. On the one hand, the area of ​​the pedestrian shadow is larger than that of the pedestrian entity, and the effect is more obvious. On the other hand, it is because the pedestrian is at an angle with the camera when walking, which leads to the deviation of the imaging projection, and the shadow is the projection of the pedestrian entity on the ground, and the sun's rays can be approximately regarded as parallel rays, so the projection of the shadow and the camera imaging angle are less affected. In general, the shadow gait has the property of being more stable than the pedestrian entity gait, especially when the shadow is longer.

[0097] 3. Pedestrian shadow and pedestrian entity joint gait recognition based on fixed camera

[0098] In view of the different weather conditions in actual situations, the pedestrian entity gait and pedestrian shadow gait based on fixed cameras can be flexibly used in various actual situations. In the presence of sunlight and in certain circumstances, the captured pedestrian video has two gaits, pedestrian and shadow. These two gaits can be regarded as the separate imaging of two cameras (fixed camera and sun). Therefore, from the perspective of multi-source data fusion, the fusion of the two gait recognitions can theoretically improve the recognition rate and robustness. In order to verify the joint recognition situation, this patent combines the pedestrian shadow gait and pedestrian entity gait data at the feature level for experimental verification. The specific method is to directly combine the feature components of the above two shadow gait energy maps after dimensionality reduction as new feature components, and also use the nearest neighbor principle for classification and recognition.

[0099] Fig.12 is a comparison of the above two fusion methods at different dimensionality reduction amounts. Overall, the fusion of pedestrian entity + ideal pedestrian shadow gait is stable at 96.08% in the 7th dimension. The fusion of pedestrian entity + ideal pedestrian shadow gait has the same highest correct recognition rate as the ideal pedestrian shadow gait, which is equivalent to the two pedestrians not being correctly identified, but is better than the highest recognition rate of 88.24% of pedestrian entity. However, the fused gait has reached stability in the 7th dimension, indicating that the method of fusing the two gait features can improve the gait recognition rate.

[0100] The gait recognition method of the present invention may be replaced by other periodic gait recognition algorithms besides the dynamic energy graph.

[0101] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A joint gait recognition method based on a fixed camera pedestrian body and shadow, characterized in that: The steps include: 1) Using the mixed Gaussian method to obtain a stable video background scene in the video stream obtained by a fixed camera; 2) Obtain the pedestrian gait and pedestrian shadow gait foreground through background subtraction; 3) Obtain the binary image sequences of the pedestrian's gait and shadow's gait in the foreground of the pedestrian's gait and the pedestrian's shadow's gait respectively from the fixed camera; 4) Perform morphological corrosion and dilation operations on the binary image sequences of pedestrian gait and shadow gait to eliminate holes; 5) Normalizing the centroid of the binary image sequences of the pedestrian's gait and shadow's gait to generate centroid-normalized binary image sequences of the pedestrian's gait and shadow's gait; 6) Centroiding the binary image sequences of the pedestrian's gait and shadow gait, and aligning the binary image sequences according to the gait cycle to generate the pedestrian's gait energy map and shadow gait energy map for each pedestrian sequence; 7) The pedestrian's main gait energy map and the shadow gait energy map are respectively reduced in dimension using a two-dimensional principal component analysis algorithm, and the reduced feature vectors are fused at the feature level to generate new gait features; 8) Based on the newly generated gait features, the feature vector obtained after dimensionality reduction is used as the recognition feature; the recognition feature is compared and recognized with the samples in the pedestrian gait library by using the nearest neighbor classifier.

2. The method for joint gait recognition based on fixed camera pedestrian body and shadow according to claim 1 is characterized in that: In step 3), the pedestrian's main gait and shadow gait binary image are manually annotated as Groud-truth data, or the pedestrian entity and shadow gait as a whole are obtained by background subtraction, and then automatic segmentation is performed to obtain the pedestrian's main gait and shadow gait.

3. The method for joint gait recognition based on fixed camera pedestrian body and shadow according to claim 1, characterized in that: In step 6), the method for generating the pedestrian body gait energy map and the shadow gait energy map is: after obtaining all the centroid normalized images of one cycle of gait of all pedestrians, they are superimposed and averaged according to the definition of the gait energy map to obtain the gait energy map.

4. The method for joint gait recognition of pedestrian body and shadow by fixed camera according to claim 1, characterized in that: In step 7), the method for fusing the pedestrian's main gait energy map and the shadow's gait energy map at the feature level is: using 2D-PCA to reduce the dimensions of the pedestrian's main gait energy map and the shadow's gait energy map respectively, and using the joint feature component after dimensionality reduction as the gait feature.

5. The method for joint gait recognition of pedestrian body and shadow by fixed camera according to claim 1, characterized in that: In step 8), the newly generated gait features are reduced in dimension using a two-dimensional principal component analysis algorithm, and the feature vector obtained after the dimension reduction is used as the recognition feature; the method of comparing and identifying the recognition features with the samples in the pedestrian gait library using the nearest neighbor classifier is as follows: the feature dimension after the feature components of the pedestrian's main body gait energy map and the shadow gait energy map after 2D-PCA dimension reduction are fused as the similarity measure, and the Euclidean distance is used as the standard unit. The distance between the components after dimension reduction is directly compared, and the sample with the closest distance in the pedestrian's main body and the shadow gait library is selected as the recognition matching pair to achieve gait recognition.

6. The method for joint gait recognition based on fixed camera pedestrian body and shadow according to claim 1, characterized in that: In step 7), the pedestrian's main body gait energy map or the shadow gait energy map is used for identification, so as to be applicable to the environment where no shadow is formed.

Images