A human gait recognition method and system based on surveillance video

Abstract

The present invention relates to the technical field of gait recognition, and in particular to a method and system for human gait recognition based on surveillance video. The method for human gait recognition based on surveillance video includes the following steps: obtaining a human target in surveillance video data and tracking the human target; extracting the body features of the human target; comparing the body features with the body features in the memory to obtain an initial target; judging whether the body features match; obtaining an enlarged image of the initial target and tracking the initial target; extracting the gait features of a cycle of image sequences; comparing the obtained gait features with the gait features in the memory to obtain a final target. First, the initial targets that meet the body features among the human targets are preliminarily screened by the body feature extraction method, which greatly reduces the recognition difficulty of directly obtaining a large number of target gait features and improves the reliability of recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of gait recognition, and in particular to a method and system for human gait recognition based on surveillance video. Background Art

[0002] Gait recognition technology uses a camera to acquire, detect, and segment data on the walking process of the target to be identified. That is, after visually detecting the entire walking process and completing a walking cycle, the data is extracted based on the features, and the gait data is input into the database to be compared for comparison to confirm the identity of the detected target. Gait recognition can achieve long-distance and cross-perspective recognition. There is a lot of information that can be extracted from the human body during movement. One type is internal features that can include physiological information such as height, head shape, leg bones, joints, etc.; the other type is the dynamic characteristics of human walking, including walking posture, arm swing amplitude, shoulder and head movement amplitude during walking, etc.

[0003] Current gait recognition technology requires obtaining a person's gait characteristics through a camera to identify the person's identity. However, the acquisition and analysis of gait characteristics involves a large number of algorithms. The results are obtained through comprehensive processing of multiple data. When faced with a large amount of data that needs to be analyzed, it is difficult to directly obtain a person's gait characteristics, and the accuracy and reliability of determining a person's identity are low. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a human gait recognition method and system based on surveillance video, which solves the problem that gait recognition requires a large amount of data to be analyzed, resulting in low accuracy and reliability in determining the identity of a person.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for human gait recognition based on surveillance video, comprising the following steps:

[0007] S1 obtains surveillance video data through the first monitoring device group, obtains the human target in the surveillance video data and tracks the human target;

[0008] S2. Extracting body features of human targets;

[0009] S3. The extracted body features of the human target are compared with the body features in the memory to obtain the initial target;

[0010] S4. Determine whether the body features match. If yes, execute step S5; if not, execute step S1;

[0011] S5. The surveillance video data acquisition path is switched from the first surveillance device group to the second surveillance device group, the second surveillance device group acquires surveillance video data, obtains a magnified image of the initial target and tracks the initial target;

[0012] S6. extracting gait features from a period of the image sequence to obtain the range of variation of the arm swing angle α and the stride angle β;

[0013] S7. Compare the obtained ranges of change of the arm swing angle α and the stride angle β with the gait features in the memory to obtain the final target and generate an alarm message.

[0014] Specifically, the first monitoring equipment group includes a first wide-angle camera and a second wide-angle camera, and the first wide-angle camera and the second wide-angle camera are symmetrically arranged. The first wide-angle camera obtains distant front monitoring video data, and the second wide-angle camera obtains distant back monitoring video data. An overlapping monitoring area is formed between the first wide-angle camera and the second wide-angle camera.

[0015] Specifically, the first monitoring device group is accompanied by a second monitoring device group, the second monitoring device group includes a first zoom camera and a second zoom camera, the first wide-angle camera is accompanied by the first zoom camera, and the second wide-angle camera is accompanied by the second zoom camera.

[0016] Specifically, in step S5, the surveillance video acquisition path is transferred from the first wide-angle camera and the second wide-angle camera to the first zoom camera and the second zoom camera, respectively. The first zoom camera and the second zoom camera track the initial target using a geometric zoom tracking method and obtain a magnified image of the initial target. The geometric zoom tracking method obtains an estimated value of the trajectory curve by linear interpolation based on the two trajectory curves of the near and far targets. The tracking position formula is:

[0017]

[0018] in, To estimate the location point, is the initial position point, is the focus motor position point when the zoom motor position is z at close object distance, is the focus motor position when the zoom motor is z at a long object distance, z init is the initial zoom motor position when the object distance is d.

[0019] Specifically, the focal lengths of the first wide-angle camera and the second wide-angle camera are set to F, the pixel width of the human target is set to P, the distance between the human target and the wide-angle camera is measured to be L, and the body features of the human target are extracted, including height h, shoulder width w, and leg length m. The calculation formulas for the height h, shoulder width w, and leg length m of the human target are respectively:

[0020] h=(P1*L) / F, w=(P2*L) / F, m=(P3*L) / F

[0021] P1 is the pixel width of the human body target height, P2 is the pixel width of the human body target shoulder width, and P3 is the pixel width of the human body target leg length.

[0022] Specifically, in step S6, the gait feature extraction method is a fusion feature recognition method, which includes a structured feature method and an unstructured feature method; the structured feature method simulates a structured model of a person walking by establishing a 2D or 3D model, and the pendulum model based on the leg stride action and the pendulum model based on the arm swing action conform to a certain angle change, the angle of the swing action is set to α, and the angle of the stride action is set to β. The swing action and the stride action have a certain periodicity, and the movement of each person conforms to a certain pattern, and structured feature extraction is performed. The unstructured feature method extracts the human body contour from the gait sequence, and expresses the gait feature by the information change of the connecting line between the points constituting the contour and the center. The wavelet packet transform is applied to this one-dimensional gait signal to extract the gait feature to enhance robustness.

[0023] Specifically, in step S6, the gait feature extraction method includes one or more of an inter-frame difference method or a background subtraction method. The inter-frame difference method extracts moving targets by combining brightness changes between continuous image sequences, and extracts the gait features by combining morphological and human body structure features. The background subtraction method uses the current frame image to compare with the background image, and defines the area with small difference as the background area and the area with large difference as the moving area, thereby subtracting the background and extracting only the gait features.

[0024] In the second aspect, the present invention also provides a human gait recognition system based on surveillance video, including a wide-angle image acquisition module, a zoom image acquisition module, a central control module, a wireless network transmission module, an imaging display module and an audio-visual alarm module. The wide-angle image acquisition module and the zoom image acquisition module are used to acquire image information and transmit the image information to the central control module. After the target information is identified and integrated by the central control module, the identification information is transmitted to the imaging display module through the wireless network transmission module. The target person information is displayed by the imaging display module, and the audio-visual alarm module issues an alarm at the same time.

[0025] Specifically, the central control module includes a data storage and an image processor. The image processor is used to process the acquired image information, and the data storage is used to store a gait feature library.

[0026] Specifically, the wireless network transmission module transmits information to the imaging display module and the sound and light alarm module through Ethernet. The imaging display module is a color display screen, and the sound and light alarm module includes a buzzer and an alarm light.

[0027] The present invention provides a method and system for human gait recognition based on surveillance video, which has the following beneficial effects:

[0028] 1. The present invention first extracts human targets that meet the body shape characteristics from the monitored human targets using a body shape feature extraction method to obtain an initial target. Then, the initial target is tracked using a zoom lens tracking method and its gait features are extracted. Finally, the gait features of the initial target are further analyzed and integrated to obtain the final target and generate an alarm. This allows the gait recognition process to be screened through body shape recognition, thereby reducing the number of human targets and obtaining the initial target. When performing gait recognition on the initial target, there is no need to deal with a large amount of data to analyze, reducing the difficulty of directly acquiring gait features from a large number of targets and improving recognition reliability.

[0029] 2. The present invention symmetrically arranges the first wide-angle camera and the second wide-angle camera in the monitoring equipment group so that the first wide-angle camera and the second wide-angle camera form overlapping monitoring areas, thereby enabling multi-directional monitoring of human targets, preventing human targets from being obscured by other human targets, and at the same time compensating for the single-direction monitoring and recognition, thereby further improving the recognition accuracy.

[0030] 3. By setting up the first zoom camera and the second zoom camera, the present invention can magnify the initial target, thereby making the initial target clearer, so that the variation range of the swing angle α and the stride angle β can be obtained more accurately, thereby improving the accuracy and reliability of gait recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the steps of the human gait recognition method based on surveillance video of the present invention;

[0032] Figure 2 Flowchart of the human gait recognition method based on surveillance video of the present invention;

[0033] Figure 3 A distribution map of monitoring areas of the present invention;

[0034] Figure 4 A schematic diagram of the body characteristics of a human target of the present invention;

[0035] Figure 5 A schematic diagram of gait characteristics of the initial target of the present invention;

[0036] Figure 6 This is a system diagram of the human gait recognition method based on surveillance video of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0038] Reference Figures 1 to 5 As shown, an embodiment of the present invention provides a method for human gait recognition based on surveillance video, comprising the following steps:

[0039] S1 obtains surveillance video data through the first monitoring device group, obtains human targets in the surveillance video data and tracks human targets;

[0040] S2. Extracting body features of human targets;

[0041] S3. The extracted body features of the human target are compared with the body features in the memory to obtain the initial target;

[0042] S4. Determine whether the body features match. If yes, proceed to step S5; if not, proceed to step S1;

[0043] S5. The surveillance video data acquisition path is switched from the first monitoring device group to the second monitoring device group, the second monitoring device group acquires surveillance video data, obtains a magnified image of the initial target and tracks the initial target;

[0044] S6. extracting gait features from a period of the image sequence to obtain the range of variation of the arm swing angle α and the stride angle β;

[0045] S7. Compare the obtained range of change of the arm swing angle α and the stride angle β with the gait characteristics in the memory to obtain the final target and generate an alarm message.

[0046] This embodiment first extracts human targets from the monitored population that meet the body shape characteristics using a posture feature extraction method to obtain an initial target. Then, using a zoom lens tracking method, the initial target is tracked and its gait features are extracted. Finally, the initial target's gait features are further analyzed and integrated to obtain the final target and generate an alarm. This divides the gait recognition process into a first screening step using posture recognition, thereby reducing the number of human targets and obtaining the initial target. Then, when performing gait recognition on the initial target, there is no need to deal with a large amount of data to analyze, reducing the difficulty of directly acquiring gait features from a large number of targets and improving recognition reliability.

[0047] Reference Figure 3 As shown, the first monitoring equipment group includes a first wide-angle camera and a second wide-angle camera. The first wide-angle camera and the second wide-angle camera are symmetrically arranged. The first wide-angle camera obtains distant front monitoring video data, and the second wide-angle camera obtains distant back monitoring video data. An overlapping monitoring area is formed between the first wide-angle camera and the second wide-angle camera.

[0048] By symmetrically positioning the first and second wide-angle cameras within the monitoring system, the overlapping monitoring areas of the first and second wide-angle cameras enable multi-directional monitoring of human subjects, preventing obstruction of human subjects by other human subjects. This also compensates for the limitations of single-directional monitoring and recognition, further improving recognition accuracy. Furthermore, since the wide-angle cameras have a viewing angle of over 120 degrees, the first and second wide-angle cameras can monitor a wider area.

[0049] Specifically, the first monitoring device group is accompanied by a second monitoring device group, the second monitoring device group includes a first zoom camera and a second zoom camera, the first wide-angle camera is accompanied by the first zoom camera, and the second wide-angle camera is accompanied by the second zoom camera.

[0050] By attaching a first zoom camera to the first wide-angle camera and a second zoom camera to the second wide-angle camera, the first zoom camera and the second zoom camera are symmetrically arranged. Thus, when the viewing angle of either zoom camera is blocked when the first zoom camera and the second zoom camera shoot the initial target, the other zoom camera can also obtain complete monitoring video data of the initial target.

[0051] Specifically, in step S5, the surveillance video acquisition path is transferred from the first wide-angle camera and the second wide-angle camera to the first zoom camera and the second zoom camera respectively. The first zoom camera and the second zoom camera track the initial target through the geometric zoom tracking method and obtain a magnified image of the initial target. The geometric zoom tracking method obtains an estimated value of the trajectory curve through linear interpolation based on the two trajectory curves of the near and far targets. The tracking position formula is:

[0052]

[0053] in, To estimate the location point, is the initial position point, is the focus motor position point when the zoom motor position is z at close object distance, is the focus motor position when the zoom motor is z at a long object distance, z init is the initial zoom motor position when the object distance is d.

[0054] By setting up the first zoom camera and the second zoom camera, the initial target can be tracked and magnified, making the initial target clearer, so that the variation range of the swing angle α and the stride angle β can be obtained more accurately, thereby improving the accuracy and reliability of gait recognition.

[0055] Specifically, the focal lengths of the first wide-angle camera and the second wide-angle camera are set to F, the pixel width of the human target is set to P, the distance between the human target and the first wide-angle camera or the second wide-angle camera is measured to be L, and the body features of the human target are extracted, including height h, shoulder width w, and leg length m. The calculation formulas for the height h, shoulder width w, and leg length m of the human target are respectively:

[0056] h=(P1*L) / F, w=(P2*L) / F, m=(P3*L) / F

[0057] P1 is the pixel width of the target human body height, P2 is the pixel width of the target human body shoulder width, and P3 is the pixel width of the target human body leg length.

[0058] Since the height, shoulder width and leg length of the human body are not the same, by extracting the body features of height, shoulder width and leg length, the range of human targets can be narrowed down, and the human targets with the same height, shoulder width and leg length can be used as the initial targets for further gait recognition. Because the range is narrowed and the body features of height, shoulder width and leg length have been extracted, the amount of data processing can be reduced during gait recognition.

[0059] For example, the focal length F of the wide-angle camera is 16 mm, the pixel width P1 of the human target's height is 64 MP, the distance L between the human target and the wide-angle camera is 40 m, and the calculated height h of the human target is 160 cm; the pixel width P2 of the human target's shoulder width is 16 MP, and the calculated shoulder width w of the human target is 40 cm; the pixel width P3 of the human target's leg length is 36 MP, and the calculated leg length m of the human target is 90 cm.

[0060] Specifically, in step S6, the gait feature extraction method is a fusion feature recognition method, which includes a structured feature method and an unstructured feature method. The structured feature method simulates a structured model of a person walking by establishing a 2D or 3D model. The pendulum model based on the leg stride action and the pendulum model based on the arm swing action conform to a certain angle change. The angle of the swing action is set to α, and the angle of the stride action is set to β. The swing action and the stride action have a certain periodicity. The movement of each person conforms to a certain pattern, and structured feature extraction is performed. The unstructured feature method extracts the human body contour from the gait sequence, and uses the information change of the connecting line between the points that make up the contour to the center to express the gait characteristics. Based on this one-dimensional gait signal, wavelet packet transform is applied to extract gait features to enhance robustness.

[0061] Specifically, in step S6, the gait feature extraction method includes one or more of the inter-frame difference method and the background subtraction method. The inter-frame difference method extracts the moving target by combining the brightness changes between continuous image sequences, and extracts the gait features by combining morphology and human body structure features; the background subtraction method uses the current frame image to compare with the background image, and defines the area with small difference as the background area and the area with large difference as the moving area, thereby subtracting the background and extracting only the gait features.

[0062] Specifically, in step S7, the range of the swing angle α and the stride angle β is compared with the gait features in the memory, and a linear classification algorithm can be selected. The linear classifier consists of a scoring function and a loss function. It performs classification through a linear combination of features. During the optimization process, the parameters of the scoring function are continuously changed to minimize the loss function value. The scoring function is a simple linear mapping, and the expression of the scoring function is:

[0063] f(x i , W, b) = Wx i +b

[0064] Among them, f is a function, x is a variable, W and b are the parameters of the function, W is usually called the weight, and b is usually called the bias.

[0065] By learning the input training data, appropriate parameters W and b are obtained so that the classification calculated for each training data point matches the true value of the image data in the training set. The loss function calculates the degree of discrepancy between the score calculated by the scoring function using given parameters and the true value of the data, thereby quantitatively representing the quality of the current parameters. When the loss function value is high, it indicates a large degree of discrepancy. In this case, the parameter values need to be adjusted appropriately based on the degree of discrepancy. The optimal parameters are found by continuously adjusting the parameter weight W and the bias b. The loss function can use the multi-class support vector machine loss function or the regularized loss function.

[0066] Reference Figure 6 As shown, an embodiment of the present invention also provides a human gait recognition system based on surveillance video, including a wide-angle image acquisition module, a zoom image acquisition module, a central control module, a wireless network transmission module, an imaging display module and an audio-visual alarm module. The wide-angle image acquisition module and the zoom image acquisition module are used to obtain image information and transmit the image information to the central control module. After the target information is identified and integrated by the central control module, the identification information is transmitted to the imaging display module through the wireless network transmission module. The target person information is displayed through the imaging display module, and the audio-visual alarm module issues an alarm.

[0067] Specifically, the wide-angle image acquisition module includes a wide-angle image acquisition module 1 and a wide-angle image acquisition module 2, and the zoom image acquisition module includes a zoom image acquisition module 1 and a zoom image acquisition module 2.

[0068] Specifically, the central control module includes a data storage and an image processor. The image processor is used to process the acquired image information, and the data storage is used to store a gait feature library.

[0069] Specifically, the wireless network transmission module transmits information to the imaging display module and the sound and light alarm module through Ethernet. The imaging display module is a color display screen, and the sound and light alarm module includes a buzzer and an alarm light.

[0070] The above description is merely a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for human gait recognition based on surveillance video, characterized in that: The following steps are involved: S1. Acquire surveillance video data through a first surveillance device group, acquire a human target in the surveillance video data, and track the human target; wherein the first surveillance device group includes a first wide-angle camera and a second wide-angle camera, the first wide-angle camera and the second wide-angle camera being symmetrically arranged, the first wide-angle camera acquiring distant front-facing surveillance video data, the second wide-angle camera acquiring distant rear-facing surveillance video data, and the first wide-angle camera and the second wide-angle camera forming an overlapping surveillance area; the first surveillance device group is accompanied by a second surveillance device group, the second surveillance device group including a first zoom camera and a second zoom camera, the first wide-angle camera being accompanied by the first zoom camera, and the second wide-angle camera being accompanied by the second zoom camera; S2. Extracting body features of human targets; S3. The extracted body features of the human target are compared with the body features in the memory to obtain the initial target; S4. Determine whether the body features match. If yes, execute step S5; if not, execute step S1; S5. Switching the surveillance video data acquisition path from the first surveillance device group to the second surveillance device group, the second surveillance device group acquiring surveillance video data, obtaining a magnified image of the initial target, and tracking the initial target; wherein the surveillance video acquisition path is transferred from the first wide-angle camera and the second wide-angle camera to the first zoom camera and the second zoom camera, respectively, the first zoom camera and the second zoom camera track the initial target using a geometric zoom tracking method and obtain a magnified image of the initial target, the geometric zoom tracking method obtaining an estimated value of the trajectory curve by linear interpolation based on two trajectory curves of the near and far targets, and the tracking position formula is: in, To estimate the location point, is the initial position point, is the focus motor position point when the zoom motor position is z at close object distance, is the focus motor position point when the zoom motor is z at a long object distance, z init is the initial zoom motor position when the object distance is d; S6. extracting gait features from a period of the image sequence to obtain the range of variation of the arm swing angle α and the stride angle β; S7. Compare the obtained ranges of change of the arm swing angle α and the stride angle β with the gait features in the memory to obtain the final target and generate an alarm message.

2. The method for human gait recognition based on surveillance video according to claim 1, characterized in that: The focal lengths of the first wide-angle camera and the second wide-angle camera are set to F, the pixel width of the human target is set to P, the distance between the human target and the first wide-angle camera is measured to be L, and the body features of the human target are extracted, including height h, shoulder width w, and leg length m. The calculation formulas for the height h, shoulder width w, and leg length m of the human target are respectively: h=(P1*L) / F, w=(P2*L) / F, m=(P3*L) / F P1 is the pixel width of the target human body height, P2 is the pixel width of the target human body shoulder width, and P3 is the pixel width of the target human body leg length.

3. The method for human gait recognition based on surveillance video according to claim 1, characterized in that: In step S6, the gait feature extraction method is a fusion feature recognition method, which includes a structured feature method and an unstructured feature method; the structured feature method simulates a structured model of a person walking by establishing a 2D or 3D model, and the pendulum model based on the leg stride action and the pendulum model based on the arm swing action conform to a certain angle change, the angle of the swing action is set to α, and the angle of the stride action is set to β. The swing action and the stride action have a certain periodicity, and the movement of each person conforms to a certain pattern, and structured feature extraction is performed. The unstructured feature method extracts the human body contour from the gait sequence, and expresses the gait feature by the information change of the connecting line between the points constituting the contour and the center. The wavelet packet transform is applied to this one-dimensional gait signal to extract the gait feature to enhance robustness.

4. The method for human gait recognition based on surveillance video according to claim 1, characterized in that: In step S6, the gait feature extraction method includes one or more of an inter-frame difference method and a background subtraction method, wherein the inter-frame difference method extracts the moving target by combining brightness changes between continuous image sequences, and the gait feature is extracted by combining morphological and human body structure features; The background subtraction method compares the current frame image with the background image, defines the area with small difference as the background area, and defines the area with large difference as the motion area, thereby subtracting the background and extracting only the gait features.

5. A system for the method for human gait recognition based on surveillance video according to any one of claims 1 to 4, characterized in that: It includes a wide-angle image acquisition module, a zoom image acquisition module, a central control module, a wireless network transmission module, an imaging display module and an audio-visual alarm module. The wide-angle image acquisition module and the zoom image acquisition module are used to obtain image information and transmit the image information to the central control module. After the target information is identified and integrated by the central control module, the identification information is transmitted to the imaging display module through the wireless network transmission module. The target person information is displayed through the imaging display module, and the audio-visual alarm module issues an alarm at the same time.

6. The human gait recognition system based on surveillance video according to claim 5, characterized in that: The central control module includes a data storage and an image processor. The image processor is used to process the acquired image information, and the data storage is used to store a gait feature library.

7. The human gait recognition system based on surveillance video according to claim 5, characterized in that: The wireless network transmission module transmits information to the imaging display module and the sound and light alarm module through Ethernet. The imaging display module is a color display screen, and the sound and light alarm module includes a buzzer and an alarm light.

Images