Hierarchical human body activity identification method and system based on multiple position sensors

Abstract

The invention discloses a hierarchical human body activity recognition method and system based on multiple position sensors. The system comprises a plurality of sensor modules, a data fusion module, a feature vector extraction module, a time sequence feature generation module and a human body activity recognition and judgment module. The plurality of sensor modules are used for acquiring and collecting detection data; the data fusion module performs data layer fusion on the collected detection data; a feature vector extraction module extracts representation vectors matched with human body activities from the fused data; a time sequence feature generation module generates hierarchical time sequence features of the human body activity according to the representation vector matched with the human body activity and the time change; the human body activity recognition and judgment module judges and recognizes human body activities according to the time sequence characteristics of the hierarchical human body activities. According to the method, a series of recognition technologies such as data fusion, feature extraction and action recognition are adopted to quickly recognize human body actions, and the actions are analyzed by adopting a plurality of sections of continuous time axes, so that subsequent action prediction is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of hierarchical human activity recognition, and in particular to a hierarchical human activity recognition method and system based on multi-position sensors. Background Art

[0002] Human activity recognition is widely used in fields such as smart homes and sports and health. In order to make smart homes and sports and health better serve people, it is necessary to identify and predict human activities. Currently, common human activity recognition technologies are not perfect and cannot achieve accurate recognition. They cannot predict activities within multiple consecutive biological time periods, making it impossible for smart machines such as smart homes and sports and health to respond in time. Summary of the Invention

[0003] In order to at least solve or partially solve the above problems, a hierarchical human activity recognition method and system based on multi-position sensors are provided, which can quickly recognize human activities and have a certain activity prediction capability.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] The present invention provides a hierarchical human activity recognition system based on multi-position sensors, comprising a plurality of sensor modules, a data fusion module, a feature vector extraction module, a time series feature generation module, and a human activity recognition and judgment module; the plurality of sensor modules are used to acquire and collect detection data; the data fusion module performs data layer fusion on the collected detection data; the feature vector extraction module extracts a characterization vector matching the human activity from the fused data; the time series feature generation module generates time series features of hierarchical human activities based on the characterization vector matching the human activity and time changes; and the human activity recognition and judgment module judges and recognizes human activities based on the time series features of the hierarchical human activities.

[0006] As a preferred technical solution of the present invention, the multiple sensor modules adopt accelerometers, gyroscopes, and GNNs.

[0007] As a preferred technical solution of the present invention, the data fusion module fuses the acceleration of the accelerometer and the angular velocity of the gyroscope through ins, gnns provides position information, and fuses the fusion data and position information respectively to obtain a fusion vector; after the fusion vector is optimized, the result is fed back to the fusion data for correction and output.

[0008] As a preferred technical solution of the present invention, the feature vector extraction module includes a human activity feature unit, a historical feature activity unit, and a feature vector activity unit. The human activity feature unit generates human activity features based on a fusion vector, and the historical feature activity unit extracts historical feature data based on its position in the historical data; the feature vector activity unit combines human activity features and historical feature data based on machine learning to calculate a representation vector that matches human activity.

[0009] As a preferred technical solution of the present invention, the time series feature generation module divides multiple continuous times into time periods with equal intervals, and generates hierarchical time series features of human activities according to the characterization vector and weight of each time period.

[0010] As a preferred technical solution of the present invention, the human activity recognition and judgment module inputs the temporal features of hierarchical human activities into the LSTM neural network model, wherein the LSTM neural network model has learned the mapping relationship between each feature and human activity, and outputs the judgment result.

[0011] The present invention also provides the following technical solution: A hierarchical human activity recognition method based on multi-position sensors comprises the following steps:

[0012] A: Use multi-position sensors to detect and collect data;

[0013] B: Perform data layer fusion on the data;

[0014] C: Extracting representation vectors that match human activities based on the fused data;

[0015] D: Generate hierarchical temporal features of human activities based on the representation vectors matching human activities and time changes;

[0016] E: Judge and identify human activities based on the temporal characteristics of hierarchical human activities.

[0017] As a preferred technical solution of the present invention, the data layer fusion step includes fusing the acceleration of the accelerometer and the angular velocity of the gyroscope through ins, gnns provides position information, and the fused data and position information are fused respectively to obtain a fusion vector; after the fusion vector is optimized, the result is fed back to the fusion data to correct it and then output.

[0018] As a preferred technical solution of the present invention, the step of extracting a characterization vector that matches human activities includes combining human activity characteristics and historical feature data based on machine learning to calculate a characterization vector that matches human activities.

[0019] As a preferred technical solution of the present invention, the step of judging and identifying human activities includes inputting the temporal features of hierarchical human activities into an LSTM neural network model, wherein the LSTM neural network model has learned the mapping relationship between each feature and human activity, and outputs the judgment result.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention adopts a series of recognition technologies such as data fusion, feature extraction, and action recognition to quickly identify human body movements, and uses multiple continuous time axes to analyze the movements, which facilitates subsequent prediction of the movements. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0023] Figure 1 It is a system module diagram of the present invention;

[0024] Figure 2 is a flow chart of the method of the present invention;

[0025] In the figure: 1. Sensor module; 2. Data fusion module; 3. Feature vector extraction module; 4. Time series feature generation module; 5. Human activity recognition and judgment module; 6. Human activity feature unit; 7. Historical feature activity unit; 8. Feature vector activity unit. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0027] Furthermore, if detailed description of known art is not necessary to illustrate the characteristics of the present invention, it will be omitted.

[0028] Example 1

[0029] like Figure 1As shown, the present invention provides a hierarchical human activity recognition system based on multi-position sensors, including multiple sensor modules 1, a data fusion module 2, a feature vector extraction module 3, a time series feature generation module 4, and a human activity recognition and judgment module 5; the multiple sensor modules 1 are used to acquire and collect detection data; the data fusion module 2 performs data layer fusion on the collected detection data; the feature vector extraction module 3 extracts a representation vector matching the human activity from the fused data; the time series feature generation module 4 generates a time series feature of the hierarchical human activity according to the representation vector matching the human activity and the time change; the human activity recognition and judgment module 5 judges and recognizes the human activity according to the time series feature of the hierarchical human activity.

[0030] First, the observation data of the sensors are fused, and then the feature vector is extracted from the fused data. The observation data of the sensors used by multiple sensor modules 1 include acceleration of the accelerometer, angular velocity of the gyroscope, and positioning data of the GNNS.

[0031] The data fusion module 2 fuses the acceleration of the accelerometer and the angular velocity of the gyroscope through the ins, and the gnns provides the position information. The fusion data and the position information are fused to obtain the fusion vector respectively; after the fusion vector is optimized, the result is fed back to the fusion data to correct it and then output.

[0032] The feature vector extraction module 3 includes a human activity feature unit 6, a historical feature activity unit 7, and a feature vector activity unit 8. The human activity feature unit 6 generates human activity features based on the fusion vector, and the historical feature activity unit 7 extracts historical feature data according to its position in the historical data; the feature vector activity unit 8 combines the human activity features and the historical feature data based on machine learning to calculate a representation vector that matches the human activity, where the machine learning algorithm can be but is not limited to logistic regression, gradient decision tree, recurrent neural network, convolutional neural network, etc.

[0033] The time series feature generation module 4 divides multiple continuous times into time periods with equal intervals, and generates hierarchical time series features of human activities according to the representation vector and weight of each time period.

[0034] The human activity recognition and judgment module 5 inputs the temporal features of the hierarchical human activities into the LSTM neural network model, where the LSTM neural network model has learned the mapping relationship between each feature and the human activity, and outputs the judgment result.

[0035] like Figure 2 As shown, a hierarchical human activity recognition method based on multi-position sensors is characterized by comprising the following steps:

[0036] A: Use multi-position sensors to detect and collect data;

[0037] B: Perform data layer fusion on the data;

[0038] C: Extracting representation vectors that match human activities based on the fused data;

[0039] D: Generate hierarchical temporal features of human activities based on the representation vectors matching human activities and time changes;

[0040] E: Judge and identify human activities based on the temporal characteristics of hierarchical human activities.

[0041] The data layer fusion step includes fusing the acceleration of the accelerometer and the angular velocity of the gyroscope through ins, gnns provides position information, and fusing the fused data and position information to obtain a fusion vector; after optimizing the fusion vector, the result is fed back to the fusion data and corrected for output.

[0042] The step of extracting the characterization vector that matches the human activity includes combining the human activity characteristics and historical feature data based on machine learning to calculate the characterization vector that matches the human activity.

[0043] The step of judging and identifying human activities includes inputting the temporal features of hierarchical human activities into the LSTM neural network model, wherein the LSTM neural network model has learned the mapping relationship between each feature and human activity and outputs the judgment result.

[0044] The present invention adopts a series of recognition technologies such as data fusion, feature extraction, and action recognition to quickly identify human body movements, and uses multiple continuous time axes to analyze the movements, which facilitates subsequent prediction of the movements.

[0045] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A hierarchical human activity recognition system based on multi-position sensors, characterized in that: The system comprises a plurality of sensor modules, a data fusion module, a feature vector extraction module, a time series feature generation module, and a human activity recognition and judgment module; the plurality of sensor modules are used to acquire and collect detection data; the data fusion module performs data layer fusion on the collected detection data; the feature vector extraction module extracts a representation vector matching human activity from the fused data; the time series feature generation module generates hierarchical time series features of human activity based on the representation vector matching human activity and time changes; The human activity recognition and judgment module judges and recognizes human activities according to the temporal characteristics of hierarchical human activities.

2. The hierarchical human activity recognition system based on multi-position sensors according to claim 1, characterized in that: The multiple sensor modules include accelerometers, gyroscopes, and GNNS.

3. The hierarchical human activity recognition system based on multi-position sensors according to claim 1, characterized in that: The data fusion module fuses the acceleration of the accelerometer and the angular velocity of the gyroscope through ins, and gnns provides position information. The fusion data and position information are fused separately to obtain a fusion vector; after the fusion vector is optimized, the result is fed back to the fusion data and corrected for output.

4. The hierarchical human activity recognition system based on multi-position sensors according to claim 1, characterized in that: The feature vector extraction module includes a human activity feature unit, a historical feature activity unit, and a feature vector activity unit. The human activity feature unit generates human activity features based on the fusion vector, and the historical feature activity unit extracts historical feature data based on the position in the historical data. The feature vector activity unit combines human activity features and historical feature data based on machine learning to calculate a representation vector that matches the human activity.

5. The hierarchical human activity recognition system based on multi-position sensors according to claim 1, characterized in that: The time series feature generation module divides a plurality of continuous times into time periods with equal intervals, and generates hierarchical time series features of human activities according to the representation vector and weight of each time period.

6. The hierarchical human activity recognition system based on multi-position sensors according to claim 1, characterized in that: The human activity recognition and judgment module inputs the temporal features of hierarchical human activities into the LSTM neural network model, wherein the LSTM neural network model has learned the mapping relationship between each feature and human activity, and outputs the judgment result.

7. A hierarchical human activity recognition method based on multi-position sensors, characterized in that: The following steps are involved: A: Use multi-position sensors to detect and collect data; B: Perform data layer fusion on the data; C: Extracting representation vectors that match human activities based on the fused data; D: Generate hierarchical temporal features of human activities based on the representation vectors matching human activities and time changes; E: Judge and identify human activities based on the temporal characteristics of hierarchical human activities.

8. The hierarchical human activity recognition method based on multi-position sensors according to claim 7, characterized in that: The data layer fusion step includes fusing the acceleration of the accelerometer and the angular velocity of the gyroscope through ins, gnns provides position information, and fusing the fused data and position information to obtain a fusion vector; after optimizing the fusion vector, the result is fed back to the fusion data and corrected for output.

9. The hierarchical human activity recognition method based on multi-position sensors according to claim 7, characterized in that: The step of extracting the characterization vector that matches the human activity includes combining the human activity characteristics and historical feature data based on machine learning to calculate the characterization vector that matches the human activity.

10. The hierarchical human activity recognition method based on multi-position sensors according to claim 7, characterized in that: The step of judging and identifying human activities includes inputting the temporal features of hierarchical human activities into the LSTM neural network model, wherein the LSTM neural network model has learned the mapping relationship between each feature and human activity and outputs the judgment result.

Images