A human body posture detection system, method and device

Through dynamic hybrid variational autoencoding classification model and bidirectional hybrid door unit technology, the problem that existing human posture detection methods cannot accurately detect 3D spatial position changes and posture changes is solved, and the deep information of dynamic nonlinear human space data is realized Mining improves detection accuracy.

CN118395296BActive Publication Date: 2024-11-29YANGU (HANGZHOU) E-COMMERCE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410393508.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-02
Publication Date
2024-11-29
Estimated Expiration
2044-04-02

AI Technical Summary

Technical Problem

The existing human posture detection methods cannot accurately detect changes in 3D spatial position and posture changes, and the performance of the threshold-based detection method is affected by threshold selection, so it is impossible to fully tap deep hidden information in dynamic nonlinear human space data, resulting in low detection accuracy. .

Method used

Using dynamic hybrid variational autoencoding classification (DMVAEC) model, combining bidirectional hybrid gate unit (MIXGU) and variational autoencoder (VAE) frameworks, global hidden variables and local information are extracted from nonlinear dynamic human posture data, using neural The network is reconstructed and classified, and manifold learning is introduced to reduce the impact of noise and improve detection accuracy.

Benefits of technology

It realizes accurate detection of human posture, fully explores hidden information in dynamic nonlinear data, improves detection accuracy, and reduces the impact of noise. It is suitable for deep learning models of complex dynamic data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118395296B_ABST
    Figure CN118395296B_ABST
Patent Text Reader

Abstract

The present invention discloses a human posture detection system, method and device. The present invention proposes a human posture detection method of a dynamic hybrid variational autoencoder classification DMVAEC model, which uses a bidirectional MIXGU to extract global latent variable information and classification information from all sample data sets, and based on the variational autoencoder VAE framework, uses MIXGU as an autoencoder to extract global information and multimodal local information from nonlinear dynamic human posture data, uses a neural network as a decoder to achieve reconstruction of input data, uses a softmax classifier to predict human posture categories, introduces manifold learning to well describe the intrinsic geometric structure information of human posture data, and obtains the optimal model parameters through a variational reasoning method. This technology fully mines the hidden nonlinear local dynamic and global geometric structure information of the data set by introducing the MIXGU deep learning model, and better reduces the influence of human posture data noise by introducing variational reasoning technology, thereby improving detection accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of human posture detection, and specifically refers to a human posture detection system, method and equipment. Background Art

[0002] The safety of infrastructure construction and industrialization has always been the focus of the country. For a long time, the Chinese government has issued different regulations and rules for construction project safety, which has greatly promoted the improvement of construction site safety management level by governments at all levels and construction managers. However, due to the limitation of project management cost and the number of management personnel, it is difficult to accurately control the on-site personnel in real time, and it is even more impossible to manage the working status of personnel and the existing safety hazards.

[0003] In order to improve the safety of on-site personnel and better protect the health of workers, some research results on smart helmets have emerged. In 2022, a team composed of Liang Kaibo and others from the School of Electronic Information Engineering of Jinling Institute of Technology used Beidou positioning technology and Alibaba Cloud platform, combined with MPU6050 inertial sensors, to design a smart helmet that can detect falls and alarm through the Alibaba Cloud platform. In 2022, Liu Huan from Xidian University designed an intelligent helmet system based on the GB / 228181 protocol. The system combines Internet of Things technology, spatial positioning technology, mobile communications and multimedia technologies to realize video surveillance, voice intercom, collaborative positioning and safety alarms. In 2022, Zhao Peng from Harbin University of Science and Technology combined ZigBee technology, OneNET cloud platform and Android software to produce a smart helmet for limited space working environments, which includes functions such as temperature detection, toxic gas concentration detection, heart rate and blood oxygen detection, and power detection. In 2022, Fahrurrasyid and others from Telkom University designed a smart helmet using heart pulse sensors, GPS positioning modules, etc. in response to the sharp increase in motorcycle accidents in Indonesia. The helmet can detect drowsiness through heart rate and issue an alarm, and can also locate and track the injured through Google Maps. In addition, the smart helmet is also equipped with an SOS call function.

[0004] However, GPS can only collect the 2D coordinate position of the human body, and cannot detect the change of 3D spatial position, resulting in the inability to accurately detect some human postures (such as standing, leaning forward and backward, leaning left and right, falling) and posture changes. In general, domestic and foreign research mainly focuses on the research and application of environmental parameter detection, human positioning and physiological parameter detection. However, the performance of the current threshold-based human posture detection method is affected by the selection of thresholds. The existing human posture detection method cannot fully explore the deep hidden information in dynamic nonlinear human spatial data, and the detection accuracy is not satisfactory. Summary of the invention

[0005] The purpose of the present invention is to provide a human posture detection system, method and device that can accurately detect human posture, fully explore deep hidden information in dynamic nonlinear human body spatial data, and have high detection accuracy.

[0006] To achieve the above-mentioned purpose, the present invention provides a human body detection system, including a single-chip microcomputer, an infrared obstacle avoidance sensor, a pressure sensor, a photoresistor sensor, a temperature sensor, a temperature and humidity sensor, a toxic gas sensor, an air pressure sensor, a dust sensor, a GPS positioning module, a speech synthesis module, a gyroscope module, an acceleration sensor, a pulse blood oxygen sensor, a camera module, an OLED display module, and a WIFI module, wherein the gyroscope module and the acceleration sensor belong to an inertial sensor IMU; the single-chip microcomputer collects environmental, physiological and position parameters from various sensors and modules, processes the data by the single-chip microcomputer, and uploads the data to a server through the WIFI module, and makes corresponding decisions and controls on the single-chip microcomputer and the server.

[0007] As a further solution of the present invention: the temperature sensor is a DS18B20 temperature sensor, the temperature and humidity sensor is a DHT11 temperature and humidity sensor, the speech synthesis module is a SYN6288 speech synthesis module, the gyroscope module is an MPU6050 gyroscope module, the acceleration sensor is an ADXL345 acceleration sensor, the pulse oximeter sensor is a MAX30102 pulse oximeter sensor, and the WIFI module is an ESP8266 WIFI module.

[0008] A human posture detection method of the present invention comprises an offline modeling stage and an online human posture detection stage, wherein the offline modeling stage comprises the following steps:

[0009] Step 1: Data preprocessing: the inertial sensor IMU data in the human posture detection system is processed through the encoder Bi-MIXGU model to generate a sample set;

[0010] Step 2: Generate a data series based on sample autocorrelation and sample set;

[0011] Step 3: Standardize the data sequence generated in step 2;

[0012] Step 4: Determine the structure of the dynamic hybrid variational autoencoder classification DMVAEC model and the parameters of the objective function of the optimization problem, and randomly generate model parameters as initial values ​​of the model parameters;

[0013] Step 5: Use the stochastic gradient descent SGD method to solve the optimization problem and save the optimal parameter values ​​of the DMVAEC model;

[0014] The online human pose detection stage includes the following steps:

[0015] Step 1: Collect human posture data from IMU online and process it to generate samples;

[0016] Step 2: Generate sequence data corresponding to the new sample and standardize it;

[0017] Step 3: Calculate the classification probability of the DMVAEC model output;

[0018] Step 4: Determine the human body posture category and send it to the microcontroller system;

[0019] Step 5: Determine whether the posture is abnormal. If so, issue an alert and send it to the administrator. Otherwise, continue to detect new data.

[0020] As a further solution of the present invention: the Bi-MIXGU model is a bidirectional mixed gate unit MIXGU, and the MIXGU is constructed by adaptively weighting the recursive unit neural network GRU and the streamlined model minimum gate unit MGU.

[0021] As a further solution of the present invention: the dynamic hybrid variational autoencoder classification DMVAEC model adopts bidirectional MIXGU to extract global latent variable information and classification information from all sample data sets, based on the variational autoencoder VAE framework, uses MIXGU as an autoencoder to extract global information and multimodal local information from nonlinear dynamic human posture data, uses a neural network as a decoder to realize the reconstruction of the input data, uses a softmax classifier to predict the human posture category, introduces manifold learning to well describe the intrinsic geometric structure information of the human posture data, and obtains the optimal model parameters through the variational reasoning method.

[0022] A human body detection device of the present invention is a safety helmet, which includes a human body detection system. The single-chip microcomputer is arranged at the upper part inside the safety helmet. The safety helmet includes a lighting switch and a lighting lamp. The lighting switch and a speech synthesis module are respectively placed at the middle position of the outer side of the safety helmet; a photoresistor sensor is located just above the brim of the safety helmet, and the lighting lamp and an OLED display screen module are respectively placed just in front of the safety helmet; a camera module is arranged at the middle position of the brim of the safety helmet; a pressure sensor is arranged at the top inside the safety helmet, and directly contacts with the head to judge whether the safety helmet is worn correctly; a DS18B20 temperature sensor and a MAX30102 pulse blood oxygen sensor are placed at the back of the safety helmet and are in close contact with the head; a GPS positioning module is arranged at the upper position of the front of the safety helmet; an infrared obstacle avoidance sensor is located below the brim of the safety helmet; a sensor integration box is arranged on the inner side of the rear part of the safety helmet, and a temperature and humidity sensor, a toxic gas sensor, an air pressure sensor, a dust sensor, a gyroscope module, an acceleration sensor, and a WIFI module are arranged in the integration box for measuring external environmental parameters and human body posture position data.

[0023] As a further solution of the present invention: the working power supply of the safety helmet is provided by a lithium battery.

[0024] Compared with the prior art, the present invention proposes a human posture detection method based on a dynamic hybrid variational autoencoder classification DMVAEC model. The model uses bidirectional MIXGU to extract global latent variable information and classification information from all sample data sets. Based on the variational autoencoder VAE framework, MIXGU is used as an autoencoder to extract global information and multimodal local information from nonlinear dynamic human posture data. A neural network is used as a decoder to achieve reconstruction of input data, a softmax classifier is used to predict human posture categories, manifold learning is introduced to well describe the intrinsic geometric structure information of human posture data, and the optimal model parameters are obtained through variational reasoning methods. This technology fully mines the hidden nonlinear local dynamic and global geometric structure information of the data set by introducing the MIXGU deep learning model, and better reduces the influence of human posture data noise by introducing variational reasoning technology, thereby improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the structure of the human body posture detection system of the present invention.

[0026] Figure 2 It is a schematic diagram of some basic sensor circuits of the human posture detection system of the present invention.

[0027] Figure 3 It is a schematic diagram of a complex sensor circuit of a human posture detection system of the present invention.

[0028] Figure 4 (a) is the data flow and operation diagram in the GRU unit.

[0029] Figure 4 (b) is the data flow and operation diagram in the MGU unit.

[0030] Figure 4 (c) is the data flow and operation diagram in the MIXGU unit.

[0031] Figure 5 It is the MIXGU diagram of the present invention.

[0032] Figure 6 It is a bidirectional MIXGU network structure diagram of the present invention.

[0033] Figure 7 It is a Bi-MIXGU network structure diagram for extracting global information of human posture dynamic data according to the present invention.

[0034] Figure 8 It is the DMVAEC human posture detection model based on MIXGU of the present invention.

[0035] Fig. 9 It is a network structure diagram of the encoder MIXGU of the present invention.

[0036] Fig.10 It is a flow chart of human posture detection modeling based on DMVAEC model of the present invention.

[0037] In the figure: 1. OLED display module circuit, 2. DHT11 temperature and humidity sensor circuit, 3. DS18B20 temperature sensor circuit, 4. BMP180 air pressure sensor circuit, 5. GPS positioning module circuit, 6. Dust sensor, 7. Infrared obstacle avoidance sensor circuit, 8. MQ135 toxic gas sensor circuit, 9. Photoresistor sensor circuit, 10. Pressure sensor circuit, 11. SYN6288 speech synthesis module circuit, 12. MAX30102 pulse oximeter sensor, 13. ADXL345 acceleration sensor, 14. MPU6050 gyroscope module. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the human posture detection system includes a single-chip microcomputer, an infrared obstacle avoidance sensor, a pressure sensor, a photoresistor sensor, a temperature sensor, a temperature and humidity sensor, a toxic gas sensor, an air pressure sensor, a dust sensor, a GPS positioning module, a speech synthesis module, a gyroscope module, an acceleration sensor, a pulse blood oxygen sensor, a camera module, an OLED display module, and a WIFI module, wherein the gyroscope module and the acceleration sensor belong to the inertial sensor IMU; the single-chip microcomputer is electrically connected to various sensors and modules, and the single-chip microcomputer collects environmental, physiological and position parameters from various sensors and modules, processes the data, and uploads the data to the server through the WIFI module, and makes corresponding decisions and controls on the single-chip microcomputer and the server.

[0040] The temperature sensor in the system is DS18B20 temperature sensor, the temperature and humidity sensor is DHT11 temperature and humidity sensor, the toxic gas sensor is MQ135 toxic gas sensor, the pressure sensor is BMP180 pressure sensor, the speech synthesis module is SYN6288 speech synthesis module, the gyroscope module is MPU6050 gyroscope module, the acceleration sensor is ADXL345 acceleration sensor, the pulse blood oxygen sensor is MAX30102 pulse blood oxygen sensor, and the WIFI module is ESP8266WIFI module. The circuit diagrams of each sensor and module are as follows: Figure 2 , Figure 3 shown.

[0041] The detection method used in the human posture detection system is specifically described as follows.

[0042] Recurrent neural networks (RNNs) are a commonly used method for dynamic data mining and analysis, but they have problems such as gradient vanishing and gradient exploding. The long short-term memory (LSTMs) model improves the problem that RNNs cannot obtain the long-term temporal dependencies between data samples through a gating mechanism, and has achieved great success in complex dynamic data modeling and analysis. However, the internal structure of the LSTM unit is very complex, and the mechanism by which the internal components affect the training model effect is not obvious, which affects its practical application. Existing studies have shown that gated recurrent unit neural networks (GRUs) can achieve similar accuracy to LSTMs, such as Figure 4 As shown in (b), the simplified model minimum gate unit (MGU) still shows considerable advantages in some sequence analysis and image processing, and the computational complexity is greatly reduced. Figure 4 As shown in (a), the GRU state output is updated by the gate z t , reset gate t And the internal hidden state According to the update gate output, the internal hidden state and the hidden state at time t-1 are linearly combined to obtain the GRU output hidden state The update calculation method for GRU is as follows:

[0043] z t =σ(W z ·[h t-1 ,x t ]+b z ) (1)

[0044] r t =σ(W r [h t-1 ,x t ]+b r ) (2)

[0045]

[0046]

[0047] Among them, h t-1 represents the hidden state of the GRU unit output at time t-1, h t-1 With x t represents the GRU unit input at time t, ⊙ represents the vector element-by-element multiplication operator, W z , W r and W h is the weight matrix of the corresponding gate, b z 、b r and b his the bias vector of the corresponding gate, tanh(·) is the hyperbolic tangent function, and σ(·) is the nonlinear activation function, usually sigmoid and ReLu functions. Based on the importance of the forget gate in the gating mechanism, the MGU unit is simply composed of a forget gate and an internal hidden state gate. The calculation method is:

[0048] f t =σ(W f ·[h t-1 ,x t ]+b f ) (5)

[0049]

[0050]

[0051] Among them, W f and b f are the weight matrix and bias vector of the forget gate respectively, W h 'and b' h are the weight matrix and bias vector of the internal hidden state gate, respectively.

[0052] In order to make full use of the obvious advantages of GRU's high precision and MGU's simplicity and physical significance, the present invention constructs a mixed gate unit (MIXGU) for complex nonlinear dynamic data regression modeling by adaptively weighting GRU and MGU, thereby achieving a reasonable compromise between complexity and precision. Figure 4 (c) The MIXGU shown consists of GRU, MGU and weight adjustment module (AdjustmentModule), assuming Input x t The hidden states of the GRU and MGU of the MIXGU module are used as the output of the adjustment module. Right now:

[0053]

[0054]

[0055] Then the predicted output of the MIXGU unit is

[0056]

[0057] Among them, f(·) is the activation function, g(·) is the nonlinear regression function, which is generally the sigmoid function, and V represents the regression coefficient matrix. After the adjustment module, the new hidden state as input to the next unit.

[0058] Based on MIXGU units, the MIXGU network modeling structure of dynamic data is as follows Figure 5 shown.

[0059] The bidirectional MIXGU (Bi-MIXGU) network is composed of a forward layer and a backward layer network, such as Figure 6 As shown, the corresponding hidden states are recorded as Then input x t The output y is calculated by the following nonlinear function t

[0060]

[0061] Among them, for time step t, the hidden state of the Bi-MIXGU unit is The calculation process is

[0062]

[0063]

[0064] In view of the problem of human posture detection, the present invention proposes a human posture detection method based on a dynamic mixture variation autoencoder classification (DMVAEC) model. The model uses a bidirectional MIXGU to extract global latent variable information and classification information from all sample data sets. Based on the variation autoencoder-decoder (VAE) framework, MIXGU is used as an autoencoder to extract global information and multimodal local information from nonlinear dynamic human posture data. A neural network is used as a decoder to reconstruct the input data, a softmax classifier is used to predict the human posture category, manifold learning is introduced to well describe the intrinsic geometric structure information of the human posture data, and the optimal model parameters are obtained through a variational reasoning method. The specific steps are as follows:

[0065] Step 1: Fusion of MIXGU and VAE human posture detection models;

[0066] 1) Global information extraction based on Bi-MIXGU

[0067] Assume that at time t, the acceleration and angular velocity measurements of the accelerometer and gyroscope of the inertial sensor IMU on the (x, y, z) axis relative to the sensor coordinate system are recorded as a x (t),a y (t),a z (t) and ω x (t),ω y(t),ω z (t), t=1,2,…,T', let M t =[a x (t),a y (t),a z (t),ω x (t),ω y (t),ω z (t)] T "T" represents the transpose of a vector or matrix. Affected by the manufacturing process, working environment and the accumulation of measurement errors, the IMU measurement data contains a lot of noise, so the data needs to be filtered. The present invention uses the unscented Kalman filter algorithm UKF to Filtering is performed, and the filtered data is recorded as The filtered acceleration data According to the sampling period, the displacement of the human body in the (x, y, z) coordinate axis direction xp(t), yp(t), zp(t)(t=1,2,…,M) can be obtained by integrating the sampling period. According to the sampling period, the integral operation can be performed to obtain the angles xa(t), ya(t), and za(t) of the human body in the direction of the (x, y, z) coordinate axis. Let x t =[xp(t),yp(t),zp(t),xa(t),ya(t),za(t)] T Represents the position and angle value of the inertial sensor IMU in the (x, y, z) axis direction at time t, t = 1, 2, …, M, M represents the number of samples.

[0068] Considering that Bi-MIXGU can effectively extract local information of dynamic data while ignoring the global information of the data, this patent adopts the following method: Figure 7 The encoder Bi-MIXGU model shown in the figure obtains the global information of the dynamic variable data of the human posture. s And category information c, where the category information c of the data sample is obtained by softmax, and assuming z s Obeying normal distribution, then the category information c and global information z s Obtained from the following formula

[0069]

[0070]

[0071] in, The forward and backward MIXGU networks are respectively from the data sequence x1, x2, ..., x T The extracted hidden state, T is the length of the sequence, and its value is based on x t The autocorrelation coefficient of is determined; express The purpose is to ensure The value is positive. is the linear transformation weight matrix, is the linear transformation bias; N(μs,diag(σs)) means the mean is μ s , the covariance matrix is ​​diagonal Normal distribution of w c represents the weight vector, C is the number of categories, c=1,2,…,C, and f(·) represents a nonlinear function. Obviously, z s and c utilize the global hidden state of the bidirectional MIXGU Therefore, global information and category information of dynamic data can be extracted.

[0072] 2) Fusion of MIXGU and VAE human posture detection model structure

[0073] The global information and local information of the data can fully describe the overall and local structure of the data. Making full use of the global information and local information of the data can effectively improve the performance of the deep learning model. Considering that the actual IMU data has obvious random uncertainty due to the environment and IMU, and the long short-term memory network LSTM calculation is relatively complex, the present invention proposes a human posture detection method based on the DMVAEC model. The method uses the advantage of variational autoencoder VAE to effectively describe the random uncertainty of dynamic data, uses VAE and uses the global information of dynamic human posture data to better extract the local information of dynamic data; considering that the complexity of MIXGU is much lower than LSTM but has comparable performance to LSTM, the present invention uses MIXGU network as VAE encoder; Gaussian mixture model GMM is used in latent space to fit the latent feature distribution of human dynamic posture data, overcomes the defect of traditional VAE model assuming that latent variables obey Gaussian distribution, and can better characterize complex distribution data sets. The method uses the global information and local information of dynamic data to realize the classification detection of human posture. Compared with the prior art, this method not only effectively improves the detection accuracy of the model, but also reduces the calculation complexity of the model, and is easy to be used in actual production. In order to better understand the DMVAE model, the model structure is first briefly introduced.

[0074] The model variational autoencoder (VAE) can well describe the dynamic uncertainty of the input data. The global information z of the data set can be obtained by using the above calculation. s , c and helmet detection human body dynamic variables x t-1 As the encoder MIXGU input, the encoder output is the hidden variable mean and variance normal distribution parameter μ t , And draw samples z from a normal distribution t, and then construct a decoder and a classifier to realize the human body posture category decision. Based on the above ideas, the present invention adopts Figure 5 The human motion posture detection model structure shown in the figure consists of an encoder, a decoder and a classifier.

[0075] like Figure 8 As shown in the figure, the encoder is implemented using the MIXGU network. c represents the class variable that encodes the class label in a one-hot manner. “NN” represents the decoder, which is approximated by a multi-layer neural network. “Softmax” represents the classification function for hidden features, and a neural network can also be used as a classifier. The input of each MIXGU unit is composed of the global hidden feature z of the sequence. s , data sample x t And the previous MIXGU unit outputs the hidden state h t-1 The specific form can be described as:

[0076] h t =MIXGU(c,z s ,x t ,h t-1 ) (15)

[0077] μ t =f1(h t ) (16)

[0078]

[0079]

[0080]

[0081] y t =softmax(z t ) (20)

[0082] Among them, the function MIXGU(.) represents the calculation and realization by the MIXGU unit, f1(·) and f2(·) are both mapping functions, and the present invention takes linear functions. express The natural logarithm of is a positive number; Repar_sampling(·) represents the reparameter sampling function, and the output of the neural network NN(zt) is the sample x t The reconstructed output of The present invention is implemented using a three-layer neural network.

[0083] from Figure 8It can be seen that the model utilizes the global information of human posture data and the dynamic correlation between data. In the encoder stage, each MIXGU unit not only utilizes the hidden state information and global information of the previous moment, but also utilizes the hidden variable information of the previous moment, so that the hidden variable at the current moment contains the local and global information of the dynamic data and the hidden state information related to it, and deeply mines the distribution of the hidden space features of human posture, thereby ensuring the high quality characteristics of the hidden features of the human state sequence data.

[0084] Step 2: DMVAEC modeling;

[0085] The diversity of human posture movement leads to a very complex distribution of dynamic data latent space features. The traditional VAE latent space feature distribution assumes a Gaussian distribution, which is difficult to describe the complexity of human posture data. Therefore, during the encoding process, the model of the present invention uses a Gaussian mixture model GMM to describe the data latent space feature distribution, and assumes that the global latent variable features of the model and the dynamic dependency between local latent variables follow a normal distribution, that is,

[0086] p(c)=π c (twenty one)

[0087]

[0088]

[0089] Among them, π c represents the prior distribution of class c, z <t represents the set of hidden variables before the current time t, X t represents the time series at time t, μ s , is the global hidden variable z s The mean and variance of p(z s |c k =1) represents the conditional distribution of the global latent variable. Formula (7) represents the conditional distribution of the global latent variable of the sample, and its value is independent of the local components of the data. c Represents the local latent variable at time t, including the global latent variable, the latent variable before time t, and the time series X t Since it is impossible to directly obtain the local hidden variable conditional distribution p(z c |z <t ,z s ,X t ), so the MIXGU network output is used as p(z c |z <t ,z s ,X t ) approximation, that is, the output of the MIXGU network is μ c and σ cAssume that the output of the MIXGU unit at time t is

[0090] h t =f s (h t-1 ,z s ,z t-1 ,x t ) (twenty four)

[0091] and x t The hidden variable z t Normal distribution Its mean μ t and variance It can be calculated by the following formula:

[0092] μ t =W μ h t +b μ (25)

[0093]

[0094] Among them, W μ , W σ is the linear transformation weight matrix, b σ 、b μ is the bias of the linear transformation. express The natural logarithm of Is a positive number.

[0095] The decoder of the model mainly reconstructs and classifies the input data based on latent variables. θ (x t |z t ) part is approximated by a 3-layer MLP neural network, and the data classifier outputs p θ (y t |z t ) is implemented using the fully connected softmax function, which is expressed as:

[0096]

[0097] p θ (y t |z t )=softmax(z t ) (28)

[0098] Where θ represents the decoder parameters, Represented by vector Let be a diagonal matrix with diagonal elements. C is the number of data categories, The form is

[0099]

[0100] Where c = 1, 2, ..., C, f(z t ) is a nonlinear function, which is calculated by a neural network whose activation function is the sigmoid function.

[0101] Using probability graph theory, the full probability distribution of the model is

[0102]

[0103] Among them, y 1:T represents y1,…,y T The dynamic data sequence composed of , the other similar expressions have the same meaning, θ represents the decoder parameter, T is the length of the data sequence. The present invention uses the variational inference method to solve the optimal latent variables and model parameters. In the data encoding stage, the present invention uses the MIXGU network and the bidirectional MIXGU network, such as Fig. 9 As shown, the approximate local latent variable variational posterior distribution is:

[0104]

[0105] here, The encoder MIXGU network is implemented to approximate the true conditional distribution of local dynamic hidden variables. is the global variable z s With class c conditional distribution, use Figure 7 The bidirectional MIXGU network shown is approximately represents the posterior distribution of the sample sequence, Calculated by BI-MIXGU We get c = [1, 2, ..., C], represents the approximate distribution model parameters.

[0106] According to variational inference theory, the log-likelihood function lnp(x 1:T ,y 1:T ) is bounded by:

[0107]

[0108] Among them, D KL (p||q)=E p (ln p-ln q) represents the KL-divergence distance between probability distributions p and q, E p (x) represents the mathematical expectation of a random variable x with distribution p, and ELBO represents the evidence lower bound. In order to learn the optimal parameters of the model, it is necessary to maximize the sample set likelihood function, which is equivalent to maximizing ELBO.

[0109] Assume the decoder reconstructs the data distribution And the classifier p θ (y t |z t )=soft max(z t ), then the final optimization objective of the model can be rewritten as:

[0110]

[0111] in, Represents x t The reconstruction of is calculated by the decoder neural network, noting that Calculating this term requires the posterior probability The present invention uses the reparameterization technique to calculate this term, which is to first sample the random variable ω from the normal distribution N(0,1) l ~N(0,I), and then obtain the final sampling result through parameter transformation:

[0112]

[0113] Where ⊙ represents the point-by-point multiplication of vector elements, l = 1, 2, ..., L, and L represents the number of random samples. t The true distribution of L is generally greater than 10 of the variable dimension. 2 ~10 3 times. Calculated as follows

[0114]

[0115]

[0116] here, Represents classification probability, calculated by softmax classifier, μ c represents the center of class c. Note that p θ (z t |z s ,z <t ) obeys Gaussian distribution, Represents the similarity distance between the true distribution of the latent variable and the approximate distribution of the latent variable, which is calculated as follows:

[0117]

[0118] Note that the encoder global latent variable distribution The encoder global latent variable conditional distribution is So

[0119]

[0120] For the last term of the objective function, p(c) represents the prior distribution of the latent variable distribution component, which is continuously updated in the optimization process; the global mode posterior distribution is calculated by BI-MIXGU We get, noting that the category indicator variable c is a random variable, then The calculation method is

[0121]

[0122] Manifolds can be viewed as low-dimensional spaces that partially look like some Euclidean spaces, on which calculus operations can be performed. In general, actual data may come from adjoint spaces with fewer degrees of freedom than the adjoint dimension. Considering the smoothness of the conditional probability distribution in the geometric direction of the data, this paper proposes a new human posture detection model, whose optimization problem is in the form of

[0123]

[0124] in, Indicates that the decoder gets x t Reconstruction of local hidden variables z i Given by the re-parameterized trick sampling shown in formula (34), is a close proximity similarity matrix whose elements S ij It is calculated by the following heat kernel function, that is,

[0125]

[0126] Here, kNN(x j ) is the sample point x j The nearest neighbor set, scale parameter t o Generally choose x j With the nearest neighbor set kNN(x j ) is the average distance between samples. It is worth noting that there are other methods to define the matrix S, such as 0-1 weighting, dot product weighting, etc. The second term on the right side of the optimization problem objective function equation (40) is used to keep the geometric structure of relatively close neighboring data points unchanged, so that the distance between similar data is as close as possible, thereby maintaining the geometric manifold of the latent variable space. Using stochastic gradient descent SGD, we can find a given new sample x n

[0127]

[0128] Among them, the posterior probability distribution q(c|x n-T+1:n ) represents the probability that the sample belongs to the cth mixture component, x n-T+1:n Represents the current window data, Can be Approximately, among which is a mixed Gaussian distribution q(z n |z g ,z <n , x ≤i )The mean of the cth Gaussian component, z g is a global hidden variable, Indicates about The first predicted label probability. The final predicted label probability p(y n |x n ) is the weighted fusion of the predictions of each mixed component. The following decision function determines the sample x n Posture category:

[0129]

[0130] Among them, p(y n |x n )=[p(1|x n ),p(2|x n ),p(C|x n )].

[0131] Implementation process of human posture detection based on DMAEC

[0132] In the actual human posture detection task, Fig.10 The application process of human posture detection model training based on DMAEC is given, which is mainly divided into two parts: offline modeling stage and online human posture detection stage.

[0133] Offline modeling phase:

[0134] Step 1. Data preprocessing: Collect data from the inertial sensor IMU of the human detection system according to the global information extraction method based on Bi-MIXGU and perform integral calculation to generate a data set Then use UKF to Perform filtering fusion, and the data after filtering fusion is Iteration number n = 1;

[0135] Step 2. Generate sequence data samples: Calculate The autocorrelation ρ(l) of , the l value corresponding to the maximum autocorrelation coefficient is the sequence length T, that is The value of l is usually 0 to 10. The overlap between adjacent sequences is l'. Generate a sequence data set with a sequence length of T represents the i-th sequence, i = 1, 2, ..., K, K represents the number of sequences;

[0136] Step 3. Sequence data standardization: Sequence data set Standardization is performed, and the standardized sequence data set is recorded as Determine the class label of the sequence Where C represents the number of human posture categories C (forward leaning, backward leaning, left leaning, right leaning, bending over, lying down, squatting), x i represents the human body posture data sequence at the i-th moment, the human body posture data x from the i-T+1 moment to the i-th moment i-T+1:i Composition, y i represents the human body posture category label sequence at the i-th moment. If x i belongs to the c'th category, then y i The c'th element of is 1, and the rest are 0.

[0137] Step 4. Determine the structure and parameter initialization of the DMVAEC model:

[0138] Determine the dimension d of the latent space, the number of GMM subcomponents in the latent space, the number of nodes in the softmax fully connected layer of the latent variable classification model, the initial state of the VAE encoder (MIXGU network), the number of hidden layer nodes of the VAE decoder (3-layer neural network), and the number of neighbors;

[0139] Randomly initialize the network connection weights and bias parameters of the DMVAEC model: including the data global information extraction model Bi-MIXGU network, VAE encoder MIXGU network parameter initialization: MIXGU unit connection weights and bias parameters, initial state as shown in formula (1)-(7), MIXGU unit weight adjustment module weight matrix and bias adjacent as shown in formula (8), MIXGU unit prediction output weight matrix as shown in formula (10), VAE decoder neural network, DMVAEC classification layer softmax model network full connection weights and bias parameters, calculate the neighbor similarity matrix S shown in formula (41);

[0140] Step 5. DMVAEC model training

[0141] The preprocessed sequence dataset X is used as the input of the DMVAEC model to calculate the category prediction value and reconstruction value of each sample;

[0142] 1) Calculate the global information and hidden state of sequence data:

[0143] a. Calculate the forward layer hidden state of the Bi-MIXGU network:

[0144] The sequence data X i middle and the forward initial state As an encoder Bi-MIXGU network Figure 3 As shown, the input of the forward layer calculates the hidden state of the first MIXGU unit according to formulas (1)-(9) and (11): Then with and As the second MIXGU unit input to calculate the hidden state The calculation is repeated until the final MIXGU unit outputs the hidden state

[0145] b. Calculate the backward hidden state and predicted output of the Bi-MIXGU network:

[0146] The sequence data X i middle and the initial hidden state As the input of the MIXGU unit in the backward layer, the hidden state of the unit is calculated according to formulas (1)-(9) and (12) Calculate hidden state and The nonlinear function of the input is used as the unit according to formula (10) The predicted output Then with and As the input of the subsequent MIXGU unit, calculate the hidden state of the unit Calculate hidden state and The nonlinear function of the input is used as the unit The predicted output The calculation is repeated until the hidden state of the front MIXGU unit output is obtained. and about The predicted output

[0147] c. Determine the global pattern of sequence data and the distribution of latent variables:

[0148] Based on the hidden state obtained in steps a and b and Use formula (13) to calculate the sequence X i The categorical variable c i , use formula (14) to calculate the global latent variable The mean of the normal distribution With the covariance matrix

[0149] 2) Calculate the hidden variables of VAE:

[0150] Use formula (34) to get the normal distribution The random variables are resampled to obtain the sequence X iThe global latent variable With sequence X i of c i , and the hidden state h of the j-1th MIXGU unit in the VAE encoder j-1 and Hidden variables As the input of the j-th MIXGU unit, according to the j-th MIXGU unit hidden state h j Calculate using formula (25)-(26) Hidden variables Mean vector of the distribution and the covariance matrix

[0151] 3) GMM modeling of latent variables:

[0152] The latent variables are generated from the normal distribution using resampling techniques as in formula (34) Then the dataset composed of latent variables Establish a Gaussian mixture model GMM and obtain the subcomponent category prior probability π of GMM c And the distribution of each subcomponent

[0153] 4) According to GMM clustering Using the reparameterized sampling method, we can obtain Hidden variables

[0154] 5) As the input of the softmax classification network and the decoder neural network, respectively, softmax is used to calculate Probability of belonging to the category The final classification output of the sample The posterior distribution Determine the sample Categories:

[0155] 6) As the input of the decoder NN (3-layer neural network), the decoder output Weighted fusion as The reconstruction value of

[0156] 7) Calculate the DMVAEC loss function value J of the model of the present invention shown in formula (39) according to the calculation results of steps 5) and 6) n ,

[0157] If |J n -J n-1 |>ε, then n←n+1;

[0158] And update the model parameters through back propagation of stochastic gradient descent SGD method; go to step 5), otherwise save the model parameters.

[0159] 8) Calculate the reconstruction error Calculate the mean of the reconstruction error and variance Exit model training.

[0160] Online human pose detection stage

[0161] Step 1. At time t', data is collected from the IMU, temperature sensor, pulse oximeter and GPS in the human body detection system and sent to the server through the single-chip system. The server filters and fuses the collected IMU data according to the method in step 1 of the offline modeling phase.

[0162] Step 2. Generate a sequence of length T based on the data processed in step 1 Standardize according to the offline modeling step 3 method, and the standardized sequence is recorded as X t' ;

[0163] Step 3. Calculate the classification probability of the DMVAEC model output

[0164] 1) Put X t' As the input of the DMVAEC model and forward propagation calculation, the sequence X is obtained t' middle The corresponding hidden variable

[0165] 2) According to the GMM model and reparameterized sampling There is no hidden variable corresponding to the subspace

[0166] 3) Calculate The decoder output is the input And the classification layer softmax output Determine the sample Categories:

[0167] If c* belongs to an abnormal category and the physiological information is abnormal, the execution server sends the abnormal category and personnel identity and location information to the MCU and the administrator, and the MCU issues corresponding voice and light warnings according to the abnormal category;

[0168] 4) As the input of the decoder NN (3-layer neural network), the decoder output Weighted fusion as The reconstruction value of

[0169] 5) If the reconstruction error The execution server sends warnings and personnel information to the MCU system and the administrator, and the MCU sends voice and light warnings; otherwise, go to step 1 and continue monitoring.

[0170] The human posture detection device of the present invention is a safety helmet. In order to accurately collect relevant data such as environment, helmet wearing, body posture detection, etc., it is necessary to collect data from various types of sensors, and the human body detection system is reasonably arranged on the safety helmet. The single-chip microcomputer is arranged at the upper part inside the helmet. The helmet includes a lighting switch and a lighting lamp. The lighting switch and the speech synthesis module are respectively placed at the middle position of the outer side of the helmet, which is convenient for workers to switch on and off. The photoresistor sensor is located directly above the brim of the helmet, and the lighting lamp and the OLED display module are respectively placed directly in front of the helmet. The camera module is arranged at the middle position of the brim of the helmet. The pressure sensor is arranged at the top of the helmet and directly contacts the head to judge whether the helmet is worn correctly. The DS18B20 temperature sensor and the MAX30102 pulse blood oxygen sensor are placed at the back of the helmet and need to be in close contact with the head to measure the head temperature and blood oxygen saturation parameters. The GPS positioning module is arranged at the upper position of the front of the helmet. The infrared obstacle avoidance sensor is located below the brim of the helmet. A sensor integrated box is arranged on the inner side of the rear of the helmet, and a temperature and humidity sensor, a toxic gas sensor, an air pressure sensor, a dust sensor, a gyroscope module, an acceleration sensor, and a WIFI module are arranged in the integrated box to measure external environmental parameters and human posture position data. The working power supply of the helmet is provided by a lithium battery. The present invention achieves the purpose of accurately collecting external data by reasonably placing sensors.

Claims

1. A detection method for a human posture detection system, comprising an offline modeling stage and an online human posture detection stage. It is characterized in that The offline modeling stage includes the following steps: Step 1: Data preprocessing, processing the inertial sensor IMU data in the human posture detection system through the encoder Bi-MIXGU model and generating a sample set; the Bi-MIXGU model is a bidirectional mixed gate unit MIXGU, and the MIXGU is constructed by adaptively weighting the recurrent unit neural network GRU and the simplified model minimum gate unit MGU; Step 2: Generate a data series based on sample autocorrelation and sample set; Step 3: Standardize the data sequence generated in step 2; Step 4: Determine the structure of the dynamic hybrid variational autoencoder classification DMVAEC model and the parameters of the objective function of the optimization problem, and randomly generate model parameters as initial values ​​of the model parameters; Determine the dimension d of the latent space, the number of GMM subcomponents in the latent space, the number of nodes in the softmax fully connected layer of the latent variable classification model, the initial state of the VAE encoder (MIXGU network), the number of hidden layer nodes in the VAE decoder, and the number of neighbors; Randomly initialize the network connection weights and bias parameters of the DMVAEC model: including the data global information extraction model Bi-MIXGU network, VAE encoder MIXGU network parameter initialization: MIXGU unit connection weights and bias parameters, MIXGU unit weight adjustment module weight matrix and bias adjacent, MIXGU unit prediction output weight matrix, VAE decoder neural network, DMVAEC classification layer softmax model network full connection weights and bias parameters, calculate the neighbor similarity matrix S; Step 5: DMVAEC model training: The preprocessed sequence dataset X is used as the input of the DMVAEC model to calculate the category prediction value and reconstruction value of each sample; 1) Calculate the global information and hidden state of sequence data: a. Calculate the forward layer hidden state of the Bi-MIXGU network: The sequence data X i middle and the forward initial state As the input of the encoder Bi-MIXGU forward layer, according to the hidden state of the Bi-MIXGU unit Calculation formula to calculate the hidden state of the first MIXGU unit Then with and As the second MIXGU unit input to calculate the hidden state The calculation is repeated until the final MIXGU unit outputs the hidden state b. Calculate the backward hidden state and predicted output of the Bi-MIXGU network: The sequence data X i middle and the initial hidden state As the input of the backward layer MIXGU unit, according to the hidden state of the Bi-MIXGU unit The calculation formula calculates the hidden state of the unit Calculate hidden state and The nonlinear function of the input is used as the prediction output formula of the MIXGU unit as the unit The predicted output Then with and As the input of the subsequent MIXGU unit, calculate the hidden state of the unit Calculate hidden state and The nonlinear function of the input is used as the unit The predicted output The calculation is repeated until the hidden state of the front MIXGU unit output is obtained. and about The predicted output c. Determine the global pattern of sequence data and the distribution of latent variables: Based on the hidden state obtained in steps a and b and Calculate the sequence X using the calculation formula of category information c i The categorical variable c i , calculate the global latent variable The mean of the normal distribution With the covariance matrix ); 2) Calculate the hidden variables of VAE: The final sampling result formula is used to follow the normal distribution The random variables are resampled to obtain the sequence X i The global latent variable With sequence X i of i c. and the hidden state h of the j-1th MIXGU unit in the VAE encoder j-1 and Hidden variables As the input of the j-th MIXGU unit, according to the j-th MIXGU unit hidden state h j Using the mean μ t and variance Formula calculation Hidden variables Mean vector of the distribution and the covariance matrix ); 3) GMM modeling of latent variables: Generating latent variables from a normal distribution using resampling techniques Then the dataset composed of latent variables Establish a Gaussian mixture model GMM and obtain the subcomponent category prior probability π of GMM c And the distribution of each subcomponent 4) According to GMM clustering Using the reparameterized sampling method, we can obtain Hidden variables c=1,2,…,C; 5) As the input of the softmax classification network and the decoder neural network, respectively, softmax is used to calculate Probability of belonging to the category The final classification output of the sample The posterior distribution Determine the sample Categories: 6) As the input of the decoder NN (3-layer neural network), the decoder output Weighted fusion as The reconstruction value of 7) Calculate the DMVAEC loss function value J of the model of the present invention according to the calculation results of steps 5) and 6) n , If |J n -J n-1 |>ε, then n←n+1; And update the model parameters through back propagation of stochastic gradient descent SGD method; go to step 5), otherwise save the model parameters; 8) Calculate the reconstruction error Calculate the mean of the reconstruction error and variance Exit model training; The online human pose detection stage includes the following steps: Step 1: Collect human posture data from IMU online and process it to generate samples; Step 2: Generate sequence data corresponding to the new sample and standardize it; Step 3: Calculate the classification probability of the DMVAEC model output; Step 4: Determine the human body posture category and send it to the microcontroller system; Step 5: Determine whether the posture is abnormal. If so, issue an alert and send it to the administrator. Otherwise, continue to detect new data.

2. A human posture detection system, comprising a single chip microcomputer, It is characterized in that It also includes infrared obstacle avoidance sensors, pressure sensors, photoresistor sensors, temperature sensors, temperature and humidity sensors, toxic gas sensors, air pressure sensors, dust sensors, GPS positioning modules, speech synthesis modules, gyroscope modules, acceleration sensors, pulse oximeter sensors, camera modules, OLED display modules, and WIFI modules, wherein the gyroscope modules and acceleration sensors belong to the inertial sensor IMU; the single-chip microcomputer collects environmental, physiological, and position parameters from various sensors and modules, processes the data, and uploads the data to the server through the WIFI module, and applies the detection method of a human posture detection system as described in claim 1 to make corresponding decisions and controls on the single-chip microcomputer and the server.

3. A human posture detection system according to claim 2, It is characterized in that The temperature sensor is a DS18B20 temperature sensor, the temperature and humidity sensor is a DHT11 temperature and humidity sensor, the toxic gas sensor is an MQ135 toxic gas sensor, the air pressure sensor is a BMP180 air pressure sensor, the speech synthesis module is a SYN6288 speech synthesis module, the gyroscope module is an MPU6050 gyroscope module, the acceleration sensor is an ADXL345 acceleration sensor, the pulse blood oxygen sensor is a MAX30102 pulse blood oxygen sensor, and the WIFI module is an ESP8266WIFI module.

4. A device for implementing the detection method of a human posture detection system as claimed in claim 1, It is characterized in that The device includes a human posture detection system as described in any one of claims 2 or 3, wherein the device is a safety helmet, the single-chip microcomputer is arranged at the upper part inside the safety helmet, the safety helmet includes a lighting switch and a lighting lamp, and the lighting switch and the speech synthesis module are respectively placed in the middle position of the outer side of the safety helmet; the photoresistor sensor is located directly above the brim of the safety helmet, and the lighting lamp and the OLED display module are respectively placed directly in front of the safety helmet; the camera module is arranged in the middle position of the brim of the safety helmet; the pressure sensor is arranged at the top inside the safety helmet, directly contacting the head to determine whether the safety helmet is worn correctly; the DS18B20 temperature sensor and the MAX30102 pulse blood oxygen sensor are placed at the back inside the safety helmet; the GPS positioning module is arranged at the upper front position of the safety helmet; the infrared obstacle avoidance sensor is located below the brim of the safety helmet; a sensor integration box is provided on the inner side of the rear of the safety helmet, and the integrated box is provided with a temperature and humidity sensor, a toxic gas sensor, an air pressure sensor, a dust sensor, a gyroscope module, an acceleration sensor, and a WIFI module.

5. A device according to claim 4, It is characterized in that The working power source of the safety helmet is provided by a lithium battery.