Dimensional adaptive human behavior prediction method and device

Abstract

The application discloses a kind of dimension self-adapting human behavior prediction method and equipment, the method first acquires the D-dimensional time series containing human behavior, carries out modal decomposition processing, obtains each dimension mode.Secondly, for each mode, the internal relationship between the implied state, activity contribution and observation event is constructed based on the dynamics equation.Then estimate each parameter of the dynamics equation, and construct total parameter set.Finally, the prediction sequence of each mode is inferred based on the total parameter set, and is reconstructed as D-dimensional time series, realize the prediction of human behavior.In the equipment, the central processing module processes the human behavior data collected by the perception module, the abnormal behavior control interaction module alarms, the communication module is used to send the abnormal data back to the personal computer end, and the data is transmitted to the storage module for saving when power failure or network interruption.The application does not need prior knowledge when carrying out human behavior prediction, low cost and high accuracy.

Description

Technical Field

[0001] This invention belongs to the field of human behavior prediction, specifically relating to a dimension-adaptive human behavior prediction method and device. Background Technology

[0002] With the rapid development of artificial intelligence technology, human behavior prediction, as a core technology in human-computer interaction and intelligent sensing, has demonstrated significant application value in scenarios such as intelligent monitoring, rehabilitation medicine, and virtual reality. This technology predicts behavioral intentions by analyzing human motion characteristics. Its core challenge lies in effectively extracting information from multi-sensor data and establishing a behavior representation model with strong generalization capabilities. Current mainstream methods mainly revolve around two major technical routes: computer vision and wearable sensors. Vision-based methods capture human posture sequences through cameras and extract spatiotemporal features using deep learning models; inertial measurement unit (IMU)-based methods acquire biomechanical signals through devices such as accelerometers and gyroscopes to establish motion pattern recognition models.

[0003] Current research still faces significant technical bottlenecks in terms of the accuracy and real-time performance of behavior prediction. Traditional temporal modeling methods, such as Hidden Markov Models (HMMs), are limited by their shallow feature representation capabilities, making it difficult to capture the spatiotemporal correlations of complex actions. While convolutional neural network-based solutions can extract local motion features, their ability to model long-term temporal dependencies is insufficient. Recurrent neural networks and their improved long short-term memory (LSTM) network architectures, which have emerged in recent years, have improved temporal modeling performance, but they still suffer from inherent problems such as vanishing gradients and high computational complexity. Transformer models overcome the sequence length limitation through self-attention mechanisms, but their computational complexity poses a serious challenge to real-time systems. Therefore, how to achieve low-cost, low-complexity, and high-real-time human behavior prediction based on multi-sensor data remains an urgent problem to be solved. Summary of the Invention

[0004] In view of the shortcomings of the aforementioned background technology, the purpose of this invention is to provide a dimension-adaptive human behavior prediction method and device. This method first performs modal decomposition on multi-dimensional time series collected by human behavior sensors, then constructs hidden states for each modality, thereby constructing parameter sets for different modes. Simultaneously, it determines the switching patterns between modes by utilizing the contribution of different modes, and finally achieves human behavior prediction by fusing the prediction results of each modality.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A dimension-adaptive human behavior prediction method, comprising:

[0007] Step 1: Collect a D-dimensional time series X containing human behavior.

[0008] Step 2: Perform modal decomposition on the time series X to obtain the modes in each dimension.

[0009] Step 3: For each mode, construct the intrinsic relationship between the hidden state s, the activity contribution w, and the observed event v based on the dynamic equations.

[0010] Step 4: Estimate the parameters of the dynamic equations and construct the total parameter set M.

[0011] Step 5: Based on the total parameter set M, infer the prediction sequence of each modality and reconstruct it into a D-dimensional time series to achieve the prediction of human behavior.

[0012] In step one, the value of D is determined by the dimension of the data collected by the sensor.

[0013] The mode decomposition method in step two can employ the Multivariate Empirical Mode Decomposition (MEMD) algorithm.

[0014] In step four, when estimating parameters, the Expectation Maximization (EM) algorithm can be used for linear parameters, while the Levenberg-Marquardt (LM) algorithm can be used for nonlinear parameters. Furthermore, the total parameter set M contains parameters for different modes under different modalities.

[0015] A device suitable for the above-mentioned dimensional adaptive human behavior prediction method includes: a perception module, a central processing module, a communication module, a storage module, and an interaction module.

[0016] The sensing module is connected to the central processing module and is used to collect human behavior data and transmit it to the central processing module.

[0017] The central processing module is connected to the sensing module, communication module, storage module, and interaction module, respectively. It is used to process human behavior data collected by the sensing module, specifically including: modal decomposition, estimation of modal parameters, and prediction sequence, thereby realizing human behavior prediction. Once abnormal behavior is detected, the interaction module can be controlled to issue an alarm, or the abnormal data can be sent back to the personal computer (PC) using the communication module. Alternatively, the data can be transferred to the storage module for storage in the event of a power outage or network disconnection.

[0018] The communication module is connected to the central processing module and is used to communicate with the PC or other terminals that conform to the communication protocol. It can send abnormal data to the terminal, or receive instructions from the terminal and transmit them to the central processing module.

[0019] The storage module is connected to the central processing module and is used to store data in the event of a power outage or network interruption.

[0020] The interactive module is connected to the central processing module and is used to display the device's working status and working mode, and can also be used to alarm for abnormal situations.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] (1) The human behavior prediction method proposed in this invention can be used for real-time prediction of human behavior and can detect abnormal behavior in a timely manner, such as falling down and other abnormal behaviors in daily monitoring tasks.

[0023] (2) This invention does not require any prior knowledge when predicting human behavior, such as the type of human behavior, the duration of the behavior, etc., nor does it require any training, verification and test datasets.

[0024] (3) This invention is a low-cost human behavior prediction method that does not require high-cost human behavior acquisition equipment or high-performance data processing equipment, and achieves high-accuracy behavior prediction. Attached Figure Description

[0025] Figure 1 A flowchart of a human behavior prediction method;

[0026] Figure 2 Here is a flowchart of the modal decomposition method;

[0027] Figure 3 A block diagram of a human behavior data acquisition device;

[0028] Figure 4 The result of the mode decomposition is shown in the figure.

[0029] Figure 5 This is a diagram showing the original data and prediction results of human behavior. Detailed Implementation

[0030] The present invention will now be described in detail with reference to specific examples. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention and are not intended to limit the invention; any modifications or substitutions made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0031] like Figure 1 As shown, it illustrates a flowchart of a dimension-adaptive human behavior prediction method, which includes the following steps:

[0032] Step 101: Collect a D-dimensional time series X containing human behavior, where the value of D is determined by the dimension of the data collected by the sensor.

[0033] Step 102: Perform mode decomposition on the time series X to obtain modes in each dimension. The mode decomposition can be implemented using the multivariate empirical mode decomposition (MEMD) algorithm.

[0034] Step 103: For each mode, construct the intrinsic relationship between the hidden state s, the activity contribution w, and the observed event v based on the dynamic equation.

[0035] Step 104: Estimate the parameters of the dynamic equations and construct the total parameter set M.

[0036] Step 105: Infer the prediction sequence of each mode based on the total parameter set M.

[0037] Step 106: Reconstruct the prediction of D-dimensional time series based on the prediction sequences of each modality, and finally realize the prediction of human behavior.

[0038] It should be noted that the specific implementation method of time series mode decomposition based on the MEMD algorithm in step 102 is as follows: Figure 2 As shown. For example, the intrinsic mode function (IMF) of a D-dimensional time series is calculated as follows:

[0039] Step 201: The collected D-dimensional time series can be represented as X(t)={x1(t),x2(t),…,x D (t)}.

[0040] Step 202: Based on the Hammersley sequence, generate a uniform set of sampling points on the D-dimensional hypersphere to obtain the D-dimensional spatial direction vector set. Where K is the number of direction vectors. Let be the angle in the k-th direction.

[0041] Step 203: Calculate the multidimensional time series X(t) in each direction vector Projection on

[0042] Step 204: Find each projection vector The time corresponding to the maximum value Where i is the index of the maximum point.

[0043] Step 205: Use spline functions to find the extreme points Interpolation is performed to obtain K extreme value envelope curves.

[0044] Step 206: For K projection direction vectors, the mean of their envelope curves is:

[0045]

[0046] Step 207: Extract the IMF component g(t) using g(t) = X(t) - m(t). If g(t) satisfies the multivariate IMF standard, then use X(t) - g(t) as the input signal to continue the iterative calculation of steps 203-207 to extract the multivariate IMF component; otherwise, use g(t) as the input signal to continue the iterative calculation of steps 203-207.

[0047] Step 208: Until the stopping condition is met, the multivariate IMF components are finally obtained. The correlation between each order of IMF components in D dimensions and the original signal is calculated. The top L IMF components with high correlation are selected for subsequent analysis. The correlation calculation can preferably use the Pearson correlation coefficient.

[0048] In step 102, the mode decomposition of the multidimensional time series can be performed not only using the MEMD algorithm, but also using Empirical Mode Decomposition (EMD), Ensemble Empirical Mode Decomposition (EEMD), Noise-Assisted Multivariate Empirical Mode Decomposition (NA-MEMD), and their derivative algorithms.

[0049] The stopping condition in step 208 can typically be achieved using the following methods:

[0050] (1) Maximum number of iterations limit: Set a fixed upper limit for the number of iterations to prevent infinite loops, for example, set it to 10;

[0051] (2) IMF stability test: Stop when the difference between the number of extreme points and the number of zero points of the IMF component in continuous iteration does not exceed 1, and the envelope mean approaches zero;

[0052] (3) Residual energy convergence: Referring to the energy difference tracking idea of ​​optimized recursive VMD, the process terminates when the error between the total signal energy and the sum of IMF energy is lower than the convergence threshold, for example, the threshold is set to 1*10^-5;

[0053] It should be noted that in step 103, the intrinsic relationship between the hidden state s, the activity contribution w, and the observed event v is constructed based on the dynamic equation. Taking the i-th mode of the l-th mode of the d-th dimension time series as an example, the specific details are as follows:

[0054]

[0055] in, Let s represent the implicit state of the i-th pattern. ij (t) represents the j-th dimension value of the hidden state of the i-th pattern, k′ represents the dimension of the hidden state, and S i (t)=s i (t) T s i (t) represents the hidden state matrix of the i-th mode, and c represents the number of modes. w represents the activity contribution of the pattern. i The weighting coefficients are set to 1 when estimating model parameters, and v(t) represents the observed events of the current mode. The parameter set for mode switching, t c p represents the sequence length. i Q i and A i U represents the parameters of the state iteration equation. i and V i This represents the parameters of the observation equation.

[0056] It should be noted that, in step 104, the estimation of each parameter of the dynamic equation, taking the i-th mode of the l-th mode of the d-th dimension time series as an example, is specifically done as follows:

[0057] Step 401: Define the actual sequence in the current window as X C The sequence estimated based on the aforementioned kinetic equation is V. C .

[0058] Step 402: Divide the parameters of the dynamic equation into linear dynamic parameters and nonlinear dynamic parameters, wherein the linear dynamic parameters are defined as follows: The nonlinear dynamic parameter is defined as follows:

[0059]

[0060] Step 403: The linear dynamic parameters Iterative optimization algorithms can be used for estimation, such as the EM algorithm, weighted least squares estimation (WLS), or maximum likelihood estimation (MLE). The objective function can be expressed as:

[0061]

[0062] in, The initial state value, s i ′ o For the estimation of the initial state value, θ i ′ L f represents the estimate of the linear dynamic parameters. C This is a generating function.

[0063] Step 404: The nonlinear dynamic parameters Nonlinear least squares method can be used.

[0064] Estimating using the Nonlinear Least Squares (NLS) or LM algorithm, the optimization objective function can be expressed as:

[0065]

[0066] in, The initial state value, s i ′ o For the estimation of the initial state value, θ i ′ N

[0067] f represents the estimation of nonlinear dynamic parameters. C This is a generating function.

[0068] Step 405: The final parameters for the i-th mode are obtained as follows The mode parameters are then added to the total parameter set M.

[0069] It should be noted that different X values ​​can be obtained by moving the current window. C After steps 401-405

[0070] The parameters of multiple different modes can be estimated, thereby enriching the total parameter set M.

[0071] It should be noted that in step 105, the prediction sequence of each mode is inferred based on the total parameter set M. Taking the l-th mode of the d-th dimension time series as an example, the specific method is as follows:

[0072] Step 501: Define the actual sequence in the current window as X C The sequence generated based on the i-th pattern is V. C,i .

[0073] Step 502: Iteratively calculate the sequence generated based on the mode parameters in each mode and the current window sequence X. C The initial state value when the deviation between the two is minimized, where the optimization objective function of the i-th mode can be expressed as follows:

[0074]

[0075] in, The initial state value, f is an estimate of the initial state value. C This is the generating function. If, after traversing all patterns, even the smallest pattern deviation is unacceptable, then consider generating a new pattern through steps 401-405 and updating the total parameter set M.

[0076] Step 503: Obtain the generated sequence V for each mode C,i The objective function for optimizing the activity contribution w(t) of different modes can be expressed as follows:

[0077]

[0078] By iteratively optimizing to find the optimal w, then according to Inferring the predicted sequence of the l-th mode of the d-th dimension time series And add the mode switching parameter set R to the total parameter set M.

[0079] The iterative optimization algorithm in steps 502 and 503 can preferably be the EM algorithm or the MLE algorithm.

[0080] It should be noted that the prediction of reconstructing the D-dimensional time series based on the prediction sequences of each modality in step 106, taking the d-th dimension time series as an example, can be expressed as follows:

[0081]

[0082] After obtaining the predicted sequences for each dimension, combining them yields the predicted D-dimensional time series. Ultimately, this will enable the prediction of human behavior.

[0083] like Figure 3 As shown, it demonstrates a dimension-adaptive human behavior prediction device, including: a perception module 301, a central processing module 302, a communication module 303, a storage module 304, and an interaction module 305.

[0084] The sensing module 301 is connected to the central processing module 302 and is used to collect human behavior data and transmit it to the central processing module 302. The central processing module 302 is connected to the sensing module 301, communication module 303, storage module 304, and interaction module 305, respectively, and is used to process the human behavior data collected by the sensing module 301, specifically including: modal decomposition, estimation of modal parameters, and prediction sequence, thereby realizing human behavior prediction. Once abnormal behavior is detected, the interaction module 305 can be controlled to trigger an alarm, or the communication module 303 can be used to send the abnormal data back to the personal computer (PC). Alternatively, the data can be transmitted to the storage module 304 for storage in the event of a power outage or network interruption. The communication module 303 is connected to the central processing module 302 and is used to communicate with the PC or other terminals conforming to the communication protocol. It can send human behavior data to the terminal or receive instructions from the terminal and transmit them to the central processing module 302. The storage module 304 is connected to the central processing module 302 and is used to store data in the event of a power outage or network interruption. The interactive module 305 is connected to the central processing module 302 and is used to display the working status and working mode of the device, and can also be used to alarm abnormal situations.

[0085] The sensing module 301 can be selected from one or more of the following sensors:

[0086] The triaxial accelerometer is preferably Freescale MMA7361.

[0087] The magnetic sensor is preferably a HONEYWELL HMC100X or HMC102X series magnetic sensor.

[0088] The gyroscope can preferably be an MPU6050 module.

[0089] Central processing module 302 may be selected from one or more of the following modules:

[0090] (1) Low-power microprocessor MSP430.

[0091] (2) Circuit modules designed using FPGA.

[0092] (3) Circuit modules made using microprocessor chips (e.g., Atmel 328P, STM32).

[0093] Communication module 303 can be one or more of the following:

[0094] Wireless communication method: The module supports IEEE 802.15.4 Zigbee wireless transmission or IEEE 802.15.1 Bluetooth wireless transmission mode.

[0095] Wired communication methods: Modules that support serial communication, CAN communication, and IIC communication.

[0096] Storage module 304 can use a 1GB Micro SD card as external storage.

[0097] The interactive module 305 can select three different colored LED indicators to display different working statuses and working modes. It can use rapid flashing to trigger an alarm or use a buzzer to trigger an alarm.

[0098] In addition, motion capture systems can also be applied to the aforementioned dimensional adaptive human behavior prediction methods, including: sensors, signal capture devices, data transmission devices, and data processing devices.

[0099] It should be noted that the sensors, as tracking devices at specific locations on the human body, provide the motion capture system with crucial information about the human's movement position. The number of trackers typically depends on the required level of detail in the capture. The signal capture device varies depending on the type of motion capture system and is primarily used to capture position signals. In a mechanical system, it can be a circuit board for capturing electrical signals; while in an optical motion capture system, a high-resolution infrared camera can be used. The data transmission device is used to rapidly and accurately transmit large amounts of motion data from the signal capture device to the computer system for subsequent processing. The data processing device is used to correct and process the captured data to predict human behavior.

[0100] To further illustrate the effectiveness of the method of this invention, the collected human behavior data was analyzed (3-dimensional time series), such as... Figure 5 The upper part is shown. Based on the method of this invention, MEMD processing of the 3D time series can first yield 7 IMF components for each dimension, as shown in the figure. Figure 4 As shown. Then, the proposed human behavior prediction method is used to construct a dynamic equation for each IMF component, estimate the parameters of the equation, construct the intrinsic relationship between the hidden state, activity contribution, and observed events of the IMF component, and perform sequence prediction, such as... Figure 4 The curve that appears after the horizontal axis reaches point 125 is shown. The final prediction result is as follows. Figure 5 As shown, the upper part represents the original human behavior sequence, and the lower part represents the human behavior sequence predicted based on the present invention, illustrating that the method of the present invention can reliably complete the task of predicting human behavior.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dimension-adaptive human behavior prediction method, characterized in that, Includes the following steps: Step 1: Collect a D-dimensional time series X containing human behavior using sensors; Step 2: Perform mode decomposition on the time series X to obtain the modes in each dimension; Step 3: For each mode, construct the intrinsic relationship between the hidden state s, the activity contribution w, and the observed event v based on the dynamic equations. The specific implementation process is as follows: Based on the dynamic equations, the intrinsic relationship between the hidden state s, the activity contribution w, and the observed event v is constructed. The i-th mode of the l-th mode of the d-th dimension time series is as follows: in, Let s represent the implicit state of the i-th pattern. ij (t) represents the j-th dimension value of the hidden state of the i-th pattern, k′ represents the dimension of the hidden state, and S i (t)=s i (t) T s i (t) represents the hidden state matrix of the i-th mode, and c represents the number of modes. w represents the activity contribution of the pattern. i The weighting coefficients are set to 1 when estimating model parameters, and v(t) represents the observed events of the current mode. The parameter set for mode switching, t c p represents the sequence length. i Q i and A i U represents the parameters of the state iteration equation. i and V i Represents the parameters of the observation equation; Step 4: Estimate the parameters of the dynamic equations and construct the total parameter set M; Step 5: Based on the total parameter set M, infer the prediction sequence of each modality and reconstruct it into a D-dimensional time series to achieve the prediction of human behavior; The specific implementation process of inferring the prediction sequences of each modality based on the total parameter set M is as follows: Step 501: In the l-th mode of the d-th dimension time series, define the true sequence in the current window as X. C The sequence generated based on the i-th pattern is V. C,i ; Step 502: Iteratively calculate the sequence generated based on the mode parameters in each mode and the current window sequence X. C The initial state value when the deviation between the two is minimized, where the optimization objective function of the i-th mode is expressed as: in, The initial state value, f is an estimate of the initial state value. C For generating functions; if the smallest pattern deviation is also accepted after traversing all patterns, then proceed through steps 401-405 to generate a new pattern and update M; Step 503: Obtain the generated sequence V for each mode C,i The contribution of different activity modes, w(t). The optimization objective function is expressed as follows: By iteratively optimizing to find the optimal w, then according to Inferring the predicted sequence of the l-th mode of the d-th dimension time series And add the mode switching parameter set R to the total parameter set M.

2. The dimension-adaptive human behavior prediction method according to claim 1, characterized in that, The specific implementation process of step four is as follows: Step 401: Define the actual sequence in the current window as X C The sequence estimated based on the aforementioned kinetic equation is V. C ; Step 402: Divide the parameters of the dynamic equation into linear dynamic parameters and nonlinear dynamic parameters, wherein the linear dynamic parameters are defined as follows: The nonlinear dynamic parameter is defined as follows: Step 403: The linear dynamic parameters An iterative optimization algorithm is used for estimation, and the objective function is expressed as: in, The initial state value, For the estimation of the initial state value, f represents the estimate of the linear dynamic parameters. C For generating functions; Step 404: Nonlinear dynamic parameters The objective function is estimated using the nonlinear least squares method or the LM algorithm, and is expressed as follows: in, The initial state value, For the estimation of the initial state value, f represents the estimation of nonlinear dynamic parameters. C For generating functions; Step 405: The final parameters for the i-th mode are obtained as follows The mode parameters are then added to the total parameter set M.

3. The dimension-adaptive human behavior prediction method according to claim 2, characterized in that, Step four also includes obtaining different X values ​​by moving the current window. C After steps 401-405, parameters of multiple different modes are estimated, enriching the total parameter set M.

4. The dimension-adaptive human behavior prediction method according to claim 3, characterized in that, The reconstruction is a D-dimensional time series, used to predict human behavior, specifically: After obtaining the predicted sequences for each dimension, combining them yields the predicted D-dimensional time series. Ultimately, this will enable the prediction of human behavior.

5. A dimension-adaptive human behavior prediction device, used to implement the human behavior prediction method according to any one of claims 1 to 3, characterized in that, include: The system comprises a perception module, a central processing module, a communication module, a storage module, and an interaction module. The sensing module is used to collect human behavior data and transmit it to the central processing module. The central processing module is connected to the perception module, communication module, storage module and interaction module respectively, and is used to process the human behavior data collected by the perception module, including: mode decomposition, estimation of mode parameters and prediction sequence, to realize human behavior prediction; The abnormal behavior control interaction module will trigger an alarm, or the communication module will send the abnormal data back to the personal computer PC. In the event of a power outage or network outage, the data will be transferred to the storage module for storage. The communication module is connected to the central processing module and is used to communicate with the PC or other terminals that conform to the communication protocol, send abnormal data to the terminal, or receive instructions from the terminal and transmit them to the central processing module. The storage module is connected to the central processing module and is used to store data in the event of a power outage or network interruption. The interactive module is connected to the central processing module and is used to display the device's working status and working mode, as well as to provide alarms for abnormal situations.

Images