A Human Behavior Segmentation Method and Device without Prior Knowledge

Abstract

The present invention discloses a human behavior segmentation method and device without prior knowledge. The method first collects a time series containing human behaviors, and extracts the first d im dimensions according to the importance degree to construct the time series X. Secondly, by iterating the probability of the time series X at time n with pattern m and state s, traverse and retrieve the pattern and state corresponding to the maximum probability. If a pattern transition occurs at time n, save this time as a candidate segmentation point and perform segmentation. Then classify the segmented time series according to different pattern types, and fit the model parameters of pattern m. Finally, based on the model parameters, with the cost function as a constraint, loop and iterate to traverse and retrieve the optimal value range of pattern m and state s to obtain the best human behavior segmentation result. The present invention does not require any prior knowledge when performing human behavior segmentation, and realizes low-cost and high-precision human behavior segmentation.

Description

Technical Field

[0001] The present invention belongs to the field of human behavior segmentation, and particularly relates to a method and device for human behavior segmentation without prior knowledge. Background Art

[0002] With the increasingly serious problem of population aging, due to the decline of the physical functions of the elderly, they may face many safety risks in daily life, such as bumps, falls, etc. Therefore, the demand for daily health monitoring and safety guarantee of the elderly is increasing day by day. Human behavior recognition technology can predict the health status and timely detect abnormal situations by analyzing the daily behavior patterns of the elderly, such as gait, posture, activity frequency, etc. Once abnormal behaviors are detected, such as staying still for a long time, suddenly falling down, etc., alarm information can be immediately sent to family members or medical institutions to provide timely safety guarantee for the elderly. In addition, human behavior recognition technology can also be applied to smart home systems to realize natural interaction between the elderly and smart homes. The elderly can control home appliances through simple actions or voice commands, improving the convenience and comfort of life. For example, the elderly can control the switches and adjustments of devices such as TVs and air conditioners through gestures without complex operation processes.

[0003] However, the existing human behavior recognition technologies mainly focus on the recognition of single-behavior type and non-continuous human behaviors. For example, when a human body is in a certain behavior type for a long time, it can be discriminated by this technology. However, in reality, human behaviors are of multiple behavior types and continuous changes, and may switch from one behavior type to another at any time without any prior conditions. To achieve continuous human behavior recognition with high accuracy, a very important prerequisite is to detect the time points of conversion between different human behavior types, that is, it is necessary to segment continuous human behaviors. High-precision human behavior segmentation is the cornerstone of subsequent accurate recognition of human behavior types and also the guarantee for the daily health monitoring of the elderly. Summary of the Invention

[0004] In view of the deficiencies of the above background art, the purpose of the present invention is to provide a method and device for human behavior segmentation without prior knowledge. This method first performs implicit mode modeling on the multi-dimensional time series collected by a human behavior sensor, then finds the optimal segmentation point based on the dynamic programming algorithm, and finally realizes the segmentation of human behavior by minimizing the cost function, providing prior knowledge for the accurate recognition of human behavior.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A method for human behavior segmentation without prior knowledge, comprising:

[0007] Step 1: Collect d containing human behavior org-dimensional time series, sorted according to the importance of the information contained in the said d org -dimensional time series, and the first d im dimensions are extracted to construct the time series X.

[0008] Step 2: By iteratively calculating the probability that the said d im -dimensional time series X is in mode m and state s at time n, traverse and retrieve the mode and state corresponding to the maximum probability. If a mode transition occurs at time n, save this time as a candidate segmentation point and perform segmentation.

[0009] Step 3: Classify the segmented time series according to different mode types, and fit the model parameters θ of mode m m .

[0010] Step 4: Based on the model parameters θ m , with the cost function as a constraint, iteratively traverse and retrieve the optimal value ranges of mode m and state s, and finally obtain the best segmentation result of human behavior.

[0011] In the said Step 1, the value of d org is determined by the data dimension collected by the sensor. The evaluation criteria for the importance of the information contained in the time series can be the entropy value, sample entropy value, variance, or mean square error of the sequence.

[0012] In the said Step 2, traversing and retrieving the mode and state corresponding to the maximum probability can be implemented using a dynamic programming algorithm, such as the Viterbi algorithm.

[0013] In the said Step 3, fitting parameters based on the classified sub-time series can use the Expectation Maximization (EM) algorithm or the Baum-Welch algorithm.

[0014] In the said Step 4, iteratively traversing and retrieving the optimal value ranges of mode m and state s can be implemented using a greedy algorithm.

[0015] A device applicable to the above human behavior segmentation method without prior knowledge, including: a sensing module, a control module, a communication module, a storage module, and an interaction module;

[0016] The sensing module is connected to the control module and is used to collect human behavior data and transmit it to the control module.

[0017] The control module is respectively connected to the sensing module, the communication module, the storage module and the interaction module, and is used to process the human body behavior data collected by the sensing module, and use the communication module to send the data back to the personal computer (PC) side. In the case of power failure or network disconnection, the human body behavior data can also be transmitted to the storage module for storage. In addition, the control interaction module displays the working state of the device, and an alarm can also be given through the interaction module once the device has an abnormality.

[0018] The communication module is connected to the control module and is used to communicate with the PC side or other terminals that conform to the communication protocol. It can send the human body behavior data to the terminal and can also receive instructions from the terminal in the reverse direction and transmit them to the control module.

[0019] The storage module is connected to the control module and is used to store data in the case of power failure or network interruption.

[0020] The interaction module is connected to the control module and is used to display the working state and working mode of the device, and can also be used for alarming in case of abnormal situations.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] (1) The human body behavior segmentation method proposed by the present invention can be used to process continuous human body behaviors and segment them into independent types of human body behavior data segments, which is convenient for improving the recognition accuracy of subsequent human body behavior types.

[0023] (2) When the present invention performs human body behavior segmentation, no prior knowledge is required, such as the type of human body behavior, the number of each behavior type, the duration of the behavior, etc., and no training, verification and test data sets are required either.

[0024] (3) The present invention is a low-cost human body behavior segmentation method, which does not require high-cost human body behavior acquisition equipment, nor high-performance data processing equipment. Description of the Drawings

[0025] Figure 1 is the flowchart of the human body behavior segmentation method;

[0026] Figure 2 is the flowchart of the time series sample entropy calculation;

[0027] Figure 3 is the flowchart of the time series segmentation point retrieval;

[0028] Figure 4 is the block diagram of the human body behavior data acquisition device;

[0029] Figure 5 is the diagram of the human body behavior segmentation result. Detailed implementation mode

[0030] The present invention will be described in detail below with reference to specific examples. The description of these examples is only for helping to understand the method and core idea of the present invention, and is not used to limit the present invention; any modification or replacement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0031] As Figure 1 shown, it shows a flowchart of a human behavior segmentation method without prior knowledge, specifically including the following steps:

[0032] Step 101: Collect a d org -dimensional time series containing human behaviors, where the value of d org is determined by the data dimension collected by the sensor.

[0033] Step 102: Sort according to the importance degree of the information contained in the d org -dimensional time series, and extract the first d im dimensions to construct a time series X. The importance degree of the information contained in the series can be measured by the sample entropy value of the series.

[0034] Step 103: Iteratively calculate the probability that the d im -dimensional time series is in mode m and state s at time n.

[0035] Step 104: Traverse and retrieve the mode and state corresponding to the maximum probability through the Viterbi algorithm.

[0036] Step 105: If a mode conversion occurs at time n, save this time as a candidate segmentation point.

[0037] Step 106: Classify the segmented time series according to different mode types, and fit the model parameter θ m of mode m based on the Baum-Welch algorithm.

[0038] Step 107: Based on the model parameter θ m , with the cost function as a constraint, iteratively traverse and retrieve the optimal value range of mode m and state s. The specific formula of the cost function is as follows:

[0039]

[0040] where, Cost T represents the total cost, Cost C (X|Θ) represents the coding cost, Cost M (Θ) represents the model cost, Cost L represents the sequence length cost, CostP represents the parameter cost, α represents the weight coefficient of the encoding cost, β represents the weight coefficient of the model cost, λ represents the weight coefficient of the sequence length cost, X represents the input time series, Θ={θ1,θ2,L,θ g , Δ} represents the model parameter set, θ m represents the parameters of the mth mode, where m∈{1,2,L,g}, Δ represents the mode conversion matrix, seg i represents the i-th data segment after segmentation, where i∈[1,2,L,l], N represents the length of the time series, and d im represents the time series dimension, l represents the number of data segments after segmentation, g represents the number of patterns, log * The universal encoding length for integers can be defined as log * (x) = log2(x) + log2log2(x) + L, where the sum contains only positive terms.

[0041] Step 108: Obtain the best segmentation result of human behavior.

[0042] It should be noted that in step 102, the first d im The specific implementation method of dimensional time series is as follows Figure 2 For example, for a 1-dimensional time series x(n) = {x(1), x(2), L, x(N)} consisting of N data, the calculation method of its sample entropy is as follows:

[0043] Step 201: Reorganize the time series into X c (1),X c (2),L,X c (N-c+1), where X c (i) = {x(i), x(i+1), L, x(i+c-1)}, 1≤i≤N-c+1, these vectors represent c consecutive x values starting from point i.

[0044] Step 202: Define vector X c (i) With X c (j) The distance d between c [X c (i),X c (j)] is the absolute value of the maximum difference between the corresponding elements of the two, that is:

[0045] d c [X c (i),X c (j)]=max k＝0,1,L,c-1 (x(i+k)-x(j+k))

[0046] Step 203: For a given Xc (i), count X c (i) and X c The number of j whose distance from (j) is less than or equal to the similarity tolerance h is denoted as B i . For 1 ≤ i ≤ N - c, define:

[0047]

[0048] Define B c (h) as:

[0049]

[0050] Step 204: Increase the dimension to c + 1, and count the number of j whose distance between X c+1 (i) and X c+1 (j) is less than or equal to the similarity tolerance h, denoted as A i . Similarly is defined as:

[0051]

[0052] Define A c (h) as:

[0053]

[0054] Step 205: In this way, B c (h) is the probability that two sequences match c points under the similarity tolerance h, and A c (h) is the probability that two sequences match c + 1 points, then the sample entropy is:

[0055]

[0056] When N is a finite value, it can be estimated by the following formula:

[0057]

[0058] It should be noted that the specific implementation methods of the iterative processes in Step 103, Step 104, and Step 105 are as Figure 3 shown. Assume that the optimal value ranges of patterns m and m′ are [1, g], and m ≠ m′, and the value range of n is [1, N].

[0059] Step 301: Input the sequence X and initialize n = 1, m = 1.

[0060] Step 302: Iteratively calculate the probability The specific calculation formula is as follows:

[0061]

[0062] Among them, δ m′m represents the probability that the mode transitions from m' to m; represents that at the (n - 1)th moment, the mode is m' and the state is s' j when obtaining the optimal probability; s i , s j ∈{1, 2, …, k m} represents different states in mode m, and s i ≠s j ; s i ', s' j ∈{1, 2, …, k m′} represents different states in mode m', and s i '≠s' j ; represents the initialization probability that the mode is m and the state is s i ; represents the output probability of x i when the mode is m and the state is s n ; represents the transition probability from state s j to s i when the mode is m.

[0063] Step 303: Update the set of candidate segmentation points That is, when the mode changes in Step 302, save this moment as a candidate segmentation point.

[0064] Step 304: Determine whether m is less than or equal to g. If it holds, let m = m + 1 and enter the next round of iteration; otherwise, enter Step 305.

[0065] Step 305: Determine whether n is less than or equal to N. If it holds, let n = n + 1, m = 1, and enter the iterative calculation at the (n + 1)th moment; otherwise, enter Step 306.

[0066] Step 306: Obtain the set of optimal candidate segmentation points P cut , and obtain the set of sub-time series X sub based on the optimal candidate segmentation points.

[0067] It should be noted that the specific implementation method of the fitting process of the model parameter θ m ={Π m , A m , B m} of mode m in Step 106 is as follows. Taking the sub-time series O = {O1, O2, …, O D} as an example, where the sample of the d-th observation sequence is Od = {xd(1), xd(2), …, xd(Nd)}, and the model parameters are θ1 = {Π1, A1, B1}:

[0068] Step 1: The initialization of the model parameters can be carried out in one of the following ways:

[0069] (1) Initialize the model parameters randomly.

[0070] (2) Randomly select sub-time series to estimate the initialization parameters of the model.

[0071] (3) Uniformly sample sub-time series to estimate the initialization parameters of the model.

[0072] Step 2: Update the model parameters:

[0073]

[0074]

[0075]

[0076] Among them, represents the probability that the model parameters θ1 and the observation sequence O d are in state s at time n; i ; represents the probability that the model parameters θ1 and the observation sequence O d are in state s at time n i and in state s at time n + 1; v j represents the observation state. kk ;

[0077] Step 3: If converges, output the model parameters; otherwise, execute Step 2 to enter the next round of iterative calculation. The model parameter fitting process for the remaining modes is the same as that of Mode 1.

[0078] It should be noted that the specific calculation method of the coding cost in Step 107 can be expressed as:

[0079]

[0080] Among them, p(X[seg i |θ m ) represents the likelihood function of seg i ; θ m is the best mode parameter of seg i ; The weight coefficient α of the coding cost is used to optimize the proportion of the coding cost in the total cost and can be set to 0.8. The specific calculation method of the model parameter cost in the model cost function can be expressed as:

[0081]

[0082] Among them, Fc represents the floating-point cost, which can be set to 32; k m represents the number of states in mode m. The specific calculation method of the mode conversion cost can be expressed as Cost M (Δ) = Fc·g 2 , the weight coefficient β of the model cost is used to optimize the proportion of the model parameter cost and the mode conversion cost in the total cost, and can be set to 1.2. The weight coefficient λ of the sequence length cost is used to balance the cost between the length and the number of sub-time series, and can be set to 1.

[0083] As Figure 4 shown, it shows a device applicable to the above-mentioned human behavior segmentation method without prior knowledge, including: a sensing module 401, a control module 402, a communication module 403, a storage module 404, and an interaction module 405.

[0084] Among them, the sensing module 401 is connected to the control module 402 and is used to collect human behavior data and transmit it to the control module 402. The control module 402 is respectively connected to the sensing module 401, the communication module 403, the storage module 404, and the interaction module 405, and is used to process the human behavior data collected by the sensing module 401, and use the communication module 403 to send the data back to the PC side. It can also transmit the human behavior data to the storage module 404 for storage in the case of power failure or network disconnection. In addition, the control interaction module 405 displays the working state of the device, and can also give an alarm through the interaction module 405 once the device has an abnormality. The communication module 403 is connected to the control module 402 and is used to communicate with the PC side or other terminals that conform to the communication protocol. It can send the human behavior data to the terminal, or receive instructions from the terminal in reverse and transmit them to the control module 402. The storage module 404 is connected to the control module 402 and is used to store data in the case of power failure or network interruption. The interaction module 405 is connected to the control module 402 and is used to display the working state and working mode of the device, and can also be used for alarming abnormal situations.

[0085] The sensing module 401 can select one or more of the following multiple sensors:

[0086] A three-axis acceleration sensor, which can preferably be the Freescale MMA7361.

[0087] A magnetic sensor, which can preferably be the HMC100X and HMC102X series magnetic sensors of HONEYWELL.

[0088] A gyroscope, which can preferably be the MPU6050 module.

[0089] The control module 402 can select one or more of the following modules:

[0090] (1) Low-power microprocessor MSP430.

[0091] (2) Circuit module designed using FPGA.

[0092] (3) Circuit module made using microprocessor chips (such as Atmel 328P, STM32).

[0093] The communication module 403 can select one or more of the following:

[0094] Wireless communication method: Modules that support IEEE 802.15.4 Zigbee wireless transmission or IEEE 802.15.1 Bluetooth wireless transmission mode.

[0095] Wired communication method: Modules that support serial communication, CAN communication, and IIC communication functions.

[0096] The storage module 404 can select 1GB Micro SD as external storage.

[0097] The interaction module 405 can select three different color LED indicators to display different working states and working modes, can alarm in a fast strobe manner, or alarm with a buzzer.

[0098] In addition, the motion capture system can also be applied to the above-mentioned prior-knowledge-free, low-cost, and fast human behavior segmentation method, including: sensors, signal capture devices, data transmission devices, and data processing devices.

[0099] It should be noted that the sensor, as a tracking device at specific positions on the human body, is used to provide key information about the human body's movement position to the motion capture system. The number of the trackers usually depends on the required capture detail. The signal capture device varies depending on the type of the motion capture system and is mainly used to capture position signals. In a mechanical system, it can be a circuit board for capturing electrical signals; while in an optical motion capture system, a high-resolution infrared camera can be used. The data transmission device is used to quickly and accurately transmit a large amount of motion data from the signal capture device to the computer system for subsequent processing. The data processing device is used to correct and process the captured data to achieve the segmentation of human behavior.

[0100] Based on the above-mentioned prior-knowledge-free, low-cost, and fast human behavior segmentation method and its device, the final human behavior segmentation effect is as Figure 5 shown.

[0101] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A human body behavior segmentation method without prior knowledge, characterized in that, It includes the following steps: Step 1: Collect a d-dimensional time series containing human behaviors, sort it according to the importance of the information contained in the d-dimensional time series, and extract the first d dimensions to construct a time series X; org d org d im dimensions to construct a time series X; Step 2: Calculate d through iteration im The probability that the d-dimensional time series X has a pattern of m and a state of s at time n. Traverse the pattern and state corresponding to the maximum retrieval probability. If a pattern transition occurs at time n, save this time as a candidate segmentation point and perform segmentation; Step 3: Classify the segmented time series according to different pattern types and fit the model parameters θ of pattern m m ; The model parameters θ of the said mode m m ={Π m , A m , B m} The fitting process is specifically as follows: The sub-time series O of mode 1 = {O1, O2, …, O D}, where the sample of the d-th observation sequence is O d = {x d (1), x d (2), …, x d (N d )}, and the model parameters are θ1 = {Π1, A1, B1}: Step 3.1: Initialize the model parameters; Step 3.2: Update the model parameters, and the formula is as follows: Among them, represents the probability that the model parameter θ1 and the observation sequence O d are in state s at time n; i The probability; represents the probability that the model parameter θ1 and the observation sequence O d are in state s at time n i and in state s at time n + 1; v j The probability; kk represents the observation state; Step 3.3: If converges, output the model parameters; otherwise, execute Step 3.2 to enter the next round of iterative calculation; the model parameter fitting process for the remaining patterns is the same as that of Pattern 1; Step 4: Based on the model parameter θ m , with the cost function as the constraint, iteratively traverse the optimal value ranges of the retrieval pattern m and the state s in a loop to obtain the optimal segmentation result of human behavior; The specific formula of the cost function is as follows: Among them, Cost T represents the total cost, Cost C (X|Θ) represents the encoding cost, Cost M (Θ) represents the model cost, Cost L represents the sequence length cost, Cost P represents the parameter cost, α represents the weight coefficient of the encoding cost, β represents the weight coefficient of the model cost, λ represents the weight coefficient of the sequence length cost, X represents the input time series, Θ = {θ1, θ2, …, θ g , Δ} represents the set of model parameters, θ m represents the parameter of the m-th mode, where m ∈ {1, 2, …, g}, Δ represents the mode transition matrix, seg i represents the i-th data segment after segmentation, where i ∈ [1, 2, …, l], N represents the time series length, d im represents the time series dimension, l represents the number of data segments after segmentation, g represents the number of modes, log * represents the universal coding length of an integer, defined as log * (x) = log2(x) + log2log2(x) + …, where the sum only contains positive terms.

2. The human body behavior segmentation method without prior knowledge according to claim 1, wherein The said d org is determined by the data dimension collected by the sensor; the evaluation criteria for the importance degree of the information contained in the time series are the entropy value, sample entropy value, variance or mean square error of the series.

3. The human body behavior segmentation method without prior knowledge according to claim 1, characterized in that, Traverse the patterns and states corresponding to the maximum value of the retrieval probability, and implement it using the dynamic programming algorithm.

4. The method for human behavior segmentation without prior knowledge according to claim 1, characterized in that The specific implementation process of the second step is as follows: Step 2.1: Set the optimal value ranges of patterns m and m′ to be [1, g], and m≠m′. The value range of n is [1, N]. Input the sequence X, and initialize n = 1, m = 1; Step 2.2: Iteratively calculate the probability The specific calculation formula is as follows: where, δ m′m represents the probability of the mode transitioning from m′ to m; represents that at the (n - 1)th moment, the mode is m′ and the state is s j ′, achieving the optimal probability; s i , s j ∈ {1, 2, …, k m} represents different states in mode m, and s i ≠ s j ; s i ′, s j ′ ∈ {1, 2, …, k m′} represents different states in mode m′, and s i ′ ≠ s j ′; represents the initialization probability when the mode is m and the state is s i ; represents the output probability of x i when the mode is m and the state is s n ; represents the transition probability of the state from s j to s i when the mode is m; Step 2.3: Update the candidate segmentation point set s i ∈ {1, 2, …, k m}, that is, when the pattern is converted in Step 2.2, save this moment as a candidate segmentation point; Step 2.4: Determine whether m is less than or equal to g. If it holds, then let m = m + 1 and enter the next round of iteration; otherwise, enter Step 2.5; Step 2.5: Determine whether n is less than or equal to N. If it holds, then let n = n + 1, m = 1, and enter the iterative calculation at the (n + 1)-th moment; otherwise, enter Step 2.6; Step 2.6: Obtain the set P of the optimal candidate segmentation points cut , and obtain the set X of sub-time series based on the optimal candidate segmentation points sub .

5. The method for human behavior segmentation without prior knowledge according to claim 4, wherein The initialization of the model parameters includes the following three methods: (1) Randomly initialize the model parameters; (2) Randomly select a sub-time series to estimate the initialization parameters of the model; (3) Uniformly sample the sub-time series to estimate the initialization parameters of the model.

6. The method for segmenting human body behaviors without prior knowledge according to claim 5, characterized in that The specific calculation method of the encoding cost is expressed as: Among them, p(X[seg i |θ m ) represents the likelihood function of seg i ; θ m is the optimal mode parameter of seg i ; the weight coefficient α of the encoding cost is used to optimize the proportion of the encoding cost in the total cost; The specific calculation method of the model parameter cost in the model cost function is expressed as: Among them, Fc represents the floating-point cost; k m represents the number of states in pattern m; the specific calculation method of the pattern conversion cost is expressed as Cost M (Δ) = Fc·g 2 , the weight coefficient β of the model cost is used to optimize the proportion of the model parameter cost and the pattern conversion cost in the total cost, and the weight coefficient λ of the sequence length cost is used to balance the cost between the length and the number of sub-time series.

7. The method for human behavior segmentation without prior knowledge according to claim 1, characterized in that In the fourth step, loop through and iterate to traverse the optimal value ranges of the retrieval pattern m and the state s, and implement it using the greedy algorithm.

8. A human body behavior segmentation device without prior knowledge, used to implement the segmentation method described in any one of claims 1 to 7, characterized in that, It includes a sensing module, a control module, a communication module, a storage module, and an interaction module; The sensing module is connected to the control module and is used to collect human behavior data and transmit it to the control module; The control module is respectively connected to the sensing module, the communication module, the storage module, and the interaction module. It is used to process the human behavior data collected by the sensing module, and use the communication module to send the data back to the personal computer. In the case of power failure or network disconnection, it transmits the human behavior data to the storage module for storage. In addition, it controls the interaction module to display the working state of the device, and alarms through the interaction module when the device has an abnormality; The communication module is connected to the control module and is used to communicate with the PC or other terminals that conform to the communication protocol, send the human behavior data to the terminal, and receive instructions from the terminal in reverse and transmit them to the control module; The storage module is connected to the control module and is used to store data in the case of power failure or network interruption; The interaction module is connected to the control module and is used to display the working state and working mode of the device, and for alarming in case of abnormal situations.

Images