Multi-type tumble situation detection method, system, device and medium

Abstract

The invention belongs to the technical field of tumble detection, and discloses a multi-type tumble situation detection method, system and device and a medium, and the method comprises the steps: obtaining a tumble data set, building an initial tumble situation detection model based on a time convolution network, carrying out the weight optimization and training of the initial tumble situation detection model based on the tumble data set, and obtaining a tumble situation detection result. Obtaining a trained tumble situation detection model; performing hyper-parameter optimization on the trained tumble situation detection model based on a Bayesian optimization method to obtain an optimized tumble situation detection model; and deploying the optimized tumble situation detection model to the intelligent wearable device, and monitoring the tumble situation of the user in real time. According to the technical scheme, the method has the advantages of being high in descent speed and good in global convergence, and meanwhile the defects that the Newton method is large in calculation amount and sensitive to initialization weight are overcome.

Description

Technical Field

[0001] The present invention belongs to the technical field of fall detection, and in particular relates to a method, system, device and medium for detecting multiple types of fall situations. Background Art

[0002] There are two main problems with current fall detection algorithms: insufficient differentiation of fall types and lack of fall severity assessment. Traditional fall detection algorithms can usually only identify two situations: falls and normal, and cannot classify fall types in detail, which limits the application of algorithms in medical assistance, rehabilitation assessment and other fields. At the same time, the algorithm is also unable to assess the severity of a fall based on factors such as the posture, speed, and impact force of the fall, which is crucial for emergency rescue judgment.

[0003] In deep learning classification models, the sigmoid function is often used as the output layer activation function for binary classification problems, but is not suitable for multi-classification problems. In contrast, the softmax function can output a probability distribution, which is suitable for multi-classification problems. However, the currently commonly used stochastic gradient descent (SGD) algorithm has shortcomings such as unstable updates, slow convergence, the need to adjust the learning rate, and easy to fall into local optimality when used as an optimizer.

[0004] In order to overcome the shortcomings of the SGD algorithm, the Newton method as an optimizer has the advantages of fast convergence speed, strong global optimization ability and adaptability. However, the Newton method also has shortcomings such as high computational complexity and large memory consumption, especially when dealing with large-scale deep learning models.

[0005] In summary, in order to improve the accuracy and practicality of fall detection algorithms, it is necessary to develop algorithms that can identify multiple types of falls and evaluate the severity of falls. At the same time, when selecting an optimizer, it is necessary to weigh factors such as computational complexity, memory consumption, and convergence speed to find the optimizer that best suits the current task. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-type fall situation detection method, system, device and medium to solve the problems existing in the above-mentioned prior art.

[0007] To achieve the above object, the present invention provides a multi-type fall situation detection method, comprising: Acquire a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; Building an initial fall situation detection model based on a temporal convolutional network, wherein an output layer activation function of the initial fall situation detection model is a softmax activation function; performing weight optimization and training on the initial fall situation detection model based on the fall data set to obtain a trained fall situation detection model; Based on the Bayesian optimization method, hyperparameter optimization is performed on the trained fall detection model to obtain an optimized fall detection model; The optimized fall detection model is deployed on smart wearable devices to monitor the user's fall situation in real time.

[0008] Optionally, the inertial measurement unit sensor signal data includes three-axis gravity acceleration and gyroscope signals.

[0009] Optionally, the weight optimization and training of the fall situation detection model based on the fall data set specifically includes: S11: Initializing the model weight of the initial fall situation detection model; S12: inputting the sensor training data into the initial fall situation detection model for classification prediction, and calculating error data between the initial training result after classification prediction and the fall type label corresponding to the sensor training data; S13: Calculating the update amount of the model weight based on the small batch increment rule, and updating and adjusting the model weight based on the update amount; S14: Repeat steps S12 to S13 to iteratively train the initial fall situation detection model until the error data is less than a preset threshold, stop iteration, and output the trained fall situation detection model.

[0010] Optionally, the updating amount of the model weight is calculated based on the small batch increment rule, specifically including: , In the formula, k=1,...,M,...,N For the k The weights of the training data are updated by samples. N ≥ M , α is the control factor, w ij is the weight, ∆ w ij Update the value for the weight, e i is the error data, soft max() is the activation function, To reduce the control value, x i is the ith input, x j is the jth output.

[0011] Optionally, the performing hyperparameter optimization on the trained fall situation detection model based on the Bayesian optimization method specifically includes: S21: Building a surrogate model for simulating the objective function based on the falls dataset; S22: Using the uncertainty of the surrogate model prediction and the expected improvement of the objective function, select the next hyperparameter sampling point; S23: Train the deep learning model and evaluate the performance under the selected hyperparameter settings, and update the proxy model based on the performance feedback; S24: Repeat steps S22-S23, stop when a predetermined number of sampling times is reached or the objective function value is less than a preset threshold, and output the optimized fall situation detection model.

[0012] A multi-type fall situation detection system, comprising: A data acquisition module, used to obtain a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; A model building module, used to build an initial fall situation detection model according to a temporal convolutional network, wherein an output layer activation function of the initial fall situation detection model is a softmax activation function; The model optimization and application module is used to optimize and train the weights of the initial fall situation detection model according to the fall data set to obtain a trained fall situation detection model; optimize the hyperparameters of the trained fall situation detection model based on the Bayesian optimization method to obtain an optimized fall situation detection model; and deploy the optimized fall situation detection model on the smart wearable device to monitor the user's fall situation in real time.

[0013] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform a multi-type fall situation detection method.

[0014] A computer-readable storage medium stores a computer program, wherein the computer program implements the multi-type fall situation detection method when executed by a processor.

[0015] The technical effects of the present invention are: The present invention proposes an SSNND method that combines softmax as the output layer activation function and the improved Newton method as the descent optimization; the SSNND method not only fully retains the advantages of the Newton method's fast descent speed and has the advantage of good global convergence, but also overcomes the shortcomings of the Newton method's large amount of computation and sensitivity to initialization weights; the model SSNND-TCN proposed in the present invention has good performance in the detection of multiple types of fall situations based on IMU signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of updating weights using node means of part of the sample data in an embodiment of the present invention; Figure 2 A schematic diagram of weight update using the node mean of all sample data in an embodiment of the present invention; Figure 3 A schematic diagram of weight update using node values ​​of all single sample data in an embodiment of the present invention; Figure 4 A schematic diagram comparing the training processes of the SSNND optimization algorithm and the SGD algorithm in an embodiment of the present invention; Figure 5 FIG. 4 is a flow chart of fall situation detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0019] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0020] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] like Figure 1 - Figure 5As shown, in this embodiment, a multi-type fall situation detection method is provided, including: obtaining a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; constructing an initial fall situation detection model based on a temporal convolutional network, wherein the output layer activation function of the initial fall situation detection model is a softmax activation function; performing weight optimization and training on the initial fall situation detection model based on the fall data set to obtain a trained fall situation detection model; performing hyperparameter optimization on the trained fall situation detection model based on a Bayesian optimization method to obtain an optimized fall situation detection model; and deploying the optimized fall situation detection model on a smart wearable device to perform real-time monitoring of the user's fall situation.

[0023] This embodiment proposes a method for accelerating the training process of a multi-type fall situation detection model by fusing multiple sensor signals, aiming to solve the problem that when smart wearable devices represented by smart watches realize multi-type fall recognition, the collected three-axis gravity acceleration and gyroscope signals need to be processed simultaneously, resulting in a slow convergence speed of the deep learning model training process and low accuracy of the inference model.

[0024] In order to overcome the problems of slow convergence speed and low accuracy of deep learning models based on commonly used optimizers in fusing multi-sensor signals for multi-type fall situations and in the training process of detection models, this embodiment proposes an SSNND method that combines softmax as the output layer activation function and the improved Newton method as the descent optimization method; the SSNND method fully retains the advantages of the Newton method's fast descent speed and good global convergence, while overcoming the shortcomings of the Newton method's large amount of computation and sensitivity to initialization weights; the model SSNND-TCN proposed in this embodiment has good performance in the detection of multi-type fall situations based on IMU signals.

[0025] Among them, the softmax function (Here i Indicates i categories, the total number of classification categories) is one of the commonly used activation functions in deep learning multi-classification models. It has a relatively smooth curve, has a good representation ability for the nonlinear characteristics of the input, and has multi-order differentiability.

[0026] The first-order derivative of the softmax function is: , The corresponding second-order derivative is: .

[0027] Therefore, this embodiment trains the deep learning model by retaining the advantages of Newton's method as an optimizer in the deep learning classification model training process as much as possible, and overcoming its shortcomings in large amount of computation and memory consumption. At the same time, it overcomes the problem that SGD is prone to local convergence (that is, the accuracy of the model can be improved), and improves the convergence speed. This embodiment uses Newton's method combined with the softmax function as the activation function of the deep learning model to propose an improved deep learning optimizer method. Since the initialization in the deep learning model process is also random, this embodiment is named as the randomly improved Newton gradient method (SSNND) combined with the softmax function as the output layer activation function.

[0028] Therefore, the SSNND proposed in this embodiment is applied to the training of the classification model for multi-type fall situation detection, which not only overcomes the problem that SGD is prone to local convergence (that is, it can improve the accuracy of the model), but also improves the convergence speed. It can also overcome the shortcomings of Newton's method such as high computational complexity, convergence problems and sensitivity restrictions on the initial point, and has the complexity advantage of being simple and easy to implement similar to SGD.

[0029] In view of the above content, smart wearable devices represented by smart watches use three-axis gravity acceleration and gyroscope signals as inputs of the classification model for multi-type fall situation detection, and the model training process converges slowly, and the inference model obtained after training is not accurate. This embodiment provides a method that can overcome the slow convergence speed of the multi-type fall situation detection model training process and the low accuracy of the inference process. At the same time, the classic temporal convolutional network (TCN) is used as the network structure of the classification model for multi-type fall situation detection, and the hyperparameters of TCN (including the convolution layer of TCN, learning rate) are optimized based on SGD using Bayesian In order to fully verify the effectiveness of the SSNND proposed in this embodiment. This embodiment discloses a method for accelerating the training process of a multi-type fall situation detection model by fusing multiple sensor signals to improve the Newton optimizer method, including The principle of generalized incremental rule, the principle of small batch rule, TCN multi-classification model construction, the principle of improved Newton optimizer algorithm based on softmax as the output layer activation function, and the principle of Bayesian optimization of deep learning model hyperparameters.

[0030] Principle of generalized incremental rule: The training steps of single-layer neural network using generalized incremental rule include: (1) Initialize weight values; (2) Obtain "input" from the training data. The training data format is {"input", "target input"}, and then pass the "input" to the deep learning network model to calculate the target output d i With model outputy i The error between e i ,Right now e i = d i - y i ; (3) Calculate the weight update according to the generalized incremental rule, that is, , here The first-order derivative of the softmax function is used, that is, the SGD method updates the weights. v i represents the weighted sum of the output nodes; (4) Adjust the weight, that is ; (5) Repeat steps (2) to (4) for all training data; (6) Repeat steps (2) to (5) until the error e i Less than or equal to the set threshold.

[0031] Principle of small batch learning: Small batch learning is between batch learning and online learning. It is a common learning rule in the deep learning model training process. The differences between the three are: Mini-batch learning processes data in groups and uses the node mean of some sample data to update the weights, that is, ,in k=1,...,M,...,N , number of samples N ≥ M ,like Figure 1 shown.

[0032] Batch learning does not need to group data, and uses the node mean of all sample data to update weights, that is, ,in k=1,...,M,...,N , number of samples N ,like Figure 2 shown.

[0033] Online learning only targets one set of data at a time, and uses the node values ​​of all single sample data to update the weights, that is, ,in k=1,...,M,...,N , number of samples N ,like Figure 3 shown.

[0034] TCN multi-classification model construction: First, the input of the model is determined to be a 6-second IMU sensor signal, that is, a sequence of dimension [5*fs, 6]; the output of the model is activated by softmax to be the probability value of 7 neurons, and the sum is 1; the middle layer of the model includes a continuous time convolution layer and a fully connected layer. The specific parameters include Improved Newton optimizer algorithm based on softmax as output layer activation function: Based on the improvement of Newton method, softmax is used as the output layer activation function to replace the weight update formula in the small batch rule. , that is, the weight update formula based on Newton's method is as follows: , In the formula, due to The value range is (0,1), so The value range is (-1,1). The value selection method includes negative values, and division by 0 may occur. Based on the iterative descent in the optimization algorithm (i.e. The value of the denominator is less than 0) and the denominator cannot have a value of 0. In this embodiment, the denominator in the above formula is corrected to ensure that its value range is greater than 0. One idea is , where the control factor is the same as in SGD α The value range is (0,1], which is consistent with the positive number, which is convenient for comparison with the performance of SGD. Here, compared with the two control factors of the commonly used ADAM method, one control factor is simplified, and less manual intervention is required.

[0035] The principles of Bayesian optimization of deep learning model hyperparameters include: 1) Based on the existing multi-classification sample data of falls, a proxy model TCN of the objective function is constructed; 2) According to the uncertainty of the proxy model TCN and the expectation of the objective function, the expected improvement strategy is used to select the next sampling point; 3) After selecting the sampling points, sample the objective function and update the proxy model; 4) Repeat (2)-(3) and stop when the predetermined number of sampling times is reached or the objective function value is less than the threshold.

[0036] In order to achieve the above purpose, this embodiment adopts the following technical solutions: The input of the training data is an IMU signal of dimension 5*fs*6, and the corresponding target input is {{fall forward}, {fall backward}, {fall to the left}, {fall to the right}, {trip}, {faint}, {other}}.

[0037] Specific training steps: 1) Initialize weight values; 2) Get the "input" from the training data. The training data format is {"input", "target input"}, and then pass the "input" to the neural network model to calculate the target output d i With model output y i The error between e i ,Right now e i = d i - y i ; 3) Calculate the weight update according to the small batch increment rule, that is ,in k= 1,...,M,...,N , number of samples N ≥ M , , where k Indicates k The weight update of sample training data is α Set to 0.5, which is commonly used by SGD; To reduce the control value, x i is the ith input, x j is the jth output; 4) Adjust the weight, that is ; 5) Repeat steps 2) to 4) for all training data; 6) Repeat steps 2) to 5) until the error e i Reaching a value less than or equal to the set threshold.

[0038] As shown in Table 1 below, this embodiment proposes a SSNND model for real-time multi-type fall situation recognition. A smart watch product is worn as a collection device for three-axis gravity acceleration and gyroscope signals. The model is verified by marking forward falls, backward falls, left-side falls, right-side falls, stumbling, fainting and other seven categories of sample data. Accurate performance indicators are used: accuracy, recall, specificity and Matthew correlation coefficient.

[0039] The calculation formula of accuracy (Ac for short) is: .

[0040] The calculation formula of recall rate (abbreviated as Re) is: .

[0041] The calculation formula of specificity (Sp) is: .

[0042] The calculation formula of Matthew correlation coefficient (Mcc) is: , Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative instances, respectively. The value range of the Matthew correlation coefficient MCC is from -1 to 1, MCC of -1 represents the worst possible prediction, and a value of 1 represents the best possible prediction scheme. In addition, MCC of 0 represents a random prediction. The 20,000 sample points based on three-axis gravity acceleration collected and labeled are divided into training sets and test sets according to a ratio of 8:2, wherein the number of sample points in the training set and the test set are 16,000 and 4,000, respectively. The classification results of the test set of the trained SSNND-TCN model compared with the TCN using the SGD optimizer, that is, SGD-TCN, are expressed by performance indicators as shown in Table 1. It can be seen that compared with the SGD-TCN model, the SSNND-TCN proposed in this embodiment has a significant performance improvement on the test set used for multi-type fall situation recognition.

[0043] Table 1 Performance indicators , The SSNND-TCN model training process proposed in this embodiment uses the loss function curve as the evaluation index of training speed and accuracy, where the SSNND and SGD method parameters are Set it to the commonly used value of 0.5 for SGD method. Figure 4 As shown in the figure, in the training process of the TCN model for multi-type fall detection based on IMU signals, the SSNND optimization algorithm has a faster convergence speed and more accurate performance than the SGD algorithm.

[0044] In order to overcome the problems of slow convergence speed and low accuracy of deep learning models based on commonly used optimizers in fusing multi-sensor signals for multi-type fall situations and in the training process of detection models, this embodiment proposes an SSNND method that combines softmax as the output layer activation function and the improved Newton method as the descent optimization method; the SSNND method fully retains the advantages of the Newton method's fast descent speed and good global convergence, while overcoming the shortcomings of the Newton method's large amount of computation and sensitivity to initialization weights; the model SSNND-TCN proposed in this embodiment has good performance in the detection of multi-type fall situations based on IMU signals.

[0045] A multi-type fall situation detection system, comprising: A data acquisition module, used to obtain a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; A model building module, used to build an initial fall situation detection model according to a temporal convolutional network, wherein an output layer activation function of the initial fall situation detection model is a softmax activation function; The model optimization and application module is used to optimize and train the weights of the initial fall situation detection model according to the fall data set to obtain a trained fall situation detection model; optimize the hyperparameters of the trained fall situation detection model based on the Bayesian optimization method to obtain an optimized fall situation detection model; and deploy the optimized fall situation detection model on the smart wearable device to monitor the user's fall situation in real time.

[0046] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform a multi-type fall situation detection method.

[0047] A computer-readable storage medium stores a computer program, wherein the computer program implements the multi-type fall situation detection method when executed by a processor.

[0048] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multi-type fall situation detection method, characterized in that: include: Acquire a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; Building an initial fall situation detection model based on a temporal convolutional network, wherein an output layer activation function of the initial fall situation detection model is a softmax activation function; performing weight optimization and training on the initial fall situation detection model based on the fall data set to obtain a trained fall situation detection model; Based on the Bayesian optimization method, hyperparameter optimization is performed on the trained fall detection model to obtain an optimized fall detection model; The optimized fall detection model is deployed on smart wearable devices to monitor the user's fall situation in real time.

2. A multi-type fall situation detection method according to claim 1, characterized in that: The inertial measurement unit sensor signal data includes three-axis gravity acceleration and gyroscope signals.

3. A multi-type fall situation detection method according to claim 1, characterized in that: The weight optimization and training of the fall situation detection model based on the fall data set specifically includes: S11: Initializing the model weight of the initial fall situation detection model; S12: inputting the sensor training data into the initial fall situation detection model for classification prediction, and calculating error data between the initial training result after classification prediction and the fall type label corresponding to the sensor training data; S13: Calculating the update amount of the model weight based on the small batch increment rule, and updating and adjusting the model weight based on the update amount; S14: Repeat steps S12 to S13 to iteratively train the initial fall situation detection model until the error data is less than a preset threshold, stop iteration, and output the trained fall situation detection model.

4. A multi-type fall situation detection method according to claim 3, characterized in that: The updating amount of the model weight is calculated based on the small batch incremental rule, specifically including: ， In the formula, k=1,...,M,...,N For the k The weights of the training data are updated by samples. N ≥ M , α is the control factor, w ij is the weight, ∆ w ij Update the value for the weight, e i is the error data, soft max() is the activation function, To reduce the control value, x i is the ith input, x j is the jth output.

5. A multi-type fall situation detection method according to claim 1, characterized in that: The method of performing hyperparameter optimization on the trained fall detection model based on the Bayesian optimization method specifically includes: S21: Building a surrogate model for simulating the objective function based on the falls dataset; S22: Using the uncertainty of the surrogate model prediction and the expected improvement of the objective function, select the next hyperparameter sampling point; S23: Train the deep learning model and evaluate the performance under the selected hyperparameter settings, and update the proxy model based on the performance feedback; S24: Repeat steps S22-S23, stop when a predetermined number of sampling times is reached or the objective function value is less than a preset threshold, and output the optimized fall situation detection model.

6. A multi-type fall situation detection system, characterized in that: include: A data acquisition module, used to obtain a fall data set, wherein the fall data set includes inertial measurement unit sensor signal data and corresponding fall type labels; A model building module, used to build an initial fall situation detection model according to a temporal convolutional network, wherein an output layer activation function of the initial fall situation detection model is a softmax activation function; The model optimization and application module is used to optimize and train the weights of the initial fall situation detection model according to the fall data set to obtain a trained fall situation detection model; optimize the hyperparameters of the trained fall situation detection model based on the Bayesian optimization method to obtain an optimized fall situation detection model; and deploy the optimized fall situation detection model on the smart wearable device to monitor the user's fall situation in real time.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform a multi-type fall situation detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored therein, and when the computer program is executed by a processor, a multi-type fall situation detection method as described in any one of claims 1 to 5 is implemented.

Images