Apparatus for fall detection and method thereof

Abstract

This disclosure provides an apparatus and method for fall detection. The apparatus for fall detection includes: a feature extractor configured to extract feature vectors based on a user's acceleration data; and a classifier configured to determine whether a user has fallen based on the extracted feature vectors. The apparatus and method of this disclosure improve the detection rate of falls occurring in different directions and at different times, and can accurately determine non-fall behaviors in various real-life scenarios.

Description

TECHNICAL FIELD

[0001] The disclosure relates to the field of artificial intelligence, and more particularly, to a device for detecting whether a user falls and a method thereof. BACKGROUND

[0002] Recently, with the development of microelectronics and sensing technology, wearable fall detection systems based on multi-sensor such as acceleration sensor, gyroscope sensor, barometric pressure sensor, etc. have become the main trend of fall detection research at home and abroad. Many companies, universities and research institutions have carried out a lot of development on fall detection, and some fall detection devices have been developed into products and put into use.

[0003] However, the existing fall detection model still has many defects. For example, the features extracted by the existing sensor fall feature extraction method cannot truly distinguish between fall and non-fall behaviors. Mainly because most of the existing fall feature extraction methods are based on researchers' experiments and experience of manual extraction, so the extracted features are limited by researchers' cognition of fall behavior. In addition, the existing fall detection model performs well on the experimental data set, but in many real-life scenarios, the model detection effect is very poor, which is due to the lack of model generalization ability (the ability of the model to make accurate judgments on data that have not appeared in the training sample). Therefore, the feature vector extracted by the existing feature extraction method cannot maintain applicability in various scenarios. SUMMARY

[0004] The exemplary embodiments of the disclosure provide a device for detecting whether a user falls and a method thereof, which at least solve the above-mentioned technical problems and other technical problems not mentioned above, and provide the following beneficial effects.

[0005] An aspect of the disclosure is to provide a device for fall detection, which can include a feature extractor configured to extract a feature vector based on acceleration data of a user, and a classifier configured to determine whether the user falls according to the extracted feature vector.

[0006] The feature extractor can be configured to extract the feature vector using a max-min pooling layer based on the acceleration data of the user, wherein the max-min pooling layer includes a max pooling layer and a min pooling layer, and the feature vector is extracted according to an output of the max pooling layer and an output of the min pooling layer.

[0007] Optionally, the feature extractor can be further configured to input the feature vector extracted using the max-min pooling layer to a feature ordering layer for feature alignment processing, wherein the feature ordering layer includes a flattening layer and an alignment layer, the flattening layer transforms the input data into a one-dimensional feature vector, and the alignment layer sorts the one-dimensional feature vector in a specific order.

[0008] Further, the feature extractor can be configured to extract the feature vector based on the acceleration data of the user using a deep learning network, wherein the deep learning network comprises: a convolutional layer; and at least one of a max-min pooling layer and a feature ordering layer.

[0009] In the case of using the deep learning network, the feature extractor can be configured to input the acquired acceleration data to the convolutional layer to perform preliminary feature extraction, and input the obtained preliminary features to at least one of the max-min pooling layer and the feature ordering layer to further process.

[0010] When the obtained preliminary features are input to the max-min pooling layer, the obtained preliminary features are further subjected to feature extraction to obtain gradient features.

[0011] The parameters of the deep learning network are trained by a specific loss function based on the class separability between the fall behavior and the non-fall behavior and the intra-class compactness of the fall behavior. Another aspect of the disclosure is to provide a method for fall detection, which can include the steps of: extracting a feature vector based on the acceleration data of the user; and determining whether the user falls according to the extracted feature vector.

[0012] The step of extracting the feature vector can include extracting the feature vector based on the acceleration data of the user using a max-min pooling layer, wherein the max-min pooling layer includes a max pooling layer and a min pooling layer, and the feature vector is extracted from the output of the max pooling layer and the output of the min pooling layer.

[0013] Optionally, the step of extracting the feature vector further includes inputting the feature vector extracted using the max-min pooling layer to a feature ordering layer for feature alignment processing, wherein the ordering layer includes a flattening layer and an alignment layer, the flattening layer transforms the input data into a one-dimensional feature vector, and the alignment layer sorts the one-dimensional feature vector in a specific order.

[0014] Further, the step of extracting the feature vector can include extracting the feature vector based on the acceleration data of the user using a deep learning network, wherein the deep learning network comprises: a convolutional layer; a max-min pooling layer and a feature ordering layer.

[0015] In the case of using the deep learning network, the step of extracting the feature vector can include inputting the acceleration data of the user to a convolution layer to perform preliminary feature extraction, and inputting the obtained preliminary feature to at least one of a max-min pooling layer and a feature ordering layer to further process. When the obtained preliminary feature is input to the max-min pooling layer, the obtained preliminary feature is further subjected to feature extraction to obtain gradient features. When the obtained preliminary feature is input to the feature ordering layer, the obtained preliminary feature is subjected to feature alignment processing.

[0016] The parameters of the deep learning network are trained by a specific loss function based on the class separability between the fall behavior and the non-fall behavior and the intra-class compactness of the fall behavior.

[0017] Another aspect of the disclosure is to provide a computer-readable storage medium storing a program, which can include instructions for executing the above-described method for fall detection.

[0018] Another aspect of the disclosure is to provide a computer including a readable medium storing a computer program and a processor, which, when the processor executes the computer program, executes the above-described method for fall detection.

[0019] Based on the above-described method and apparatus, the detection rate of fall behavior occurring in different directions and at different times can be effectively improved, and non-fall behavior in various scenarios in actual life can also be accurately judged, thereby reducing the missed detection rate and misjudgment rate in actual application, and greatly improving the practicability of the fall detection system. BRIEF DESCRIPTION OF DRAWINGS

[0020] These and / or other aspects and advantages of the disclosure will become apparent and more readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1 is a block diagram of an apparatus for fall detection according to an exemplary embodiment of the disclosure;

[0022] Figure 2 is a diagram of the structure of a max-min pooling layer according to an exemplary embodiment of the disclosure;

[0023] Figure 3 is a diagram of the structure of a feature ordering layer according to an exemplary embodiment of the disclosure;

[0024] Figure 4 is a flowchart of a method for fall detection according to an exemplary embodiment of the disclosure;

[0025] Figure 5is an acceleration signal graph of a fall behavior in different directions according to an exemplary embodiment of the disclosure;

[0026] Figure 6 is an acceleration signal graph of a fall behavior at different times according to an exemplary embodiment of the disclosure;

[0027] Figure 7 is a process for fall detection using a deep learning network according to an exemplary embodiment of the disclosure;

[0028] Figure 8 is a diagram when a user falls according to an exemplary embodiment of the disclosure. DETAILED DESCRIPTION

[0029] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of embodiments of the disclosure as defined by the claims and their equivalents. Various specific details are included to assist in understanding but are not intended to limit the disclosure. Therefore, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0030] Hereinafter, devices and methods according to various embodiments of the disclosure will be described with reference to the accompanying drawings.

[0031] Figure 1 is a block diagram of a device for fall detection according to an exemplary embodiment of the disclosure.

[0032] The device 100 for fall detection according to the disclosure can be implemented as or applied to at least one of a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop computer, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player, an MP3 player, a mobile medical device, a camera, or a wearable device, etc., but is not limited thereto. According to an embodiment of the disclosure, the wearable device can include at least one of a accessory-type device (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device), a fabric or clothing-integrated device (e.g., electronic clothes), a body-attached device (e.g., a skin pad or a tattoo), or a body-implantable device (e.g., an implantable circuit), etc., but is not limited thereto.

[0033] Reference Figure 1The device 100 for fall detection can include a feature extractor 101 and a classifier 102. The feature extractor 101 can extract a feature vector using a deep learning network from acceleration data of a user. The classifier 102 can determine whether a user falls according to the extracted feature vector. Each element in the device 100 according to the disclosure can be implemented by one or more elements, and the name of the corresponding element can vary according to the type of the element. In various embodiments, some elements in the device 100 can be omitted, or additional elements can be further included. Furthermore, elements according to various embodiments of the disclosure can be combined to form a single entity, and thus can equivalently perform the functions of the corresponding elements before the combination.

[0034] In the process of extracting features, the existing feature extraction method is mainly a manual feature extraction method, including a time domain feature extraction method, a frequency domain feature extraction method, and a time-frequency feature extraction method, and the like statistical feature extraction method. Although the feature vectors extracted by the frequency domain feature extraction method and the time-frequency feature extraction method are relatively comprehensive, the extraction process is complex and slow, and the time domain feature extraction method has a relatively simple extraction process, small calculation amount, and fast speed, but it performs poorly in detection rate. Therefore, these methods will fail to perform features in many real-life scenarios, resulting in poor model detection effect. The reason is that these manual feature extraction methods are mainly based on the data collected by the acceleration sensor according to the researchers' experiments and experience, which is limited by the researchers' cognition of the fall behavior. Although the scenes included in the experimental data perform well, the scenes not included in the experimental data are prone to inapplicable situations.

[0035] In the prior art, acceleration signal amplitude vector, acceleration signal amplitude domain, and the like features are generally used to solve the problem of fall direction uncertainty, but these feature extraction methods have the disadvantage of losing part of the useful information, especially the negative signal information; for the fall time uncertainty problem, time-independent statistical features (such as maximum value, minimum value, mean value, median, etc.) are generally used, but statistical features are discrete features, and acceleration signals are continuous data, so information loss will also occur, resulting in a decrease in the accuracy of the detection result.

[0036] The disclosure extracts a feature vector from the acceleration data of a user by borrowing the idea of a deep learning method, so that, since a data-driven feature extraction method is used, prior knowledge of the fall behavior of researchers is not required, so automatic feature extraction can be performed for various scenarios in real life, and the extracted feature vector can more effectively distinguish between fall behavior and non-fall behavior.

[0037] According to an embodiment of the disclosure, the feature extractor 101 can extract a feature vector using a max-min pooling layer based on the acceleration data of the user, as the feature vector to be input into the classifier. As an example, the acceleration data of the user is raw data of three-dimensional acceleration. For example, when the sampling frequency of the acceleration data is 100 Hz and the time window is 4.5 s, the dimension of the acceleration data is 3x450. However, the above example is merely exemplary, and the disclosure is not limited thereto. Here, the max-min pooling layer includes a max-pooling layer and a min-pooling layer in a convolutional network, and a feature vector is extracted from the output of the max-pooling layer and the output of the min-pooling layer.

[0038] When a fall occurs, the data features of the behavior are generally reflected by one or more axes of a 3-dimensional acceleration sensor, depending on the uncertainty of the fall direction. To capture the fall direction, we propose a max-min pooling layer to extract features with rotational invariance and inverse direction invariance.

[0039] Figure 2 The structure of the max-min pooling layer is shown.

[0040] As Figure 2 shown, the max-min pooling layer 200 includes a max-pooling layer 201 and a min-pooling layer 202, and subtracts the output of the min-pooling layer 202 from the output of the max-pooling layer 201 to obtain the output of the max-min pooling layer 200 as the extracted feature vector. For the structural design of the max-min pooling layer 200, the detection rate of the fall behavior in different directions can be improved, thereby solving the problem of uncertainty of the fall direction.

[0041] For a three-dimensional acceleration input signal (x, y, z), the output of the max-min pooling layer 200 can be calculated using equation (1):

[0042] f(x, y, z) = max(x, y, z) - min(x, y, z) (1)

[0043] where max(x, y, z) represents the output of the max-pooling layer 201, and min(x, y, z) represents the output of the min-pooling layer 202.

[0044] The max-min pooling layer 200 of the disclosure can solve the problem of uncertainty of the user fall direction as Figure 5 shown. Figure 5 The fluctuations of the acceleration signal in the x, y, z three axes when the user falls in different directions are shown. The horizontal axis represents the time when the acceleration signal occurs, and the vertical axis represents the size of the acceleration. For example, Figure 5 (a) of FIG. 1 shows the fluctuations of the acceleration signal when the user falls forward, and it can be seen that the fluctuations of the acceleration signal in the y and z axes are more intense, Figure 5(b) shows the fluctuation of the acceleration signal when the user falls forward and backward, and Figure 5 (c) shows the case when the user falls sideways. As can be seen, the acceleration signals of different falling directions show different fluctuation curves on the x, y, z three axes, thereby determining the falling direction of the user.

[0045] Since f(x, y, z) = f(x, z, y) = f(z, y, x), and f(x, y, z) = f(-x, -y, -z). The above two equations show that the max-min pooling layer is symmetric to the x, y, z three-axis data (i.e. f(x, y, z) = f(x, z, y) = f(z, y, x)) and has inversion invariance (i.e. f(x, y, z) = f(-x, -y, -z)), so the max-min pooling layer of the present disclosure can solve the uncertainty of the falling direction of the user.

[0046] In addition, the feature extractor 101 can also use the feature sorting layer for feature alignment processing based on the feature vectors extracted by the max-min pooling layer. Here, the feature sorting layer includes a flattening layer and an alignment layer, the flattening layer transforms the input data into a one-dimensional feature vector, and the alignment layer sorts the one-dimensional feature vector in a certain order.

[0047] Falls can occur at any time. Therefore, we designed the feature sorting layer to align the extracted features in the time dimension. The feature sorting layer is a hybrid of flattening and sorting operations.

[0048] Figure 3 The structure of the feature sorting layer is shown.

[0049] As Figure 3 shown, the feature sorting layer 300 includes a flattening layer 301 and an alignment layer 302, the flattening layer 301 transforms the input data into a one-dimensional feature vector, and the alignment layer 302 sorts the one-dimensional feature vector in a certain order. For the network structure design of the feature sorting layer 300, the detection rate of the falling behavior at different times can be improved, thereby solving the problem of falling time uncertainty.

[0050] For example, after obtaining the features output by the max-min pooling layer (which can be referred to as gradient features), the gradient features are input into the feature sorting layer. The structure of the feature sorting layer 300 is shown in Figure 3 The gradient features are input into the flattening layer 301, the feature map is stretched into a one-dimensional feature vector by the flattening layer 301, and then the one-dimensional feature vector is input into the alignment layer 302, the one-dimensional feature vector is sorted in ascending or descending order by the alignment layer 302. The feature sorting layer is mainly used for aligning features.

[0051] When the time of the user falling changes, the position of the features corresponding to the falling process in the feature vector also changes, and the role of the sorting disrupts the position of all the features in the feature vector, so that the features are not sorted according to the time of occurrence, but are sorted according to the obviousness of the features, which is beneficial to solve the problem of uncertainty of the time of falling.

[0052] The feature sorting layer of the present disclosure can solve the problem of uncertainty of the time of user falling as shown in Figure 6 For example, Figure 6 (a), (b), (c), (d) of FIG. 1 show different fluctuation curves of acceleration signals on x, y, z three axes when the user falls at different times, that is, when the user falls, the fluctuation curves of the acceleration signals on the x, y, z three axes are obviously different from the fluctuation of the acceleration signals when the user does not fall. However, Figure 6 The fluctuation curves shown in (a), (b), (c), (d) of FIG. 1 are only exemplary.

[0053] In the present disclosure, the feature extractor 101 can use a deep learning network to extract a feature vector according to the acceleration data of the user. According to an embodiment of the present disclosure, the deep learning network can include at least one of a max-min pooling layer 200 and a feature sorting layer 300 and a convolutional layer, where the deep learning network uses a convolutional neural network as a backbone structure, and then adds at least one of the max-min pooling layer and the feature sorting layer.

[0054] In the case where the deep learning network includes the max-min pooling layer 200 and the convolutional layer, the feature extractor 101 can input the acceleration data of the user to the convolutional layer to obtain preliminary features, and input the preliminary features to the max-min pooling layer 200 to obtain the feature vector to be input to the classifier.

[0055] For example, after obtaining the acceleration data of the user, the acceleration data is convoluted using a convolution kernel in the deep learning network, thereby obtaining preliminary features. It should be noted that in the convolution operation, the convolution kernel and the moving step length and other parameters used are not determined artificially, but are determined by training optimization using the loss function of the present disclosure, which will be described in detail later. After obtaining the data after convolution operation (i.e., the features after the convolutional layer), the data is input into the max-min pooling layer 200. The structure of the max-min pooling layer 200 is shown in Figure 2 The input features are respectively input into the max pooling layer 201 and the min pooling layer 202, and then the output of the max pooling layer 201 and the output of the min pooling layer 202 are subtracted to obtain the output of the max-min pooling layer 200 as the feature vector to be input to the classifier 102.

[0056] In the case where the deep learning network includes the feature sorting layer 300 and the convolution layer, the feature extractor 101 can input the acceleration data of the user to the convolution layer to obtain preliminary features, and input the preliminary features to the feature sorting layer 300 to sort the obtained preliminary features.

[0057] For example, after obtaining the acceleration data of the user, the acceleration data is subjected to a convolution operation using a convolution kernel in the deep learning network, thereby obtaining preliminary features. It should be noted that in the convolution operation, the parameters such as the convolution kernel used and the moving step length are not determined artificially, but are determined by training optimization using the loss function of the present disclosure, which will be described in detail later. After obtaining the data subjected to the convolution operation (i.e., the features subjected to the convolution layer), the data is input to the feature sorting layer. The structure of the feature sorting layer 300 is shown in FIG. 3, the preliminary features are input to the flattening layer 301, the feature map is stretched into a one-dimensional feature vector by the flattening layer 301, and then the one-dimensional feature vector is input to the alignment layer 302, which sorts the one-dimensional feature vector in ascending or descending order. The feature sorting layer is mainly used to align the features. Figure 3

[0058] When the time of the user's fall changes, the position of the features corresponding to the fall process in the feature vector also changes, and the sorting disrupts the positions of all the features in the feature vector, so that these features are not sorted according to the time of occurrence, but are sorted according to the apparent degree of the features, which is beneficial to solve the problem of uncertainty of the time of fall.

[0059] In the case where the deep learning network includes the maximum-minimum pooling layer 200, the feature sorting layer 300 and the convolution layer, the feature extractor 101 can input the obtained acceleration data to the convolution layer to obtain preliminary features, input the preliminary features to the maximum-minimum pooling layer 200 and input the output of the maximum-minimum pooling layer to the feature sorting layer 300 to obtain the feature vector to be input to the classifier. In other words, in order to well extract the distinguishing features that can distinguish between the fall behavior and the normal behavior (non-fall), we use a convolutional neural network as the backbone structure to extract the bottom features, and use the maximum-minimum pooling layer and the feature sorting layer as higher layers to extract the distinguishing features.

[0060] ​For example, after obtaining the acceleration data of the user, the acceleration data is convoluted by using a convolution kernel in the deep learning network, so as to obtain preliminary features. It should be noted that in the convolution operation, the convolution kernel and the moving step length and other parameters used are not determined artificially, but are determined by training optimization using the loss function of the present disclosure, which will be described in detail later. After obtaining the data subjected to the convolution operation, the data is input into the maximum pooling layer 201 and the minimum pooling layer 202 of the maximum-minimum pooling layer 200, the data is calculated by using equation (1) to obtain a gradient feature vector (i.e., the output of the maximum-minimum pooling layer), and then the gradient feature vector is input into the flattening layer 301 of the feature sorting layer 300, the gradient feature vector is stretched into a one-dimensional feature vector by the flattening layer 301, and then the one-dimensional feature vector is input into the alignment layer 302 of the feature sorting layer, the one-dimensional feature vector is sorted in ascending order or descending order by the alignment layer 302, so as to obtain a feature vector to be input into the classifier.

[0061] After obtaining the final feature vector (the feature vector to be input into the classifier), the final feature vector is input into the classifier 102, and the classifier 102 can determine whether the user falls according to the extracted feature vector. According to an embodiment of the present disclosure, the classifier 102 can be implemented by K-Nearest Neighbor method. However, the present disclosure is not limited thereto.

[0062] In addition, in the case of using the deep learning network to extract the feature vector, the device 100 for fall detection of the present disclosure can further include a parameter optimizer 103 (not shown). The parameter optimizer 103 can train and optimize the parameters of the deep learning network, such as the parameters of the convolution kernel, the moving step length of the convolution kernel, and the like, by using a specific loss function according to the extracted feature vector.

[0063] In order to extract features that can effectively distinguish between fall behaviors and non-fall behaviors, the extracted fall features must have class separability. The most common class separability measure is the distance measure: intra-class distance and inter-class distance. When extracting features, the general principle is to make the intra-class distance as small as possible and the inter-class distance as large as possible. Since the fall behaviors of people in different scenarios and different angles have similarities, for fall behaviors, the principle of “intra-class distance minimization” can be used for feature extraction; and since non-fall behaviors are diverse, such as walking, running, jumping, etc., non-fall behaviors do not have similarities, and for non-fall behaviors, the principle of “intra-class distance minimization” is not suitable for feature extraction.

[0064] Therefore, the loss function in the present disclosure is defined based on the class separability between the fall behavior and the non-fall behavior and the intra-class compactness of the fall behavior, where the intra-class compactness represents a minimum characteristic of intra-class distance, i.e., the intra-class distance is as small as possible.

[0065] The loss function of the present disclosure can be represented by equation (2):

[0066] Loss = Loss in + Loss out (2)

[0067] wherein,

[0068]

[0069]

[0070] wherein, v represents a feature vector output by the deep learning network; x i represents a fall sample; represents a class center of the fall sample; l x is a hyperparameter for filtering the fall sample close to the class center of the fall sample; represents the number of fall samples satisfying ; y i represents a non-fall sample; l y is a hyperparameter for filtering the non-fall sample far from the class center of the fall sample, and satisfies 0≤l x < l y ≤1; represents the number of non-fall samples satisfying .

[0071] For example, assuming that the feature vector output by the deep learning network is v = {v1, v2,..., v p}, the fall sample in each training process is {x1, x2,..., x m}, and the non-fall sample is {y1, y2,..., y n}, the class center of the fall sample is Then, according to the definitions of the above Loss in and Loss out , the network parameters are optimized by minimizing the total loss function Loss.

[0072] The design of the loss function of the present disclosure actually draws on the idea of "outlier detection", i.e., the fall behavior is regarded as a class, and the non-fall behavior is regarded as an outlier of the fall behavior, so that the extracted feature vector can be applied to various non-fall scenarios, thereby improving the robustness of the fall detection model.

[0073] Optionally, the device 100 may also include an acceleration sensor 104 (not shown), so that the device 100 can directly detect the user's acceleration and obtain the user's acceleration data.

[0074] Furthermore, the fall detection device 100 disclosed herein may also include an alarm 105 (not shown). When a user is detected to have fallen, the alarm 105 automatically sends an alarm message so that the fallen user can receive timely assistance.

[0075] Figure 4 This is a flowchart of a method for fall detection according to an exemplary embodiment of the present disclosure.

[0076] Reference Figure 4 In step S401, the accelerometer acquires acceleration data from the user. The acquired acceleration data is the raw data of three-dimensional acceleration. For example, when the sampling frequency of the acceleration data is 100Hz and the time window is 4.5s, the dimension of the acceleration data is 3×450.

[0077] In step S402, feature vectors are extracted based on the acquired acceleration data.

[0078] According to embodiments of this disclosure, feature vectors to be input into the classifier can be obtained from user acceleration data using only max-min pooling layers. Optionally, the user's acceleration data can be first input into a max-min pooling layer to extract feature vectors, and then the output of the max-min pooling layer can be input into a feature ranking layer to align the extracted features. Compared to using deep learning methods to extract features, the computational load can be significantly reduced and the computation speed accelerated when using only max-min pooling layers or both max-min pooling layers and feature ranking layers.

[0079] Optionally, when using a deep learning network to extract feature vectors, the deep learning network according to this disclosure may include at least one of a max-min pooling layer 200 and a feature sorting layer 300, as well as a convolutional layer; however, this disclosure does not exclude other layers in existing convolutional networks.

[0080] When a deep learning network includes convolutional layers and max-min pooling layers, the feature extractor 101 inputs the acquired acceleration data into the convolutional layer to obtain preliminary features, and then inputs the preliminary features into the max-min pooling layer to obtain gradient feature vectors, which will be used as feature vectors input into the classifier. For example, after obtaining the features through the convolutional layer, equation (1) can be used to calculate the output of the max-min pooling layer 200, which will be used as the feature vectors input into the classifier.

[0081] When a deep learning network includes convolutional layers and feature ranking layers, the feature extractor 101 inputs the acquired acceleration data into the convolutional layers to obtain preliminary features, and then inputs these preliminary features into the feature ranking layer for feature alignment to obtain the feature vector that will be input into the classifier. For example, after obtaining the convolutionally processed features, a flattening layer can be used to transform the convolutionally processed features into a one-dimensional feature vector, and then this one-dimensional feature vector is ranked to obtain the feature vector that will be input into the classifier.

[0082] When a deep learning network includes convolutional layers, max-min pooling layers, and feature sorting layers, the feature extractor 101 inputs the acquired acceleration data into the convolutional layer to obtain preliminary features. The preliminary features are then input into the max-min pooling layer for further feature extraction to obtain gradient feature vectors. Finally, the gradient feature vectors are input into the feature sorting layer for feature alignment to obtain feature vectors that will be input into the classifier.

[0083] According to embodiments of this disclosure, corresponding feature vectors are obtained based on different structures of deep learning networks. The above situation is consistent with reference to... Figure 1 The feature extractor 101 described above performs the same operations, so they will not be repeated here.

[0084] In step S403, it is determined whether the user has fallen based on the extracted feature vector. For example, a classifier can be used to determine whether the user has fallen based on the extracted feature vector. The K-nearest neighbor method can be used as a classifier for fall detection. However, this disclosure is not limited thereto.

[0085] When a user is confirmed to have fallen, the system proceeds to step S404, where alarm 105 automatically sends an alarm message to ensure the fallen user receives timely assistance. If it is determined that the user has not fallen, the system returns to step S401 to continue acquiring the user's acceleration signal for real-time detection of the user's fall behavior.

[0086] The process of fall detection using a deep learning network is illustrated using a wearable device (e.g., a smartwatch) worn by a user as an example. It is assumed that the fall detection device according to this disclosure is applied to a smartwatch, that is, the fall detection device is integrated into the smartwatch, referring to... Figure 7The acceleration signal of the user wearing the smartwatch can be obtained through the accelerometer sensor in the smartwatch. After obtaining the user's acceleration signal, it is input into a convolutional layer for preliminary feature extraction. Then, the preliminary features are input into a max-min pooling layer to obtain gradient features, and the gradient features are input into a feature ranking layer for feature alignment. Finally, the obtained feature vector is input into a classifier, which determines whether the user has fallen. That is, the classifier can determine whether the user is in a normal behavioral state or has fallen based on the extracted final feature vector. When it is determined that the user has fallen, such as... Figure 8 As shown, the alarm in the smartwatch will promptly send out alert messages to help the user get assistance.

[0087] Furthermore, the method according to this disclosure may also include training and optimizing the parameters of a deep learning network using a specific loss function based on the extracted feature vectors. The specific loss function can be defined based on the inter-class separability of fall behavior and non-fall behavior, and the intra-class compactness of fall behavior. For example, the specific loss function can be represented by equation (2), and optimizing the parameters in the deep learning network by minimizing the specific loss function yields features with better class separability.

[0088] This disclosure proposes a deep learning-based solution. To address the uncertainty of fall direction, a max-min pooling layer is proposed to extract features with rotation invariance and inverse direction invariance. Simultaneously, to address the uncertainty of the fall's timing, a feature ranking layer is proposed for feature alignment. Furthermore, to improve the model's robustness to various non-fall scenarios, a unique loss function is designed based on the "outlier detection" concept. In other words, an end-to-end deep learning framework is proposed to solve the fall detection problem; robustness is improved through two new operational layers—max-min pooling and feature ranking—and a novel loss function; and a feature extraction method using only max-min pooling and feature ranking layers is proposed to effectively identify fall behavior, demonstrating significant practical value.

[0089] The device and method disclosed herein can not only improve the detection rate of user falls occurring in different directions and at different times, but also accurately distinguish non-fall behaviors in various scenarios in real life, thereby reducing the false negative rate and false positive rate in practical applications and greatly improving the practicality of the fall detection system.

[0090] The methods described above according to this disclosure can be executed according to computer program instructions. Since these program instructions can be included in a computer, a dedicated processor, or programmable or dedicated hardware, the instructions executed therein can facilitate the performance of the above-described functions. As understood by those skilled in the art, the computer, processor, or programmable hardware includes storage devices capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described in this disclosure.

[0091] Although this disclosure has been shown and described with reference to its exemplary embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this disclosure as defined by the claims and their equivalents.

Claims

1. A device for fall detection, characterized in that, The device includes: The feature extractor is configured to obtain feature vectors based on the user's acceleration data; and The classifier is configured to determine whether a user has fallen based on the acquired feature vector. The feature extractor is configured to use a deep learning network to obtain feature vectors based on the user's acceleration data. The deep learning network includes max-min pooling layers and feature ranking layers. The max-min pooling layer consists of a max pooling layer and a min pooling layer. The feature vector extracted by the max pooling layer is obtained by subtracting the output of the min pooling layer from the output of the max pooling layer. The feature sorting layer includes a flattening layer and an alignment layer. The flattening layer transforms the feature vectors extracted by the max-min pooling layer into one-dimensional feature vectors, and the alignment layer sorts the one-dimensional feature vectors in a specific order.

2. The device as described in claim 1, characterized in that, The deep learning network also includes: convolutional layers, The feature extractor is configured as follows: The user's acceleration data is input into the convolutional layer for preliminary feature extraction. The obtained preliminary features are input into a max-min pooling layer to obtain gradient features. The obtained gradient features are input into the feature sorting layer to perform feature alignment processing on the obtained gradient features.

3. The device as described in claim 1, characterized in that, The parameters of the deep learning network are obtained by training a specific loss function based on the inter-class separability of fall behavior and non-fall behavior, and the intra-class compactness of fall behavior.

4. A method for fall detection, characterized in that, The method includes: Feature vectors are obtained based on user acceleration data; and Determining whether a user has fallen based on the acquired feature vectors. The steps for obtaining feature vectors include using a deep learning network to obtain feature vectors based on the user's acceleration data. The deep learning network includes max-min pooling layers and feature ranking layers. The max-min pooling layer consists of a max pooling layer and a min pooling layer. The feature vector extracted by the max pooling layer is obtained by subtracting the output of the min pooling layer from the output of the max pooling layer. The feature sorting layer includes a flattening layer and an alignment layer. The flattening layer transforms the feature vectors extracted by the max-min pooling layer into one-dimensional feature vectors, and the alignment layer sorts the one-dimensional feature vectors in a specific order.

5. The method as described in claim 4, characterized in that, The deep learning network also includes convolutional layers. The steps for obtaining feature vectors also include: The user's acceleration data is input into the convolutional layer for preliminary feature extraction. The obtained preliminary features are input into a max-min pooling layer to obtain gradient features. The obtained gradient features are input into the feature sorting layer to perform feature alignment processing on the obtained gradient features.

6. The method as described in claim 4, characterized in that, The parameters of the deep learning network are obtained by training a specific loss function based on the inter-class separability of fall behavior and non-fall behavior, and the intra-class compactness of fall behavior.

7. A computer-readable storage medium storing a program, characterized in that, The program includes instructions for performing the method as described in any one of claims 4-6.

8. A computer, comprising a readable medium storing a computer program, characterized in that, The computer program includes instructions for performing the method as described in any one of claims 4-6.

Images