Fall detection method and device, electronic equipment and storage medium
By integrating motion sensors and auxiliary sensors in wearable devices and using multi-sensor fusion model and target fall detection model, the problem of low fall detection accuracy in the prior art is solved, achieving higher detection accuracy and lower power consumption.
Patent Information
- Application Number
- CN202311750505.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-18
- Publication Date
- 2025-06-20
AI Technical Summary
Existing wearable devices are not accurate when detecting users' falls, especially when users perform daily exercises, which are prone to misjudgment.
By integrating motion sensors and auxiliary sensors in wearable devices, a multi-sensor fusion model and target fall detection model can be used to obtain more accurate fall detection results.
It improves the accuracy of fall detection, reduces misjudgment, and reduces the power consumption of wearable devices without affecting the accuracy of detection, improving the user experience.
Smart Images
Figure CN120167943A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of consumer electronics technology, and in particular to a method, apparatus, electronic device, and storage medium for fall detection. Background Art
[0002] With the rapid development of wireless communication terminal technology, wearable devices have increasingly appeared in people's sight. Wearable electronic devices have various forms and can be applied to many fields, such as medical, education, entertainment, industry, and even military fields, etc. Common wearable devices can be, for example, smart watches, smart bracelets, smart helmets, or smart glasses and other products.
[0003] A common personal health application of wearable devices is fall detection, which is used to timely detect whether the user wearing the wearable device has an accidental fall, facilitating timely treatment and rescue. In related technologies, when using wearable devices for fall detection, there is a technical problem of low accuracy. For example, false judgments of falls are likely to occur when the user is performing daily exercises (such as playing badminton). Summary of the Invention
[0004] The purpose of the embodiments of this application is to provide a method, apparatus, electronic device, and storage medium for fall detection.
[0005] To solve the above technical problems, the embodiments of this application are achieved through the following aspects.
[0006] According to a first aspect of the embodiments of the present disclosure, a method for fall detection is provided, which is applied to a wearable device. The wearable device includes a motion sensor and at least one auxiliary sensor. The method includes: obtaining motion sensor data collected by the motion sensor, and obtaining at least one motion index data from the motion sensor data; in the case where any of the at least one motion index data exceeds a corresponding preset index threshold, obtaining auxiliary sensor data collected by the at least one auxiliary sensor; obtaining a target fall detection model from a plurality of fall detection models, inputting the motion sensor data into the target fall detection model to obtain a first detection result; and inputting the first detection result and the auxiliary sensor data into a multi-sensor fusion model to obtain a fall detection result.
[0007] According to a second aspect of the embodiments of the present disclosure, there is provided a device for fall detection, which is applied to a wearable device. The wearable device includes a motion sensor and at least one auxiliary sensor. The device includes: a first acquisition module configured to acquire motion sensor data collected by the motion sensor and obtain at least one motion index data from the motion sensor data; a second acquisition module configured to acquire auxiliary sensor data collected by the at least one auxiliary sensor when any of the at least one motion index data exceeds a corresponding preset index threshold; a first detection module configured to obtain a target fall detection model from a plurality of fall detection models and input the motion sensor data into the target fall detection model to obtain a first detection result; and a second detection module configured to input the first detection result and the auxiliary sensor data into a multi-sensor fusion model to obtain a fall detection result.
[0008] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a memory, a processor, and computer executable instructions stored on the memory and executable on the processor. When the computer executable instructions are executed by the processor, the following steps are implemented: acquiring motion sensor data collected by the motion sensor and obtaining at least one motion index data from the motion sensor data; acquiring auxiliary sensor data collected by the at least one auxiliary sensor when any of the at least one motion index data exceeds a corresponding preset index threshold; obtaining a target fall detection model from the plurality of fall detection models and inputting the motion sensor data into the target fall detection model to obtain a first detection result; and inputting the first detection result and the auxiliary sensor data into the multi-sensor fusion model to obtain a fall detection result.
[0009] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing computer executable instructions. When the computer executable instructions are executed by a processor, the following steps are implemented: acquiring motion sensor data collected by the motion sensor and obtaining at least one motion index data from the motion sensor data; acquiring auxiliary sensor data collected by the at least one auxiliary sensor when any of the at least one motion index data exceeds a corresponding preset index threshold; obtaining a target fall detection model from a plurality of fall detection models and inputting the motion sensor data into the target fall detection model to obtain a first detection result; and inputting the first detection result and the auxiliary sensor data into a multi-sensor fusion model to obtain a fall detection result.
[0010] By adopting the above technical means, the first detection result corresponding to the motion sensor data is obtained through the target fall detection model in the wearable device, and the first detection result and the auxiliary sensor data are input into the multi-sensor fusion model to obtain the fall detection result. The target fall detection model corresponding to the user wearing the wearable device can obtain a more accurate first detection result. Combining the auxiliary sensor data and obtaining the fall detection result through the multi-sensor fusion model can further improve the accuracy of the fall detection result. At the same time, when any of the above motion index data exceeds the corresponding preset index threshold, obtaining the auxiliary sensor data can improve the accuracy while reducing the power consumption of the wearable device and enhancing the user experience.
[0011] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.
[0012] Other features and advantages of the present disclosure will be described in detail in the following specific implementation section. Brief Description of the Drawings
[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0014] Figure 1 A schematic flowchart showing a method for fall detection provided by an embodiment of the present application;
[0015] Figure 2 Another schematic flowchart showing a method for fall detection provided by an embodiment of the present application;
[0016] Figure 3 Another schematic flowchart showing a method for fall detection provided by an embodiment of the present application;
[0017] Figure 4 A schematic diagram showing the first time period and the second time period in the acceleration data provided by an embodiment of the present application;
[0018] Figure 5 Another schematic flowchart showing a method for fall detection provided by an embodiment of the present application;
[0019] Figure 6 A schematic diagram of a display interface showing a method for fall detection provided by an embodiment of the present application;
[0020] Figure 7Another schematic flowchart showing the method for fall detection provided by the embodiments of the present application;
[0021] Figure 8 A block diagram showing a fall detection device provided by the embodiments of the present application;
[0022] Figure 9 A block diagram showing another fall detection device provided by the embodiments of the present application;
[0023] Figure 10 A schematic hardware structure diagram of an electronic device for executing the fall detection method provided by the embodiments of the present application. Detailed implementation manners
[0024] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
[0025] First, briefly elaborate on the terms involved in the present application.
[0026] A wearable device is a portable device with computing functions that can be connected to a terminal through a wireless communication method (such as a wireless communication network or a short - range communication technology such as Bluetooth or WiFi). The product forms of wearable devices are diverse. For example, portable devices worn on the wrist or finger (such as smart watches, smart bracelets, smart wristbands or smart rings), portable devices worn on the foot or leg (such as smart shoes, socks), and portable devices worn on the head (such as smart glasses, smart helmets). The present application does not limit the types and forms of specific wearable devices. For new types or forms of wearable devices that appear in the future, the technical solutions obtained by those of ordinary skill in the art based on the new types or forms of wearable devices without creative efforts shall fall within the protection scope of the present application.
[0027] In the related art, due to the fact that wearable devices usually use fewer sensors considering power consumption, and the fall detection models are relatively single, without distinguishing the significant differences between the fall characteristics of different populations, the accuracy of fall detection is not high. The present application proposes a fall detection method. Through the corresponding target fall detection model, a more accurate first detection result can be obtained, and the accuracy of the fall detection result can be further improved by combining the auxiliary sensor data through a multi - sensor fusion model.
[0028] Figure 1A schematic flowchart showing a method for fall detection provided by an embodiment of the present application. This method can be applied to a wearable device. The wearable device includes a motion sensor and at least one auxiliary sensor. The wearable device may include multiple fall detection models and a multi-sensor fusion model. It can be understood that the multiple fall detection models and the multi-sensor fusion model can be models set in the storage space of the wearable device, or models provided in an application in the wearable device. The present application does not limit this. In some embodiments, the wearable device can receive information from a server to update the fall detection model and / or the multi-sensor fusion model, or can obtain information from the server in response to a user's update instruction to update the fall detection model and / or the multi-sensor fusion model.
[0029] As Figure 1 shown, the method may include the following steps:
[0030] In step S101, obtain the motion sensor data collected by the motion sensor, and obtain at least one motion index data from the motion sensor data.
[0031] In some embodiments, the wearable device may be provided with one or more motion sensors for obtaining one or more types of motion sensor data corresponding to a user of a smart watch and capturing one or more types of motion index data.
[0032] Taking a smart watch as an example, an accelerometer and / or a gyroscope may be provided in the smart watch. Among them, the accelerometer can obtain the acceleration data during the movement of the smart watch, and the gyroscope is a device for detecting angular motion around one or more axes orthogonal to the self-rotation axis in a relative inertial space and can obtain the angular velocity data during the movement of the smart watch.
[0033] In step S102, when any motion index data exceeds the corresponding preset index threshold, obtain the auxiliary sensor data collected by at least one auxiliary sensor.
[0034] In some embodiments, at least one auxiliary sensor includes at least one environmental sensor and / or at least one vital sign sensor. The auxiliary sensor data includes the environmental sensor data collected by the at least one environmental sensor and / or the vital sign sensor data collected by the at least one vital sign sensor.
[0035] Taking a smart watch as an example, the environmental sensor may be a microphone for collecting environmental audio data in the environment, and the vital sign sensor may be an electrocardiogram sensor for collecting the electrocardiogram sensor data of the user. The electrocardiogram sensor data may specifically be, for example, one or more of heart rate, heart rate variability, or electrocardiogram waveform abnormality.
[0036] It can be understood that the environmental sensor may further include other environmental sensors that can significantly characterize the user's fall, such as a barometric pressure sensor. The vital sign sensor may include other vital sign sensors that can significantly characterize the user's fall, such as a body temperature sensor. Those skilled in the art can set more environmental sensors and / or vital sign sensors according to business needs.
[0037] In step S103, a target fall detection model is obtained from multiple fall detection models, and the motion sensor data is input into the target fall detection model to obtain a first detection result.
[0038] In some embodiments, the multiple fall detection models include fall detection models respectively corresponding to multiple different types of behavioral ability disorders and / or fall detection models respectively corresponding to multiple different types of movements.
[0039] The fall detection models respectively corresponding to different types of behavioral ability disorders are fall detection models constructed for different special groups in daily life. Such different special groups may be, for example, people with different degrees of limb disabilities, hemilateral limb disorders caused by cardiovascular and cerebrovascular diseases, unilateral or bilateral crutch - using groups caused by sports injuries, and pregnant women, etc. It can be understood that the characteristics of the motion sensor data corresponding to the accidental falls of different special groups will vary due to the differences of the special groups.
[0040] The fall detection models respectively corresponding to different types of movements are fall detection models constructed when ordinary users other than the above - mentioned special groups are performing a certain movement (such as walking, running, badminton or football). It can be understood that when the user accidentally falls while performing different types of movements, the characteristics of the corresponding motion sensor data will vary due to the current movement. For example, the characteristics of the motion sensor data corresponding to accidentally falling while walking and accidentally falling while playing badminton will be different.
[0041] In some embodiments, the fall detection model may be a convolutional neural network model. To make the model more suitable for wearable devices, the fall detection model may be a depth - separable convolutional neural network model. This depth - separable convolutional neural network model can use DSC (Depthwise Separable Convolution) instead of the convolution operation. Since the depth - separable convolutional layer uses a step - by - step convolution operation and fewer parameters, the calculation speed is faster than that of the standard convolution. The depth - separable convolution can significantly reduce the amount of calculation while maintaining a certain model performance, and is better suitable for wearable devices.
[0042] Compared with the related art in which a fall detection result is obtained through a single fall detection model, the first detection result obtained by obtaining a target fall detection model from multiple fall detection models and inputting motion sensor data into the target fall detection model has higher accuracy.
[0043] In some embodiments, the motion sensor data may include acceleration data and / or angular velocity data.
[0044] When the motion sensor data includes acceleration data and angular velocity data, the dimension of the acceleration data is m / s 2 , while the dimension of the angular velocity data is degree / second. Due to the different dimensions, the range of values thereof will also be greatly different. In some embodiments, the acceleration data and the angular velocity data may be transformed by a preset scale transformation function, and the scale-transformed acceleration data and the angular velocity data may be input into the target fall detection model to obtain the first detection result.
[0045] For example, the acceleration data may be a sequence (a1, a2, ..., a n ), the angular velocity data can be a sequence (ω1, ω2, ..., ω m ), the scale transformation function may be a logarithmic function, and the acceleration data after scale transformation may be (loga1, loga2, ..., loga n ), the angular velocity data after scale transformation can be (logω1, logω2,…,logω m ), the acceleration data and angular velocity data after scale change are equivalent, which can further improve the accuracy of the first detection result obtained.
[0046] In step S104, the first detection result and the auxiliary sensor data are input into a multi-sensor fusion model to obtain a fall detection result.
[0047] For example, after obtaining the first detection result, the first detection result and the auxiliary sensor data obtained by the auxiliary sensor may be further input into a multi-sensor fusion model to obtain a fall detection result.
[0048] By adopting the above-mentioned technical method, a more accurate first detection result can be obtained through the target fall detection model corresponding to the user wearing the wearable device. The fall detection result obtained by combining the auxiliary sensor data through a multi-sensor fusion model can further improve the accuracy of the fall detection result. At the same time, when any of the motion index data exceeds the corresponding preset index threshold, the auxiliary sensor data is obtained. When the motion index data does not exceed the corresponding preset index threshold, the auxiliary sensor data can be kept closed, thereby improving the accuracy of fall detection while reducing the power consumption of the wearable device and improving the user experience.
[0049] Figure 2 Another flowchart showing the method for fall detection provided by the embodiments of the present application is as follows: Figure 2 As shown, step S103 may include the following steps:
[0050] In step S1031, when the wearable device includes the user-predefined type of behavioral ability disorder, according to the user-predefined type of behavioral ability disorder, obtain the fall detection model corresponding to the user-predefined type of behavioral ability disorder from multiple fall detection models as the target fall detection model.
[0051] Exemplarily, the user-predefined type of behavioral ability disorder may be obtained by the wearable device in response to the user's setting operation, and the setting operation may be an operation directly performed by the user on the wearable device or an operation performed by the user on the terminal (such as a mobile phone) connected to the wearable device.
[0052] The wearable device may include fall detection models corresponding to multiple different types of behavioral ability disorders respectively. In some possible implementation manners, after obtaining the user-predefined type of behavioral ability disorder, the fall detection model corresponding to the user-predefined type of behavioral ability disorder may be obtained from multiple fall detection models as the target fall detection model according to the user-predefined type of behavioral ability disorder and the first model correspondence.
[0053] In some possible implementation manners, the first model correspondence may be, for example, the correspondence between the type of behavioral ability disorder and the fall detection model shown in Table 1 below.
[0054] Table 1
[0055] Types of behavioral ability disorders Fall detection model Physical disability Fall detection model 1 Hemilateral limb disorder Fall detection model 2 Unilateral crutch walking Fall detection model 3 Bilateral crutch walking Fall detection model 4 Pregnant woman Fall detection model 5
[0056] Exemplarily, in response to the user setting the type of behavioral ability disorder as "using a single crutch", it may be determined that the corresponding target fall detection model is fall detection model 3.
[0057] It should be noted that the above Table 1 is an example of the first correspondence between the user-predefined type of behavioral ability disorder and the fall detection model. Those skilled in the art can make adaptive changes to Table 1 based on the example of Table 1. It can be understood that based on the characteristics of the motion sensor data during fall detection, different types of behavioral ability disorders may also correspond to the same fall detection model. For example, hemilateral limb disorder and using a single crutch may also correspond to the same fall detection model. The present application does not limit this.
[0058] In some embodiments, the server may train fall detection models corresponding to multiple different types of behavioral ability disorders respectively by the following steps, and send the trained fall detection models corresponding to multiple different types of behavioral ability disorders to the wearable device.
[0059] Step 10: Obtain fall data sets of a medical institution for multiple different types of behavioral ability disorders.
[0060] Step 11: Use one-hot encoding to label the training data in the data set. For example, the fall label is 1 and the non-fall label is 0.
[0061] Step 12: Use the fall data sets of the above multiple different types of behavioral ability disorders and the corresponding labels to train multiple deep separable convolutional neural network models to be trained respectively, and obtain the trained fall detection models corresponding to multiple different types of behavioral ability disorders respectively.
[0062] Step 13: Send the trained fall detection models corresponding to multiple different types of behavioral ability disorders respectively to the wearable device.
[0063] In step S1032, when the wearable device does not include the type of behavioral ability disorder preset by the user, input the motion sensor data into the motion type detection model to obtain the current motion type of the user.
[0064] In some embodiments, the motion type detection model may be a convolutional neural network detection model. Specifically, DSC depth separable convolution may also be used to replace the convolution operation to optimize the convolutional neural network detection model. When specifically training the motion type detection model, the data set publicly available from a sports research institution may be used for training, and then the motion sensor data is input into the trained motion type detection model to obtain the current motion type of the user. For example, the user is currently walking.
[0065] In another embodiment, one or more of the personal information, location information, and auxiliary sensor data (such as electrocardiogram sensor data) preset by the user may be further input into the motion type detection model together with the motion sensor data to obtain the current motion type of the user, which can improve the accuracy of the obtained motion type.
[0066] In step S1033, obtain the fall detection model corresponding to the current motion type of the user from multiple fall detection models as the target fall detection model.
[0067] In some possible implementations, after obtaining the user's current motion type, the corresponding fall detection model for the user's current motion type can be obtained from multiple fall detection models according to the correspondence between the user's current motion type and the second model as the target fall detection model.
[0068] In some possible implementations, the correspondence between the second model can be, for example, the correspondence between the user's current motion type and the fall detection model shown in Table 2 below.
[0069] Table 2
[0070] User's current exercise type Fall detection model Walking Fall detection model 6 Running Fall detection model 7 Badminton Fall detection model 8 Football Fall detection model 9 Cycling Fall detection model 10
[0071] Exemplarily, when the user's current motion type is "badminton", it can be determined that the corresponding target fall detection model is the fall detection model 8.
[0072] It should be noted that the above Table 2 is an example of the second correspondence between the user's current motion type and the fall detection model, and those skilled in the art can make adaptive changes to Table 2 based on the example in Table 2.
[0073] By adopting the above technical means, a more accurate first detection result can be obtained through the target fall detection model corresponding to the user wearing the wearable device.
[0074] Figure 3 Another flowchart showing the fall detection method provided by the embodiment of the present application is shown as Figure 3 shown, step S104 may include the following steps:
[0075] In step S1041, the corresponding first time period during the fall and the corresponding second time period after the fall are obtained according to the motion sensor data.
[0076] It can be understood that during the first time period and the second time period, the motion sensor data will show characteristics different from those before the fall.
[0077] Figure 4 A schematic diagram showing the first time period and the second time period in the acceleration data provided by the embodiment of the present application is shown as Figure 4 shown, before the fall occurs, the acceleration data shows a relatively stable fluctuation characteristic. During the first time period corresponding to the fall, there are multiple cases where the instantaneous acceleration exceeds the acceleration threshold in the acceleration data, while during the second time period corresponding to after the fall, the fluctuation amplitude of the acceleration data is significantly smaller than the fluctuation amplitude of the acceleration data before the fall.
[0078] The time lengths of the first time period and the second time period can be determined based on experience or experiments, and the present application does not limit this.
[0079] In step S1042, first auxiliary sensor data corresponding to a first time period and second auxiliary sensor data corresponding to a second time period are obtained from the auxiliary sensor data.
[0080] Exemplarily, taking a smartwatch as an example, the auxiliary sensor data may include environmental audio data. The first auxiliary sensor data corresponding to the first time period may be the first environmental audio data corresponding to the first time period, and the second auxiliary sensor data corresponding to the second time period may be the second environmental audio data corresponding to the second time period.
[0081] The auxiliary sensor data may include electrocardiogram sensor data. The first auxiliary sensor data corresponding to the first time period may be the first electrocardiogram sensor data corresponding to the first time period, and the second auxiliary sensor data corresponding to the second time period may be the second electrocardiogram sensor data corresponding to the second time period.
[0082] The auxiliary sensor data may further include environmental audio data and electrocardiogram sensor data. The first auxiliary sensor data corresponding to the first time period includes the first environmental audio data and the first electrocardiogram sensor data, and the second auxiliary sensor data corresponding to the second time period includes the second environmental audio data and the second electrocardiogram sensor data.
[0083] In step S1043, an auxiliary sensor quantization result is obtained according to the first auxiliary sensor data and the second auxiliary sensor data.
[0084] In some embodiments, when the auxiliary sensor data is environmental audio data, the auxiliary sensor quantization result is the environmental audio data quantization result. The auxiliary sensor quantization result can be obtained according to the first auxiliary sensor data and the second auxiliary sensor data through the following steps.
[0085] Step 20: Obtain the maximum amplitude ratio.
[0086] The maximum amplitude ratio is the ratio of a first maximum amplitude value corresponding to the first environmental audio data and a second maximum amplitude value corresponding to the second environmental audio data.
[0087] Exemplarily, the maximum amplitude ratio can be determined by the following formula (1).
[0088]
[0089] Wherein, R is the maximum amplitude ratio, A2 is the amplitude sequence corresponding to the second environmental audio data, A1 is the amplitude sequence corresponding to the first environmental audio data, and Max is to take the maximum value in the sequence. It can be understood that the smaller the maximum amplitude ratio, the greater the probability of a fall occurring; conversely, the larger the maximum amplitude ratio, the smaller the probability of a fall occurring.
[0090] Step 21: Use the first scale transformation algorithm to transform the maximum amplitude ratio into the quantization result of the environmental audio data.
[0091] To further improve the efficiency and accuracy of the multi-sensor fusion model, the maximum amplitude ratio can be mapped to a preset value range through the first scale change algorithm. The preset value range can be, for example, [0, 1]. In some possible implementation manners, the first scale transformation algorithm can be, for example, the following formula two.
[0092]
[0093] Where, R is the maximum amplitude ratio, and S(R) is the quantization result of the environmental audio data.
[0094] In some embodiments, when the auxiliary sensor data is electrocardiogram sensor data, the auxiliary sensor quantization result is the quantization result of the electrocardiogram sensor data. The auxiliary sensor quantization result can be obtained according to the first auxiliary sensor data and the second auxiliary sensor data through the following steps.
[0095] Step 30: Obtain the DTW distance between the first electrocardiogram sensor data and the second electrocardiogram sensor data through the dynamic time warping algorithm DTW.
[0096] Exemplarily, the first electrocardiogram sensor data can include the first time series Q(q1, q2,..., q m ), and the second electrocardiogram sensor data can include the second time series H(h1, h2,..., h m ). The DTW distance between the first electrocardiogram sensor data and the second electrocardiogram sensor data can be obtained through the DTW (Dynamic Time Warping) algorithm. In some possible implementation manners, the DTW distance between the i-th element in the first time series and the j-th element in the second time series can be determined through the following formula three. D(m, n) is the DTW distance between the first electrocardiogram sensor data and the second electrocardiogram sensor data.
[0097] D(i, j) = Dist(i, j) + min[D(i - 1, j), D(i, j - 1), D(i - 1, j - 1)] (Formula three)
[0098] Where, Dist(i, j) is the absolute value of the difference between the i-th element in the first time series and the j-th element in the second time series. Min is to take the minimum value. For the technical description of the specific DTW distance, further reference can be made to the records of related technologies, which will not be elaborated here.
[0099] Step 31: Use the second scale transformation algorithm to transform the DTW distance into the quantization result of the electrocardiogram sensor data.
[0100] To further improve the efficiency and accuracy of the multi-sensor fusion model, the DTW distance can be mapped to a preset value range through the second scale change algorithm. The preset value range can be, for example, [0, 1]. In some possible implementation manners, the second scale transformation algorithm can be, for example, the hyperbolic tangent function shown in Formula 4 below.
[0101]
[0102] Where tanh(x) is the quantization result of the electrocardiogram sensor data, x is the DTW distance, and e is the natural constant.
[0103] In step S1044, the first detection result and the quantization result of the auxiliary sensor are input into the multi-sensor fusion model to obtain the fall detection result.
[0104] Where the quantization result of the auxiliary sensor can include the quantization result of the environmental audio data and / or the quantization result of the electrocardiogram sensor data.
[0105] Exemplarily, the first detection result and the quantization result of the auxiliary sensor can be input into a pre-trained multi-sensor fusion model to obtain the fall detection result.
[0106] By adopting the above technical means, a more accurate first detection result can be obtained through the corresponding target fall detection model. Combining the auxiliary sensor data and obtaining the fall detection result through the multi-sensor fusion model can further improve the accuracy of the fall detection result.
[0107] Since the input parameters of the multi-sensor fusion model are fewer and the value ranges have been mapped to the preset value range, compared with the fall detection model in the related art, the performance of the fall detection model can be significantly improved. In some possible implementation manners, the multi-sensor fusion model can adopt a simpler BP (back propagation) neural network model, which can further improve the performance of the fall detection model.
[0108] Figure 5 Another flowchart showing the fall detection method provided by the embodiment of the present application is shown in Figure 5 As shown, the method may further include the following steps:
[0109] In step S105, when the fall detection result is a fall, a fall warning message is sent to the terminal corresponding to the preset emergency contact.
[0110] Among them, the fall warning message includes the fall detection result, and also includes at least one of the device identifier of the wearable device, the fall time, and the fall location.
[0111] The wearable device can set at least one emergency contact in response to the user's setting operation. In the case where the fall detection result is a fall, the fall warning information can be displayed to the terminals corresponding to one or more emergency contacts.
[0112] In some possible implementation manners, the wearable device can be connected to the server through a wireless communication module, and the server can send the fall warning information to the terminals corresponding to one or more emergency contacts, or the wearable device can directly send the fall warning information to the terminals corresponding to one or more emergency contacts. This application does not limit this.
[0113] Figure 6 FIG. shows a schematic diagram of a display interface for the fall detection method provided by an embodiment of the present application. In the case where the fall detection result is a fall, the confirmation interface as shown in Figure 6 (a) can be displayed on the interface of the wearable device, and the user can also be reminded by vibration, so that the user can click "Cancel" or "Alarm" on the interface according to the actual situation. For example, in the case where the fall detection result is a false alarm, the confirmation interface can be closed in response to the user's "Cancel" operation. Otherwise, in response to the user's "Alarm" operation, the fall warning information can be sent to the terminals corresponding to one or more emergency contacts. In some possible implementation manners, after the above confirmation interface is displayed and the time without receiving the user's operation exceeds a preset duration threshold, the fall warning information can be directly sent to the terminals corresponding to one or more emergency contacts.
[0114] In some embodiments, after receiving the user's "Cancel" operation or "Alarm" operation, the wearable device can send the sensor data corresponding to the fall detection and the operation information of the user on the fall detection result to the server as training data, so that the server can update the training data set to train and update the corresponding target fall detection model and / or multi-sensor fusion model. It can be understood that the operation information of the user on the fall detection result can be used as the label of the corresponding sensor data, and the sensor data misreported by the target fall detection model and / or multi-sensor fusion model (for example, the user clicks "Cancel" indicating that the fall detection result is a false alarm) can be given a higher weight during training to further improve the accuracy of the target fall detection model and / or multi-sensor fusion model.
[0115] After receiving the fall warning information, the terminals corresponding to one or more emergency contacts can display as Figure 6The fall warning interface shown in (b). In some possible implementation manners, the fall warning interface can be displayed with the highest priority so that the emergency contacts can handle it in time.
[0116] Adopting the above implementation manner, when the result of fall detection is a fall, in response to the user's confirmation operation or when the user confirmation times out, a fall warning message can be sent to the terminals corresponding to one or more emergency contacts, facilitating the emergency contacts to handle the accidental fall situation in time and improving the user experience of the wearable device.
[0117] Figure 7 Another flow schematic diagram showing the fall detection method provided by the embodiments of the present application. The motion sensor is an accelerometer, and the auxiliary sensors are a microphone and an electrocardiogram sensor. As Figure 7 shown, the method includes the following steps:
[0118] In step 100, the acceleration data collected by the accelerometer is obtained, and the acceleration index data is obtained from the acceleration data.
[0119] The acceleration index data can be, for example, the maximum acceleration in the acceleration time series of the acceleration data.
[0120] In step 101, when the wearable device does not include the user-predefined type of behavioral ability disorder, the current motion type of the user is determined.
[0121] Exemplarily, the acceleration data can be input into the motion type detection model to obtain the current motion type of the user.
[0122] In step 102, a target fall detection model is obtained from multiple fall detection models.
[0123] When the wearable device includes the user-predefined type of behavioral ability disorder, the fall detection model corresponding to the user-predefined type of behavioral ability disorder can be obtained from multiple fall detection models as the target fall detection model. Otherwise, the fall detection model corresponding to the current motion type of the user can be obtained from multiple fall detection models as the target fall detection model.
[0124] In step 103, when the acceleration index data exceeds the preset index threshold, the electrocardiogram sensor data collected by the electrocardiogram sensor and the environmental audio data collected by the microphone are obtained.
[0125] In step 104, the acceleration data is input into the target fall detection model to obtain a first detection result.
[0126] In step 105, the first detection result, the electrocardiogram sensor data, and the environmental audio data are input into the multi-sensor fusion model to obtain the fall detection result.
[0127] In step 106, when the fall detection result is a fall and no cancellation operation from the user is received within a preset duration threshold, a fall warning message is sent to the terminal corresponding to the preset emergency contact.
[0128] By adopting the above technical method, a more accurate first detection result can be obtained through the corresponding target fall detection model. Combining the auxiliary sensor data to obtain the fall detection result through the multi-sensor fusion model can further improve the accuracy of the fall detection result. When the fall detection result is a fall, a fall warning message can also be sent to the terminal corresponding to the emergency contact in a timely manner, improving the user experience.
[0129] Figure 8 The block diagram of a fall detection device provided by an embodiment of the present application is shown. It is applied to a wearable device. The wearable device includes a motion sensor and at least one auxiliary sensor. The wearable device includes multiple fall detection models and a multi-sensor fusion model. As Figure 8 shown, the fall detection device 200 includes:
[0130] A first acquisition module 210, configured to acquire motion sensor data collected by the motion sensor and obtain at least one motion index data from the motion sensor data;
[0131] A second acquisition module 220, configured to acquire auxiliary sensor data collected by at least one auxiliary sensor when any motion index data exceeds the corresponding preset index threshold;
[0132] A first detection module 230, configured to obtain a target fall detection model from multiple fall detection models and input the motion sensor data into the target fall detection model to obtain a first detection result;
[0133] A second detection module 240, configured to input the first detection result and the auxiliary sensor data into the multi-sensor fusion model to obtain a fall detection result.
[0134] Optionally, the wearable device includes user-predefined types of behavioral ability disorders. The multiple fall detection models include fall detection models corresponding to multiple different types of behavioral ability disorders. The first detection module 230 is further configured to:
[0135] According to the user-predefined type of behavioral ability disorder, obtain the fall detection model corresponding to the user-predefined type of behavioral ability disorder from the multiple fall detection models as the target fall detection model.
[0136] Optionally, the multiple fall detection models include fall detection models corresponding to multiple different types of motion. The first detection module 230 is further configured to:
[0137] Input the motion sensor data into the motion type detection model to obtain the user's current motion type;
[0138] Obtain the fall detection model corresponding to the user's current motion type from multiple fall detection models as the target fall detection model.
[0139] Optionally, the wearable device also stores the user's preset personal information, and the first detection module 230 is further configured to:
[0140] Input the user's preset personal information and motion sensor data into the motion type detection model to obtain the user's current motion type.
[0141] Optionally, the second detection module 240 is further configured to:
[0142] Obtain the corresponding first time period during the fall and the corresponding second time period after the fall according to the motion sensor data;
[0143] Obtain the first auxiliary sensor data corresponding to the first time period and the second auxiliary sensor data corresponding to the second time period from the auxiliary sensor data;
[0144] Obtain the auxiliary sensor quantization result according to the first auxiliary sensor data and the second auxiliary sensor data;
[0145] Input the first detection result and the auxiliary sensor quantization result into the multi-sensor fusion model to obtain the fall detection result.
[0146] Optionally, the first auxiliary sensor data is the first environmental audio data, the second auxiliary sensor data is the second environmental audio data, the auxiliary sensor quantization result is the environmental audio data quantization result, and the second detection module 240 is further configured to:
[0147] Obtain the maximum amplitude ratio, where the maximum amplitude ratio is the ratio of the second maximum amplitude value corresponding to the second environmental audio data to the first maximum amplitude value corresponding to the first environmental audio data;
[0148] Use the first scale transformation algorithm to transform the maximum amplitude ratio into the environmental audio data quantization result.
[0149] Optionally, the first auxiliary sensor data is the first electrocardiogram sensor data, the second auxiliary sensor data is the second electrocardiogram sensor data, the auxiliary sensor quantization result is the electrocardiogram sensor data quantization result, and the second detection module 240 is further configured to:
[0150] Obtain the DTW distance between the first electrocardiogram sensor data and the second electrocardiogram sensor data through the dynamic time warping algorithm DTW;
[0151] The DTW distance is transformed into the quantization result of the electrocardiogram sensor data by using a second scaling algorithm.
[0152] The device 200 provided in the embodiments of the present application can execute the various methods in the foregoing method embodiments, and realize the functions and beneficial effects of the various methods in the foregoing method embodiments, which will not be elaborated herein.
[0153] Figure 9 The block diagram of a fall detection device provided by an embodiment of the present application is shown, as Figure 9 shown, the fall detection device 200 further includes:
[0154] A warning module 250, configured to send a fall warning message to the terminal corresponding to a preset emergency contact when the fall detection result is a fall.
[0155] Figure 10 The schematic hardware structure diagram of an electronic device provided by an embodiment of the present application is shown, as Figure 10 shown, at the hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (Random-Access Memory, RAM), and may also include non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.
[0156] The processor, network interface, and memory can be interconnected through an internal bus, and the internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a bidirectional arrow is used to represent it in this figure, but it does not mean that there is only one bus or one type of bus.
[0157] The memory stores a program. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include internal memory and non-volatile memory, and provide instructions and data to the processor.
[0158] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a device for locating the target user at the logical level. The processor executes the program stored in the memory and specifically executes: Figures 1 - 7 The method disclosed in the illustrated embodiment and realizes the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be elaborated herein.
[0159] The above as in this application Figures 1 - 7 The method disclosed in the illustrated embodiment can be applied in the processor or implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, the various steps of the above method can be completed by the integrated logic circuit in the hardware of the processor or by instructions in the form of software. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.
[0160] The electronic device can also execute the various methods described in the foregoing method embodiments and realize the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be elaborated herein.
[0161] Of course, in addition to the software implementation method, the electronic device of the present application does not exclude other implementation methods, such as a logic device or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.
[0162] Embodiments of the present application also propose a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, cause the electronic device to execute Figures 1 - 7 the method disclosed in the illustrated embodiment and realize the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be elaborated herein.
[0163] Among them, the computer-readable storage medium includes a read-only memory (ROM for short), a random access memory (RAM for short), a magnetic disk, an optical disc, etc.
[0164] Furthermore, embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, implement the following process: Figures 1 - 7 the method disclosed in the illustrated embodiment and realize the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be elaborated herein.
[0165] In summary, the above are only the preferred embodiments of the present application, and do not limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
[0166] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.
[0167] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0168] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0169] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
Claims
1. A method for fall detection, characterized in that, Applied to a wearable device, the wearable device includes a motion sensor and at least one auxiliary sensor, and the method includes: Obtain motion sensor data collected by the motion sensor, and obtain at least one motion index data from the motion sensor data; When any of the at least one motion index data exceeds a corresponding preset index threshold, obtain auxiliary sensor data collected by the at least one auxiliary sensor; Obtain a target fall detection model from multiple fall detection models, and input the motion sensor data into the target fall detection model to obtain a first detection result; Input the first detection result and the auxiliary sensor data into a multi-sensor fusion model to obtain a fall detection result.
2. The method according to claim 1, characterized in that, The wearable device includes a type of behavioral ability disorder preset by the user, and the multiple fall detection models include fall detection models corresponding to multiple different types of behavioral ability disorders; The obtaining the target fall detection model from the multiple fall detection models includes: According to the type of behavioral ability disorder preset by the user, obtain the fall detection model corresponding to the type of behavioral ability disorder preset by the user from the multiple fall detection models as the target fall detection model.
3. The method according to claim 1, characterized in that, The multiple fall detection models include fall detection models corresponding to multiple different motion types; the obtaining the target fall detection model from the multiple fall detection models includes: Input the motion sensor data into a motion type detection model to obtain the current motion type of the user; Obtain the fall detection model corresponding to the current motion type of the user from the multiple fall detection models as the target fall detection model.
4. The method according to claim 3, characterized in that, The wearable device also stores personal information preset by the user, and the inputting the motion sensor data into a preset motion type detection model to obtain the current motion type of the user includes: Input the personal information preset by the user and the motion sensor data into a motion type detection model to obtain the current motion type of the user.
5. The method according to claim 1, characterized in that, The inputting the first detection result and the auxiliary sensor data into the multi-sensor fusion model to obtain a fall detection result includes: Obtain a corresponding first time period during the fall process and a second time period after the fall according to the motion sensor data; Obtain first auxiliary sensor data corresponding to the first time period and second auxiliary sensor data corresponding to the second time period from the auxiliary sensor data; Obtain an auxiliary sensor quantization result according to the first auxiliary sensor data and the second auxiliary sensor data; Input the first detection result and the auxiliary sensor quantization result into a multi-sensor fusion model to obtain a fall detection result.
6. The method according to claim 5, characterized in that, The at least one auxiliary sensor includes at least one environmental sensor and / or at least one vital sign sensor; the auxiliary sensor data includes environmental sensor data collected by the at least one environmental sensor and / or vital sign sensor data collected by the at least one vital sign sensor.
7. The method according to claim 6, characterized in that, The first auxiliary sensor data is first ambient audio data, the second auxiliary sensor data is second ambient audio data, and the auxiliary sensor quantization result is the ambient audio data quantization result. Obtaining the auxiliary sensor quantization result according to the first auxiliary sensor data and the second auxiliary sensor data includes: Obtaining a maximum amplitude ratio, where the maximum amplitude ratio is the ratio of a second maximum amplitude value corresponding to the second ambient audio data to a first maximum amplitude value corresponding to the first ambient audio data; Using a first scale transformation algorithm to transform the maximum amplitude ratio into the ambient audio data quantization result.
8. The method according to claim 6, characterized in that, The first auxiliary sensor data is first electrocardiogram sensor data, the second auxiliary sensor data is second electrocardiogram sensor data, and the auxiliary sensor quantization result is the electrocardiogram sensor data quantization result. Obtaining the auxiliary sensor quantization result according to the first auxiliary sensor data and the second auxiliary sensor data includes: Obtaining the DTW distance between the first electrocardiogram sensor data and the second electrocardiogram sensor data through the dynamic time warping algorithm DTW; Using a second scale transformation algorithm to transform the DTW distance into the electrocardiogram sensor data quantization result.
9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: When the fall detection result is a fall, sending a fall warning message to a terminal corresponding to a preset emergency contact. The fall warning message includes the fall detection result, and also includes at least one of the device identifier, fall time, and fall location of the wearable device.
10. A device for fall detection, characterized in that, Applied to a wearable device, the wearable device includes a motion sensor and at least one auxiliary sensor, and the device includes: A first acquisition module configured to acquire motion sensor data collected by the motion sensor and obtain at least one motion index data from the motion sensor data; A second acquisition module configured to, when any of the at least one motion index data exceeds a corresponding preset index threshold, acquire auxiliary sensor data collected by the at least one auxiliary sensor; A first detection module configured to obtain a target fall detection model from a plurality of fall detection models and input the motion sensor data into the target fall detection model to obtain a first detection result; A second detection module configured to input the first detection result and the auxiliary sensor data into a multi-sensor fusion model to obtain a fall detection result.
11. A wearable device, comprising: A processor; And A memory arranged to store computer-executable instructions that, when executed, use the processor to execute the steps of the fall detection method according to any one of claims 1-10.
12. A computer-readable storage medium storing one or more programs which, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the steps of the method for fall detection according to any one of claims 1-10.