Stroke tumble early warning method based on flexible sensor and related device

Abstract

The invention discloses a flexible sensor-based stroke falling early warning method and a related device. The method comprises the following steps: according to acceleration, angular velocity, pressure, heart rate and myoelectricity data of a patient; calculating a resultant acceleration, resultant angular velocity and resultant pressure time sequence collected by the waist flexible sensor and each ankle flexible sensor according to the acceleration, angular velocity, pressure, heart rate and myoelectricity data of the patient; calculating a stability coefficient of the waist acceleration sequence and the normal gait template sequence and a balance coefficient between the left and right ankle acceleration sequences according to a multi-scale DTW algorithm; fusing the stability coefficient, the balance coefficient and the pressure distribution data, the heart rate data and the myoelectricity data collected by the flexible sensor to form a multi-modal feature vector; and inputting the multi-modal feature vector into an ANFIS tumble model, and predicting the tumble risk probability of the patient. According to the method, multi-modal data fusion and gait analysis are carried out through fuzzy logic and the neural network, and the accuracy of patient falling risk prediction is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical assistance technology, and in particular to a stroke fall warning method and device based on a flexible sensor, and a computing device. Background Art

[0002] Stroke patients often face problems such as balance dysfunction, decreased muscle strength, and abnormal sensation, making them more likely to fall.

[0003] Traditional fall risk assessment mainly relies on the subjective judgment of doctors or simple clinical tests (such as the stand-and-walk test). It has defects such as strong subjectivity, low efficiency, and inability to perform real-time monitoring, making it difficult to meet clinical needs.

[0004] With the development of artificial intelligence technology, some studies have begun to try to use machine learning or deep learning models to predict the risk of falls by analyzing sensor data. However, most of them only rely on single sensor data (such as accelerometer) or single feature (such as walking speed) for fall detection, ignoring the complexity of falls and failing to fully reflect the patient's fall risk.

[0005] To solve the above problems, the present invention proposes a stroke fall warning method based on flexible sensors, which uses fuzzy logic and neural networks to fuse multimodal data and analyze gait to improve the accuracy of patient fall risk prediction. Summary of the invention

[0006] In view of the above problems, the present invention provides a stroke fall warning method and device, and a computing device based on a flexible sensor.

[0007] According to one aspect of the present invention, a stroke fall warning method based on a flexible sensor is provided, comprising:

[0008] Acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space are collected in real time according to a plurality of flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to a plurality of preset positions of the patient's body, wherein the preset positions include the waist, ankle, elbow joint and knee joint;

[0009] According to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space, the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexible sensor and each ankle flexible sensor are calculated; the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences are calculated according to the multi-scale DTW algorithm;

[0010] The stability coefficient, balance coefficient and pressure distribution data, heart rate data and electromyography data collected by the flexible sensor are integrated to form a multimodal feature vector;

[0011] The multimodal feature vector is input into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

[0012] In an optional manner, the calculating of the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexibility sensor and each ankle flexibility sensor further comprises:

[0013] Calculate the total acceleration once according to the three components of acceleration at each time point to obtain the total acceleration time series;

[0014] The combined angular velocity is calculated once according to the three components of the angular velocity at each time point to obtain the combined angular velocity time series;

[0015] For a multi-region discrete pressure sensor, the pressure value at each time point is added to obtain a combined pressure time series; for a multi-region continuous pressure sensor, the average pressure value of each pixel region at each time point is calculated to obtain a combined pressure time series;

[0016] Among them, the flexible pressure sensor includes a single-point pressure sensor, a multi-area discrete pressure sensor and a multi-area continuous pressure sensor; the multi-area discrete pressure sensor contains multiple independent pressure-sensitive areas, each sensitive area provides a pressure value, and the multi-area continuous pressure sensor provides a pressure value for each pixel area in the form of a matrix.

[0017] In an optional manner, the calculating the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm further includes:

[0018] Multi-scale smoothing of the waist and ankle combined acceleration time series;

[0019] The stability coefficient of each gait cycle is calculated based on the MS-DTW distance between the waist combined acceleration time series and the normal gait template sequence;

[0020] The balance coefficient of each gait cycle was calculated based on the MS-DTW distance of the combined acceleration time series of the left and right ankles.

[0021] In an optional manner, the fusing of the stability coefficient, the balance coefficient, and the pressure distribution data, the heart rate data, and the electromyographic data collected by the flexible sensor to form a multimodal feature vector further includes:

[0022] The equilibrium state is calculated based on the stability coefficient and the balance coefficient, the maximum pressure, average pressure and the center of pressure are obtained based on the pressure distribution data, the heart rate variability is calculated based on the heart rate data, and the activity intensity is calculated based on the electromyography data;

[0023] According to the set activity intensity weight, balance state weight, and heart rate variability weight, a multimodal feature vector is obtained.

[0024] In an optional manner, the fuzzification layer includes a plurality of input nodes, a fuzzy rule layer and an output node; wherein each input node is connected to at least one membership function; the membership function includes a Gaussian function, a bell function and a trapezoidal function; the fuzzy rule layer calculates the trigger strength of each rule according to the fuzzy rule;

[0025] The stacked hourglass prediction layer is used for joint posture prediction and supplementing joint missing data, including a preliminary convolution layer, multiple hourglass modules and a terminal convolution layer, wherein each hourglass module includes a downsampling path encoding layer, a low-resolution feature map network layer and an upsampling path decoding layer;

[0026] The consequence layer calculates the fall risk probability according to the trigger intensity and the consequent parameters of the fuzzy rule.

[0027] In an optional manner, the pressure distribution data includes the pressure center offset, the plantar pressure distribution symmetry, the pressure peak position within the gait cycle, and the dynamic pressure distribution range;

[0028] The heart rate data features include heart rate variability index, heart rate acceleration / deceleration rate and heart rate abnormal fluctuation frequency;

[0029] The electromyographic data includes muscle activation intensity, muscle co-activation degree and muscle fatigue index.

[0030] In an optional manner, the calculating of the fall risk probability according to the trigger intensity and the consequent parameter of the fuzzy rule further comprises:

[0031] Define the state set S = {stable gait, mildly unstable gait, moderately unstable gait, high risk of falling, fall occurred};

[0032] Define observation symbol V = [p1, p2, p3, p4, h1, h2, h3, e1, e2, e3, st, b]; where p1 is the pressure center offset; p2 is the symmetry of plantar pressure distribution; p3 is the peak pressure position in the gait cycle; p4 ​​is the dynamic pressure distribution range; h1 is the heart rate variability index; h2 is the heart rate acceleration / deceleration rate; h3 is the heart rate abnormal fluctuation frequency; e1 is the muscle activation intensity; e2 is the degree of muscle synergistic activation; e3 is the muscle fatigue index; st is the stability coefficient; b is the balance coefficient;

[0033] Define the transition probability matrix A of each state set S, the initial state probability distribution π, and the observation probability matrix B of the observation symbol V;

[0034] The forward algorithm is used to calculate the probability of each state from the first observation value to the current observation value, and the backward algorithm is used to calculate the probability of each state from the current observation value to the last observation value;

[0035] Adjust the trigger strength according to the probability of each state, and incorporate the probability of each state into the consequent parameter of the fuzzy rule;

[0036] The fall risk probability is calculated based on the adjusted trigger intensity and rule consequent parameters.

[0037] In an optional manner, the calculation formula for the fall risk probability is:

[0038]

[0039] Among them, P i is the consequent parameter of the ith rule; α i_adjusted is the trigger strength of the i-th rule; α i_adjusted =α i ·∑ s∈S P(q t =s|O)·w s ; xi is the i-th input variable, L xi is the membership value of xi; P(q t =s|O) is the probability of being in state s at time point t given the observation sequence O; w s is the weight of state s; S is the state set.

[0040] According to another aspect of the present invention, a stroke fall warning device based on a flexible sensor is provided, comprising:

[0041] A data acquisition module, for collecting acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space in real time according to a plurality of flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to a plurality of preset positions of the patient's body, wherein the preset positions include the waist, ankle, elbow joint and knee joint;

[0042] The dynamic time warping feature extraction module is used to calculate the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexible sensor and each ankle flexible sensor based on the patient's acceleration, angular velocity, pressure, heart rate and electromyography data in three-dimensional space; calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences based on the multi-scale DTW algorithm;

[0043] A multimodal data fusion module is used to fuse the stability coefficient, balance coefficient, and pressure distribution data, heart rate data, and electromyography data collected by the flexible sensor to form a multimodal feature vector;

[0044] The fall prediction module is used to input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

[0045] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0046] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned stroke fall warning method based on flexible sensors.

[0047] According to the solution provided by the present invention, the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space are collected in real time according to multiple flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to multiple preset positions of the patient's body, and the preset positions include the waist, ankles, elbow joints and knee joints; according to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space, the combined acceleration collected by the waist flexible sensor and each ankle flexible sensor is calculated The invention is to calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm; integrate the stability coefficient, balance coefficient and the pressure distribution data, heart rate data and electromyography data collected by the flexible sensor to form a multimodal feature vector; input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer. The invention improves the accuracy of the patient's fall risk prediction by fusion of multimodal data and gait analysis through fuzzy logic and neural network.

[0048] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0050] Figure 1 A schematic diagram showing a flow chart of a stroke fall warning method based on a flexible sensor according to an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a calculation process of a stability coefficient and a balance coefficient according to an embodiment of the present invention is shown;

[0052] Figure 3 A schematic diagram of an ANFIS fall model according to an embodiment of the present invention is shown;

[0053] Figure 4 A schematic diagram of a fall risk probability calculation process according to an embodiment of the present invention is shown;

[0054] Figure 5 A 2D-3D posture schematic diagram of an embodiment of the present invention is shown;

[0055] Figure 6 A schematic diagram of posture missing supplementation according to an embodiment of the present invention is shown;

[0056] Figure 7 A schematic diagram of a frame of a method and device for early warning of stroke and fall based on flexible sensor according to an embodiment of the present invention is shown;

[0057] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0059] Figure 1 FIG. 1 is a flow chart of a stroke fall warning method based on a flexible sensor according to an embodiment of the present invention. Specifically, Figure 1 As shown, the following steps are included:

[0060] Step S101, collecting the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space in real time according to multiple flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to multiple preset positions of the patient's body, and the preset positions include the waist, ankles, elbows and knees.

[0061] In this embodiment, the flexible three-axis accelerometer measures the acceleration of the patient in three axes (front and back, left and right, up and down) to reflect the movement and posture changes of the patient's body. The flexible three-axis gyroscope measures the angular velocity of the patient in three axes, that is, the speed of body rotation, to reflect the balance and posture changes of the patient's body. The flexible pressure sensor measures the pressure between the patient's body and the contact surface (such as a plantar pressure sensor, which measures the pressure distribution on the sole of the patient's foot when walking or standing). The flexible heart rate sensor measures the patient's heart rate. The flexible electromyography sensor measures the electrical activity of the patient's muscles, and the electromyography signal reflects the activation intensity and fatigue degree of the muscles.

[0062] To ensure that the sensor can be used conveniently and comfortably for a long time, the present invention uses a flexible sensor and integrates it into a wearable device to transmit data via a wireless network, for example:

[0063] Strap / armband type: Flexible accelerometers, gyroscopes, electromyography sensors, etc. are integrated into adjustable straps or armbands and fixed at locations such as the waist, ankles, elbows and knees.

[0064] Insole type: Flexible pressure sensors are embedded in the insole to facilitate monitoring of plantar pressure.

[0065] Patch type: The heart rate sensor is attached to the chest or wrist in the form of a patch.

[0066] Step S102, based on the patient's acceleration, angular velocity, pressure, heart rate and electromyography data in three-dimensional space, calculate the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexible sensor and each ankle flexible sensor; calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences based on the multi-scale DTW algorithm.

[0067] In this embodiment, Figure 2 As shown in the figure, wearable sensors are used to collect human motion data, gait features are extracted through data processing, and the stability and balance of gait are evaluated using a multi-scale dynamic time warping (DTW) algorithm. The multi-scale DTW algorithm provides quantitative gait stability and balance indicators, which facilitate objective evaluation and tracking. By comparing with the normal gait template, personalized gait analysis is achieved. Among them, the combined acceleration is the square root of the sum of the square acceleration values ​​of each axis, and the combined pressure can be obtained by the average value of the pressure sensor data or other weighted methods.

[0068] Specifically, the gait data of normal people are collected in advance, and their combined acceleration time series are extracted as a normal gait template.

[0069] The waist acceleration sequence and the normal gait template sequence are decomposed in multiple scales, for example, using wavelet decomposition to decompose the data into components of different frequencies (approximate components and detail components).

[0070] At each scale, the distance between the waist acceleration sequence and the normal gait template sequence is calculated using the DTW algorithm. The DTW algorithm can handle sequences of different lengths and rhythms and can allow local time warps between sequences.

[0071] The stability coefficient is defined as the inverse of the DTW distance or the normalized value. The higher the stability coefficient, the closer the gait is to the normal gait template.

[0072] The multi-scale DTW distance between the combined acceleration sequences of the left and right ankles is calculated. The balance coefficient is defined as the inverse of the DTW distance or calculated based on the offset of the DTW distance. The higher the balance coefficient, the more symmetrical the movements of the left and right ankles.

[0073] In an optional manner, the calculating of the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexibility sensor and each ankle flexibility sensor further comprises:

[0074] Calculate the total acceleration once according to the three components of acceleration at each time point to obtain the total acceleration time series;

[0075] The combined angular velocity is calculated once according to the three components of the angular velocity at each time point to obtain the combined angular velocity time series;

[0076] For a multi-region discrete pressure sensor, the pressure value at each time point is added to obtain a combined pressure time series; for a multi-region continuous pressure sensor, the average pressure value of each pixel region at each time point is calculated to obtain a combined pressure time series;

[0077] Among them, the flexible pressure sensor includes a single-point pressure sensor, a multi-area discrete pressure sensor and a multi-area continuous pressure sensor; the multi-area discrete pressure sensor contains multiple independent pressure-sensitive areas, each sensitive area provides a pressure value, and the multi-area continuous pressure sensor provides a pressure value for each pixel area in the form of a matrix.

[0078] In this embodiment, different methods of calculating the combined pressure are adopted for different types of pressure sensors (single point, multi-region discrete, multi-region continuous), and the standard vector modulus is used to calculate the combined acceleration and the combined angular velocity, and finally a time series is formed.

[0079] Specifically, for each time point t, the three components of acceleration are Ax(t), Ay(t) and Az(t), and the total acceleration is A_mag(t): Arrange the resultant acceleration values ​​at each time point in chronological order to obtain a resultant acceleration time series, and similarly obtain a resultant angular velocity time series.

[0080] For a single-point pressure sensor, at each time point t, the pressure sensor provides a pressure value P(t), which is directly used as the total pressure at that time point.

[0081] For a multi-region discrete pressure sensor, the sensor has n independent pressure sensitive regions, and the pressure value provided by each region is P1(t), P2(t), ..., Pn(t). Then the combined pressure P_mag(t): P1(t)+P2(t)+...+Pn(t). The combined pressure values ​​at each time point are arranged in chronological order to obtain the combined pressure time series.

[0082] For a multi-region continuous pressure sensor, the sensor has m×n pixel regions, and the pressure value of each region is Pij(t), where i represents the row and j represents the column. Then the combined pressure P_mag(t) is the average of the pressure values ​​of all pixel regions. The combined pressure values ​​at each time point are arranged in chronological order to obtain the combined pressure time series.

[0083] Arrange the total acceleration value at each time point in chronological order to obtain the total acceleration time series.

[0084] In an optional manner, the calculating the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm further includes:

[0085] Multi-scale smoothing of the waist and ankle combined acceleration time series;

[0086] The stability coefficient of each gait cycle is calculated based on the MS-DTW distance between the waist combined acceleration time series and the normal gait template sequence;

[0087] The balance coefficient of each gait cycle was calculated based on the MS-DTW distance of the combined acceleration time series of the left and right ankles.

[0088] In this embodiment, since traditional DTW is very sensitive to local distortion of time series, MS-DTW can better tolerate noise, individual differences and changes in pace speed in gait data by performing smoothing at multiple scales. By comparing the similarity of time series at multiple scales, more comprehensive gait features are captured, so as to more accurately calculate the difference between gait and normal template and the coordination of left and right ankle movements. In addition, gait features of different scales are extracted through multi-scale smoothing. For example, the coarse scale reflects the smoothness of the overall gait, and the fine scale reflects the slight changes in the details of the pace. For example, the MS-DTW distance of a certain gait cycle is 0.2, while the standard for normal people is 0.05, indicating that the movement of the left and right ankles in this gait cycle is not coordinated and the balance is poor.

[0089] Step S103, integrating the stability coefficient, the balance coefficient, and the pressure distribution data, the heart rate data, and the electromyography data collected by the flexible sensor to form a multimodal feature vector.

[0090] In an optional manner, the fusing of the stability coefficient, the balance coefficient, and the pressure distribution data, the heart rate data, and the electromyographic data collected by the flexible sensor to form a multimodal feature vector further includes:

[0091] The equilibrium state is calculated based on the stability coefficient and the balance coefficient, the maximum pressure, average pressure and the center of pressure are obtained based on the pressure distribution data, the heart rate variability is calculated based on the heart rate data, and the activity intensity is calculated based on the electromyography data;

[0092] According to the set activity intensity weight, balance state weight, and heart rate variability weight, a multimodal feature vector is obtained.

[0093] In this embodiment, multimodal fusion can capture subtle changes that are difficult to detect with a single modality. For example, through pressure distribution and electromyography data, the relationship between muscle activity and plantar pressure can be analyzed more precisely.

[0094] For example, the stability coefficient is 0.4, the balance coefficient is 0.3, the pressure distribution extracts the pressure peak (left foot: 1.2kPa, right foot: 1.1kPa), the discrete degree of the pressure center trajectory (0.05cm), and the symmetry of the pressure distribution (0.1). Heart rate: the average heart rate is 90bpm, and the heart rate variability is 40ms. Electromyography: the average activation amplitude of the quadriceps femoris is 0.5mV, and the activation time of the tibialis anterior is 0.2 seconds. All the extracted features are concatenated into a vector to form a multimodal feature vector: [0.4, 0.3, 1.2, 1.1, 0.05, 0.1, 90, 40, 0.5, 0.2].

[0095] In an optional manner, the pressure distribution data includes the pressure center offset, the plantar pressure distribution symmetry, the pressure peak position within the gait cycle, and the dynamic pressure distribution range;

[0096] The heart rate data features include heart rate variability index, heart rate acceleration / deceleration rate and heart rate abnormal fluctuation frequency;

[0097] The electromyographic data includes muscle activation intensity, muscle co-activation degree and muscle fatigue index.

[0098] In this embodiment, the pressure center offset reflects the movement trajectory and stability of the pressure center during gait, and a large offset indicates an unstable gait or poor center of gravity control. The symmetry of plantar pressure distribution reflects the degree of symmetry of the feet in load-bearing and pressure distribution. The position of the peak pressure within the gait cycle reflects the area and time of pressure concentration. The dynamic pressure distribution range reflects the changes in the plantar force range during gait, revealing the motor control and muscle coordination ability. The heart rate acceleration / deceleration rate reflects how fast the heart rate changes over time. The frequency of abnormal heart rate fluctuations reflects the frequency of irregular heart rate fluctuations. The degree of muscle synergistic activation reflects the coordinated activities of different muscles during exercise. The muscle fatigue index reflects the degree of muscle fatigue, which can be evaluated using specific parameters (median frequency) in the electromyographic signal.

[0099] Step S104, inputting the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

[0100] In this embodiment, the ANFIS model combines the self-learning ability of neural networks and the ability of fuzzy logic to handle uncertainty, where the neural network part learns the pattern of data and the fuzzy logic part handles the ambiguity and uncertainty in the data. Unlike black-box neural networks, the ANFIS model has a certain degree of interpretability, and the fuzzy rules reveal the relationship between features and fall risk.

[0101] Among them, the input layer receives a multimodal feature vector, and the number of input neurons is equal to the dimension of the feature vector. The stacked hourglass prediction layer is used to improve the feature learning ability of the model. The stacked hourglass structure contains an encoder and a decoder. The network fuzzification layer converts the input features into fuzzy memberships, and each input feature is mapped to a different fuzzy set through a predefined membership function (such as Gaussian function, triangular function, etc.). The rule layer defines a set of fuzzy rules, each of which represents the relationship between the input feature combination and the risk of falling. The consequence layer calculates the contribution of each rule to the output based on the activation strength of the rule and the consequence parameter of the rule. The output layer performs a weighted summation of the contributions of all rules to obtain the final probability of falling risk.

[0102] In an optional manner, the fuzzification layer includes a plurality of input nodes, a fuzzy rule layer and an output node; wherein each input node is connected to at least one membership function; the membership function includes a Gaussian function, a bell function and a trapezoidal function; the fuzzy rule layer calculates the trigger strength of each rule according to the fuzzy rule;

[0103] The stacked hourglass prediction layer is used for joint posture prediction and supplementing joint missing data, including a preliminary convolution layer, multiple hourglass modules and a terminal convolution layer, wherein each hourglass module includes a downsampling path encoding layer, a low-resolution feature map network layer and an upsampling path decoding layer;

[0104] The consequence layer calculates the fall risk probability according to the trigger intensity and the consequent parameters of the fuzzy rule.

[0105] In this embodiment, Figure 3 As shown in the figure, each input node can be connected to multiple membership functions, and one input feature expresses multiple fuzzy concepts in the fuzzification process, thereby more carefully depicting the complex relationship between features and fall risk. For example, the Gaussian function smoothly represents the central value, while the trapezoidal function expresses the interval value. The use of stacked hourglass prediction layers can more effectively learn the human joint posture information, complete the missing joint data, and learn information representations that are more informative than the original features. Through downsampling and upsampling operations, the hourglass module learns features of different scales to better capture the global and local information of the joint posture. The low-resolution feature map network layer performs convolution and activation operations on the low-resolution feature map. The terminal convolution layer convolves the output of the last hourglass module to obtain the final joint posture feature representation, and performs joint posture prediction and supplements the missing joint data (such as Figure 5 , Figure 6 The consequence layer calculates the fall risk probability based on the trigger strength of the rule and the consequent parameters of the rule.

[0106] In an optional manner, the calculating of the fall risk probability according to the trigger intensity and the consequent parameter of the fuzzy rule further comprises:

[0107] Define the state set S = {stable gait, mildly unstable gait, moderately unstable gait, high risk of falling, fall occurred};

[0108] Define observation symbol V = [p1, p2, p3, p4, h1, h2, h3, e1, e2, e3, st, b]; where p1 is the pressure center offset; p2 is the symmetry of plantar pressure distribution; p3 is the peak pressure position in the gait cycle; p4 ​​is the dynamic pressure distribution range; h1 is the heart rate variability index; h2 is the heart rate acceleration / deceleration rate; h3 is the heart rate abnormal fluctuation frequency; e1 is the muscle activation intensity; e2 is the degree of muscle synergistic activation; e3 is the muscle fatigue index; st is the stability coefficient; b is the balance coefficient;

[0109] Define the transition probability matrix A of each state set S, the initial state probability distribution π, and the observation probability matrix B of the observation symbol V;

[0110] The forward algorithm is used to calculate the probability of each state from the first observation value to the current observation value, and the backward algorithm is used to calculate the probability of each state from the current observation value to the last observation value;

[0111] Adjust the trigger strength according to the probability of each state, and incorporate the probability of each state into the consequent parameter of the fuzzy rule;

[0112] The fall risk probability is calculated based on the adjusted trigger intensity and rule consequent parameters.

[0113] In this embodiment, Figure 4 As shown in the figure, a series of states related to fall risk are defined, such as "stable gait" to "fall occurrence", which can capture the dynamic changes of patients' fall risk rather than just a static assessment. The set of observation symbols V includes indicators of multiple dimensions such as pressure center, plantar pressure distribution, heart rate variability, muscle activation, etc., which evaluate the patient's fall risk from multiple angles. By inferring the patient's hidden state (for example, "moderately unstable gait") through observed data, the evolution of fall risk can be captured, rather than just analyzing independent observations. The state probability is calculated through the forward-backward algorithm and integrated into the fuzzy rule consequent parameters of ANFIS. The consequent parameters of the rule are no longer static, but change according to the real-time state of the patient. Dynamically adjusting the trigger strength according to the probability of the state can make more accurate predictions based on the patient's current state.

[0114] For example, the observation symbol V only uses the pressure center offset p1 as the observation symbol, and the state set S is: {stable gait, high risk of falling state}.

[0115] Observation sequence: Patient’s p1 changes over time. [0.2, 0.3, 0.4, 0.7, 0.8, 0.9, 0.5, 0.4].

[0116] Fuzzy rules: IF p1 IS high THEN the fall risk IS high, IF p1 IS low THEN the fall risk IS low.

[0117] Initialize A, π, B, and set the transition probability from "stable gait" to "high risk of falling state" to 0.2, and vice versa to 0.1. The initial probability of "stable gait" of π is 0.9. The B matrix can be initialized according to the training data. For example, in the "high risk of falling state", the mean value of p1 is higher.

[0118] Use the training data to train the ANFIS parameters, and continuously update the membership function and consequent parameters of ANFIS through back propagation and gradient descent. For the observation sequence [0.2, 0.3, 0.4, 0.7, 0.8, 0.9, 0.5, 0.4], the state probability at each time step is calculated through the forward-backward algorithm. For example, when p1 = 0.9, the probability of "high risk of falling state" is higher. Dynamically adjust the consequent parameters of the rule according to the state probability. For example, if the probability of "high risk of falling state" at the current time step is high, the consequent parameters of the rule "IF p1 IS high THEN the risk of falling IS high" will be amplified. According to the adjusted rules and trigger strength, calculate the final risk of falling probability.

[0119] In an optional manner, the calculation formula for the fall risk probability is:

[0120]

[0121] Among them, P i is the consequent parameter of the ith rule; α i_adjusted is the trigger strength of the i-th rule; α i_adjusted =α i ·∑ s∈S P(q t =s|O)·w s ; xi is the i-th input variable, L xi is the membership value of xi; P(q t =s|O) is the probability of being in state s at time point t given the observation sequence O; w s is the weight of state s; S is the state set.

[0122] In this embodiment, the risk values ​​of multiple rules are weighted averaged, and the weights are determined by the rule trigger strength and the probability of the current state, so that personalized risk assessment can be performed for different groups of people. Fuzzy logic handles the uncertainty of input variables and rules, which is more in line with the actual situation. The rule form has good interpretability, which makes it easy to understand the reasons for risk assessment.

[0123] According to the solution provided by the present invention, the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space are collected in real time according to multiple flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to multiple preset positions of the patient's body, and the preset positions include the waist, ankles, elbow joints and knee joints; according to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space, the combined acceleration collected by the waist flexible sensor and each ankle flexible sensor is calculated The invention is to calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm; integrate the stability coefficient, balance coefficient and the pressure distribution data, heart rate data and electromyography data collected by the flexible sensor to form a multimodal feature vector; input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer. The invention improves the accuracy of the patient's fall risk prediction by fusion of multimodal data and gait analysis through fuzzy logic and neural network.

[0124] Figure 7 The schematic diagram of the framework of the flexible sensor-based stroke fall warning device according to an embodiment of the present invention is shown. The flexible sensor-based stroke fall warning device comprises:

[0125] The data acquisition module 710 is used to collect acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space in real time according to a plurality of flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to a plurality of preset positions of the patient's body, wherein the preset positions include the waist, ankle, elbow joint and knee joint;

[0126] The dynamic time warping feature extraction module 720 is used to calculate the acceleration, angular velocity and pressure time series collected by the waist flexible sensor and each ankle flexible sensor according to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space; calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm;

[0127] A multimodal data fusion module 730 is used to fuse the stability coefficient, the balance coefficient, and the pressure distribution data, the heart rate data, and the electromyographic data collected by the flexible sensor to form a multimodal feature vector;

[0128] The fall prediction module 740 is used to input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

[0129] Figure 8 The schematic diagram of the structure of the computing device embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0130] like Figure 5 As shown, the computing device may include: a processor (processor) 802 , a communication interface (Communications Interface) 804 , a memory (memory) 806 , and a communication bus 808 .

[0131] The processor 802, the communication interface 804, and the memory 806 communicate with each other via a communication bus 808. The communication interface 804 is used to communicate with other devices such as a client or other server network elements. The processor 802 is used to execute a program 810, which can specifically execute the relevant steps in the above-mentioned flexible sensor-based stroke fall warning method embodiment.

[0132] Specifically, the program 810 may include program codes, which include computer operation instructions.

[0133] The processor 802 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0134] The memory 806 is used to store the program 810. The memory 806 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0135] According to the solution provided by the present invention, the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space are collected in real time according to multiple flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to multiple preset positions of the patient's body, and the preset positions include the waist, ankles, elbow joints and knee joints; according to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space, the combined acceleration collected by the waist flexible sensor and each ankle flexible sensor is calculated The invention is to calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm; integrate the stability coefficient, balance coefficient and the pressure distribution data, heart rate data and electromyography data collected by the flexible sensor to form a multimodal feature vector; input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer. The invention improves the accuracy of the patient's fall risk prediction by fusion of multimodal data and gait analysis through fuzzy logic and neural network.

[0136] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose. In addition, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means being within the scope of the present invention and forming different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination. The present invention can be implemented by hardware including several different elements and by appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The steps in the above embodiments should not be understood as limiting the execution order unless otherwise specified.

Claims

1. A stroke fall warning method based on flexible sensor, characterized in that: include: Acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space are collected in real time according to a plurality of flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to a plurality of preset positions of the patient's body, wherein the preset positions include the waist, ankle, elbow joint and knee joint; According to the acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space, the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexible sensor and each ankle flexible sensor are calculated; the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences are calculated according to the multi-scale DTW algorithm; The stability coefficient, balance coefficient and pressure distribution data, heart rate data and electromyography data collected by the flexible sensor are integrated to form a multimodal feature vector; The multimodal feature vector is input into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a stacked hourglass prediction layer, a network fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

2. The stroke fall warning method based on flexible sensor according to claim 1 is characterized in that: The calculating of the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexibility sensor and each ankle flexibility sensor further comprises: Calculate the total acceleration once according to the three components of acceleration at each time point to obtain the total acceleration time series; The combined angular velocity is calculated once according to the three components of the angular velocity at each time point to obtain the combined angular velocity time series; For a multi-region discrete pressure sensor, the pressure value at each time point is added to obtain a combined pressure time series; for a multi-region continuous pressure sensor, the average pressure value of each pixel region at each time point is calculated to obtain a combined pressure time series; Among them, the flexible pressure sensor includes a single-point pressure sensor, a multi-area discrete pressure sensor and a multi-area continuous pressure sensor; the multi-area discrete pressure sensor contains multiple independent pressure-sensitive areas, each sensitive area provides a pressure value, and the multi-area continuous pressure sensor provides a pressure value for each pixel area in the form of a matrix.

3. The stroke fall warning method based on flexible sensor according to claim 1, characterized in that: The step of calculating the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences according to the multi-scale DTW algorithm further includes: Multi-scale smoothing of the waist and ankle combined acceleration time series; The stability coefficient of each gait cycle is calculated based on the MS-DTW distance between the waist combined acceleration time series and the normal gait template sequence; The balance coefficient of each gait cycle was calculated based on the MS-DTW distance of the combined acceleration time series of the left and right ankles.

4. The stroke fall warning method based on flexible sensor according to claim 1, characterized in that: The fusing of the stability coefficient, the balance coefficient, and the pressure distribution data, the heart rate data, and the electromyographic data collected by the flexible sensor to form a multimodal feature vector further includes: The equilibrium state is calculated based on the stability coefficient and the balance coefficient, the maximum pressure, average pressure and the center of pressure are obtained based on the pressure distribution data, the heart rate variability is calculated based on the heart rate data, and the activity intensity is calculated based on the electromyography data; According to the set activity intensity weight, balance state weight, and heart rate variability weight, a multimodal feature vector is obtained.

5. The stroke fall warning method based on flexible sensor according to claim 1, characterized in that: The fuzzification layer includes a plurality of input nodes, a fuzzy rule layer and an output node; wherein each input node is connected to at least one membership function; the membership function includes a Gaussian function, a bell function and a trapezoidal function; the fuzzy rule layer calculates the trigger strength of each rule according to the fuzzy rule; The stacked hourglass prediction layer is used for joint posture prediction and supplementing joint missing data, including a preliminary convolution layer, multiple hourglass modules and a terminal convolution layer, wherein each hourglass module includes a downsampling path encoding layer, a low-resolution feature map network layer and an upsampling path decoding layer; The consequence layer calculates the fall risk probability according to the trigger intensity and the consequent parameters of the fuzzy rule.

6. The stroke fall warning method based on flexible sensor according to claim 5 is characterized in that: The pressure distribution data includes the pressure center offset, the plantar pressure distribution symmetry, the pressure peak position in the gait cycle and the dynamic pressure distribution range; The heart rate data features include heart rate variability index, heart rate acceleration / deceleration rate and heart rate abnormal fluctuation frequency; The electromyographic data includes muscle activation intensity, muscle co-activation degree and muscle fatigue index.

7. The stroke fall warning method based on flexible sensor according to claim 6 is characterized in that: The calculating of the fall risk probability according to the trigger intensity and the consequent parameter of the fuzzy rule further comprises: Define state set S = {stable gait, mildly unstable gait, moderately unstable gait, high risk of falling, fall occurred}; Define observation symbol V = [p1, p2, p3, p4, h1, h2, h3, e1, e2, e3, st, b]; where p1 is the pressure center offset; p2 is the symmetry of plantar pressure distribution; p3 is the peak pressure position in the gait cycle; p4 ​​is the dynamic pressure distribution range; h1 is the heart rate variability index; h2 is the heart rate acceleration / deceleration rate; h3 is the heart rate abnormal fluctuation frequency; e1 is the muscle activation intensity; e2 is the degree of muscle synergistic activation; e3 is the muscle fatigue index; st is the stability coefficient; b is the balance coefficient; Define the transition probability matrix A of each state set S, the initial state probability distribution π, and the observation probability matrix B of the observation symbol V; The forward algorithm is used to calculate the probability of each state from the first observation value to the current observation value, and the backward algorithm is used to calculate the probability of each state from the current observation value to the last observation value; Adjust the trigger strength according to the probability of each state, and incorporate the probability of each state into the consequent parameter of the fuzzy rule; The fall risk probability is calculated based on the adjusted trigger intensity and rule consequent parameters.

8. The stroke fall warning method based on flexible sensor according to claim 7 is characterized in that: The calculation formula for the fall risk probability is: Among them, P i is the consequent parameter of the ith rule; α i_adjusted is the trigger strength of the i-th rule; α i_adjusted =α i ·∑ s∈S P(q t =s|O)·w s ; xi is the i-th input variable, L xi is the membership value of xi; P(q t =s|O) is the probability of being in state s at time point t given the observation sequence O; w s is the weight of state s; S is the state set.

9. A stroke fall warning device based on flexible sensor, characterized in that: include: A data acquisition module, for collecting acceleration, angular velocity, pressure, heart rate and electromyography data of the patient in three-dimensional space in real time according to a plurality of flexible sensors, wherein the flexible sensors include a flexible three-axis accelerometer, a flexible three-axis gyroscope, a flexible pressure sensor, a flexible heart rate sensor and a flexible electromyography sensor, which can be wearably attached to a plurality of preset positions of the patient's body, wherein the preset positions include the waist, ankle, elbow joint and knee joint; The dynamic time warping feature extraction module is used to calculate the combined acceleration, combined angular velocity and combined pressure time series collected by the waist flexible sensor and each ankle flexible sensor based on the patient's acceleration, angular velocity, pressure, heart rate and electromyography data in three-dimensional space; calculate the stability coefficient of the waist acceleration sequence and the normal gait template sequence and the balance coefficient between the left and right ankle acceleration sequences based on the multi-scale DTW algorithm; A multimodal data fusion module is used to fuse the stability coefficient, balance coefficient, and pressure distribution data, heart rate data, and electromyography data collected by the flexible sensor to form a multimodal feature vector; The fall prediction module is used to input the multimodal feature vector into the ANFIS fall model to predict the patient's fall risk probability; wherein the ANFIS fall model includes an input layer, a fuzzification layer, a rule layer, a normalization layer, a consequence layer and an output layer.

10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned stroke fall warning method based on flexible sensors.

Images