Old people fall detection alarm device based on multi-sensor fusion

Abstract

This invention discloses a fall detection and alarm device for the elderly based on multi-sensor fusion, comprising: a rigid inner insole, a soft outer insole, and an integrated module. The rigid inner insole is placed inside the shoe cavity, and the soft outer insole is attached to the upper side of the rigid inner insole. The integrated module is located inside the rigid inner insole and includes a posture sensor, a pressure sensor, a microcontroller, a communication module, and a power module. The posture sensor collects posture data of the human foot, and the pressure sensor collects pressure data of the human foot. The posture and pressure data are respectively input to the microcontroller, which infers and determines a fall based on the posture and pressure data and sends the fall determination information through the communication module. Therefore, it has the advantages of multi-sensor fusion effectively reducing the false alarm rate, real-time proactive alarm improving rescue timeliness, and flexible use.

Description

Technical Field

[0001] This invention relates to the field of smart wearables, specifically to a fall detection and alarm device for the elderly based on multi-sensor fusion. Background Technology

[0002] Currently, fall detection technology based on smart wearable devices has made great strides. These devices continuously sense the human body's movement state by embedding various miniature sensors, and use signal processing and pattern recognition methods to identify and alert on fall events in real time, becoming an effective technical approach to solve the problem of caregiving difficulties for elderly people living alone.

[0003] In existing technology, a wearable solution integrating a fall detection module into the insole has emerged. A typical implementation involves fabricating an insole made entirely of flexible foam material, milling an installation cavity in the arch area, and placing a rigid printed circuit board containing a triaxial accelerometer, a microcontroller chip, a low-power Bluetooth module, and a button battery into this cavity, then encapsulating it with flexible sealant. During normal operation, the triaxial accelerometer outputs triaxial acceleration values ​​of the foot movement at a specified sampling rate. The microcontroller chip runs a preset threshold determination program, calculating the resultant acceleration vector amplitude in real time. Once the amplitude exceeds a pre-set alarm threshold and remains there for a period of time, a fall is detected, triggering the Bluetooth module to wirelessly send an alarm command to a paired smartphone. The smartphone then sends a distress message to the caregiver via an application.

[0004] However, the aforementioned existing technical solutions have a key technical problem: because they only use a single acceleration sensor as the sensing source and fix the rigid circuit unit directly in the flexible insole body, the device cannot obtain the core feature of the rapid drop in height of the human foot during a fall. It is difficult to accurately distinguish in principle between a real fall and everyday actions such as suddenly squatting or sitting down violently, which have high acceleration impact but no obvious change in height. Summary of the Invention

[0005] The present invention aims to at least partially solve the technical problems in the above-mentioned technologies.

[0006] Therefore, this invention discloses a fall detection and alarm device for the elderly based on multi-sensor fusion, comprising:

[0007] The shoe consists of a rigid inner core insole, a soft outer insole, and an integrated module.

[0008] The rigid inner core insole is used to lay inside the shoe cavity, and the soft outer insole is attached to the upper side of the rigid inner core insole;

[0009] The integrated module is disposed inside the rigid inner core insole, and the integrated module is provided with an attitude sensor, a pressure sensor, a microcontroller, a communication module and a power module;

[0010] The attitude sensor, the pressure sensor, and the communication module are each electrically connected to the microcontroller, and the power supply module is electrically connected to the attitude sensor, the pressure sensor, the microcontroller, and the communication module.

[0011] The posture sensor is used to collect posture data of the human foot, and the pressure sensor is used to collect real-time pressure data of the human foot. The posture data and the pressure data are respectively input to the microcontroller. The microcontroller infers and determines whether the human has fallen based on the posture data and the pressure data, and sends the human fall determination information through the communication module.

[0012] In addition, the fall detection alarm device for the elderly based on multi-sensor fusion disclosed in this invention may also have the following additional technical features:

[0013] Furthermore, the microcontroller is configured to perform the following operations:

[0014] The attitude data output by the attitude sensor is acquired. The attitude data includes triaxial acceleration data and triaxial angular velocity data, forming a six-channel time series.

[0015] The real-time tilt angle of the human foot is obtained by fusing triaxial acceleration and triaxial angular velocity data, which serves as a representation of the body posture tilt angle.

[0016] According to the preset time window length and sliding step size, multiple sampling points are continuously extracted from the six-channel time series to form the current window sequence;

[0017] Each channel sequence of the current window sequence is transformed into a Gram angular field image, and the images of all channels are stacked into a multi-channel input image;

[0018] The multi-channel input image is input into a pre-trained convolutional neural network, which outputs fall detection results, motion state monitoring results, and abnormal trend warning results.

[0019] The pressure time series output by the pressure sensor is acquired synchronously and aligned with the time window of the attitude data. Combined with the calculated real-time tilt angle, they serve as auxiliary verification data for fall detection.

[0020] Further, the step of transforming each channel sequence of the current window sequence into a Gram angle field image specifically includes:

[0021] For each length of One-dimensional sequence Perform a normalization operation to obtain a normalized sequence. ;

[0022] Normalized sequence Each value in Mapped to polar coordinates, where angle ,radius ;

[0023] Based on angle Calculate Gram angle and field matrix and Gram angular difference field matrix Each generates an image with a size of Images;

[0024] For each channel, a GASF image and a GADF image are generated. All channel images are stacked along the channel dimension to obtain a multi-channel input image with a total of 12 channels and an image size of [missing information]. .

[0025] Furthermore, the convolutional neural network includes:

[0026] The input layer is used to receive the multi-channel input image;

[0027] Multiple convolutional blocks are connected in sequence, each containing a two-dimensional convolutional layer, a batch normalization layer, and a ReLU non-linear activation function;

[0028] A downsampling layer is set after at least some convolutional blocks to reduce the spatial size of the feature map;

[0029] The global average pooling layer, located after the last convolutional block, is used to average the multi-channel feature maps along the spatial dimension to obtain a one-dimensional feature vector of fixed length.

[0030] The three parallel fully connected branches connected to the one-dimensional feature vector are used to output the fall detection result, the motion state monitoring result, and the abnormal trend warning result, respectively.

[0031] Furthermore, the plurality of convolutional blocks and downsampling layers are constructed in the following order:

[0032] The first convolutional block uses 64 7×7 convolutional kernels with a stride of 1, along with edge padding to maintain spatial size, and outputs a 64-channel feature map. Then, a 2×2 max pooling layer is set as downsampling to halve the spatial size.

[0033] The second convolutional block uses 32 5×5 convolutional kernels with a stride of 1, along with edge padding, to output a 32-channel feature map. Then, a 2×2 max pooling layer is set to halve the spatial size again.

[0034] The third convolutional block uses 16 3×3 convolutional kernels with a stride of 1, along with edge padding, to output a 16-channel feature map without further downsampling.

[0035] The fourth convolutional block uses eight 1×1 convolutional kernels with a stride of 1, and outputs an 8-channel feature map.

[0036] The global average pooling layer receives the feature maps of the 8 channels, calculates the average value of all pixels in each feature map, and obtains an 8-dimensional feature vector.

[0037] Furthermore, the three parallel fully connected branches are configured as follows:

[0038] The first branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of fall categories, including no fall category and various fall posture categories. The category with the highest probability is taken as the fall detection result, and the highest probability value is used as the confidence level.

[0039] The second branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of motion pattern categories, and the category with the highest probability is taken as the current motion state.

[0040] The third branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is 2, representing the probability of normal gait and abnormal gait, respectively. When the abnormal probability exceeds a set threshold, an abnormal trend warning is triggered.

[0041] Furthermore, the microcontroller is also configured to:

[0042] When the confidence level of the fall detection result exceeds the preset alarm threshold and the detection result is not "no fall", and the calculated foot tilt angle exceeds the preset angle threshold, and the pressure sensor detects a sudden drop in pressure below the preset pressure threshold, an alarm message is generated. The alarm message includes a timestamp and the fall type.

[0043] The alarm information is sent to the pre-bound guardian's terminal via the communication module.

[0044] Furthermore, the convolutional neural network employs a multi-task weighted cross-entropy loss function during the training phase, with a total loss of:

[0045] ,in, For cross-entropy loss, These are the real labels for fall type, movement pattern, and gait abnormality, respectively. For the corresponding predicted probability, The preset task weights.

[0046] Furthermore, the microcontroller is also used to receive the elderly person's weight parameters input by the supporting terminal APP, and dynamically calibrate the reference threshold for pressure and the fall pressure judgment threshold based on the weight parameters.

[0047] Furthermore, the pressure data-assisted fall detection workflow specifically includes:

[0048] The microcontroller acquires in real time the change in plantar pressure, the absolute value of the pressure, the foot tilt angle, and the acceleration amplitude within the current time window;

[0049] If a sudden drop in plantar pressure exceeding a preset pressure drop threshold is detected within a preset short time window, and the acceleration amplitude collected by the posture sensor simultaneously exceeds a preset acceleration threshold and the foot tilt angle exceeds a preset angle threshold, then it is determined to be a suspected fall event, and a fall is confirmed by combining the convolutional neural network inference results of the posture data.

[0050] If the plantar pressure data shows periodic fluctuations synchronized with the gait cycle, and the foot tilt angle is within the normal standing or walking range, it is determined to be a normal daily action such as walking, going up and down stairs, or standing, thus suppressing false alarms caused by abnormal single posture.

[0051] The fall detection and alarm device for the elderly based on multi-sensor fusion disclosed in this invention has at least the following beneficial effects:

[0052] (1) By integrating both posture sensor and plantar pressure sensor in the insole, the system can collect posture data such as foot acceleration and angular velocity, and also acquire plantar pressure change data in real time. The microcontroller integrates the two data sources and combines them with a pre-trained AI model to make fall inference judgments. It can accurately capture the core feature of the rapid drop in foot height during a real fall, thereby effectively distinguishing daily actions such as sudden squatting and sitting down with high acceleration impact but no significant change in height. This significantly improves the accuracy of fall detection and reduces false alarms.

[0053] (2) After the microcontroller determines that a person has fallen, it automatically sends an alarm signal containing the fall determination information to the guardian's terminal through the communication module, realizing the active remote alarm when an elderly person living alone falls, so that the elderly can receive help as soon as possible and effectively reduce the risk of secondary injury caused by delayed discovery.

[0054] (3) The integrated module is embedded inside the insole. The overall structure is compact and can be easily laid in various shoe cavities without requiring special modifications to the shoes. It is convenient for the elderly to change and clean daily, and has good versatility and concealment.

[0055] Additional features and advantages of this invention will be set forth in the description which follows, or may be learned by practicing the invention. Attached Figure Description

[0056] The technical solution and beneficial effects of the present invention will become apparent and readily understood from the following description in conjunction with the accompanying drawings, wherein:

[0057] Figure 1 This is a schematic diagram of the rigid inner core insole, the soft outer surface insole, and the integrated module of the present invention;

[0058] Figure 2 This is another schematic diagram of the rigid inner core insole, the soft outer surface insole, and the integrated module of the present invention;

[0059] Figure 3 This is another schematic diagram of the rigid inner core insole, the soft outer surface insole, and the integrated module of the present invention;

[0060] Figure 4 This is a flowchart of the process of the present invention;

[0061] Figure 5 An architectural diagram for the image construction of this invention;

[0062] Figure 6 This is a diagram illustrating the architecture of the convolutional neural network of the present invention.

[0063] Figure 7 This is an architectural diagram of the fully connected layer of the present invention;

[0064] As shown in the figure:

[0065] 101-Hard inner core insole, 102-Soft outer surface insole, 103-Integrated module. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0067] The fall detection and alarm device for the elderly based on multi-sensor fusion disclosed in this invention will now be described with reference to the accompanying drawings.

[0068] like Figure 1 , Figure 2 and Figure 3 As shown, a fall detection and alarm device for the elderly based on multi-sensor fusion includes:

[0069] The shoe includes a rigid inner core insole 101, a soft outer insole 102, and an integrated module 103, wherein...

[0070] A rigid inner core insole 101 is used to lay inside the shoe cavity, and a soft outer insole 102 is attached to the upper side of the rigid inner core insole 101.

[0071] The integrated module 103 is located inside the rigid inner core insole 101. The integrated module 103 includes an attitude sensor, a pressure sensor, a microcontroller, a communication module, and a power module.

[0072] The attitude sensor, pressure sensor, and communication module are electrically connected to the microcontroller, and the power supply module is electrically connected to the attitude sensor, pressure sensor, microcontroller, and communication module.

[0073] The posture sensor is used to collect posture data of the human foot, and the pressure sensor is used to collect pressure data of the human foot. The posture data and pressure data are respectively input to the microcontroller. The microcontroller infers and determines whether the human has fallen based on the posture data and pressure data, and sends the fall determination information through the communication module.

[0074] like Figure 4 , Figure 5 , Figure 6 and Figure 7 As shown, the microcontroller is configured to perform the following operations:

[0075] The attitude data output by the attitude sensor is acquired. The attitude data includes three-axis acceleration data and three-axis angular velocity data, forming a six-channel time series.

[0076] The real-time tilt angle of the human foot is obtained by fusing triaxial acceleration and triaxial angular velocity data, which serves as a representation of the body posture tilt angle.

[0077] According to the preset time window length and sliding step size, multiple sampling points are continuously extracted from the six-channel time series to form the current window sequence;

[0078] Each channel sequence of the current window sequence is transformed into a Gram angular field image, and the images of all channels are stacked into a multi-channel input image.

[0079] The multi-channel input image is fed into a pre-trained convolutional neural network, which outputs fall detection results, motion state monitoring results, and abnormal trend warning results.

[0080] The pressure time series output by the pressure sensor is acquired synchronously, aligned with the time window of the attitude data, and combined with the calculated real-time tilt angle to serve as auxiliary verification data for fall detection.

[0081] Transform each channel of the current window sequence into a Gram angle field image, specifically including:

[0082] For each length of One-dimensional sequence Perform a normalization operation to obtain a normalized sequence. ;

[0083] Normalized sequence Each value in Mapped to polar coordinates, where angle ,radius ;

[0084] Based on angle Calculate Gram angle and field matrix and Gram angular difference field matrix Each generates an image with a size of Images;

[0085] For each channel, a GASF image and a GADF image are generated. All channel images are stacked along the channel dimension to obtain a multi-channel input image with a total of 12 channels and an image size of [image size not specified]. .

[0086] Convolutional neural networks include:

[0087] The input layer is used to receive multi-channel input images;

[0088] Multiple convolutional blocks are connected in sequence, each containing a two-dimensional convolutional layer, a batch normalization layer, and a ReLU non-linear activation function;

[0089] A downsampling layer is set after at least some convolutional blocks to reduce the spatial size of the feature map;

[0090] The global average pooling layer, located after the last convolutional block, is used to average the multi-channel feature maps along the spatial dimension to obtain a one-dimensional feature vector of fixed length.

[0091] Three parallel fully connected branches connected to the one-dimensional feature vector are used to output fall detection results, motion state monitoring results, and abnormal trend warning results, respectively.

[0092] Multiple convolutional blocks and downsampling layers are constructed in the following order:

[0093] The first convolutional block uses 64 7×7 convolutional kernels with a stride of 1, along with edge padding to maintain spatial size, and outputs a 64-channel feature map. Then, a 2×2 max pooling layer is set as downsampling to halve the spatial size.

[0094] The second convolutional block uses 32 5×5 convolutional kernels with a stride of 1, along with edge padding, to output a 32-channel feature map. Then, a 2×2 max pooling layer is set to halve the spatial size again.

[0095] The third convolutional block uses 16 3×3 convolutional kernels with a stride of 1, along with edge padding, to output a 16-channel feature map without further downsampling.

[0096] The fourth convolutional block uses eight 1×1 convolutional kernels with a stride of 1, and outputs an 8-channel feature map.

[0097] The global average pooling layer receives 8-channel feature maps, calculates the average value of all pixels in each feature map, and obtains an 8-dimensional feature vector.

[0098] The three parallel fully connected branches are configured as follows:

[0099] The first branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of fall categories, including no fall category and various fall posture categories. The category with the highest probability is taken as the fall detection result, and the highest probability value is used as the confidence level.

[0100] The second branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of motion pattern categories, and the category with the highest probability is taken as the current motion state.

[0101] The third branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is 2, representing the probability of normal gait and abnormal gait, respectively. When the abnormal probability exceeds a set threshold, an abnormal trend warning is triggered.

[0102] The microcontroller is also configured as follows:

[0103] When the confidence level of the fall detection result exceeds the preset alarm threshold and the detection result is not "no fall", and the calculated foot tilt angle exceeds the preset angle threshold, and the pressure sensor detects a sudden drop in pressure below the preset pressure threshold, an alarm message is generated. The alarm message includes a timestamp and the fall type.

[0104] The alarm information is sent to the pre-bound guardian's terminal via the communication module.

[0105] The convolutional neural network uses a multi-task weighted cross-entropy loss function during the training phase, with a total loss of:

[0106] ,in, For cross-entropy loss, These are the real labels for fall type, movement pattern, and gait abnormality, respectively. For the corresponding predicted probability, The preset task weights.

[0107] The microcontroller is also used to acquire pressure data collected by the pressure sensor, and to assist in the fall detection process based on this pressure data, specifically including:

[0108] The workflow for using data and posture angle data to assist in fall detection specifically includes:

[0109] The microcontroller acquires in real time the change in plantar pressure, the absolute value of the pressure, the foot tilt angle, and the acceleration amplitude within the current time window;

[0110] If a sudden drop in plantar pressure exceeding a preset pressure drop threshold is detected within a preset short time window, and the acceleration amplitude collected by the posture sensor simultaneously exceeds a preset acceleration threshold and the foot tilt angle exceeds a preset angle threshold, then it is determined to be a suspected fall event. The fall is then confirmed by combining the convolutional neural network inference results of the posture data.

[0111] If the plantar pressure data shows periodic fluctuations synchronized with the gait cycle, and the foot tilt angle is within the normal standing and walking range, it is judged as normal daily activities such as walking, going up and down stairs, or standing, thus suppressing false alarms caused by abnormal single posture.

[0112] In one embodiment of the present invention:

[0113] The microcontroller, as the core processing unit of the entire device, is internally programmed with a pre-trained convolutional neural network model and corresponding pre-processing and post-processing programs. During device operation, the microcontroller executes the following inference and alarm processes.

[0114] The attitude sensor continuously acquires triaxial acceleration and triaxial angular velocity signals from the feet at a sampling frequency of 16Hz, forming a six-channel real-time data stream. The microcontroller maintains a circular buffer in memory, always retaining the most recent 128 sampling points (corresponding to an 8-second time interval). The sliding step size is set to 1 sampling point; that is, each time a new sampling point is acquired, the microcontroller retrieves the latest 128 sampling points from the buffer to form a time window. Each window contains 6 one-dimensional sequences of length 128, denoted as... All subsequent transformations and inferences are based on this window of data.

[0115] For each one-dimensional sequence of length 128 within the window The microcontroller performs the following operations in sequence:

[0116] Calculate the minimum value of the sequence. and maximum value Map each value to The interval is used to obtain the normalized sequence. ;

[0117] The normalized values ​​are converted to polar coordinates, where the angles are... Encoded signal amplitude, radius Encoding time sequence: ;

[0118] Generate the Gram angle sum field (GASF) matrix and the Gram angle difference field (GADF) matrix: Both matrices are 128×128 in size, and the row and column directions of the matrices correspond to the time dimension.

[0119] Repeat the above steps to generate GASF and GADF images for each sequence in the six channels, resulting in a total of 12 single-channel images of 128×128. Stack these 12 images along the channel dimension to obtain a multi-channel input image of shape 12×128×128, which will be used as the input to the subsequent convolutional neural network.

[0120] The convolutional neural network deployed on the microcontroller adopts a pure two-dimensional convolutional structure, and its detailed construction is shown in the table below:

[0121]

[0122] Table 1 Convolutional Neural Network Structure

[0123] After the above convolution and pooling processes, the fourth convolutional block outputs an 8×32×32 feature cube. The global average pooling layer averages the values ​​of all pixels in each channel feature map of this cube to obtain an 8-dimensional feature vector. This vector highly condenses the core features of the multi-sensor attitude information within the entire window.

[0124] In the eigenvector The network then splits into three parallel fully connected branches, each responsible for a different classification task:

[0125] First branch (fall detection):

[0126] Fully connected layer (output 64 dimensions), ReLU, Dropout (dropout rate 0.3), fully connected layer (output number of categories, such as 5), Softmax.

[0127] Output vector This indicates the probability of each category, which includes "no fall", "fall forward", "fall backward", "fall to the left" and "fall to the right".

[0128] During inference, the category with the highest probability is taken as the detection result, and this highest probability value is used as the fall confidence level.

[0129] Second branch (motion status monitoring):

[0130] Fully connected layer (output 64 dimensions), ReLU, Dropout (dropout rate 0.3), fully connected layer (output number of categories, such as 6), Softmax.

[0131] The categories include "Still", "Walking", "Running", "Going Upstairs", "Going Downstairs", and "Lying Down", and output the motion mode of the current window.

[0132] Third branch (abnormal trend warning):

[0133] Fully connected layer (64-dimensional output), ReLU, Dropout (dropout rate 0.3), fully connected layer (2-dimensional output), Softmax.

[0134] Output the probability of normal gait and abnormal gait. When the abnormal probability exceeds a threshold (e.g., 0.8), trigger an alert.

[0135] The three branches share the same backbone network, and all three types of results are obtained simultaneously in a single forward computation.

[0136] The training of this convolutional neural network is completed offline on a host computer. After training, the model parameters are fixed in a format that can be read by the microcontroller.

[0137] The training phase uses a multi-task weighted cross-entropy loss function:

[0138] ,in, For cross-entropy loss, These are the real labels for fall type, movement pattern, and gait abnormality, respectively. For the corresponding predicted probability, The preset task weights.

[0139] The optimizer used was Adam with an initial learning rate of 0.001. The training data included collected samples from various daily activities and fall scenarios.

[0140] To improve the accuracy of fall detection and reduce false alarms, this embodiment uses triaxial acceleration and triaxial angular velocity data collected by the attitude sensor to fuse and solve the data through a first-order complementary filtering algorithm, and outputs the pitch and roll angles of the foot in real time. The larger of the two absolute values ​​is taken as the current body tilt angle. At the same time, the plantar pressure sensor collects real-time plantar pressure data at the same 16Hz frequency as the attitude sensor.

[0141] The microcontroller receives the elderly person's weight parameters from the accompanying terminal APP in advance, and automatically calibrates the plantar pressure reference threshold and fall pressure judgment threshold based on the parameters. The reference threshold is the average plantar pressure of the elderly person when standing still, and the fall pressure judgment threshold is set to 20% of the reference threshold. In this embodiment, the preset body tilt angle alarm threshold is 15° to 25°, and the acceleration summation alarm threshold is 2g.

[0142] If, within an 8-second window, the microcontroller detects a sudden drop in plantar pressure exceeding a preset pressure drop threshold, and the combined acceleration amplitude collected by the attitude sensor simultaneously exceeds a preset acceleration threshold, and the calculated body tilt angle continuously exceeds a preset angle threshold, the microcontroller will mark the window as a suspected fall event and make a comprehensive judgment based on the fall confidence output by the convolutional neural network.

[0143] A fall is only confirmed when all three physical rules—pressure change condition, acceleration amplitude condition, and tilt angle condition—are met simultaneously, and the confidence level of the attitude inference result meets the standard.

[0144] In addition, if the plantar pressure data shows periodic fluctuations synchronized with the gait cycle (such as the pressure changes when the foot alternates between landing and leaving the ground while walking, and the phased fluctuations of plantar pressure when going up and down stairs), and the body tilt angle is within the normal standing and walking angle range, it is judged as normal daily activities such as walking, going up and down stairs, or standing. At this time, even if there is a momentary abnormality in the posture, the microcontroller will suppress false alarms.

[0145] Pressure change features and tilt angle change features are also used as auxiliary contextual information to help distinguish between walking on flat ground, going upstairs and going downstairs in motion pattern recognition.

[0146] The power module of this device uses a rechargeable lithium battery and is equipped with a magnetic contact charging solution, avoiding waterproof and dustproof issues caused by exposed interfaces. Specifically, it is implemented as follows:

[0147] A recessed magnetic charging contact group is set on the non-stepping side of the hard inner core insole (preferably the outer area corresponding to the arch of the foot). The insole end is concave, and the contacts are made of gold-plated copper pillar material to improve conductivity and wear resistance.

[0148] The matching charger is equipped with a convex charging contact group, with the charger end being the convex end, and the number, spacing, and size of the contacts are precisely matched.

[0149] Two to four miniature neodymium iron boron permanent magnets are embedded around the charging contact group of the insole, and permanent magnets with opposite polarities are embedded in the corresponding positions of the charger. The charging contacts are automatically aligned and tightly fitted by magnetic attraction to prevent poor contact during charging.

[0150] The power module within the integrated module is equipped with a dedicated magnetic charging management circuit, which features overcharge protection, overcurrent protection, short circuit protection, and overtemperature protection. When the battery is fully charged, the charging circuit is automatically cut off, extending the battery's lifespan.

[0151] To facilitate manual control of the device's operation and avoid wasting power when not in use, this device integrates a small thin-film pressure sensor on the side of the rigid inner insole (on the same side as the magnetic charging contacts, away from the stepping and charging areas). This sensor is used only for device start-stop control and is hardware-independent and data channels do not interfere with each other from the foot pressure sensor used for fall detection.

[0152] The microcontroller monitors the output value of the side pressure sensor in real time. When the detected pressure value exceeds the preset start / stop threshold and lasts for 1.5 seconds, it triggers the device to switch its operating state.

[0153] If the current state is normal detection, then wake up all functional modules and restore the normal fall detection state;

[0154] If the current mode is low-power standby, the attitude sensor, foot pressure sensor and computing module will be turned off, and the system will enter low-power standby mode.

[0155] Through a dual determination mechanism of pressure threshold and duration, it effectively avoids accidental start-stop caused by slight compression during daily wear and walking.

[0156] Furthermore, if the height change exhibits periodic fluctuations synchronized with the gait, and the body tilt angle shows regular small fluctuations with the gait cycle (such as the angle change corresponding to the alternating lifting and landing of the feet during walking), it is determined to be normal walking or climbing stairs. In this case, even if there is a momentary abnormality in posture, the microcontroller will suppress false alarms. The height change features and tilt angle change features are also used as auxiliary contextual information to help distinguish between walking on flat ground, climbing stairs, and climbing stairs in motion pattern recognition.

[0157] It should be noted that when the microcontroller confirms a fall, it immediately generates an alarm message, including the current timestamp, the geographical location information obtained through the communication module, and the type of fall. The alarm message is sent via the communication module to the pre-bound guardian's mobile terminal application through the mobile network. Simultaneously, the device's own buzzer or LED light emits a local audible and visual alarm to attract the attention of those nearby and facilitate timely assistance.

[0158] It should also be noted that if the abnormal trend warning branch outputs a gait abnormality probability exceeding the threshold for multiple consecutive windows (e.g., within 3 consecutive seconds), the microcontroller will also send a warning notification to the caregiver, indicating that the elderly person's gait stability has recently declined and there may be potential health risks, and further examination is recommended.

[0159] In summary, the fall detection and alarm device for the elderly based on multi-sensor fusion disclosed in this invention has at least the following beneficial effects:

[0160] (1) By integrating both posture sensor and plantar pressure sensor in the insole, the system can collect posture data such as foot acceleration and angular velocity, and also acquire plantar pressure change data in real time. The microcontroller integrates the two data sources and combines them with a pre-trained AI model to make fall inference judgments. It can accurately capture the core feature of the rapid drop in foot height during a real fall, thereby effectively distinguishing daily actions such as sudden squatting and sitting down with high acceleration impact but no significant change in height. This significantly improves the accuracy of fall detection and reduces false alarms.

[0161] (2) After the microcontroller determines that a person has fallen, it automatically sends an alarm signal containing the fall determination information to the guardian's terminal through the communication module, realizing the active remote alarm when an elderly person living alone falls, so that the elderly can receive help as soon as possible and effectively reduce the risk of secondary injury caused by delayed discovery.

[0162] (3) The integrated module is embedded inside the insole. The overall structure is compact and can be easily laid in various shoe cavities without requiring special modifications to the shoes. It is convenient for the elderly to change and clean daily, and has good versatility and concealment.

[0163] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A fall detection and alarm device for the elderly based on multi-sensor fusion, characterized in that, include: The shoe consists of a rigid inner core insole, a soft outer insole, and an integrated module. The rigid inner core insole is used to lay inside the shoe cavity, and the soft outer insole is attached to the upper side of the rigid inner core insole; The integrated module is disposed inside the rigid inner core insole, and the integrated module is provided with an attitude sensor, a pressure sensor, a microcontroller, a communication module and a power module; The attitude sensor, the pressure sensor, and the communication module are each electrically connected to the microcontroller, and the power supply module is electrically connected to the attitude sensor, the pressure sensor, the microcontroller, and the communication module. The posture sensor is used to collect posture data of the human foot, and the pressure sensor is used to collect real-time pressure data of the human foot. The posture data and the pressure data are respectively input to the microcontroller. The microcontroller infers and determines whether the human has fallen based on the posture data and the pressure data, and sends the human fall determination information through the communication module.

2. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 1, characterized in that, The microcontroller is configured to perform the following operations: The attitude data output by the attitude sensor is acquired. The attitude data includes triaxial acceleration data and triaxial angular velocity data, forming a six-channel time series. The real-time tilt angle of the human foot is obtained by fusing triaxial acceleration and triaxial angular velocity data, which serves as a representation of the body posture tilt angle. According to the preset time window length and sliding step size, multiple sampling points are continuously extracted from the six-channel time series to form the current window sequence; Each channel sequence of the current window sequence is transformed into a Gram angular field image, and the images of all channels are stacked into a multi-channel input image; The multi-channel input image is input into a pre-trained convolutional neural network, which outputs fall detection results, motion state monitoring results, and abnormal trend warning results. The pressure time series output by the pressure sensor is acquired synchronously and aligned with the time window of the attitude data. Combined with the calculated real-time tilt angle, they serve as auxiliary verification data for fall detection.

3. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 2, characterized in that, The step of transforming each channel sequence of the current window sequence into a Gram angle field image specifically includes: For each length of One-dimensional sequence Perform a normalization operation to obtain a normalized sequence. ; Normalized sequence Each value in Mapped to polar coordinates, where angle ,radius ; Based on angle Calculate Gram angle and field matrix and Gram angular difference field matrix Each generates an image with a size of Images; For each channel, a GASF image and a GADF image are generated. All channel images are stacked along the channel dimension to obtain a multi-channel input image with a total of 12 channels and an image size of [missing information]. .

4. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 2, characterized in that, The convolutional neural network includes: The input layer is used to receive the multi-channel input image; Multiple convolutional blocks are connected in sequence, each convolutional block containing a two-dimensional convolutional layer, a batch normalization layer, and a ReLU non-linear activation function; A downsampling layer is set after at least some convolutional blocks to reduce the spatial size of the feature map; The global average pooling layer, located after the last convolutional block, is used to average the multi-channel feature maps along the spatial dimension to obtain a one-dimensional feature vector of fixed length. The three parallel fully connected branches connected to the one-dimensional feature vector are used to output the fall detection result, the motion state monitoring result, and the abnormal trend warning result, respectively.

5. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 4, characterized in that, The multiple convolutional blocks and downsampling layers are constructed in the following order: The first convolutional block uses 64 7×7 convolutional kernels with a stride of 1, along with edge padding to maintain spatial size, and outputs a 64-channel feature map. Then, a 2×2 max pooling layer is set as downsampling to halve the spatial size. The second convolutional block uses 32 5×5 convolutional kernels with a stride of 1, along with edge padding, to output a 32-channel feature map. Then, a 2×2 max pooling layer is set to halve the spatial size again. The third convolutional block uses 16 3×3 convolutional kernels with a stride of 1, along with edge padding, to output a 16-channel feature map without further downsampling. The fourth convolutional block uses eight 1×1 convolutional kernels with a stride of 1, and outputs an 8-channel feature map. The global average pooling layer receives the feature maps of the 8 channels, calculates the average value of all pixels in each feature map, and obtains an 8-dimensional feature vector.

6. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 4, characterized in that, The three parallel fully connected branches are configured as follows: The first branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of fall categories, including no fall category and various fall posture categories. The category with the highest probability is taken as the fall detection result, and the highest probability value is used as the confidence level. The second branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is the preset number of motion pattern categories, and the category with the highest probability is taken as the current motion state. The third branch consists of a fully connected layer, ReLU activation, Dropout layer, and Softmax output layer. The output dimension is 2, representing the probability of normal gait and abnormal gait, respectively. When the abnormal probability exceeds a set threshold, an abnormal trend warning is triggered.

7. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 2, characterized in that, The microcontroller is also configured to: When the confidence level of the fall detection result exceeds the preset alarm threshold and the detection result is not "no fall", and the calculated foot tilt angle exceeds the preset angle threshold, and the pressure sensor detects a sudden drop in pressure below the preset pressure threshold, an alarm message is generated. The alarm message includes a timestamp and the fall type. The alarm information is sent to the pre-bound guardian's terminal via the communication module.

8. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 2, characterized in that, The convolutional neural network employs a multi-task weighted cross-entropy loss function during the training phase, with a total loss of: ,in, For cross-entropy loss, These are the real labels for fall type, movement pattern, and gait abnormality, respectively. For the corresponding predicted probability, The preset task weights.

9. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 1, characterized in that, The microcontroller is also used to receive the elderly person's weight parameters input by the supporting terminal APP, and dynamically calibrate the reference threshold for pressure and the fall pressure judgment threshold based on the weight parameters.

10. The fall detection and alarm device for the elderly based on multi-sensor fusion according to claim 9, characterized in that, The pressure data-assisted fall assessment workflow specifically includes: The microcontroller acquires in real time the change in plantar pressure, the absolute value of the pressure, the foot tilt angle, and the acceleration amplitude within the current time window; If a sudden drop in plantar pressure exceeding a preset pressure drop threshold is detected within a preset short time window, and the acceleration amplitude collected by the posture sensor simultaneously exceeds a preset acceleration threshold and the foot tilt angle exceeds a preset angle threshold, then it is determined to be a suspected fall event, and a fall is confirmed by combining the convolutional neural network inference results of the posture data.

Images