Multi-modal data fusion abnormal behavior early warning system before falling of high-risk patient

Abstract

The invention relates to the technical field of medical health data analysis, in particular to a multi-modal data fusion abnormal behavior early warning system before falling of a high-risk patient. According to the method, for any patient, a plurality of gait cycles formed by time frames are obtained according to ankle coordinate distribution of different time frames; according to the ankle coordinate distribution of different time frames in each gait cycle, gait high risk of each gait cycle is obtained; according to the hip coordinate distribution of different time frames in each gait cycle, obtaining the center-of-gravity shift degree of each gait cycle, and obtaining the high-risk center-of-gravity shift reference degree of all patients in each gait cycle; for any patient, the falling probability in each sliding window is obtained according to the gravity center shift degrees of different gait cycles, the high-risk gravity center shift reference degree and various physiological parameter data at different moments; and early warning is carried out on abnormal falling behaviors. According to the invention, the accuracy of early warning of abnormal falling behaviors is improved by accurately analyzing the falling probability of the high-risk patient.

Description

Technical Field

[0001] This invention relates to the field of medical and health data analysis technology, specifically to a multimodal data fusion-based early warning system for abnormal behavior before falls in high-risk patients. Background Technology

[0002] Falls can cause serious physical injuries such as fractures and traumatic brain injury, as well as psychological fear, reduced willingness to move and quality of life. If not rescued in time after a fall, the injury may worsen or even lead to death. Therefore, it is necessary to provide timely warnings of abnormal behavior before a fall.

[0003] Existing methods, through instantaneous state detection or focusing on real-time detection of fall events, fail to capture the gradual transition from "steady gait" to "unsteady gait" in high-risk patients and the changing trends of patients' physiological indicators. This results in insufficient accuracy in predicting fall behavior in high-risk patients based on abnormal behaviors before a fall. Summary of the Invention

[0004] To address the technical problem of insufficient accuracy in fall warnings for high-risk patients due to neglecting changes in gait and physiological indicators, which leads to an ignoring of abnormal pre-fall behaviors, this invention aims to provide a multimodal data fusion-based early warning system for abnormal pre-fall behaviors in high-risk patients. The specific technical solution adopted is as follows: This invention proposes a multimodal data fusion-based early warning system for abnormal pre-fall behaviors in high-risk patients, the system comprising: The data acquisition module is used to acquire the ankle and hip coordinates on the left and right sides of each frame in the behavioral video of high-risk patients, as well as various physiological parameter data at different times. The behavioral performance quantification module is used to obtain multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames for any patient; to obtain the gait risk of each gait cycle based on the ankle coordinate distribution at different time frames within each gait cycle; to obtain the degree of center of gravity shift in each gait cycle based on the hip coordinate distribution at different time frames within each gait cycle; and to obtain the high-risk center of gravity shift baseline for all patients in each gait cycle. The fall probability assessment module is used to construct a sliding window that traverses the gait cycle for any patient. Based on the degree of center of gravity shift in different gait cycles, the baseline degree of high-risk center of gravity shift, and various physiological parameter data at different times, it obtains the comprehensive stability within each sliding window; and based on the distribution of comprehensive stability within different sliding windows, it obtains the fall probability within each sliding window. The fall warning module is used to issue warnings for abnormal fall behavior based on the probability of a patient falling within the latest sliding window.

[0005] Furthermore, the method for obtaining the gait period includes: For any ankle, the vertical values ​​of the ankle coordinates in all time frames are used to form an ankle curve. The minimum point in the ankle curve is obtained, and the range formed by the corresponding time frames between adjacent minimum points is taken as a gait cycle.

[0006] Furthermore, the method for obtaining gait risk includes: For any ankle, obtain the time frame with the minimum vertical value of the ankle coordinate in each gait cycle; Obtain the relative horizontal distance between the ankle coordinates of one ankle at the minimum time frame and the ankle coordinates of the other ankle; The gait risk level for each gait cycle is obtained based on the relative distance between the left and right ankles and the number of time frames in the gait cycle. The relative distance between the left and right ankles is negatively correlated with the gait risk level, while the number of time frames is positively correlated with the gait risk level.

[0007] Furthermore, the method for obtaining the degree of center of gravity offset includes: For any patient, the mean values ​​of the hip coordinates on the left and right sides at each time frame are obtained and used as the center coordinates. The mean value of the center coordinates between the left and right hip coordinates on all time frames is obtained as the overall center position; Based on the coordinate difference between the center coordinates of different frames in each gait cycle and the overall center position, as well as the gait risk level, the degree of center of gravity shift in each gait cycle is obtained. Both the coordinate difference and the gait risk level are positively correlated with the degree of center of gravity shift.

[0008] Furthermore, the method for obtaining the high-risk center of gravity offset reference degree includes: The mean value of the center of gravity shift of all patients in each gait cycle was obtained as the high-risk center of gravity shift benchmark for each gait cycle.

[0009] Furthermore, the method for obtaining the overall stability includes: Based on the degree of centroid shift of different time-state cycles and various physiological parameter data at different times, a multidimensional feature matrix of different time-state cycles is obtained, and the degree of behavioral stability of each dimension within each sliding window is obtained. The ratio of the high-risk center of gravity shift baseline for each gait cycle to the center of gravity shift degree for each patient is obtained as the first stability coefficient for each patient in each gait cycle. Based on the first stability coefficient of different time periods within each sliding window and the behavioral stability of different dimensions, the overall stability within each sliding window is obtained. Both the first stability coefficient and the behavioral stability are positively correlated with the overall stability.

[0010] Furthermore, the method for obtaining the multidimensional feature matrix includes: The mean value of each physiological parameter at different times in each gait cycle is obtained as the overall value of each physiological parameter in each gait cycle; The degree of center of gravity shift or the overall value of various physiological parameters in all gait cycles are used to construct a one-dimensional column vector of a multidimensional feature matrix.

[0011] Furthermore, the method for obtaining the degree of behavioral stability includes: Find the minimum value of each element in each dimension across all gait cycles within each sliding window; Obtain the cumulative difference between the element values ​​and the minimum element values ​​of each dimension in different gait cycles within the corresponding sliding window, and perform negative correlation mapping as the behavioral stability of each dimension within each sliding window.

[0012] Furthermore, the method for obtaining the fall probability includes: The average difference in overall stability among all adjacent sliding windows is used as the first fall coefficient. A negative correlation mapping is performed on the overall stability within each sliding window. The product of the negative correlation mapping result and the first fall coefficient is calculated and normalized to serve as the fall probability within each sliding window.

[0013] Furthermore, the aforementioned early warning system for abnormal fall behavior includes: If the probability of falling in the latest sliding window is less than the preset first fall threshold, the abnormal fall behavior is judged to be at the first fall level. If the probability of falling in the latest sliding window is greater than the preset first fall threshold and less than the preset second fall threshold, the abnormal fall behavior is judged to be at the second fall level. If the probability of falling in the latest sliding window is greater than the preset second fall threshold, the abnormal fall behavior is judged to be at the third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

[0014] The present invention has the following beneficial effects: This invention, for any patient, obtains multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames, segmenting continuous, non-stationary time series data into comparable and repeatable cycles. Based on the ankle coordinate distribution at different time frames within each gait cycle, the gait risk of each gait cycle is obtained, reflecting the degree of gait risk. Based on the hip coordinate distribution at different time frames within each gait cycle, the degree of center of gravity shift in each gait cycle is obtained, and a high-risk center of gravity shift baseline for all patients in each gait cycle is obtained to assess the stability of the center of gravity during walking. For any patient, a sliding window is constructed to traverse the gait cycles. Based on the degree of center of gravity shift at different gait cycles, the high-risk center of gravity shift baseline, and various physiological parameters at different times, the comprehensive stability within each sliding window is obtained. The sliding window can capture short-term stability changes and track the dynamic trend of patient stability changes in real time. Based on the distribution of comprehensive stability within different sliding windows, the probability of falling within each sliding window is obtained, quantifying the likelihood of a fall. Early warning is provided for abnormal fall behaviors. This invention improves the accuracy of early warning for abnormal fall behavior by accurately analyzing the fall probability of high-risk patients. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a structural block diagram of a high-risk patient's abnormal behavior warning system before a fall, provided in one embodiment of the present invention; Figure 2 A flowchart illustrating a method for obtaining gait risk according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating a method for obtaining overall stability according to an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a multimodal data fusion-based early warning system for abnormal pre-fall behavior in high-risk patients, as proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details the specific scheme of the high-risk patient fall pre-fall abnormal behavior early warning system provided by the present invention, which involves multimodal data fusion.

[0020] Please see Figure 1 The diagram illustrates a multimodal data fusion-based early warning system for abnormal pre-fall behavior in high-risk patients, according to an embodiment of the present invention. The system includes: a data acquisition module 101, a behavior quantification module 102, a fall probability assessment module 103, and a fall warning module 104. The data acquisition module 101 is used to acquire the ankle and hip coordinates on the left and right sides of each frame image in the behavioral video of high-risk patients, as well as various physiological parameter data at different times.

[0021] In embodiments of the present invention, high-risk patients typically refer to elderly individuals, patients suffering from neurological diseases, musculoskeletal diseases, cardiovascular diseases, or those who are weak after surgery. Considering that the analysis of instantaneous states ignores changes in the physiological parameters of patient behavior, the accuracy of fall warnings is insufficient. Therefore, it is necessary to analyze the motion behavior and physiological change trends of different frames in the behavioral video. First, behavioral videos of high-risk patients are collected and analyzed using surveillance cameras placed in key areas such as corridors. It should be noted that, in order to facilitate subsequent processing of video images, noise reduction, distortion correction, and background cropping are performed on the images of each frame in the collected video. The specific methods are well-known to those skilled in the art and will not be described in detail here.

[0022] The OpenPose model was used to collect patient posture data from behavioral videos, identifying the ankle and hip coordinates on the left and right sides of each frame at each time point. Wearable devices were used to acquire multiple physiological parameters of high-risk patients in real time, including at least heart rate, blood pressure, and acceleration.

[0023] It should be noted that the specific OpenPose model is a technical method well known to those skilled in the art, and will not be elaborated here.

[0024] The behavioral performance quantification module 102 is used to obtain multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames for any patient; to obtain the gait risk of each gait cycle based on the ankle coordinate distribution at different time frames in each gait cycle; to obtain the degree of center of gravity shift in each gait cycle based on the hip coordinate distribution at different time frames in each gait cycle; and to obtain the high-risk center of gravity shift baseline for all patients in each gait cycle.

[0025] Considering that different patients or the same patient may have different walking speeds at different times, resulting in different data durations, it is necessary to segment the gait of the ankle. For any patient, the analysis is performed based on the distribution of ankle coordinates at different time frames to obtain multiple gait cycles composed of time frames.

[0026] Preferably, in one embodiment of the present invention, the method for obtaining the gait period includes: For any ankle, the vertical values ​​of the ankle coordinates in all time frames are used to form an ankle curve. The minimum point in the ankle curve is obtained, and the range formed by the corresponding time frames between adjacent minimum points is taken as a gait cycle.

[0027] It should be noted that, in the embodiments of the present invention, the vertical value of the ankle coordinate reflects the walking rhythm, cadence, and the start and end of each step. If the vertical value of a time frame in the ankle curve is less than that of the adjacent time frames, the position of the corresponding time frame is a minimum point. At this time, the ankle is at the lowest point, reflecting the time point when the foot contacts the ground. Finding the next time point when the foot contacts the ground constitutes a complete walking action. That is, all time frames corresponding to adjacent minimum points constitute a gait cycle. For example, if there are adjacent minimum points corresponding to the 2nd and 5th time frames, then the range formed by the 2nd, 3rd, 4th, and 5th time frames corresponding to the adjacent minimum points is taken as a gait cycle.

[0028] Because high-risk patients are physically weak and have decreased muscle strength, they lack the power to propel their bodies forward, resulting in slow walking speed. Furthermore, their muscles do not have enough strength to complete large-amplitude hip flexion and knee extension movements, leading to short strides. In other words, the closer the ankle coordinates are in the horizontal time frame, the greater the gait risk. Therefore, the gait risk for each gait cycle is obtained based on the distribution of ankle coordinates in different time frames within each gait cycle.

[0029] Preferably, in one embodiment of the present invention, the method for obtaining gait risk is described in [reference needed]. Figure 2 It illustrates a flowchart of a method for obtaining gait risk, including: Step S201: For any ankle, obtain the time frame with the minimum vertical value of the ankle coordinate in each gait cycle.

[0030] The minimum value in the vertical direction reflects the moment the heel touches the ground, which is helpful for subsequent analysis of walking stride length.

[0031] Step S202: Obtain the relative distance in the horizontal direction between the ankle coordinates of one ankle at the minimum time frame and the ankle coordinates of the other ankle.

[0032] Relative distance is reflected in the horizontal distance between the two feet when one ankle touches the ground; the greater the distance, the longer the stride.

[0033] It should be noted that in the embodiments of the present invention, the relative distance is obtained by existing distance calculation methods such as Euclidean distance or Manhattan distance; the specific means are well known to those skilled in the art and will not be described in detail here.

[0034] Step S203: Based on the relative distance between the left and right ankles and the number of time frames in the gait cycle, obtain the gait risk of each gait cycle. The relative distance between the left and right ankles is negatively correlated with the gait risk, and the number of time frames is positively correlated with the gait risk.

[0035] It should be noted that the greater the relative distance, the greater the difference in horizontal direction between the two ankles, the longer the stride, the greater the power, and the lower the gait risk. The smaller the number of time frames, the greater the stride length in a short period of time, the greater the forward momentum of the body, the greater the stride speed, and the lower the gait risk. Therefore, relative distance is negatively correlated with gait risk, while the number of time frames is positively correlated with gait risk.

[0036] In one embodiment of the present invention, the mean of the corresponding relative distance between the left ankle and the right ankle is obtained as the average stride length of each gait cycle; the sum of the corresponding relative distance between the left ankle and the right ankle is obtained, and the sum is divided by the number of time frames in the gait cycle as the average gait speed of each gait cycle. The product of average stride length and average stride speed is obtained and negatively correlated to represent the gait risk for each gait cycle.

[0037] Based on this, the greater the relative distance between the left and right ankles in the horizontal direction, the longer the stride, the faster the stride, and the less likely the gait is to be considered high-risk.

[0038] It should be noted that, in the embodiments of the present invention, an exponential function with a base of the natural constant can be used. Alternatively, the reciprocal can be used for negative correlation mapping. When calculating the reciprocal, in order to avoid the formula being meaningless with a denominator of 0, an artificially set threshold, such as 0.01, is added to the denominator. The specific methods are well known to those skilled in the art and will not be elaborated here.

[0039] Due to weak core muscle strength and poor neural control, the body tends to lean forward or backward and become unstable during walking. Considering that in the mid-stability phase of the gait, the pelvis will slightly shift towards the supporting leg to maintain stability on one leg, this will cause the hip joint coordinates to shift to that side, thus directly reflecting the shift of the center of gravity towards the supporting side. Therefore, based on the hip coordinate distribution at different time frames within each gait cycle, the degree of center of gravity shift in each gait cycle is obtained, and the baseline degree of high-risk center of gravity shift for all patients in each gait cycle is obtained.

[0040] Preferably, in one embodiment of the present invention, the method for obtaining the degree of center of gravity offset includes: For any patient, the mean values ​​of the hip coordinates on the left and right sides at each time frame are obtained and used as the center coordinates. The mean value of the center coordinates between the left and right hip coordinates on all time frames is obtained as the overall center position; Based on the coordinate difference between the center coordinates of different frames in each gait cycle and the overall center position, as well as the gait risk level, the degree of center of gravity shift in each gait cycle is obtained. Both the coordinate difference and the gait risk level are positively correlated with the degree of center of gravity shift.

[0041] It should be noted that the difference represents the absolute value of the calculated difference; the coordinate difference reflects the magnitude of the deviation of the center position of each frame from the overall center position. The larger the coordinate difference, the greater the deviation, and the less it is close to the overall center position, resulting in a greater degree of center shift; the greater the gait risk, the slower the walking speed, the smaller the stride length, and the more likely the center of gravity shift will occur. Therefore, both the coordinate difference and the gait risk are positively correlated with the degree of center of gravity shift.

[0042] In one embodiment of the present invention, the cumulative coordinate difference between the center of gravity coordinates and the overall center of gravity position for all time frames in each gait cycle is obtained. The product of the cumulative coordinate difference and the gait risk level is then used as the degree of center of gravity shift in each gait cycle. Therefore, based on the above basic mathematical operations, a correlation is constructed between coordinate difference, gait risk level, and the degree of center of gravity shift; that is, the greater the coordinate difference, the greater the gait risk level and the greater the degree of center of gravity shift.

[0043] Based on this, multiple high-risk patients were selected for analysis to obtain the degree of center of gravity shift for each patient in each gait cycle.

[0044] Preferably, in one embodiment of the present invention, the method for obtaining the high-risk center of gravity offset reference degree includes: The mean value of the center of gravity shift of all patients in each gait cycle was obtained as the high-risk center of gravity shift benchmark for each gait cycle.

[0045] Based on this, the overall level of center of gravity shift of all patients in each gait cycle is quantified by taking the mean, which reflects the average standard of patients' behavioral performance. The greater the degree of center of gravity shift, the more likely the gait is to deviate, and the greater the benchmark degree of high-risk center of gravity shift.

[0046] The fall probability assessment module 103 is used to construct a sliding window to traverse the gait cycle for any patient, and obtain the comprehensive stability within each sliding window based on the degree of center of gravity shift in different gait cycles, the benchmark degree of high-risk center of gravity shift, and various physiological parameter data at different times; and obtain the fall probability within each sliding window based on the distribution of comprehensive stability within different sliding windows.

[0047] To capture the changing characteristics of local time periods in the gait cycle and thus understand the dynamic process of movement, a sliding window is constructed for any patient to sequentially traverse the gait cycles. It should be noted that, in one embodiment of the present invention, the length of the sliding window is W, that is, it contains W gait cycles, and the sliding step length is 1 gait cycle. Starting from the smallest gait cycle, the sliding window slides one gait cycle at a time, traversing to all gait cycles. In other embodiments of the present invention, the size of the sliding window can be set according to specific circumstances, which will not be limited or elaborated here.

[0048] Considering the analysis of patient behavior, quantifying the degree of center of gravity shift and the benchmark degree of high-risk center of gravity shift only reflects the instantaneous state. Combining the changing trends of various physiological data over time reflects the stability of changes in behavior and monitoring data over a period of time, and quantifies the comprehensive stability within the range; based on the degree of center of gravity shift in different time periods, the benchmark degree of high-risk center of gravity shift, and various physiological parameter data at different times, the comprehensive stability within each sliding window is obtained.

[0049] Preferably, in one embodiment of the present invention, the method for obtaining the overall stability level is described in [reference needed]. Figure 3 It illustrates a flowchart of a method for obtaining the overall stability level, including: Step S301: Based on the degree of centroid shift of different time-state cycles and various physiological parameter data at different times, obtain the multidimensional feature matrix of different time-state cycles, and obtain the degree of behavioral stability of each dimension within each sliding window.

[0050] The smaller the changes in the degree of center of gravity shift and various physiological parameters at different times, the smaller the changes in body control ability and the greater the degree of behavioral stability.

[0051] Preferably, in one embodiment of the present invention, the method for obtaining the multidimensional feature matrix includes: The mean value of each physiological parameter at different times in each gait cycle is obtained as the overall value of each physiological parameter in each gait cycle; The degree of center of gravity shift or the overall value of various physiological parameters in all gait cycles are used to construct a one-dimensional column vector of a multidimensional feature matrix.

[0052] It should be noted that a one-dimensional column vector is constructed for the degree of center of gravity shift and the overall values ​​of various physiological parameters under all gait cycles. For example, for any data point among the degree of center of gravity shift and the overall values ​​of various physiological parameters, the one-dimensional column vector is represented as follows: ,in, Indicates the number of gait cycles; Indicates the first Data values ​​corresponding to each gait cycle; Indicates the first The data values ​​corresponding to each gait cycle; construct a multidimensional feature matrix from all column vectors, and the degree of center of gravity shift in each gait cycle and the overall value of all physiological data correspond to the row vectors of the multidimensional feature matrix.

[0053] Preferably, in one embodiment of the present invention, the method for obtaining the degree of behavioral stability includes: Find the minimum value of each element in each dimension across all gait cycles within each sliding window; The cumulative difference between the element values ​​and the minimum element values ​​of each dimension in different gait cycles within the corresponding sliding window is obtained, and a negative correlation normalization mapping is performed to represent the behavioral stability of each dimension within each sliding window.

[0054] It should be noted that, in the embodiments of the present invention, an exponential function with a base of the natural constant can be used. Alternatively, the reciprocal can be used for negative correlation mapping. When calculating the reciprocal, to avoid the formula being meaningless with a denominator of 0, a manually set threshold, such as 0.01, is added to the denominator. Normalization is performed using linear normalization or a normalization function. The specific methods are well-known to those skilled in the art and will not be elaborated here.

[0055] Step S302: Obtain the ratio of the high-risk center of gravity shift baseline for each gait cycle to the center of gravity shift degree for each patient, as the first stability coefficient for each patient in each gait cycle.

[0056] It should be noted that the high-risk center of gravity shift benchmark reflects the general degree of risk in the behavioral performance of high-risk patients. The larger the high-risk center of gravity shift benchmark, the greater the overall center of gravity shift of the high-risk patient. The greater the degree of center of gravity shift relative to the high-risk center of gravity shift benchmark, the greater the center of gravity shift of the corresponding patient, and the smaller the first stability coefficient. Conversely, the smaller the degree of center of gravity shift relative to the high-risk center of gravity shift benchmark, the smaller the center of gravity shift of the corresponding patient, and the larger the first stability coefficient.

[0057] Step S303: Based on the first stability coefficient of different time periods within each sliding window and the behavioral stability of different dimensions, obtain the comprehensive stability within each sliding window. The first stability coefficient and the behavioral stability are both positively correlated with the comprehensive stability.

[0058] It should be noted that a larger first stability coefficient indicates a greater degree of behavioral stability, more stable behavioral activity, and a greater overall stability. In one embodiment of the invention, the average first stability coefficient of all gait cycles within each sliding window is obtained as the overall stability level; the cumulative value of behavioral stability across all dimensions within each sliding window is obtained as the overall behavioral stability level; and the product of the overall stability level and the overall behavioral stability level is calculated as the overall stability level within each sliding window. Therefore, based on the above fundamental mathematical operations, a correlation is established between behavioral stability, overall stability level, and overall stability level; that is, a greater degree of behavioral stability corresponds to a greater overall stability level and a greater overall stability level.

[0059] Overall stability reflects the patient's overall stability within a neighborhood. The greater the overall stability, the more stable the behavior and the lower the risk of falling. The probability of falling in each gait cycle is obtained based on the distribution of overall stability within different sliding windows.

[0060] Preferably, in one embodiment of the present invention, the method for obtaining the probability of falling includes: The average difference in overall stability among all adjacent sliding windows is used as the first fall coefficient. A negative correlation mapping is performed on the overall stability within each sliding window. The product of the negative correlation mapping result and the first fall coefficient is calculated and normalized to serve as the fall probability for each gait cycle.

[0061] It should be noted that in the embodiments of the present invention, linear normalization or normalization functions are used for normalization, such as maximum and minimum value normalization. The specific means are well known to those skilled in the art and will not be described in detail here.

[0062] Based on this, the larger the first fall coefficient, the greater the difference in the overall stability between different time-phase cycles, the more obvious the trend of changes in behavioral and physiological parameters, the more unstable the patient's state, and the greater the possibility of a fall; the greater the overall stability, the more stable the patient's state, and the lower the probability of a fall.

[0063] The fall warning module 104 is used to provide warnings for abnormal fall behavior based on the probability of the patient falling within the latest sliding window.

[0064] The greater the probability of a fall, the greater the likelihood of a fall occurring, and the greater the need for early warning, thus improving the timeliness and accuracy of warnings regarding abnormal fall behavior.

[0065] Preferably, in one embodiment of the invention, providing an early warning for abnormal fall behavior includes: If the probability of falling in the latest sliding window is less than the preset first fall threshold, the abnormal fall behavior is judged to be at the first fall level. If the probability of falling in the latest sliding window is greater than the preset first fall threshold and less than the preset second fall threshold, the abnormal fall behavior is judged to be at the second fall level. If the probability of falling in the latest sliding window is greater than the preset second fall threshold, the abnormal fall behavior is judged to be at the third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

[0066] It should be noted that the higher the probability of falling, the more likely a fall is to occur; the lower the probability of falling, the lower the risk of falling. Therefore, the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level. In one embodiment of the present invention, the preset first fall threshold is set to 0.5, and the preset second fall threshold is set to 0.7. In other embodiments of the present invention, the preset second fall threshold and the preset first fall threshold can be set according to specific circumstances, and are not limited or elaborated here.

[0067] It should be noted that the higher the fall level, the greater the need for early warning. Therefore, the system issues corresponding warnings based on the fall level. At the first fall level, the probability of a fall is low, the patient's condition is relatively stable, and the system maintains its normal monitoring mode without issuing a warning. At the second fall level, the patient's condition shows signs of instability, and the possibility of a fall increases, requiring a mild warning and close monitoring of the patient's condition. At the third fall level, the probability of a fall is relatively high, requiring immediate intervention and issuing a high-level warning. This allows for pre-fall intervention, effectively reducing the fall incidence rate, ensuring timely care for high-risk patients, and greatly improving the accuracy of fall behavior warnings.

[0068] In summary, this invention obtains multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames for any given patient; it obtains the gait risk of each gait cycle based on the ankle coordinate distribution at different time frames within each gait cycle; it obtains the degree of center of gravity shift in each gait cycle based on the hip coordinate distribution at different time frames within each gait cycle, and obtains the high-risk center of gravity shift baseline for all patients in each gait cycle; for any given patient, it obtains the fall probability within each sliding window based on the degree of center of gravity shift in different gait cycles, the high-risk center of gravity shift baseline, and various physiological parameter data at different times; and it provides early warning for abnormal fall behaviors. This invention improves the accuracy of early warning for abnormal fall behaviors by accurately analyzing the fall probability of high-risk patients.

[0069] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0070] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A multimodal data fusion-based early warning system for abnormal pre-fall behavior in high-risk patients, characterized in that, The system includes: The data acquisition module is used to acquire the ankle and hip coordinates on the left and right sides of each frame in the behavioral video of high-risk patients, as well as various physiological parameter data at different times. The behavioral performance quantification module is used to obtain multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames for any patient; to obtain the gait risk of each gait cycle based on the ankle coordinate distribution at different time frames within each gait cycle; to obtain the degree of center of gravity shift in each gait cycle based on the hip coordinate distribution at different time frames within each gait cycle; and to obtain the high-risk center of gravity shift baseline for all patients in each gait cycle. The fall probability assessment module is used to construct a sliding window that traverses the gait cycle for any patient. Based on the degree of center of gravity shift in different gait cycles, the baseline degree of high-risk center of gravity shift, and various physiological parameter data at different times, it obtains the comprehensive stability within each sliding window; and based on the distribution of comprehensive stability within different sliding windows, it obtains the fall probability within each sliding window. The fall warning module is used to issue warnings for abnormal fall behavior based on the probability of a patient falling within the latest sliding window.

2. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the gait cycle includes: For any ankle, the vertical values ​​of the ankle coordinates in all time frames are used to form an ankle curve. The minimum point in the ankle curve is obtained, and the range formed by the corresponding time frames between adjacent minimum points is taken as a gait cycle.

3. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The methods for obtaining gait risk include: For any ankle, obtain the time frame with the minimum vertical value of the ankle coordinate in each gait cycle; Obtain the relative horizontal distance between the ankle coordinates of one ankle at the minimum time frame and the ankle coordinates of the other ankle; The gait risk level for each gait cycle is obtained based on the relative distance between the left and right ankles and the number of time frames in the gait cycle. The relative distance between the left and right ankles is negatively correlated with the gait risk level, while the number of time frames is positively correlated with the gait risk level.

4. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the degree of center of gravity offset includes: For any patient, the mean values ​​of the hip coordinates on the left and right sides at each time frame are obtained and used as the center coordinates. The mean value of the center coordinates between the left and right hip coordinates on all time frames is obtained as the overall center position; Based on the coordinate difference between the center coordinates of different frames in each gait cycle and the overall center position, as well as the gait risk level, the degree of center of gravity shift in each gait cycle is obtained. Both the coordinate difference and the gait risk level are positively correlated with the degree of center of gravity shift.

5. The high-risk patient fall pre-fall abnormal behavior early warning system according to claim 1, characterized in that, The method for obtaining the high-risk center of gravity offset reference degree includes: The mean value of the center of gravity shift of all patients in each gait cycle was obtained as the high-risk center of gravity shift benchmark for each gait cycle.

6. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the overall stability includes: Based on the degree of centroid shift of different time-state cycles and various physiological parameter data at different times, a multidimensional feature matrix of different time-state cycles is obtained, and the degree of behavioral stability of each dimension within each sliding window is obtained. The ratio of the high-risk center of gravity shift baseline for each gait cycle to the center of gravity shift degree for each patient is obtained as the first stability coefficient for each patient in each gait cycle. Based on the first stability coefficient of different time periods within each sliding window and the behavioral stability of different dimensions, the overall stability within each sliding window is obtained. Both the first stability coefficient and the behavioral stability are positively correlated with the overall stability.

7. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 6, characterized in that, The method for obtaining the multidimensional feature matrix includes: The mean value of each physiological parameter at different times in each gait cycle is obtained as the overall value of each physiological parameter in each gait cycle; The degree of center of gravity shift or the overall value of various physiological parameters in all gait cycles are used to construct a one-dimensional column vector of a multidimensional feature matrix.

8. A high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 7, characterized in that, The methods for obtaining the degree of behavioral stability include: Find the minimum value of each element in each dimension across all gait cycles within each sliding window; Obtain the cumulative difference between the element values ​​and the minimum element values ​​of each dimension in different gait cycles within the corresponding sliding window, and perform negative correlation mapping as the behavioral stability of each dimension within each sliding window.

9. A high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The methods for obtaining the fall probability include: The average difference in overall stability among all adjacent sliding windows is used as the first fall coefficient. A negative correlation mapping is performed on the overall stability within each sliding window. The product of the negative correlation mapping result and the first fall coefficient is calculated and normalized to serve as the fall probability within each sliding window.

10. A high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The warning system for abnormal fall behavior includes: If the probability of falling in the latest sliding window is less than the preset first fall threshold, the abnormal fall behavior is judged to be at the first fall level. If the probability of falling in the latest sliding window is greater than the preset first fall threshold and less than the preset second fall threshold, the abnormal fall behavior is judged to be at the second fall level. If the probability of falling in the latest sliding window is greater than the preset second fall threshold, the abnormal fall behavior is judged to be at the third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

Images