High-risk patient fall pre-incident abnormal behavior warning system based on multi-modal data fusion

Abstract

The present application relates to the technical field of medical health data analysis, in particular to a high-risk patient fall abnormal behavior early warning system based on multi-modal data fusion. The present application obtains a plurality of gait cycles constituted by time frames according to ankle coordinate distribution of time frames for any patient; obtains gait high risk of each gait cycle according to ankle coordinate distribution of time frames in each gait cycle; obtains center of gravity offset degree of each gait cycle according to hip coordinate distribution of time frames in each gait cycle, and obtains high-risk center of gravity offset benchmark of all patients in each gait cycle; obtains fall probability in each sliding window according to center of gravity offset degree, high-risk center of gravity offset benchmark of different gait cycles and a plurality of physiological parameter data of different time for any patient; and early warns fall abnormal behavior. The present application improves the accuracy of fall abnormal behavior early warning by accurately analyzing fall probability of high-risk patients.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical health data analysis, and particularly relates to a high-risk patient pre-fall abnormal behavior early warning system based on multi-modal data fusion. BACKGROUND

[0002] Falls not only can cause fractures, brain injuries and other serious physical harm, but also can cause psychological fear in patients, reduce their willingness to move and quality of life, and if rescue cannot be obtained in time after falling, the injury may worsen or even lead to death; therefore, it is necessary to timely perform abnormal pre-fall behavior early warning.

[0003] Existing methods detect instantaneous state or focus on real-time detection of fall events, but considering ignoring the gradual change from "steady gait" to "unsteady gait" of high-risk patients and the change trend of physiological indicators of patients, the trend of progressive deterioration of patient physical function, such as gait stability decline and physiological parameter fluctuation, cannot be captured in advance, resulting in insufficient accuracy of high-risk patient fall behavior early warning based on pre-fall abnormal behavior. SUMMARY

[0004] In order to solve the technical problem of insufficient accuracy of high-risk patient fall behavior early warning based on pre-fall abnormal behavior due to ignoring the change trend of patient gait and physiological indicators, the purpose of the present application is to provide a high-risk patient pre-fall abnormal behavior early warning system based on multi-modal data fusion, and the technical solution adopted is as follows:

[0005] The present application provides a high-risk patient pre-fall abnormal behavior early warning system based on multi-modal data fusion, which comprises:

[0006] A data acquisition module for acquiring ankle coordinates and hip coordinates on the left and right sides of each time frame image in the behavior video of a high-risk patient, and a plurality of physiological parameter data at different times;

[0007] A behavior performance quantification module for, for any patient, obtaining a plurality of gait cycles constituted by time frames according to ankle coordinate distribution of the time frames, obtaining gait risk of each gait cycle according to ankle coordinate distribution of time frames at different times in each gait cycle, obtaining center of gravity offset degree of each gait cycle according to hip coordinate distribution of time frames at different times in each gait cycle, and obtaining high-risk center of gravity offset reference degree of all patients in each gait cycle;

[0008] The fall probability evaluation module is configured to, for any patient, construct a sliding window traversing a gait cycle, obtain a comprehensive stability degree in each sliding window according to a center of gravity deviation degree, a high-risk center of gravity deviation reference degree and a plurality of physiological parameter data at different time points in different gait cycles, and obtain a fall probability in each sliding window according to a comprehensive stability degree distribution in different sliding windows.

[0009] The fall warning module is configured to warn a fall abnormal behavior according to a fall probability of a patient in a latest sliding window.

[0010] Further, the gait cycle acquisition method comprises:

[0011] For any ankle, a vertical direction value of an ankle coordinate of all time point frames is obtained to form an ankle curve, and a range of time point frames between adjacent minimum value points in the ankle curve is taken as a gait cycle.

[0012] Further, the gait high-risk acquisition method comprises:

[0013] For any ankle, a vertical direction value of an ankle coordinate of all time point frames is obtained to form an ankle curve, and a range of time point frames between adjacent minimum value points in the ankle curve is taken as a gait cycle.

[0014] For any ankle, a vertical direction value of an ankle coordinate of all time point frames is obtained to form an ankle curve, and a range of time point frames between adjacent minimum value points in the ankle curve is taken as a gait cycle.

[0015] According to the relative distances of the left ankle and the right ankle and the number of time point frames in the gait cycle, a gait high-risk degree of each gait cycle is obtained, the relative distances of the left ankle and the right ankle are negatively correlated with the gait high-risk degree, and the number of time point frames is positively correlated with the gait high-risk degree.

[0016] Further, the center of gravity deviation degree acquisition method comprises:

[0017] For any patient, an average of left and right hip coordinates at each time point frame is obtained as a center coordinate.

[0018] An average of coordinate differences between the center coordinates of the left and right hip coordinates at all time point frames and a global center position is obtained.

[0019] According to the coordinate differences between the center coordinates of the left and right hip coordinates at different time point frames in each gait cycle and the global center position and the gait high-risk degree, a center of gravity deviation degree of each gait cycle is obtained, and the coordinate differences and the gait high-risk degree are positively correlated with the center of gravity deviation degree.

[0020] Further, the high-risk center of gravity deviation reference degree acquisition method comprises:

[0021] The mean value of the center of gravity shift of all patients in each gait cycle was obtained as the high-risk center of gravity shift benchmark for each gait cycle.

[0022] Furthermore, the method for obtaining the overall stability includes:

[0023] Based on the degree of centroid shift of different time-state cycles and various physiological parameter data at different times, a multidimensional feature matrix of different time-state cycles is obtained, and the degree of behavioral stability of each dimension within each sliding window is obtained.

[0024] The ratio of the high-risk center of gravity shift baseline for each gait cycle to the center of gravity shift degree for each patient is obtained as the first stability coefficient for each patient in each gait cycle.

[0025] Based on the first stability coefficient of different time periods within each sliding window and the behavioral stability of different dimensions, the overall stability within each sliding window is obtained. Both the first stability coefficient and the behavioral stability are positively correlated with the overall stability.

[0026] Furthermore, the method for obtaining the multidimensional feature matrix includes:

[0027] The mean value of each physiological parameter at different times in each gait cycle is obtained as the overall value of each physiological parameter in each gait cycle;

[0028] The degree of center of gravity shift or the overall value of various physiological parameters in all gait cycles are used to construct a one-dimensional column vector of a multidimensional feature matrix.

[0029] Furthermore, the method for obtaining the degree of behavioral stability includes:

[0030] Find the minimum value of each element in each dimension across all gait cycles within each sliding window;

[0031] Obtain the cumulative difference between the element values ​​and the minimum element values ​​of each dimension in different gait cycles within the corresponding sliding window, and perform negative correlation mapping as the behavioral stability of each dimension within each sliding window.

[0032] Furthermore, the method for obtaining the fall probability includes:

[0033] The average difference in overall stability among all adjacent sliding windows is used as the first fall coefficient.

[0034] A negative correlation mapping is performed on the overall stability within each sliding window. The product of the negative correlation mapping result and the first fall coefficient is calculated and normalized to serve as the fall probability within each sliding window.

[0035] Furthermore, the aforementioned early warning system for abnormal fall behavior includes:

[0036] If the fall probability in the latest sliding window is less than a preset first fall threshold, it is judged that the fall abnormal behavior is in a first fall level;

[0037] If the fall probability in the latest sliding window is greater than a preset first fall threshold and less than a preset second fall threshold, it is judged that the fall abnormal behavior is in a second fall level;

[0038] If the fall probability in the latest sliding window is greater than a preset second fall threshold, it is judged that the fall abnormal behavior is in a third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

[0039] The present application has the following beneficial effects:

[0040] The present application has the following beneficial effects: For any patient, according to the ankle coordinate distribution of different time frames, a plurality of gait cycles constituted by time frames are obtained, continuous and non-stationary time series data are cut into comparable and repeatable analysis cycles; according to the ankle coordinate distribution of different time frames in each gait cycle, the gait risk of each gait cycle is obtained, reflecting the high-risk degree of gait; according to the hip coordinate distribution of different time frames in each gait cycle, the degree of center of gravity deviation of each gait cycle is obtained, and the high-risk center of gravity deviation reference degree of all patients in each gait cycle is obtained, evaluating the stability degree of the center of gravity in the walking process; for any patient, a sliding window traverses the gait cycle, according to the degree of center of gravity deviation, the high-risk center of gravity deviation reference degree of different gait cycles, and a plurality of physiological parameter data of different time, the comprehensive stability degree in each sliding window is obtained, the sliding window can capture the stability change in a short period of time, and the dynamic change trend of the stability of the patient is tracked in real time; according to the comprehensive stability degree distribution in different sliding windows, the fall probability in each sliding window is obtained, and the possibility of falling into a falling state is quantified; the fall abnormal behavior is warned. The present application can accurately analyze the fall probability of high-risk patients, and improve the accuracy of fall abnormal behavior warning. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0042] Figure 1 The structural block diagram of a multi-modal data fusion high-risk patient fall abnormal behavior warning system provided by an embodiment of the present application;

[0043] Figure 2 A flow chart of a method for obtaining gait high risk according to an embodiment of the present application is provided.

[0044] Figure 3 A flow chart of a method for obtaining comprehensive stability degree according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0045] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined object, the following describes in detail the specific implementation, structure, features and effects of a multi-modal data fusion high-risk patient pre-fall abnormal behavior warning system according to the present application, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0047] The specific scheme of the multi-modal data fusion high-risk patient pre-fall abnormal behavior warning system provided by the present application is described in detail below in combination with the accompanying drawings.

[0048] Please refer to Figure 1 which shows the structure schematic diagram of a multi-modal data fusion high-risk patient pre-fall abnormal behavior warning system according to an embodiment of the present application, which comprises a data acquisition module 101, a behavior performance quantification module 102, a fall probability evaluation module 103, and a fall warning module 104.

[0049] The data acquisition module 101 is used to acquire the ankle coordinates and hip coordinates on the left and right sides of each frame image in the behavior video of the high-risk patient, as well as the physiological parameter data at different time points.

[0050] In the embodiment of the present application, the high-risk patient generally refers to the elderly, patients with nervous system diseases, musculoskeletal diseases, cardiovascular diseases or postoperative weakness, etc. Considering the analysis of instantaneous state neglects the physiological parameter data changes of the patient's behavior, the pre-warning accuracy of the fall behavior is insufficient, and the motion behavior and physiological change trend of different frames in the behavior video need to be analyzed. First, by arranging monitoring cameras in key areas such as corridors, the behavior video of high-risk patients is collected for analysis. It should be noted that in order to facilitate the subsequent processing of video images, the image of each frame in the collected video is denoised, distorted and background cut. The specific means is the technical means familiar to those skilled in the art, which is not described here.

[0051] The OpenPose model is used to collect the posture of the patient in the behavior video, and the ankle coordinates and hip coordinates on the left and right sides of each frame image are identified; the wearable device is used to obtain the physiological parameter data of the high-risk patient in real time, and the physiological parameter data at least includes heart rate, blood pressure and acceleration.

[0052] It should be noted that the specific OpenPose model is a technology known to those skilled in the art, which will not be described here.

[0053] The behavior performance quantification module 102 is configured to obtain a plurality of gait cycles constituted by the time frames according to the ankle coordinate distribution of the time frames for any patient; obtain the gait risk of each gait cycle according to the ankle coordinate distribution of the time frames in each gait cycle; obtain the center of gravity offset degree of each gait cycle according to the hip coordinate distribution of the time frames in each gait cycle, and obtain the high-risk center of gravity offset benchmark of all patients in each gait cycle.

[0054] Considering that different patients or the same patient has different walking speeds at different times, resulting in different data lengths, it is necessary to divide the gait of the ankle; for any patient, a plurality of gait cycles constituted by the time frames are obtained according to the ankle coordinate distribution of the time frames.

[0055] Preferably, in an embodiment of the present application, the method for obtaining the gait cycle comprises:

[0056] For any ankle, the vertical direction value of the ankle coordinates of all time frames constitutes an ankle curve, the minimum value point in the ankle curve is obtained, and the range constituted by the corresponding time frames between adjacent minimum value points is taken as a gait cycle.

[0057] It should be noted that in the embodiment of the present application, the vertical direction value of the ankle coordinates reflects the rhythm, step frequency and start and end of each step, if the vertical direction value of the time frame in the ankle curve is less than the adjacent time, the position of the corresponding time frame is the minimum value point, at this time the ankle is at the lowest point, reflecting the time point of the foot contacting the ground, finding the next time point of the foot contacting the ground constitutes a complete gait action, that is, all time frames corresponding to adjacent minimum value points constitute a gait cycle, for example, if the second time frame and the fifth time frame corresponding to adjacent minimum value points exist, the range constituted by the second, third, fourth and fifth time frames corresponding to adjacent minimum value points is taken as a gait cycle.

[0058] Because the high-risk patient is physically weak, muscle strength decreases, resulting in insufficient power to push the body forward, slow pace, and insufficient muscle strength to complete large hip flexion and knee extension movements, small step length, that is, the closer the ankle coordinates of the time frame in the horizontal direction, the greater the gait risk; therefore, according to the ankle coordinate distribution of each time frame in each gait cycle, the gait risk of each gait cycle is obtained.

[0059] Preferably, in an embodiment of the present application, the gait risk acquisition method refers to Figure 2 which shows a gait risk acquisition method flowchart, comprising:

[0060] Step S201: for any side ankle, obtain the time frame with the minimum vertical direction value of the ankle coordinate in each gait cycle.

[0061] The minimum vertical direction value reflects the heel touch ground instant, which is helpful for subsequent analysis of walking step length.

[0062] Step S202: obtain the relative distance in the horizontal direction between the ankle coordinates of one side ankle at the minimum value time frame and the other side ankle coordinates.

[0063] The relative distance reflects the distance between the two feet in the horizontal direction when one side ankle touches the bottom, and the greater the distance, the greater the step length.

[0064] It should be noted that in the embodiments of the present application, the relative distance is obtained by existing distance calculation methods such as Euclidean distance or Manhattan distance; the specific means are well-known to those skilled in the art, and will not be described here.

[0065] Step S203: according to the relative distances corresponding to the left and right ankles and the number of time frames in the gait cycle, the gait risk of each gait cycle is obtained, the relative distances corresponding to the left and right ankles are negatively correlated with the gait risk, and the number of time frames is positively correlated with the gait risk.

[0066] It should be noted that the greater the relative distance, the greater the difference between the two ankles in the horizontal direction, the greater the step length, the greater the power, and the smaller the gait risk; the smaller the number of time frames, the greater the step length in a short time, the greater the power of the body forward, the greater the pace, and the smaller the gait risk; therefore, the relative distance is negatively correlated with the gait risk, and the number of time frames is positively correlated with the gait risk.

[0067] In an embodiment of the present application, the mean value of the relative distances corresponding to the left and right ankles is obtained as the average step length of each gait cycle; the sum value of the relative distances corresponding to the left and right ankles is obtained, and the sum value is divided by the number of time frames in the gait cycle to obtain the average pace of each gait cycle;

[0068] The product of the average step length and the average step speed is obtained and negatively correlated mapping is performed as the gait risk of each gait cycle.

[0069] Based on this, the greater the relative distance between the left ankle and the right ankle in the horizontal direction, the greater the step length and the step speed, and the less the gait risk.

[0070] It should be noted that in the embodiments of the present application, the negative correlation mapping can be performed by an exponential function with a natural constant as the base or taking the reciprocal, wherein in order to avoid the denominator of the formula being 0 and the formula being meaningless, an artificially set threshold such as 0.01 is added at the denominator; the specific means are well known to those skilled in the art and will not be described here.

[0071] Due to weak core muscle strength and poor neural control ability, the body is prone to forward or backward leaning and unstable center of gravity during walking; considering that in the middle of the gait support, in order to maintain stability on a single leg, the pelvis will slightly translate to the side of the supporting leg, which will cause the hip joint coordinates to deviate to that side, thereby directly reflecting the deviation of the center of gravity to the supporting side; therefore, according to the distribution of the hip coordinates of each frame at different times in each gait cycle, the degree of center of gravity deviation of each gait cycle is obtained, and the reference degree of high-risk center of gravity deviation of all patients in each gait cycle is obtained.

[0072] Preferably, in an embodiment of the present application, the method for obtaining the degree of center of gravity deviation comprises:

[0073] For any patient, the mean value of the left and right hip coordinates at each time frame is obtained as the center coordinate;

[0074] The mean value of the center coordinates between the left and right hip coordinates at all time frames is obtained as the overall center position;

[0075] According to the coordinate difference between the center coordinates of each frame at different times in each gait cycle and the overall center position, and the gait risk, the degree of center of gravity deviation of each gait cycle is obtained, and the coordinate difference and the gait risk are positively correlated with the degree of center of gravity deviation.

[0076] It should be noted that the difference represents the absolute value of the difference; the coordinate difference reflects the size of the deviation of the center position of each frame from the overall center position, the greater the coordinate difference, the greater the deviation, the less close to the overall center position, and the greater the center deviation degree; the greater the gait risk, the slower the step speed, the smaller the step length, and the more likely the center of gravity deviation, therefore, the coordinate difference and the gait risk are positively correlated with the degree of center of gravity deviation.

[0077] In an embodiment of the present application, the coordinate difference accumulation value between the center coordinates of all frames at all time points in each gait cycle and the overall center position is obtained, and the product of the coordinate difference accumulation value and the gait high risk is obtained as the center of gravity deviation degree of each gait cycle. Therefore, the correlation between the coordinate difference and the gait high risk and the center of gravity deviation degree is established based on the above basic mathematical operation, that is, the greater the coordinate difference, the greater the gait high risk, and the greater the center of gravity deviation degree.

[0078] Based on this, a plurality of high-risk patients are selected for analysis, and the center of gravity deviation degree of each patient in each gait cycle is obtained.

[0079] Preferably, in an embodiment of the present application, the method for obtaining the high-risk center of gravity deviation reference degree comprises:

[0080] The average of the center of gravity deviation degrees of all patients in each gait cycle is obtained as the high-risk center of gravity deviation reference degree of each gait cycle.

[0081] Based on this, the overall level of the center of gravity deviation degree of all patients in each gait cycle is quantified by averaging, reflecting the average standard of the behavior performance of the patient, the greater the center of gravity deviation degree, the more likely the gait to deviate, and the greater the high-risk center of gravity deviation reference degree.

[0082] The fall probability evaluation module 103 is configured to, for any patient, construct a sliding window to traverse the gait cycle, obtain the comprehensive stability degree in each sliding window according to the center of gravity deviation degree, the high-risk center of gravity deviation reference degree of different gait cycles, and a plurality of physiological parameter data at different time points; and obtain the fall probability in each sliding window according to the comprehensive stability degree distribution in different sliding windows.

[0083] In order to capture the change characteristics of the local period in the gait cycle and understand the dynamic process of the movement, for any patient, a sliding window is constructed to traverse the gait cycle in sequence. It should be noted that in an embodiment of the present application, the length of the sliding window is W, that is, W gait cycles are contained, and the sliding step is 1 gait cycle. Starting from the minimum gait cycle, the gait cycle is slid each time, and all gait cycles are traversed. In other embodiments of the present application, the size of the sliding window can be set according to specific conditions, which is not limited or described here.

[0084] Considering that the behavior of the patient is analyzed, the center of gravity deviation degree and the high-risk center of gravity deviation reference degree are quantified, only the instantaneous state is reflected, and the change trend of the plurality of physiological data over time is combined to reflect the change stability of the behavior and the monitoring data within a period of time, and the comprehensive stability degree in the quantification range is quantified. According to the center of gravity deviation degree, the high-risk center of gravity deviation reference degree of different gait cycles, and a plurality of physiological parameter data at different time points, the comprehensive stability degree in each sliding window is obtained.

[0085] Preferably, in an embodiment of the present application, the method for obtaining the comprehensive stability degree comprises the following steps: Figure 3 , which shows a flow chart of a method for obtaining a comprehensive stability degree, comprising:

[0086] Step S301: Obtain a multi-dimensional feature matrix of different gait cycles according to the center of gravity offset degree of different gait cycles and the physiological parameter data of different time points, and obtain the behavior stability degree of each dimension in each sliding window.

[0087] The smaller the changes in the center of gravity offset degree and the physiological parameter data at different time points, the smaller the change in body control ability, and the greater the behavior stability degree.

[0088] Preferably, in an embodiment of the present application, the method for obtaining the multi-dimensional feature matrix comprises the following steps:

[0089] Obtain the data mean of each physiological parameter at different time points in each gait cycle as the overall value of each physiological parameter in each gait cycle;

[0090] Construct a one-dimensional column vector of the overall value of each physiological parameter or the center of gravity offset degree of all gait cycles as a one-dimensional column vector of the multi-dimensional feature matrix.

[0091] It should be noted that the overall value of the center of gravity offset degree and the overall value of the physiological parameter of all gait cycles are respectively constructed into a one-dimensional column vector, and for any data in the overall value of the center of gravity offset degree and the overall value of the physiological parameter, the one-dimensional column vector is represented as , wherein represents the number of gait cycles; represents the data value corresponding to the th gait cycle; represents the data value corresponding to the th gait cycle; and all dimension column vectors are constructed into a multi-dimensional feature matrix, and the center of gravity offset degree on each gait cycle and the overall value of all physiological data correspond to a row vector of the multi-dimensional feature matrix.

[0092] Preferably, in an embodiment of the present application, the method for obtaining the behavior stability degree comprises the following steps:

[0093] Obtain the minimum value of each dimension in all gait cycles in each sliding window;

[0094] Obtain the difference accumulation value between the element value of each dimension in different gait cycles and the minimum value of each dimension in the corresponding sliding window, and perform negative correlation normalization mapping as the behavior stability degree of each dimension in each sliding window.

[0095] It should be noted that in the embodiment of the present application, the negative correlation normalization mapping can be performed by an exponential function with a natural constant as the base or reciprocal to perform a negative correlation mapping, wherein, in order to avoid the denominator of the formula being 0, the formula being meaningless, a threshold value such as 0.01 is added at the denominator when the reciprocal is taken; linear normalization or normalization function is used for normalization, and specific means are well known to those skilled in the art and will not be described here.

[0096] Step S302: Obtain the ratio of the high-risk center-of-gravity deviation reference degree of each gait cycle and the center-of-gravity deviation degree of each patient as the first stability coefficient of each patient in each gait cycle.

[0097] It should be noted that the high-risk center-of-gravity deviation reference degree reflects the general degree of risk of behavior performance of high-risk patients. The greater the high-risk center-of-gravity deviation reference degree, the greater the overall center-of-gravity deviation of high-risk patients, the greater the center-of-gravity deviation degree relative to the high-risk center-of-gravity deviation reference degree, the greater the center-of-gravity deviation of the corresponding patient, and the smaller the first stability coefficient; the smaller the center-of-gravity deviation degree relative to the high-risk center-of-gravity deviation reference degree, the smaller the center-of-gravity deviation of the corresponding patient, and the greater the first stability coefficient.

[0098] Step S303: Obtain the comprehensive stability degree in each sliding window according to the first stability coefficient of different gait cycles in each sliding window and the behavior stability degree in different dimensions, and the first stability coefficient and the behavior stability degree are positively correlated with the comprehensive stability degree.

[0099] It should be noted that the greater the first stability coefficient, the greater the behavior stability degree, the more stable the behavior activity changes, and the greater the comprehensive stability degree; in an embodiment of the present application, the mean value of the first stability coefficient of all gait cycles in each sliding window is obtained as the overall stability level; the cumulative value of the behavior stability degree in all dimensions in each sliding window is obtained as the overall behavior stability degree; and the product of the overall stability level and the overall behavior stability degree is calculated as the comprehensive stability degree in each sliding window. Therefore, the correlation between the behavior stability degree, the overall stability level and the comprehensive stability degree is established based on the above basic mathematical operations, that is, the greater the behavior stability degree, the greater the overall stability level and the comprehensive stability degree.

[0100] The comprehensive stability degree reflects the overall stability state of the patient in the neighborhood range. The greater the comprehensive stability degree, the more stable the behavior, and the lower the risk of falling. According to the distribution of the comprehensive stability degree in different sliding windows, the falling probability of each gait cycle is obtained.

[0101] Preferably, in an embodiment of the present application, the method for obtaining the falling probability comprises:

[0102] obtaining the mean value of the difference between the comprehensive stability degrees of all adjacent sliding windows as the first falling coefficient;

[0103] The comprehensive stability degree in each sliding window is negatively correlated mapped, the product of the negatively correlated mapping result and the first fall coefficient is calculated, and normalization is performed to obtain the fall probability of each gait cycle.

[0104] It should be noted that in the embodiments of the present application, linear normalization or normalization functions are used for normalization, such as maximum and minimum value normalization, and the specific means are well known to those skilled in the art, which will not be described here.

[0105] Therefore, the larger the first fall coefficient is, the greater the difference between the comprehensive stability degrees of different gait cycles is, the more obvious the change trend of the behavior and physiological parameter data is, the more unstable the patient state is, and the greater the possibility of falling is; the greater the comprehensive stability degree is, the more stable the patient state is, and the smaller the fall probability is.

[0106] The fall warning module 104 is configured to warn the fall abnormal behavior according to the fall probability of the patient in the latest sliding window.

[0107] The greater the fall probability is, the greater the possibility of falling is, and the more the warning is needed, thereby improving the timeliness and accuracy of the fall abnormal behavior warning.

[0108] Preferably, in one embodiment of the present application, the warning of the fall abnormal behavior comprises:

[0109] If the fall probability in the latest sliding window is less than a preset first fall threshold, it is determined that the fall abnormal behavior is at a first fall level;

[0110] If the fall probability in the latest sliding window is greater than the preset first fall threshold and less than a preset second fall threshold, it is determined that the fall abnormal behavior is at a second fall level;

[0111] If the fall probability in the latest sliding window is greater than the preset second fall threshold, it is determined that the fall abnormal behavior is at a third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

[0112] It should be noted that the greater the fall probability is, the greater the possibility of falling is, and the smaller the fall probability is, the smaller the risk of falling is, so the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level; in one embodiment of the present application, the preset first fall threshold is set to 0.5, and the preset second fall threshold is set to 0.7; in other embodiments of the present application, the preset second fall threshold and the preset first fall threshold can be set according to specific conditions, which will not be limited or described here.

[0113] It should be noted that the greater the fall level, the more the need for early warning, so according to the fall level, the corresponding early warning is carried out on the patient behavior, that is, when the first fall level, the probability of falling is small, the patient state is relatively stable, the system maintains the normal monitoring mode, and no early warning is needed; when the second fall level, the patient state appears unstable trend, the possibility of falling increases, and mild early warning is needed, so as to timely pay attention to the patient state; when the third fall level, the probability of falling is relatively high, immediate intervention is needed, and high-level early warning is sent; so as to realize the pre-fall intervention, effectively reduce the incidence of falling, ensure the timely rescue of high-risk patients, and greatly improve the accuracy of the fall behavior early warning.

[0114] To sum up, for any patient, according to the ankle coordinate distribution of different time frames, a plurality of gait cycles constituted by time frames are obtained; according to the ankle coordinate distribution of different time frames in each gait cycle, the gait high risk of each gait cycle is obtained; according to the hip coordinate distribution of different time frames in each gait cycle, the center of gravity offset degree of each gait cycle is obtained, and the high-risk center of gravity offset reference degree of all patients in each gait cycle is obtained; for any patient, according to the center of gravity offset degree, the high-risk center of gravity offset reference degree of different gait cycles, and a plurality of physiological parameter data of different time, the fall probability in each sliding window is obtained; the fall abnormal behavior is early warned. The fall probability of the high-risk patient is accurately analyzed, and the accuracy of the fall abnormal behavior early warning is improved.

[0115] It should be noted that the above-mentioned embodiment of the present application is only for description, and does not represent the advantages and disadvantages of the embodiment. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0116] Each embodiment in the specification is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other, and each embodiment mainly describes the difference from other embodiments.

Claims

1. A multimodal data fusion-based early warning system for abnormal pre-fall behavior in high-risk patients, characterized in that, The system includes: The data acquisition module is used to acquire the ankle and hip coordinates on the left and right sides of each frame in the behavioral video of high-risk patients, as well as various physiological parameter data at different times. The behavioral performance quantification module is used to obtain multiple gait cycles composed of time frames based on the ankle coordinate distribution at different time frames for any patient; to obtain the gait risk of each gait cycle based on the ankle coordinate distribution at different time frames within each gait cycle; to obtain the degree of center of gravity shift in each gait cycle based on the hip coordinate distribution at different time frames within each gait cycle; and to obtain the high-risk center of gravity shift baseline for all patients in each gait cycle. The methods for obtaining gait risk include: For any ankle, obtain the time frame with the minimum vertical value of the ankle coordinate in each gait cycle; Obtain the relative horizontal distance between the ankle coordinates of one ankle at the minimum time frame and the ankle coordinates of the other ankle; The gait risk of each gait cycle is obtained based on the relative distance between the left and right ankles and the number of time frames in the gait cycle. The relative distance between the left and right ankles is negatively correlated with the gait risk, while the number of time frames is positively correlated with the gait risk. The method for obtaining the degree of center of gravity offset includes: For any patient, the mean values ​​of the hip coordinates on the left and right sides at each time frame are obtained and used as the center coordinates. The mean value of the center coordinates between the left and right hip coordinates on all time frames is obtained as the overall center position; The cumulative coordinate difference between the center coordinates and the overall center position of all frames in each gait cycle is obtained. The product of the cumulative coordinate difference and the gait risk is obtained as the degree of center of gravity shift in each gait cycle. The fall probability assessment module is used to construct a sliding window that traverses the gait cycle for any patient. Based on the degree of center of gravity shift in different gait cycles, the baseline degree of high-risk center of gravity shift, and various physiological parameter data at different times, it obtains the comprehensive stability within each sliding window; and based on the distribution of comprehensive stability within different sliding windows, it obtains the fall probability within each sliding window. The fall warning module is used to issue warnings for abnormal fall behavior based on the probability of a patient falling within the latest sliding window.

2. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the gait cycle includes: For any ankle, the vertical values ​​of the ankle coordinates in all time frames are used to form an ankle curve. The minimum point in the ankle curve is obtained, and the range formed by the corresponding time frames between adjacent minimum points is taken as a gait cycle.

3. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the high-risk center of gravity offset reference degree includes: The mean value of the center of gravity shift of all patients in each gait cycle was obtained as the high-risk center of gravity shift benchmark for each gait cycle.

4. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the overall stability includes: Based on the degree of centroid shift of different time-state cycles and various physiological parameter data at different times, a multidimensional feature matrix of different time-state cycles is obtained, and the degree of behavioral stability of each dimension within each sliding window is obtained. The ratio of the high-risk center of gravity shift baseline for each gait cycle to the center of gravity shift degree for each patient is obtained as the first stability coefficient for each patient in each gait cycle. Based on the first stability coefficient of different time periods within each sliding window and the behavioral stability of different dimensions, the overall stability within each sliding window is obtained. Both the first stability coefficient and the behavioral stability are positively correlated with the overall stability.

5. A high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 4, characterized in that, The method for obtaining the multidimensional feature matrix includes: The mean value of each physiological parameter at different times in each gait cycle is obtained as the overall value of each physiological parameter in each gait cycle; The degree of center of gravity shift or the overall value of various physiological parameters in all gait cycles are used to construct a one-dimensional column vector of a multidimensional feature matrix.

6. The high-risk patient fall pre-fall abnormal behavior early warning system according to claim 5, characterized in that, The methods for obtaining the degree of behavioral stability include: Find the minimum value of each element in each dimension across all gait cycles within each sliding window; Obtain the cumulative difference between the element values ​​and the minimum element values ​​of each dimension in different gait cycles within the corresponding sliding window, and perform negative correlation mapping as the behavioral stability of each dimension within each sliding window.

7. The high-risk patient fall pre-fall abnormal behavior early warning system according to claim 1, characterized in that, The methods for obtaining the fall probability include: The average difference in overall stability among all adjacent sliding windows is used as the first fall coefficient. A negative correlation mapping is performed on the overall stability within each sliding window. The product of the negative correlation mapping result and the first fall coefficient is calculated and normalized to serve as the fall probability within each sliding window.

8. The high-risk patient fall pre-fall abnormal behavior early warning system based on multimodal data fusion according to claim 1, characterized in that, The warning system for abnormal fall behavior includes: If the probability of falling in the latest sliding window is less than the preset first fall threshold, the abnormal fall behavior is judged to be at the first fall level. If the probability of falling in the latest sliding window is greater than the preset first fall threshold and less than the preset second fall threshold, the abnormal fall behavior is judged to be at the second fall level. If the probability of falling in the latest sliding window is greater than the preset second fall threshold, the abnormal fall behavior is judged to be at the third fall level; the preset second fall threshold is greater than the preset first fall threshold, the third fall level is greater than the second fall level, and the second fall level is greater than the first fall level.

Images