Computer vision-based fall detection and alarm method and system for alzheimer's patients

Abstract

This invention discloses a computer vision-based method and system for fall detection and alarm in elderly Alzheimer's patients. To address the difficulty in accurately identifying patient instability and falls in the presence of assistive devices such as canes, walkers, and wall handrails, this invention employs video preprocessing and online camera parameter estimation, implicit 3D reconstruction of the human body and assistive devices, construction of a ternary hypergraph of the human-assistive device-ground relationship with risk-gated attention, inference of contact state and grip stability using a dwell state memory unit in a contact Transformer, calculation of stability index, risk rate, and time to instability based on a differentiable physics model, and setting an adaptive alarm threshold for tiered alarm output. This achieves real-time identification of patient instability and fall risk under conditions of carrying or using assistive devices, improving accuracy and reducing false alarms.

Description

Technical Field

[0001] This invention relates to the field of fall detection, and more particularly to a method and system for fall recognition and alarm in elderly Alzheimer's patients based on computer vision. Background Technology

[0002] With the aging population, the risk of falls among Alzheimer's patients has increased significantly, leading to a growing demand for fall detection and alarm systems in both home and institutional settings. Existing technologies mainly fall into three categories: first, those based on wearable devices using accelerometers and gyroscopes, relying on thresholds or learning models to determine abnormal movements; second, those based on video-based 2D skeleton and target detection, combined with temporal classification to identify falls; and third, those incorporating depth sensors or multi-view vision for 3D pose estimation and ground detection to enhance spatial understanding.

[0003] In recent years, deep learning has made progress in human pose estimation, action recognition, and human-computer interaction understanding. Some methods attempt to use contact probability and support area to assist in judgment.

[0004] However, there are still shortcomings in fall detection for Alzheimer's patients using assistive devices such as canes, walkers, and wall handrails:

[0005] 1. Insufficient modeling of three-way interaction between human assistive devices on the ground: Many methods only focus on key points of the human body or simple planar constraints, making it difficult to accurately determine the holding and support relationship under the circumstances of assistive device occlusion and complex contact, which easily leads to false alarms and false alarms.

[0006] 2. Lack of interpretable stability assessment and time prediction: Most solutions directly trigger alarms based on the probability of action classification, without combining the relationship between the center of mass and the support domain, physical quantities such as friction and overturning, making it difficult to quantify the risk rate and the time to instability, and making it difficult to achieve early warning;

[0007] 3. The alarm threshold lacks adaptive and hierarchical strategies: The fixed threshold does not take into account behavioral cues such as grip stability and contact dwell time, and is not sufficiently adapted to the slow instability and progressive sinking that are common in Alzheimer's patients, affecting accuracy and usability.

[0008] Therefore, a method and system for patient fall detection and alarm that can overcome the shortcomings of the existing technology is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0009] One objective of this invention is to propose a fall detection and alarm method for elderly Alzheimer's patients based on computer vision. Addressing the shortcomings of existing technologies in modeling the grip and support relationship under the presence of assistive devices, and the lack of interpretability and early warning capabilities in stability assessment, this invention proposes a technical solution involving video preprocessing and online estimation of camera parameters, implicit 3D reconstruction of the human body and assistive devices, ground plane modeling, construction of a human-assistive device ground ternary hypergraph with risk-gated attention, inference of contact state and grip stability using a dwell state memory unit in a contact Transformer, updating the support domain based on a differentiable physics model and calculating energy barrier, friction margin, and overturning margin, regression hazard rate and instability time, fusing them to obtain a stability index, and outputting alarms in a graded manner based on an adaptive alarm threshold. This invention provides real-time, accurate identification and early warning of instability and fall risks under the use of assistive devices, reduces false alarms and missed alarms, and improves interpretability.

[0010] A fall detection and alarm method for elderly Alzheimer's patients based on computer vision according to an embodiment of the present invention is characterized by comprising the following steps:

[0011] S1. Acquire continuous video from at least one camera to obtain the original video frame sequence and timestamp, preprocess the frame sequence and estimate camera parameters;

[0012] S2. Using the preprocessed video frame sequence and camera parameter estimation results as input, implicit 3D reconstruction is performed to obtain the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, and the ground plane parameters. Based on this, the candidate contact point set is determined, the centroid ground projection is calculated, and the initial support domain is formed to generate the initial risk signal.

[0013] S3. The implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters and the candidate contact point set are used as inputs to form a human-assistive device-ground ternary hypergraph, and the initial risk signal is mapped into the edge-level gating coefficient to form the external input of risk gating attention.

[0014] S4. Using the external inputs of the human-assisted device-ground ternary hypergraph and risk-gated attention as inputs, reasoning is performed in the temporal Transformer to output the contact state sequence, contact dwell time and grip stability.

[0015] S5. Using the contact state sequence, contact dwell time and grip stability, as well as the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, ground plane parameters, centroid ground projection and initial support domain as input, the support domain is updated and its temporal evolution is determined. Stability is scored based on the differentiable physical model, the risk rate and the time to instability are regressed and fused to obtain the stability index.

[0016] S6. Using stability index, risk rate, time to instability, contact dwell time and grip stability as inputs, set a preset alarm threshold and generate an adaptive alarm threshold. Based on the threshold classification, judge and output an alarm signal for prompting and notification.

[0017] Optionally, step S1 specifically includes:

[0018] At least one camera continuously captures raw video frame sequences and timestamps, and binds the timestamps to the corresponding video frames to ensure time consistency.

[0019] Geometric normalization is performed on the original video frame sequence. Geometric normalization includes at least one or more of lens distortion correction, cropping and scaling, and perspective correction to obtain a geometrically normalized video frame sequence.

[0020] Lighting normalization is performed on the geometrically normalized video frame sequence. Lighting normalization includes at least one or more of white balance, brightness equalization, and contrast adjustment. Inter-frame stabilization and noise suppression are also performed to obtain the lighting normalized video frame sequence.

[0021] Based on the video frame sequence after illumination normalization, the camera intrinsic and extrinsic parameters are estimated online. The camera intrinsic parameters include at least focal length, principal point and distortion coefficient, and the camera extrinsic parameters include at least pose and displacement, so as to obtain the camera parameter estimation results.

[0022] The illumination-normalized video frame sequence and the camera parameter estimation results are combined into a preprocessed video frame sequence and camera parameter estimation results.

[0023] Optionally, step S2 specifically includes:

[0024] The preprocessed video frame sequence and camera parameter estimation results are used as input to perform implicit 3D reconstruction to obtain implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of assistive devices, and ground plane parameters.

[0025] Based on the implicit 3D reconstruction results of the human body and the implicit 3D reconstruction results of the walking aid, the candidate contact points of the foot, the tip of the cane, the wheel assembly of the walking aid and the wall handrail are located, and the contact position between the candidate contact points and the ground is determined according to the ground plane parameters to form a set of candidate contact points.

[0026] In the implicit 3D reconstruction results of the human body, the human body centroid is estimated and projected onto the ground defined by the ground plane parameters to obtain the centroid ground projection;

[0027] The initial support domain is formed based on the location of the foot and the candidate contact points on the ground.

[0028] An initial risk signal is generated based on the distance from the centroid ground projection to the boundary of the initial support domain and the time derivative of that distance;

[0029] Output implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of assistive devices, ground plane parameters, candidate contact point set, centroid ground projection, initial support domain and initial risk signal.

[0030] Optionally, step S3 specifically includes:

[0031] The implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters, and the candidate contact point set are used as inputs;

[0032] Based on the implicit 3D reconstruction results of the human body, key points of the human body are determined and a human body node set is formed. Based on the implicit 3D reconstruction results of the assistive device, key contact points of the assistive device are determined and an assistive device node set is formed. Based on the ground plane parameters, the ground area is defined and a ground node set is formed.

[0033] Calculate at least one or more of the following attributes for each node: position, pose, and velocity;

[0034] An edge set is established between the human node set, the assistive device node set, and the ground node set. The edge set includes at least an edge representing a gripping relationship, an edge representing a contact relationship, and an edge representing a relative pose and a sliding velocity. For each edge, at least one or more of the following attributes are calculated: contact probability, relative pose change, and sliding velocity.

[0035] The initial risk signal is mapped according to the candidate contact points associated with each edge and normalized into edge-level gating coefficients within a bounded interval to form the external input of risk gating attention;

[0036] The above-mentioned node set, edge set and their attributes are used to construct a human-assisted tool-ground ternary hypergraph, and the external input of the human-assisted tool-ground ternary hypergraph and risk gating attention is output.

[0037] Optionally, step S4 specifically includes:

[0038] The external inputs of the human-assisted device-ground ternary hypermap and risk-gated attention are used as inputs;

[0039] In the contact Transformer, the continuous time instances of the human-assisted device-ground ternary hypergraph are combined into a temporal input according to the timestamp, and the external input of risk-gated attention is used as the edge-level gating coefficient to adjust the temporal attention weight;

[0040] The dwell-state memory unit is initialized with a zero value and used to accumulate the continuous state dwell time of each edge and record the most recent state during the temporal reasoning process;

[0041] Based on gated temporal attention, a contact state sequence is output at the edge level, which includes at least stable support, slippage, and detachment.

[0042] The contact dwell time is calculated based on the cumulative continuous duration of the dwell-state memory unit;

[0043] The grip stability is calculated based on temporal consistency and relative pose change for the edges corresponding to the grip relationship.

[0044] Optionally, step S5 specifically includes:

[0045] The contact state sequence, contact dwell time and grip stability, implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of the walking aid, ground plane parameters, centroid ground projection and initial support domain are used as inputs.

[0046] Contact points in a stable support state are selected based on the contact state sequence and contact dwell time to form a set of stable support contact points. Contact points in a sliding state are selected based on the contact state sequence, and the sliding velocity is calculated based on the ground position change of the candidate contact point set at continuous time intervals to form a sliding velocity sequence.

[0047] The initial support domain is updated based on the location of the stable support contact point set on the ground to obtain the updated support domain. The boundary of the updated support domain is corrected over time based on the slip velocity sequence to obtain the time evolution of the support domain.

[0048] Under the differentiable physical model, the energy barrier, friction margin and overturning margin are calculated based on the geometric relationship between the centroid ground projection and the updated support domain and the ground plane parameters. The risk rate and the time to instability are regressed based on the above indicators and their time changes, with holding stability as the weight.

[0049] The stability index is obtained by integrating at least the energy barrier, friction margin, and overturning margin.

[0050] Optionally, step S6 specifically includes:

[0051] Stability index, risk rate, time to instability, contact dwell time, and grip stability are used as inputs;

[0052] Preset alarm thresholds are set based on stability index, hazard rate, and time to instability. The preset alarm thresholds are adaptively modified by contact dwell time and grip stability to form an adaptive alarm threshold. When grip stability increases, the alarm thresholds for hazard rate and time to instability are increased; when grip stability decreases, the alarm thresholds for hazard rate and time to instability are decreased; and when contact dwell time increases, the alarm threshold for stability index is increased.

[0053] Under the adaptive alarm threshold, a graded judgment is made. When the stability index is lower than its adaptive alarm threshold, an alarm signal is generated. When the danger rate exceeds its adaptive alarm threshold and the time to instability is shorter than its adaptive alarm threshold, an alarm signal is generated. Different levels are associated with alarm signals based on different conditions.

[0054] The generated alarm signal is output for prompting and notification.

[0055] A computer vision-based fall detection and alarm system for elderly Alzheimer's patients, comprising:

[0056] The video acquisition module is used to acquire continuous video and generate timestamps, perform geometric normalization and illumination normalization on the video, and estimate camera parameters online.

[0057] The reconstruction and calculation module is used to calculate the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters, the candidate contact point set, the centroid ground projection, the initial support domain, and the initial risk signal, taking the preprocessed video frame sequence and camera parameter estimation results as input.

[0058] The ternary hypergraph construction module is used to construct a human-assisted device-ground ternary hypergraph and map the initial risk signal into edge-level gating coefficients to form the external input of risk gating attention;

[0059] The temporal reasoning module is used to employ risk-gated attention and set up dwell-state memory units in the temporal Transformer, and output the contact state sequence, contact dwell time and grip stability.

[0060] The scoring module is used to update the support domain and calculate the energy barrier, friction margin and overturning margin, regress the risk rate and the time to instability and fuse them to obtain the stability index.

[0061] The decision module is used to set a preset alarm threshold and generate an adaptive alarm threshold based on the contact dwell time and grip stability. It performs hierarchical judgment based on the preset alarm threshold and the adaptive alarm threshold and outputs an alarm signal.

[0062] The beneficial effects of this invention are:

[0063] 1. This invention uses a unified implicit 3D reconstruction of the human body, walking aids, and the ground, and uses a human-aid-ground ternary hypergraph to represent the contact and gripping relationship. Combined with the contact Transformer and dwell state memory unit of risk gating attention, it can accurately determine the stable support, slippage and disengagement state and gripping stability in complex interactions and occlusions such as canes, walkers, and wall handrails, which significantly improves the recognition accuracy and robustness and reduces false alarms and false negatives.

[0064] 2. This invention constructs a differentiable physical stability scoring link, updates the support domain over time and calculates the energy barrier, friction margin and overturning margin, and then regresses the hazard rate and instability time, providing an interpretable stability index and early warning capability, realizing quantitative assessment and graded alarm of instability risk, and improving response timeliness.

[0065] 3. This invention generates adaptive alarm thresholds based on contact dwell time and grip stability, which can adapt to individual patient differences and behavioral patterns, avoid false triggering and missed triggering caused by fixed thresholds, and improve the usability and user experience of the system in home and institutional scenarios. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a flowchart of a fall detection and alarm method and system for elderly Alzheimer's patients based on computer vision, as proposed in this invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0069] refer to Figure 1 A method for fall detection and alarm in elderly Alzheimer's patients based on computer vision, characterized by the following steps:

[0070] S1. Acquire continuous video from at least one camera to obtain the original video frame sequence and timestamp, preprocess the frame sequence and estimate camera parameters;

[0071] S2. Using the preprocessed video frame sequence and camera parameter estimation results as input, implicit 3D reconstruction is performed to obtain the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, and the ground plane parameters. Based on this, the candidate contact point set is determined, the centroid ground projection is calculated, and the initial support domain is formed to generate the initial risk signal.

[0072] S3. The implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters and the candidate contact point set are used as inputs to form a human-assistive device-ground ternary hypergraph, and the initial risk signal is mapped into the edge-level gating coefficient to form the external input of risk gating attention.

[0073] S4. Using the external inputs of the human-assisted device-ground ternary hypergraph and risk-gated attention as inputs, reasoning is performed in the temporal Transformer to output the contact state sequence, contact dwell time and grip stability.

[0074] S5. Using the contact state sequence, contact dwell time and grip stability, as well as the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, ground plane parameters, centroid ground projection and initial support domain as input, the support domain is updated and its temporal evolution is determined. Stability is scored based on the differentiable physical model, the risk rate and the time to instability are regressed and fused to obtain the stability index.

[0075] S6. Using stability index, risk rate, time to instability, contact dwell time and grip stability as inputs, set a preset alarm threshold and generate an adaptive alarm threshold. Based on the threshold classification, judge and output an alarm signal for prompting and notification.

[0076] In this specific embodiment, S1 specifically refers to:

[0077] The system completes preprocessing through four stages: acquisition, normalization, image stabilization and noise reduction, and online estimation, to obtain a video data stream with consistent camera parameters and time. The system continuously acquires video frames from at least one camera and records timestamps, binding each timestamp to a corresponding frame to ensure time consistency. The acquired frame sequence is represented as follows: ;

[0078] in Indicates time Three-channel images For frame indexing, Total number of frames;

[0079] The timestamp sequence is represented as ;

[0080] in To and Corresponding time stamp;

[0081] Geometric normalization first unifies lens distortion, cropping, and scaling in the homogeneous pixel coordinate space, with the original pixel coordinates denoted as: ;

[0082] in and These are the pixel coordinates in the horizontal and vertical directions, respectively;

[0083] Normalized mapping is written as ;

[0084] in For the geometrically normalized pixel coordinates, For global scaling factor, For distortion correction functions based on distortion parameters, middle The radial distortion coefficient and Tangential distortion coefficient, The representation of clipping and translation in homogeneous coordinates and and Perspective correction can optionally be achieved by estimating the homography matrix, representing the translation in the horizontal and vertical directions. To enhance ground plane consistency, among which For a moment The homography transformation matrix from the plane to the image;

[0085] Illumination normalization performs white balance based on distortion correction and geometric normalization to correct channel gain, and completes normalization by mapping pixel intensity that combines brightness and contrast. The mapping form is as follows:

[0086] ;

[0087] in For a moment In pixels Normalized intensity at the point, For original strength, Contrast scaling factor For brightness bias, The gamma exponent is adaptively estimated by histogram statistics or a learning model.

[0088] Subsequently, inter-frame stabilization and noise suppression are performed to eliminate minor jitter and sensor noise. Image stabilization is achieved by estimating the affine matrix. right The geometric perturbation is corrected, where For a moment The two-dimensional affine transformation is represented in homogeneous coordinates, and time-domain smoothing is used for noise suppression to improve the stability of subsequent estimations;

[0089] Online camera parameter estimation follows the pinhole imaging model And the intrinsic and extrinsic parameters are solved jointly by minimizing the reprojection error;

[0090] in Homogeneous scaling factor For the first Each feature at time The pixel coordinates after distortion correction and image stabilization For feature indexing, The intrinsic parameter matrix is ​​composed of pixel focal length coordinates of the principal point Decide, and Focal lengths in the horizontal and vertical directions, respectively. and Principal coordinates To characterize the pose of the camera using the rotation matrix, The camera translation vector represents the displacement. Let be the representation of the corresponding 3D point in homogeneous coordinates and These are the three-dimensional coordinate components;

[0091] Finally, the frame sequence after illumination normalization and image stabilization is combined with the camera parameter estimation results at the current moment to form a preprocessed output, which is then provided for subsequent implicit 3D reconstruction and ground modeling.

[0092] In this specific embodiment, S2 specifically refers to:

[0093] Using preprocessed video frames and camera parameters as input, we first perform 3D reconstruction of the human body and assistive devices using implicit representations, while simultaneously robustly fitting the ground plane. For the human body and assistive devices, we use voxels or neural implicit fields to obtain dense geometry and component semantics. The ground is constrained by a planar model to obtain normals and heights. The analytical expression for the ground plane is written as:

[0094] ;

[0095] in Represents any three-dimensional point in the world coordinate system. Represents the ground normal vector, This represents the directed distance from the ground plane to the origin.

[0096] Subsequently, based on component semantics and geometric contact heuristics, candidate contact anchor points are extracted from the reconstruction results, covering possible contact segments such as the soles of the feet, the tips of the canes, the wheel sets of the walking aids, and the handrails on the walls. The ground orthogonal projection operator is defined as follows:

[0097] ;

[0098] in This represents projecting a 3D point onto the ground plane along the normal vector;

[0099] This operator is used to project the three-dimensional position of each candidate contact anchor point onto the ground to obtain the ground contact position, and then further processed according to time intervals. Aggregate all contact points and form an initial support region using a convex hull:

[0100] ;

[0101] in Indicates time initial support domain, Represents convex hull operation, Indicates the first Each candidate contact anchor point at time Three-dimensional position, For candidate indexes, For a moment The number of candidates;

[0102] The center of mass of the human body is obtained by implicitly reconstructing voxel occupancy or density field weighting and denoted as . Its ground projection is ;

[0103] in Indicates time The position of the center of mass Indicates its ground projection;

[0104] To measure the stability margin of the centroid projected onto the support domain boundary, the nearest boundary distance is calculated:

[0105] ;

[0106] in Indicates time minimum distance express Boundaries Represents any point on the boundary. Indicates taking the minimum value. Represents the Euclidean second norm;

[0107] Considering the impact of dynamic evolution on risk, the time derivative of the distance is approximated using discrete difference:

[0108] ;

[0109] in express rate of change over time Indicates the time interval between adjacent frames. Indicates time timestamp, This represents the discrete difference approximation;

[0110] Based on this, the initial risk signal is defined as follows:

[0111] ;

[0112] in Indicates time Initial risk intensity This represents a positive constant set to avoid a denominator of zero. and The weighting coefficients for the harmonic term and the velocity term are indicated. This represents the absolute value operation;

[0113] The final output of this step includes implicit 3D reconstruction results of the human body and assistive devices, as well as ground plane parameters. Set of ground locations of candidate contact points, and ground projection of the centroid. Initial support domain With initial risk signals Used for subsequent ternary hypergraph construction and temporal reasoning.

[0114] In this specific embodiment, S3 specifically refers to:

[0115] At any moment The implicit reconstruction of the human body and assistive devices, the ground plane and candidate contact cues are organized into a ternary hypergraph, and edge-level risk gating inputs are generated. Specifically, a set of nodes is first constructed and its source is distinguished so that the human body node set, assistive device node set and ground node set are uniformly represented in the union set as follows:

[0116] ;

[0117] in Indicates time The complete set of nodes Represents the set of human body nodes, Represents the set of assistive device nodes, Represents the set of ground nodes, Represents the union operator;

[0118] Then, based on the relationship between gripping, contact, and relative pose, an edge set is established and uniformly represented in a union set as follows:

[0119] ;

[0120] in Indicates time The complete set of edges Represents the set of edges with holding relationships. Represents the set of edges with contact relationships. Represents the set of edges relating relative pose and glide;

[0121] Based on this, the time-over-time graph is obtained:

[0122] ;

[0123] in Indicates time ternary hypergraph, It represents a binary tuple consisting of a set of nodes and a set of edges. Each node is assigned attributes such as position, pose, and velocity, and each edge is assigned attributes such as contact probability, relative pose change, and sliding velocity for subsequent temporal reasoning.

[0124] To map the initial risk signals to the edge level, the correlation of the edges is first aggregated using candidate contact cues, and the average value is then defined as follows:

[0125] ;

[0126] in Indicates time side Average degree of candidate association Indicates time Number of candidate contact points Representing an edge With the Each candidate at time Association weights For edge indexing, For candidate indexes, This represents the summation operation;

[0127] Then, by combining the global initial risk and the local contact semantics, the edge-level gating coefficients are obtained:

[0128] ;

[0129] in Indicates time side Gating strength, This means truncating the input to a closed interval. saturation function and and These are the weighting coefficients of the global risk term, the contact probability term, and the candidate association term, respectively. Indicates time Initial risk signals Indicates time side The probability of contact;

[0130] The final output is a ternary hypergraph. and its corresponding side-level gating sequence This serves as an external input to the risk-gated attention for subsequent contact with the Transformer.

[0131] In this specific embodiment, S4 specifically refers to:

[0132] Using the temporal hypergraph and edge-level gating coefficients as input, the contact Transformer groups edge-level features from consecutive time points into a fixed-length window based on timestamps and performs risk-gated temporal attention inference. Simultaneously, a dwell-state memory unit is initialized with zero values ​​and accumulates the continuous dwell time of each edge and records the most recent state during inference. To highlight contact cues related to instability, the gating attention weights are first calculated at the edge level, written as follows:

[0133] ;

[0134] in Indicates the edge At any moment To query candidate indexes within the window Attention weights Indicating in the candidate index dimension Normalized exponent mapping on The base attention log value obtained from query-key similarity represents... Indicates the gating scaling factor, Indicates candidate index The edge-level gating coefficient is derived from the risk gating external input in step S3. Represents edge index, Indicates time index, This represents the edge-time candidate index within the window;

[0135] Subsequently, the value vector is weighted and aggregated using this weight to obtain the edge-level temporal representation, which is written as follows:

[0136] ;

[0137] in Representing an edge At any moment Aggregated representation vector, This represents the summation over the candidate index set. Indicates candidate index The value vector;

[0138] The contact state determination based on the aggregated representation and classification head is written as follows:

[0139] ;

[0140] in Representing an edge At any moment Contact state This indicates the category corresponding to the maximum value. Represents category labels taken from a set Indicates stable support, Indicates slip, Indicates separation, Indicate category The discriminant scoring function;

[0141] Resident-State Memory Unit Cumulative Resident Duration Writing:

[0142] ;

[0143] in Representing an edge At any moment Duration of stay Indicates the duration of stay at the previous moment. The state indicator is set to 1 if the current state is the same as the previous state, otherwise it is set to 0. Indicates indicator functions, Indicates the time interval between adjacent frames. and These represent the timestamps of two adjacent frames;

[0144] The grip stability is measured by both temporal consistency and relative pose change.

[0145] ;

[0146] in Representing an edge At any moment grip stability, Represents the Sigmoid function, and These represent the weighting coefficients of the consistency term and the pose change term, respectively. Temporal consistency representing contact probability Representing an edge At any moment Contact probability, Represents absolute value operation, Indicates the magnitude of relative pose change, Representing an edge At any moment The relative pose parameter vector and The dimension of pose parameters is usually taken as Represents the Euclidean second norm;

[0147] Final output contact state sequence Contact and residence duration sequence With grip stability sequence This is for subsequent stability scoring and adaptive calculation of alarm thresholds.

[0148] In this specific embodiment, S5 specifically includes:

[0149] Using the contact state sequence, contact dwell time, grip stability, implicit 3D reconstruction results of the human body and assistive device, ground plane parameters, centroid ground projection, and initial support domain as inputs, stable support and slip contact points are first selected based on temporal criteria, and the support domain is updated accordingly. Then, the energy barrier, friction margin, and overturning margin are calculated under a differentiable physical model. Finally, the hazard rate and instability time are regressed and fused to obtain the stability index. The set of stable support and slip is defined as follows:

[0150] ;

[0151] in For time indexing, Indexing candidate contact points Associative mapping between candidate contact points and edges, For the edge At any moment The contact state and Indicates stable support. Indicates slip, To access the length of stay, To stabilize the screening threshold;

[0152] The slip velocity is obtained by the difference in ground position between adjacent frames:

[0153] ;

[0154] in For the first Each contact point at time Slip speed, Its three-dimensional position on the ground, For Euclidean second norm, For the time interval between adjacent frames, For timestamps;

[0155] Update the support domain using the convex hull based on the stable support set:

[0156] ;

[0157] in For a moment Update support domain, For convex hull operations;

[0158] Geometric margin is used to measure centroid stability:

[0159] ;

[0160] in The shortest distance from the centroid ground projection to the boundary of the support domain. For centroid ground projection, To support domain boundaries, For any point on the boundary, for The discrete-time derivative approximation;

[0161] Under a differentiable physical model, we take a simple and differentiable index:

[0162] ;

[0163] in For energy barrier, For energy coefficient, For friction margin, For the estimation of equivalent friction coefficient, For the slip penalty coefficient, For statistics on slip velocity, such as average or maximum value, For overturning margin, The critical distance for the supporting domain geometry;

[0164] By regressing risk based on grip stability, a kinematic approximation of the time to instability is given:

[0165] ;

[0166] in For risk rate, In order to reach the time of instability, For the Sigmoid function, For bias, For weight, For absolute value, The average value of grip stability over time is calculated by taking the average value of the grip edges. For regularity constants, For positive part function, This is the magnification factor for gripping.

[0167] The three physical margins are combined into a stability index:

[0168] ;

[0169] in For stability index, To integrate the weights, the final output is... and The contact dwell time and grip stability are used for the next step of adaptive alarm threshold and classification judgment.

[0170] In this specific embodiment, S6 specifically refers to:

[0171] The system receives the stability index, risk rate, and time to instability, and combines these with the contact dwell time and grip stability to perform adaptive graded alarms. Specifically, the stability index is denoted as... The risk rate is recorded as The time until instability is recorded as The time-based aggregation of contact and dwell time is recorded as follows: The moment of convergence of holding stability is denoted as ;

[0172] in indexed by time and and The time series values ​​of the contact edges or holding edges involved at the current moment are statistically aggregated to obtain, such as the mean or weighted mean;

[0173] To achieve adaptive threshold setting, a preset threshold is established, and linear correction and saturation cutoff based on reference values ​​are introduced. The stability index, hazard rate, and preset threshold for instability time are denoted as follows: and The corresponding adaptive alarm threshold is written as follows:

[0174] , ;

[0175] in and These are the stability index, the risk rate, and the adaptive alarm threshold for the time to instability. Indicates the real number Saturation cutoff to closed interval functions, and and and The upper and lower bounds of the three types of thresholds are respectively: To increase dwell time and reflect the strategy of "the longer the dwell time, the higher the stability index threshold", For reference value of stay duration and The grip gain is calculated as the risk rate and the time to instability, respectively, to reflect the strategy of "higher threshold for a more stable grip and lower threshold for a weaker grip". This is a reference value for grip stability;

[0176] Two types of trigger indicators are constructed in the hierarchical judgment:

[0177] and ;

[0178] in and These respectively represent "stability index triggering" and "joint triggering by risk rate and time to instability". This indicates that the indicator function takes 1 if the proposition within the parentheses is true, and 0 otherwise. This represents the logical AND operation;

[0179] Alarm levels are synthesized based on two types of indications:

[0180] ;

[0181] in The alarm level is set to 3 for the highest level, 2 for the high level, 1 for the medium level, and 0 for no alarm. The system maps the alarm level to the alarm method and outputs an alarm signal for local sound and light reminders and remote notifications.

[0182] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for fall detection and alarm in elderly Alzheimer's patients based on computer vision, characterized in that, Includes the following steps: S1. Acquire continuous video from at least one camera to obtain the original video frame sequence and timestamp, preprocess the frame sequence and estimate camera parameters; S2. Using the preprocessed video frame sequence and camera parameter estimation results as input, implicit 3D reconstruction is performed to obtain the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, and the ground plane parameters. Based on this, the candidate contact point set is determined, the centroid ground projection is calculated, and the initial support domain is formed to generate the initial risk signal. S3. The implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters and the candidate contact point set are used as inputs to form a human-assistive device-ground ternary hypergraph, and the initial risk signal is mapped into the edge-level gating coefficient to form the external input of risk gating attention. S4. Using the external inputs of the human-assisted device-ground ternary hypergraph and risk-gated attention as inputs, reasoning is performed in the temporal Transformer to output the contact state sequence, contact dwell time and grip stability. S5. Using the contact state sequence, contact dwell time and grip stability, as well as the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, ground plane parameters, centroid ground projection and initial support domain as input, the support domain is updated and its temporal evolution is determined. Stability is scored based on the differentiable physical model, the risk rate and the time to instability are regressed and fused to obtain the stability index. S6. Using stability index, risk rate, time to instability, contact dwell time and grip stability as inputs, set a preset alarm threshold and generate an adaptive alarm threshold. Based on the threshold classification, judge and output an alarm signal for prompting and notification.

2. The method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S1 is as follows: At least one camera continuously captures raw video frame sequences and timestamps, and binds the timestamps to the corresponding video frames to ensure time consistency. Geometric normalization is performed on the original video frame sequence. Geometric normalization includes at least one or more of lens distortion correction, cropping and scaling, and perspective correction to obtain a geometrically normalized video frame sequence. Lighting normalization is performed on the geometrically normalized video frame sequence. Lighting normalization includes at least one or more of white balance, brightness equalization, and contrast adjustment. Inter-frame stabilization and noise suppression are also performed to obtain the lighting normalized video frame sequence. Based on the video frame sequence after illumination normalization, the camera intrinsic and extrinsic parameters are estimated online. The camera intrinsic parameters include at least focal length, principal point and distortion coefficient, and the camera extrinsic parameters include at least pose and displacement, so as to obtain the camera parameter estimation results. The illumination-normalized video frame sequence and the camera parameter estimation results are combined into a preprocessed video frame sequence and camera parameter estimation results.

3. The method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S2 is as follows: The preprocessed video frame sequence and camera parameter estimation results are used as input to perform implicit 3D reconstruction to obtain implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of assistive devices, and ground plane parameters. Based on the implicit 3D reconstruction results of the human body and the implicit 3D reconstruction results of the walking aid, the candidate contact points of the foot, the tip of the cane, the wheel assembly of the walking aid and the wall handrail are located, and the contact position between the candidate contact points and the ground is determined according to the ground plane parameters to form a set of candidate contact points. In the implicit 3D reconstruction results of the human body, the human body centroid is estimated and projected onto the ground defined by the ground plane parameters to obtain the centroid ground projection; The initial support domain is formed based on the location of the foot and the candidate contact points on the ground. An initial risk signal is generated based on the distance from the centroid ground projection to the boundary of the initial support domain and the time derivative of that distance; Output implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of assistive devices, ground plane parameters, candidate contact point set, centroid ground projection, initial support domain and initial risk signal.

4. The method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S3 is as follows: The implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters, and the candidate contact point set are used as inputs; Based on the implicit 3D reconstruction results of the human body, key points of the human body are determined and a human body node set is formed. Based on the implicit 3D reconstruction results of the assistive device, key contact points of the assistive device are determined and an assistive device node set is formed. Based on the ground plane parameters, the ground area is defined and a ground node set is formed. Calculate at least one or more of the following attributes for each node: position, pose, and velocity; An edge set is established between the human node set, the assistive device node set, and the ground node set. The edge set includes at least an edge representing a gripping relationship, an edge representing a contact relationship, and an edge representing a relative pose and a sliding velocity. For each edge, at least one or more of the following attributes are calculated: contact probability, relative pose change, and sliding velocity. The initial risk signal is mapped according to the candidate contact points associated with each edge and normalized into edge-level gating coefficients within a bounded interval to form the external input of risk gating attention; The above-mentioned node set, edge set and their attributes are used to construct a human-assisted tool-ground ternary hypergraph, and the external input of the human-assisted tool-ground ternary hypergraph and risk gating attention is output.

5. A method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S4 is as follows: The external inputs of the human-assisted device-ground ternary hypermap and risk-gated attention are used as inputs; In the contact Transformer, the continuous time instances of the human-assisted device-ground ternary hypergraph are combined into a temporal input according to the timestamp, and the external input of risk-gated attention is used as the edge-level gating coefficient to adjust the temporal attention weight; The dwell-state memory unit is initialized with a zero value and used to accumulate the continuous state dwell time of each edge and record the most recent state during the temporal reasoning process; Based on gated temporal attention, a contact state sequence is output at the edge level, which includes at least stable support, slippage, and detachment. The contact dwell time is calculated based on the cumulative continuous duration of the dwell-state memory unit; The grip stability is calculated based on temporal consistency and relative pose change for the edges corresponding to the grip relationship.

6. A method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S5 is as follows: The contact state sequence, contact dwell time and grip stability, implicit 3D reconstruction results of the human body, implicit 3D reconstruction results of the walking aid, ground plane parameters, centroid ground projection and initial support domain are used as inputs. Contact points in a stable support state are selected based on the contact state sequence and contact dwell time to form a set of stable support contact points. Contact points in a sliding state are selected based on the contact state sequence, and the sliding velocity is calculated based on the ground position change of the candidate contact point set at continuous time intervals to form a sliding velocity sequence. The initial support domain is updated based on the location of the stable support contact point set on the ground to obtain the updated support domain. The boundary of the updated support domain is corrected over time based on the slip velocity sequence to obtain the time evolution of the support domain. Under the differentiable physical model, the energy barrier, friction margin and overturning margin are calculated based on the geometric relationship between the centroid ground projection and the updated support domain and the ground plane parameters. The risk rate and the time to instability are regressed based on the above indicators and their time changes, with holding stability as the weight. The stability index is obtained by integrating at least the energy barrier, friction margin, and overturning margin.

7. A method for fall detection and alarm in elderly Alzheimer's patients based on computer vision according to claim 1, characterized in that, Step S6 is as follows: Stability index, risk rate, time to instability, contact dwell time, and grip stability are used as inputs; Preset alarm thresholds are set based on stability index, hazard rate, and time to instability. The preset alarm thresholds are adaptively modified by contact dwell time and grip stability to form an adaptive alarm threshold. When grip stability increases, the alarm thresholds for hazard rate and time to instability are increased; when grip stability decreases, the alarm thresholds for hazard rate and time to instability are decreased; and when contact dwell time increases, the alarm threshold for stability index is increased. Under the adaptive alarm threshold, a graded judgment is made. When the stability index is lower than its adaptive alarm threshold, an alarm signal is generated. When the danger rate exceeds its adaptive alarm threshold and the time to instability is shorter than its adaptive alarm threshold, an alarm signal is generated. Different levels are associated with alarm signals based on different conditions. The generated alarm signal is output for prompting and notification.

8. A computer vision-based fall detection and alarm system for Alzheimer's patients, used to execute the computer vision-based fall detection and alarm method for Alzheimer's patients as described in any one of claims 1 to 7, comprising: The video acquisition module is used to acquire continuous video and generate timestamps, perform geometric normalization and illumination normalization on the video, and estimate camera parameters online. The reconstruction and calculation module is used to calculate the implicit 3D reconstruction results of the human body, the implicit 3D reconstruction results of the assistive device, the ground plane parameters, the candidate contact point set, the centroid ground projection, the initial support domain, and the initial risk signal, taking the preprocessed video frame sequence and camera parameter estimation results as input. The ternary hypergraph construction module is used to construct a human-assisted device-ground ternary hypergraph and map the initial risk signal into edge-level gating coefficients to form the external input of risk gating attention; The temporal reasoning module is used to employ risk-gated attention and set up dwell-state memory units in the temporal Transformer, and output the contact state sequence, contact dwell time and grip stability. The scoring module is used to update the support domain and calculate the energy barrier, friction margin and overturning margin, regress the risk rate and the time to instability and fuse them to obtain the stability index. The decision module is used to set a preset alarm threshold and generate an adaptive alarm threshold based on the contact dwell time and grip stability. It performs hierarchical judgment based on the preset alarm threshold and the adaptive alarm threshold and outputs an alarm signal.

Images