An old people fall prediction method and system based on physical constraint and experience pool fusion

Abstract

The application discloses an old people fall prediction method and system based on physical constraint and experience pool fusion, and belongs to the technical field of artificial intelligence health monitoring, which comprises the following steps: acquiring the position of human body joint points in the frame image of a target video, and arranging a skeleton sequence in time sequence based on time sequence; performing feature extraction on the skeleton sequence, acquiring a physical feature sequence and a physical heuristic signal sequence; performing feature coding on the skeleton sequence, and acquiring a high-risk fall feature in combination with the physical heuristic signal sequence; performing feature fusion on the physical feature sequence and the high-risk fall feature based on a cross attention mechanism, and acquiring a fusion feature; performing global pooling aggregation on the fusion feature, and performing classification and prediction to acquire a fall probability and an early warning time. The application uses a human body skeleton sequence as input, introduces physical knowledge related to human body imbalance to guide the learning process of deep features, retrieves historical high-risk fall features, and realizes a high prediction accuracy through a double-branch fusion algorithm.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence health monitoring technology, specifically involving a method and system for predicting falls in the elderly based on the fusion of physical constraints and experience pools. Background Technology

[0002] Fall prediction in the elderly is a research field of great significance and value. According to the World Health Organization, an estimated 684,000 people die from falls globally each year, with adults over 60 years of age accounting for the largest proportion of fatal falls. Therefore, a reliable fall prediction system is crucial for preventing falls in the elderly and providing emergency support. Most existing fall prediction technologies are based on RGB video frames, depth images, or other sensory data, combined with machine learning or deep learning algorithms to predict human fall risk. While these solutions can achieve early assessment of fall risk to some extent, they also have many limitations, such as low accuracy, difficulty in fully covering key dynamic changes before and after a fall, and difficulty in effectively utilizing similar risk information to assist in prediction. For fall prediction tasks in the elderly, the accuracy and timeliness of prediction are crucial. If a tendency to fall can be analyzed in advance, some time before a fall occurs, it can provide reaction time for introducing safety devices at critical moments, thereby significantly reducing the physical harm caused by falls.

[0003] Based on the above analysis, the following issues still need to be considered regarding fall prediction for the elderly: (1) Lack of prior physical knowledge: Current mainstream methods mainly rely on data-driven approaches to extract statistical features from video or sensor data. However, these methods often neglect the physical mechanisms in human kinematics and fail to fully utilize the biomechanical principles of human balance and stability (such as the analysis of critical states of human imbalance). This lack of prior physical knowledge results in the model lacking physical interpretability when predicting falls, making it difficult to distinguish between normal strenuous exercise and the risk of falls caused by imbalance.

[0004] (2) Fixed window segmentation is difficult to capture the entire fall process: A human fall is a dynamic process that includes multiple key time segments. Most existing methods use fixed window segmentation to extract time segments, but this method is often difficult to adaptively cover the key feature evolution before and after the fall, resulting in insufficient accuracy in capturing imbalance features.

[0005] (3) Lack of explicit retrieval mechanism for high-risk fall experience: Although traditional sequence prediction models (such as RNNs) have a certain memory function, for fall prediction tasks, these models lack an explicit mechanism to store and retrieve high-risk features or similar fall states that have appeared in the past. In real-world scenarios, falls are often accompanied by a series of similar warning signals, and traditional models have difficulty effectively utilizing historical experience to assist in current predictions.

[0006] In conclusion, designing a fall prediction algorithm for the elderly that combines prior physical knowledge of human biomechanics with the retrieval of high-risk features has significant social value and practical significance, and can safeguard the health and safety of the elderly. Summary of the Invention

[0007] Purpose of the invention: This application develops a method and system for predicting falls in the elderly based on the fusion of physical constraints and experience pools, aiming to solve the technical problems in the prior art.

[0008] Technical Solution: In a first aspect, embodiments of this application provide a method for predicting falls in the elderly based on the fusion of physical constraints and experience pools, including: The positions of human joints in the frame images of the target video are obtained, and a temporally continuous skeleton sequence is arranged based on the time order. Feature extraction is performed on the skeleton sequence to obtain the physical feature sequence and the physical heuristic signal sequence; The skeleton sequence is feature-encoded and combined with the physical heuristic signal sequence to obtain high-risk fall features; The physical feature sequence and the high-risk fall features are fused using a cross-attention mechanism to obtain fused features; The fused features are globally pooled and aggregated, then classified and predicted to obtain the fall probability and early warning time.

[0009] In some embodiments, the step of obtaining the physical feature sequence includes: The stability margin sequence of the current sequence frame is obtained based on the N-step captureability theory; Preliminary features are obtained based on the stability margin sequence, the position of the extrapolated centroid, and the Euclidean distance from the centroid to the ankle. The stability margin sequence is mapped to a binary discrete equilibrium state, and a difference operation is performed to obtain the switching time of the sequence frame. The initial features are divided into multiple block features based on the switching time to obtain the physical feature sequence.

[0010] In some embodiments, the positions of the human joints include the positions of the left hip joint and the right hip joint. The step of obtaining the stability margin sequence of the current sequence frame based on the N-step captureability theory includes: The average position of the left hip joint and the right hip joint is obtained as the position of the centroid; The centroid velocity and horizontal width are obtained based on the centroid position. Obtain the horizontal width of the human body's supporting foot, and combine the horizontal width of the center of gravity with the offset of the center of gravity position relative to the supporting foot; Obtain the pixel height of the human body in the frame image as a reference height; The stability margin sequence is obtained based on the centroid velocity, the reference height, and the offset. The characterization formula of the stability margin sequence includes: ; in, for The stability margin sequence of time-series frames; The normalized center-of-mass velocity; For reference height; This is the offset. The normalized centroid horizontal width, This is the normalized horizontal width of the support leg; These are learnable parameters.

[0011] In some embodiments, the step of obtaining the physical feature sequence further includes: Determine the standard value of the length of the physical feature sequence, which is used to characterize the standard value of the number of block features of the physical feature sequence; In response to the actual length of the physical feature sequence being greater than the length standard value, segments outside the range of the length standard value are deleted, and the physical feature sequence is updated; In response to the actual length of the physical feature sequence being less than the length standard value, missing block features are filled in, and the physical feature sequence is updated; Obtain the Boolean mask matrix, the representation formula of which includes: ; in, It is a Boolean mask matrix; A sorting index for the position of block features in the physical feature sequence, used to distinguish between true block features and filled features. This corresponds to the true segmented features. Corresponding characterization of filling features; The actual length of the physical feature sequence before padding; The filled physical feature sequence is aggregated into global features based on the Boolean mask matrix.

[0012] In some embodiments, the characterization formula for the physically inspired signal sequence includes: ; in, for The sequence of physical heuristic signals for time-series frames; for The stability margin sequence of time-series frames; The mean of the stability margin series; Standard deviation; Mapping for the Sigmoid activation function.

[0013] In some embodiments, the step of feature encoding the skeleton sequence and combining it with the physics-inspired signal sequence to obtain high-risk fall features includes: The skeleton sequence is mapped into a high-dimensional feature vector using a temporal convolutional network (TCN). Using the physical heuristic signal sequence as a soft mask, the high-dimensional feature vector is dynamically weighted to obtain gated features. The representation formula of the gated features includes: ; in, This is a gating feature; It is a high-dimensional feature vector; for The sequence of physical heuristic signals for time-series frames; The physical heuristic signal whose signal increase rate is greater than the rate threshold corresponding to the dangerous state is selected as the high-risk fall feature by the gating feature.

[0014] In some embodiments, the fall prediction method further includes updating the high-risk fall characteristics: Determine the capacity of the experience pool for high-risk fall features and the maximum lifecycle threshold for the high-risk fall features; In response to the existence of new high-risk fall features waiting to be stored, the aging time of the high-risk fall features in the experience pool is obtained; Delete the high-risk fall features whose aging time is greater than the maximum lifespan threshold, and store the new high-risk fall features.

[0015] In some embodiments, the step of fusing the physical feature sequence and the high-risk fall features based on a cross-attention mechanism to obtain the fused features includes: Perform layer normalization on the physical feature sequence to obtain the query vector; Perform layer normalization on the high-risk fall features to obtain key vectors and value vectors; Based on the query vector, the key vector, and the value vector, semantic relevance data of the physical feature sequence and the high-risk fall features are obtained through dot product attention. The representation formula of the semantic relevance data includes: ; in, This is semantically relevant data; For query vector; The key vector; It is a value vector; It is a normalized exponential function; For transpose; This is the scaling factor; The semantically relevant data is processed through residual connections and post-normalization operations to obtain the fused features.

[0016] In some embodiments, the steps of performing global pooling aggregation on the fused features, and classifying and predicting based on the pooling aggregation results to obtain the fall probability and early warning time include: The fused features after global average pooling are input into a multilayer perceptron (MLP) to obtain the fall probability. Determine the moment of impact between the human body and the ground, as well as the moment of fall warning; The advance warning time is obtained based on the impact time and the fall warning time, and the formula for representing the advance warning time includes: ; in, For the first A sample of falls Early warning time; For the first A sample of falls The corresponding moment of impact; For the first A sample of falls A fall warning moment; The sampling frame rate.

[0017] Secondly, embodiments of this application also provide an elderly fall prediction system based on the fusion of physical constraints and experience pools, comprising: The preprocessing module is used to obtain the positions of human joints in the frame images of the target video and arrange them into a temporally continuous skeleton sequence based on the time order. The physical module is used to extract features from the skeleton sequence to obtain a physical feature sequence and a physical heuristic signal sequence. An experience pool module is used to perform feature encoding on the skeleton sequence and combine it with the physical heuristic signal sequence to obtain high-risk fall features; A cross-attention module is used to perform feature fusion on the physical feature sequence and the high-risk fall features based on a cross-attention mechanism to obtain fused features. The prediction module is used to perform global pooling aggregation on the fused features, and then perform classification and prediction to obtain the fall probability and early warning time.

[0018] Beneficial Effects: Compared with existing technologies, this application provides a method for predicting falls in the elderly based on physical constraints and experience pool fusion. The method includes acquiring the positions of human joints in frame images of a target video and arranging a temporally continuous skeleton sequence based on time order; extracting features from the skeleton sequence to obtain physical feature sequences and physical heuristic signal sequences; encoding features from the skeleton sequence and combining them with the physical heuristic signal sequences to obtain high-risk fall features; fusing the physical feature sequences and high-risk fall features based on a cross-attention mechanism to obtain fused features; and performing global pooling aggregation on the fused features, followed by classification and prediction to obtain fall probability and early warning time. This application uses a vision-based human skeleton sequence as input, introduces physical knowledge related to human imbalance to constrain and guide the deep feature learning process, and adds an experience pool branch to retrieve historical high-risk fall features. The dual-branch fusion algorithm structure achieves higher prediction accuracy, greater prediction lead time, and faster system response speed, thereby minimizing injuries to the elderly caused by falls. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the steps of the elderly fall prediction method based on the fusion of physical constraints and experience pool provided in this application embodiment; Figure 2 A flowchart illustrating the steps of obtaining physical feature sequences in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment; Figure 3 A flowchart illustrating the steps for obtaining a stability margin sequence in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment; Figure 4A flowchart illustrating the steps for standardizing the size of the physical feature sequence in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment; Figure 5 A flowchart illustrating the steps for obtaining high-risk fall features in the elderly fall prediction method based on the fusion of physical constraints and experience pools, as provided in the application embodiment; Figure 6 A flowchart illustrating the steps of updating high-risk fall features in the elderly fall prediction method based on physical constraints and experience pool fusion provided in this application embodiment; Figure 7 A flowchart illustrating the steps for obtaining fused features in the elderly fall prediction method based on the fusion of physical constraints and experience pool provided in this application embodiment; Figure 8 A flowchart illustrating the steps for obtaining the fall probability and early warning time in the elderly fall prediction method based on the fusion of physical constraints and experience pool provided in this application embodiment; Figure 9 A module connection diagram of an elderly fall prediction system based on the fusion of physical constraints and experience pools provided in an embodiment of this application; Figure 10 Network structure diagram of the elderly fall prediction system based on the fusion of physical constraints and experience pool provided in the embodiments of this application; Figure 11 A flowchart of the algorithm for an elderly fall prediction system based on the fusion of physical constraints and experience pool provided in an embodiment of this application; Figure 12 A flowchart illustrating the algorithm for obtaining the skeleton sequence in the preprocessing module of the elderly fall prediction system based on the fusion of physical constraints and experience pools, provided in an embodiment of this application. Figure 13 A flowchart illustrating the algorithm for the physical module to acquire physical feature sequences in an elderly fall prediction system based on the fusion of physical constraints and experience pools, provided in an embodiment of this application. Figure 14 A flowchart illustrating the algorithm for obtaining high-risk fall features in the elderly fall prediction system based on the fusion of physical constraints and experience pools, provided in an embodiment of this application. Figure 15 A flowchart illustrating the algorithm for obtaining fused features in the cross-attention module of the elderly fall prediction system based on the fusion of physical constraints and experience pools, provided in an embodiment of this application. Figure 16 The flowchart shows the algorithm for obtaining the fall probability and early warning time in the prediction module of the elderly fall prediction system based on the fusion of physical constraints and experience pool provided in the embodiments of this application.

[0021] Figure labels: 10, Preprocessing module; 20, Physics module; 30, Experience pool module; 40, Cross-attention module; 50, Prediction module. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] This application provides a method for predicting falls in the elderly based on physical constraints and experience pool fusion. The method includes acquiring the positions of human joints in frame images of a target video and arranging a temporally continuous skeleton sequence; extracting features from the skeleton sequence to obtain physical feature sequences and physical heuristic signal sequences; encoding features from the skeleton sequence and combining them with the physical heuristic signal sequences to obtain high-risk fall features; fusing the physical feature sequences and high-risk fall features based on a cross-attention mechanism to obtain fused features; and performing global pooling aggregation on the fused features, followed by classification and prediction to obtain the fall probability and early warning time. This application uses a vision-based human skeleton sequence as input, introduces physical knowledge related to human imbalance to constrain and guide the deep feature learning process, and adds an experience pool branch to retrieve historical high-risk fall features. The dual-branch fusion algorithm structure achieves higher prediction accuracy, greater prediction lead time, and faster system response speed, thereby minimizing injuries to the elderly caused by falls.

[0024] Accordingly, embodiments of this application also provide a fall prediction system for the elderly based on the fusion of physical constraints and experience pools. Please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a module connection diagram of the elderly fall prediction system based on the fusion of physical constraints and experience pools provided in this application embodiment. The elderly fall prediction system based on the fusion of physical constraints and experience pools provided in this application embodiment includes: The preprocessing module 10 is used to obtain the position of human joint points in the frame image of the target video and arrange the temporally continuous skeleton sequence based on the time order. Physical module 20 is used to extract features from the skeleton sequence to obtain physical feature sequence and physical heuristic signal sequence; The experience pool module 30 is used to encode the skeleton sequence and combine it with the physical heuristic signal sequence to obtain high-risk fall features. Cross-attention module 40 is used to perform feature fusion on physical feature sequence and high-risk fall feature based on cross-attention mechanism to obtain fused features; Prediction module 50 is used to perform global pooling aggregation of fused features, and to classify and predict them to obtain the probability of falling and the early warning time.

[0025] In some embodiments, please refer to Figure 1 , Figure 10 and Figure 11 , Figure 1 This is a flowchart illustrating the steps of the fall prediction method for the elderly based on the fusion of physical constraints and experience pools provided in an embodiment of this application. Figure 10 This is a network structure diagram of the elderly fall prediction system based on the fusion of physical constraints and experience pools provided in the embodiments of this application. Figure 11 This is a flowchart of the algorithm for the elderly fall prediction system based on the fusion of physical constraints and experience pools provided in this application embodiment. The elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment is specifically implemented through steps 100 to 500: Step 100: Obtain the positions of human joints in the frame images of the target video and arrange them into a temporally continuous skeleton sequence based on time order.

[0026] Specifically, please refer to Figure 12 , Figure 12 This document presents an algorithm flowchart for obtaining the skeleton sequence in the preprocessing module of the elderly fall prediction system based on the fusion of physical constraints and experience pools, as provided in this application embodiment. This application processes the frame images of the input RGB raw video to extract the coordinates of human joints corresponding to each frame. The raw RGB video images contain irrelevant information such as background, lighting, and clothing; direct input would lead to noise in the model's learning. The preprocessing module extracts joints as input data, allowing the network to focus solely on the human posture and motion trajectory. The amount of data for joint coordinates is far less than that of RGB video frames, greatly improving the inference speed of the subsequent network and enabling it to run in real-time on edge devices. Simultaneously, the abstracted joint information does not retain facial or environmental information, perfectly avoiding the privacy leakage risk in home-based elderly care scenarios. Specifically, in this application embodiment, a 3-second video is input at a time, with a frame rate (FPS) of 30 frames per second, resulting in a total of 90 frames input at a time. for: ; in, This represents the input 90-frame video sequence. The system reads the current frame image and extracts its keypoint coordinates. For each frame image, MediaPipe generates the following keypoint coordinates: ; in, Indicates the first The set of key points corresponding to a frame image Represents the total number of joints, in this embodiment This represents the 33 nodes generated by MediaPipe. Represents the first The first frame of the image Two-dimensional coordinates of each joint point This represents the x-coordinate of the corresponding joint point in the image plane. This is represented as the ordinate of the corresponding joint point in the image plane. These joint points reflect the human body's position in space, providing necessary data support for subsequent skeletal modeling. Considering that each elderly person's body shape is different, or that their distance from the camera varies, this embodiment of the application performs coordinate normalization to improve the algorithm's generalization ability, setting all coordinate values ​​to a range of... The keypoint coordinates generated from all frames are organized in chronological order to form the overall input features. .

[0027] Step 200: Extract features from the skeleton sequence to obtain the physical feature sequence and the physical heuristic signal sequence.

[0028] In some embodiments, since the updating and storage of high-risk fall features requires receiving signals from the physics module, the module also outputs a physical heuristic signal sequence corresponding to the physical hazard score for each frame while generating the physical feature sequence. The generation of the physics-inspired signal sequence involves Z-score normalization and Sigmoid activation function mapping in the execution time dimension. The representation formula for the physics-inspired signal sequence includes: ; in, for The sequence of physical heuristic signals for time-series frames; for The stability margin sequence of time-series frames; The mean of the stability margin series; Standard deviation; Mapping for the Sigmoid activation function. Each element in this sequence Strictly corresponding to the video's number A frame, whose value intuitively represents the instantaneous probability limit of physical imbalance of the human body in that frame.

[0029] In some embodiments, please refer to Figure 2 and Figure 13 , Figure 2The flowchart illustrates the steps involved in obtaining the physical feature sequence in the elderly fall prediction method based on the fusion of physical constraints and experience pools, as provided in this embodiment of the application. Figure 13 This is a flowchart illustrating the algorithm for obtaining physical feature sequences in the physical module of the elderly fall prediction system based on the fusion of physical constraints and experience pools, as provided in this embodiment of the application. The method for obtaining physical feature sequences in this embodiment is specifically implemented through steps 210 to 240: Step 210: Obtain the stability margin sequence of the current sequence frame based on the N-step captureability theory.

[0030] Understandably, this application relates to skeletal sequences. Physical feature extraction is performed first; then, physical feature calculation is performed. This application selects the N-step captureability theory, which is related to human stability, to calculate the initial physical features. N-step captureability defines the ability of a human body to restore balance in a current state through no more than N steps of adjustment. To quantify the degree of risk, this application calculates the stability margin of the current state relative to the captureability boundary. Since the function required by this application is to detect falls, the application of the N-step captureability theory is simplified to one-step captureability (N=1). Due to the differences in body shape among individuals in reality, and in order to improve the generalization ability of the algorithm, this application standardizes the physical calculations through normalization. The core parameter of N-step captureability is the stability margin sequence. Its essence is to represent the shortest distance from the extrapolated center of mass to the boundary of the support surface, reflecting the safety margin of the human body to resist tipping.

[0031] In some embodiments, please refer to Figure 3 , Figure 3 The flowchart illustrates the steps for obtaining the stability margin sequence in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment. Specifically, the method for obtaining the stability margin sequence in this application embodiment is implemented through steps 211 to 215: Step 211: Obtain the average position of the left and right hip joints as the centroid position.

[0032] Step 212: Obtain the centroid velocity and centroid horizontal width based on the centroid position.

[0033] Step 213: Obtain the horizontal width of the human body's supporting foot, and combine it with the horizontal width of the center of gravity to obtain the offset of the center of gravity position relative to the supporting foot.

[0034] Step 214: Obtain the pixel height of the human body in the frame image as a reference height.

[0035] In some embodiments, the pixel size in a standard Kinect lens is 256×256, and the reference height is calculated by extracting the relative distance between the foot and the top joint.

[0036] Step 215: Obtain the stability margin sequence based on the centroid velocity, reference height, and offset.

[0037] In some embodiments, the characterization formula for the stability margin sequence includes: ; in, for The stability margin sequence of time-series frames; The normalized center-of-mass velocity; For reference height; The offset is used to characterize the horizontal distance of the body's center of gravity from the support point. The larger the offset, the greater the restoring torque required for the body to regain balance, and the higher the risk of instability. The normalized centroid horizontal width, This is the normalized horizontal width of the support leg; These are learnable parameters.

[0038] Step 220: Obtain preliminary features based on the stability margin sequence, the position of the extrapolated centroid, and the Euclidean distance from the centroid to the ankle.

[0039] In some embodiments, the formula for representing the position of the extrapolated centroid includes: ; in, Represents natural angular frequency. , It is the acceleration due to gravity. For reference height.

[0040] In some embodiments, and As the Euclidean distance from the center of mass to the left and right ankles, it directly quantifies the absolute distance of the center of mass from the left and right support boundaries, and both are important parameters constituting N-step catchability. The above main physical quantities are integrated to form a complete sequence of physical characteristics.

[0041] Step 230: Map the stability margin sequence to a binary discrete equilibrium state and perform difference operations to obtain the switching time of the sequence frame.

[0042] In some embodiments, the characterization formula for the binary discrete equilibrium state includes: ; in, Indicates the first The frame is in a stable state. Indicates the first The frame is in an unstable state and is at risk of falling. This represents the time length of the input sequence, in this embodiment. .

[0043] In some embodiments, the formula for representing the switching time includes: ; in, This is the switching time.

[0044] Furthermore, all the segmentation moments are determined by non-zero difference positions, and after integration, a set of segmentation moments is generated. : ; in, Indicates the first A dividing boundary, , This represents the number of detected state transitions.

[0045] Step 240: Divide the initial features into multiple block features according to the switching time and obtain the physical feature sequence.

[0046] Specifically, the initial features are determined based on the segmentation time. After dividing the features into N segments of different side lengths, a new sequence of physical features is formed. , For training batch size, For hidden layer dimensions, Here is the standard length value, where In the embodiments of this application, it is set to 8.

[0047] Understandably, due to the significant non-uniformity of human behavior sequences over time, the dynamic changes in everyday actions such as standing, walking, and sitting are usually relatively gradual. However, in critical stages of falls, such as imbalance, tipping, and impact, the body's posture and movement state undergo rapid abrupt changes within a short period. Using fixed-length slices can easily artificially fragment the critical fall process into different segments, thus disrupting the temporal continuity of the actions. Furthermore, a simple fixed-slice approach is highly likely to introduce excessive redundant information during stable phases, weakening the model's ability to focus on key abrupt changes. Therefore, based on these limitations, this invention introduces a Depatch mechanism to adaptively adjust the Patch length according to the degree of dynamic change in the sequence. Shorter slices are used for gradually changing everyday behaviors to prevent the model from focusing excessively on these activities, while longer slices are used for stages of rapid change leading to a fall to capture the complete process from imbalance to fall. This more fully represents the temporal evolution before and after a fall, improving the model's ability to extract and accurately identify fall warning features. The application of the Depatch mechanism mainly relies on stable margin sequences. This is because changes in stability margin reflect the changes in the human body's state during normal activities and falls due to imbalance, making it very reasonable to drive the Depatch mechanism through this.

[0048] In some embodiments, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the steps for standardizing the size of the physical feature sequence in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment. Specifically, the method for standardizing the size of the physical feature sequence in this application embodiment is implemented through steps 241 to 245: Step 241: Determine the standard value of the length of the physical feature sequence, which is used to characterize the standard value of the number of block features of the physical feature sequence.

[0049] Step 242: In response to the actual length of the physical feature sequence being greater than the length standard value, delete segments outside the length standard value range and update the physical feature sequence.

[0050] Step 243: In response to the actual length of the physical feature sequence being less than the length standard value, fill in the missing block features and update the physical feature sequence.

[0051] Specifically, due to the varying complexity of imbalance patterns among different samples, the actual length of the original physical feature sequence divided according to the split points varies. Differences exist between samples. To support overall batch parallel computation, in this invention example, it is necessary to align the variable-length and variable-short sequences to a standard length value. Because the variable-length sequence exceeds... Therefore, a truncation strategy is used for alignment, specifically when... At that time, only the first one is retained. Each segment. For shorter sequences containing normal daily actions, a padding strategy is used, specifically when... When the zero vector is used, the remaining positions are filled.

[0052] Step 244: Obtain the Boolean mask matrix based on the length standard value.

[0053] In some embodiments, the representation formula of the Boolean mask matrix includes: ; in, It is a Boolean mask matrix; A sorting index for the position of block features in the physical feature sequence, used to distinguish between true block features and filled features. This corresponds to the true segmented features. Corresponding characterization of filling features; The actual length of the physical feature sequence before filling.

[0054] Understandably, since physical feature sequences with actual lengths less than the standard length value are padded to the same length, a Boolean mask matrix is ​​generated to avoid the zero values ​​introduced by the padding into noise in subsequent attention calculations.

[0055] Step 245: Aggregate the filled physical feature sequence into global features based on the Boolean mask matrix.

[0056] Specifically, before being placed into the classification head, a Boolean mask matrix is ​​used to aggregate physical feature sequences whose actual length is less than the standard length value into global features. The main approach is to use a mask-based weighted average, averaging only the effective block features.

[0057] Understandably, the embodiments of this application calculate physical quantities such as stability margin, centroid coordinates, and centroid velocity, and simultaneously apply an adaptive slicing mechanism to capture the complete fall process, thereby outputting the corresponding physical feature sequence and generating a physical score sequence. Most traditional deep learning methods rely on data-driven learning of fall states. These methods typically neglect the physical mechanisms in human kinematics and fail to fully utilize the biomechanical principles of human balance and stability (such as the critical state analysis of human imbalance). This lack of prior physical knowledge results in a lack of physical interpretability in the model's fall prediction, making it difficult to distinguish between normal vigorous movement and fall risks caused by imbalance. This application introduces N-step captureability from biomechanics and explicitly transforms it into a physical calculation formula, providing physical interpretability. Furthermore, traditional fixed-window segmentation methods easily lose the complete fall process; therefore, this invention designs an adaptive slicing mechanism, DePatch, to achieve adaptive segmentation, ensuring that the final generated physical features have complete physical action semantics.

[0058] Step 300: Encode the skeleton sequence for features and combine it with the physical heuristic signal sequence to obtain high-risk fall features.

[0059] In some embodiments, please refer to Figure 5 and Figure 14 , Figure 5 The flowchart illustrates the steps involved in obtaining high-risk fall features in the elderly fall prediction method based on the fusion of physical constraints and experience pools, as provided in the application embodiment. Figure 14 This is a flowchart illustrating the algorithm for obtaining high-risk fall features in the experience pool module of the elderly fall prediction system based on the fusion of physical constraints and experience pool provided in this application embodiment. The method for obtaining high-risk fall features in this application embodiment is specifically implemented through steps 310 to 330: Step 310: Map the skeleton sequence into a high-dimensional feature vector using a temporal convolutional network (TCN).

[0060] Step 320: Use the physical heuristic signal sequence as a soft mask to dynamically weight the high-dimensional feature vector and obtain gated features.

[0061] In some embodiments, the characterization formula for the gated feature includes: ; in, This is a gating feature; It is a high-dimensional feature vector; It is a physical heuristic signal.

[0062] Step 330: Select physical heuristic signals with a signal increase rate greater than a rate threshold corresponding to dangerous states as high-risk fall features through gating features.

[0063] Understandably, when older adults are engaged in daily activities, at this time... Frames with characteristics close to 0 are considered low-risk by the gating network and are therefore automatically discarded from the experience pool. However, when imbalances or severe instability occur, An exponential surge, where the rate of increase of the signal exceeds the rate threshold, indicates that the current frame has a high-risk fall characteristic. The corresponding characteristic will be retained with greater weight and written into the experience pool first.

[0064] In some embodiments, please refer to Figure 6 , Figure 6 The flowchart illustrates the steps for updating high-risk fall features in the elderly fall prediction method based on the fusion of physical constraints and experience pools provided in this application embodiment. Specifically, the method for updating high-risk fall features in this application embodiment is implemented through steps 331 to 333: Step 331: Determine the capacity of the experience pool for high-risk fall features and the maximum lifecycle threshold for high-risk fall features.

[0065] Step 332: In response to the existence of new high-risk fall features waiting to be stored, obtain the aging time of high-risk fall features in the experience pool.

[0066] Step 333: Delete high-risk fall features whose aging time is greater than the maximum lifespan threshold, and store new high-risk fall features.

[0067] Understandably, regarding the storage and update strategy for high-risk fall characteristics, the experience pool is constructed as a fixed-capacity pool. The FIFO (First-In, First-Out) queue is used. The FIFO mechanism follows a strict first-in, first-out principle, primarily using a queue structure to facilitate updates to the experience pool. Specifically, when a physical heuristic triggers a write condition, the experience pool begins preparing to store high-risk fall features. When the experience pool is full, the latest high-risk fall feature frames generated subsequently are pushed to the tail of the queue, while the earliest feature frames are removed. This is the core principle behind the application of the FIFO first-in, first-out mechanism in the experience pool portion of this embodiment.

[0068] In continuous long-term video streams, if only a simple FIFO queue is used, historical high-risk fall features from a relatively long time ago, if not overwritten by new features, will remain in the pool for an extended period. In this case, the historical high-risk fall features stored in the experience pool are highly likely to cause contextual misleading predictions of current normal actions. To ensure that the features in the experience pool always maintain a strong recent correlation with the current state, this application innovatively introduces a timeliness mechanism on top of the FIFO update mechanism, mainly to forcibly clear historical experience pool features from a long time ago.

[0069] Specifically, a timestamp is assigned to each frame of features stored in the experience pool: ; in, The capacity of the experience pool is set to 10 in this embodiment, meaning it can store 10 high-risk fall characteristics. The current system time is also set to... Define the maximum lifespan threshold of the experience pool as: Before each storage wait, the system proactively calculates the aging time of features in the pool. And introduce a binary time mask function. To perform a hard cleanup, the main process is as follows: ; Essentially, it represents the retention / deletion flag of the features currently stored in the experience pool. This represents the first [item] stored in the experience pool. One feature. When the retention time of a feature exceeds When the feature's retention time is less than or equal to zero, its mask is reset to zero. At this point, expired feature frames will be forcibly initialized to zero vectors and removed from the experience pool. When the mask remains at 1, it is still considered a valid feature and is retained in the experience pool.

[0070] Based on this storage and update mechanism, an experience pool feature sequence representing high-risk fall characteristics is ultimately generated. .

[0071] Furthermore, step 4 is as follows Figure 6 As shown. The cross-attention module mainly implements the cross-attention mechanism to process physically guided physical feature sequences. and the experience pool feature sequence representing high-risk fall characteristics This process involves fusion to generate a combined feature that integrates physical properties and historical experience. .

[0072] Step 400: Based on the cross-attention mechanism, perform feature fusion on the physical feature sequence and high-risk fall features to obtain fused features.

[0073] In some embodiments, please refer to Figure 7 and Figure 15 , Figure 7 The flowchart illustrates the steps involved in obtaining fused features in the elderly fall prediction method based on the fusion of physical constraints and experience pools, as provided in this embodiment of the application. Figure 15The flowchart illustrates the algorithm for obtaining fusion features in the cross-attention module of the elderly fall prediction system based on the fusion of physical constraints and experience pools, as provided in the embodiments of this application. The method for obtaining fusion features in the embodiments of this application is specifically implemented through steps 410 to 440: Step 410: Perform layer normalization on the physical feature sequence to obtain the query vector.

[0074] Step 420: Perform layer normalization on the high-risk fall features to obtain the key vector and value vector.

[0075] Step 430: Based on the query vector, key vector, and value vector, obtain semantic relevance data of physical feature sequences and high-risk fall features through dot product attention.

[0076] In some embodiments, the representation formula for semantic relevance data includes: ; in, This is semantically relevant data; For query vector, , ; For key vectors, , ; For value vectors, , ; It is a normalized exponential function; For transpose; This is a scaling factor used to suppress the dot product value from becoming too large as the feature dimension increases, preventing the Softmax output from becoming too sharp, thereby improving training stability. The matrix multiplication in the numerator... The inner product similarity between the current physical action and each historical experience frame was calculated. (This is followed by a series of seemingly unrelated sentences.) After dimensional softmax normalization, the model can adaptively focus on the high-risk fall features that best match the current action, and use these as weights for empirical values. We perform weighted aggregation to extract the most effective high-risk fall features for the current state from the experience pool.

[0077] Step 440: Process semantically relevant data through residual connection and post-normalization operations to obtain fused features.

[0078] Understandably, in order to incorporate experience-enhanced features without disrupting the original action topology extracted from the physical branches, this application employs residual connections and post-normalization to complete feature fusion, generating more distinctive fused features. : .

[0079] Step 500: Perform global pooling aggregation on the fused features, and then classify and predict them to obtain the fall probability and early warning time; Specifically, this application first performs global average pooling on the fused features to compress the time dimension to complete the aggregation, and finally feeds them into the multilayer perceptron (MPL) classifier to output the final fall probability.

[0080] In some embodiments, please refer to Figure 8 and Figure 16 , Figure 8 The flowchart illustrates the steps involved in obtaining the fall probability and early warning time in the elderly fall prediction method based on the fusion of physical constraints and experience pools, as provided in this embodiment of the application. Figure 16 This is a flowchart illustrating the algorithm for obtaining the fall probability and early warning time in the elderly fall prediction system based on the fusion of physical constraints and experience pools, as provided in this application embodiment. The method for obtaining the fall probability and early warning time in this application is specifically implemented through steps 510 to 530: Step 510: Input the fused features after global average pooling into the multilayer perceptron (MLP) to obtain the fall probability.

[0081] In some embodiments, the formula for representing the probability of falling includes: ; in, The probability of falling; It is the Sigmoid activation function. This represents the global average pooling operation. The representative performs classification in a multilayer perceptron (MLP) and outputs... This indicates the probability of a fall occurring within the current time window.

[0082] Furthermore, to demonstrate the algorithm's predictive capabilities in detail, this application adds the average early warning time as a core indicator of predictive ability. Please refer to steps 520 to 530, which specifically refer to determining the actual impact time between the human body and the ground within a complete sequence of real fall video clips. Suppose that the network outputs fall warnings when multiple consecutive frames exceed the high-risk prediction threshold. Through the actual impact moment and fall warning moments Get early warning time: Step 520: Determine the moment of impact between the human body and the ground, and the moment of fall warning.

[0083] Step 530: Obtain the advance warning time based on the impact time and fall warning time.

[0084] In some embodiments, the formula for representing the advance warning time includes: ; in, For the first A sample of falls The advance warning time, if the warning lags behind the occurrence of the fall, then Recorded as 0; For the first A sample of falls The corresponding impact time, , This represents the total number of samples from the falls. For the first A sample of falls A fall warning moment; The sampling frame rate is 30 in this application.

[0085] Furthermore, in the entirety On a test set of fall samples, the average early warning time The calculation formula is: ; in, The unit of measurement is milliseconds (ms). The larger the value, the earlier the model can issue an alert before substantial harm occurs, and the more buffer time is reserved for subsequent physical intervention.

[0086] Understandably, this application relies on existing artificial intelligence and deep learning technologies to describe the limitations and problems of existing methods for predicting falls in the elderly, and proposes corresponding solutions. The specific solutions for each problem are as follows:

[0087] (1) The accuracy of fall prediction is not high: Falling is not just a visual change in posture, but also a process in which the energy boundary of the human dynamic system is broken. Attention mechanisms driven solely by appearance features are often easily misled by large-scale normal movements (such as bending over or sitting down quickly), resulting in a very high false alarm rate. Based on this, this invention combines N-Step Capturability theory to extract physical features, which can effectively improve the accuracy of fall prediction and explains the predictive effectiveness of this invention from a physical perspective.

[0088] (2) Lack of effective dynamic capture capability: The transition from steady-state walking to unbalanced fall is a non-linear transient process with an unpredictable duration. Traditional fixed segmentation window mechanisms are prone to semantic corruption, making it difficult for the model to accurately capture the features of the fall. To capture complete fall process information, this invention combines an adaptive patch mechanism with N-step captureability theory to generate physical modules, thereby achieving accurate and complete capture of fall precursor signals.

[0089] (3) Lack of explicit retrieval mechanism for high-risk fall experience: In real-world scenarios, falls are often accompanied by a series of similar warning signals, making it difficult for traditional models to effectively utilize historical experience to assist current predictions. Based on this challenge, this invention proposes an experience pool module with the function of storing high-risk fall risk features. The algorithm can retrieve information from the current input frame while also referring to historical high-risk fall features to assist in judgment, thereby effectively improving the effectiveness of fall prediction.

[0090] In summary, this application addresses the shortcomings of existing fall prediction technologies in terms of accuracy, dynamic capture capabilities, and explicit retrieval mechanisms, providing a more accurate and efficient fall prediction algorithm for the elderly. By implementing this invention, the stability boundary of human posture can be effectively quantified, and an adaptive patching mechanism can be combined to accurately capture dynamic features before and after a fall. Simultaneously, an experience pool is created to explicitly store and retrieve historical high-risk patterns. Through the effective fusion and prediction of dual-stream features, the physical interpretability and robustness of fall prediction are significantly improved. This invention provides valuable time to introduce safety devices in the very short time before a fall, thereby minimizing the harm caused by falls.

[0091] This application has provided a detailed description of a method and system for predicting falls in the elderly based on the fusion of physical constraints and experience pools, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for predicting falls in the elderly based on the fusion of physical constraints and experience pools, characterized in that, include: The positions of human joints in the frame images of the target video are obtained, and a temporally continuous skeleton sequence is arranged based on the time order. Feature extraction is performed on the skeleton sequence to obtain the physical feature sequence and the physical heuristic signal sequence; The skeleton sequence is feature-encoded and combined with the physical heuristic signal sequence to obtain high-risk fall features; The physical feature sequence and the high-risk fall features are fused using a cross-attention mechanism to obtain fused features; The fused features are globally pooled and aggregated, then classified and predicted to obtain the fall probability and early warning time.

2. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools as described in claim 1, characterized in that, The steps to obtain the physical feature sequence include: The stability margin sequence of the current sequence frame is obtained based on the N-step captureability theory; Preliminary features are obtained based on the stability margin sequence, the position of the extrapolated centroid, and the Euclidean distance from the centroid to the ankle. The stability margin sequence is mapped to a binary discrete equilibrium state, and a difference operation is performed to obtain the switching time of the sequence frame. Based on the switching time, the preliminary features are divided into multiple block features to obtain a physical feature sequence.

3. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools as described in claim 2, characterized in that, The positions of the human joints include the positions of the left hip joint and the right hip joint. The steps for obtaining the stability margin sequence of the current sequence frame based on the N-step captureability theory include: The average position of the left hip joint and the right hip joint is obtained as the position of the centroid; The centroid velocity and horizontal width are obtained based on the centroid position. Obtain the horizontal width of the human body's supporting foot, and combine the horizontal width of the center of gravity to obtain the offset of the center of gravity position relative to the supporting foot; Obtain the pixel height of the human body in the frame image as a reference height; The stability margin sequence is obtained based on the centroid velocity, the reference height, and the offset. The characterization formula of the stability margin sequence includes: ； in, for The stability margin sequence of time-series frames; The normalized center-of-mass velocity; For reference height; This is the offset. The normalized centroid horizontal width, This is the normalized horizontal width of the support leg; These are learnable parameters.

4. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools as described in claim 2, characterized in that, The steps for obtaining the physical feature sequence also include: Determine the standard value of the length of the physical feature sequence, which is used to characterize the standard value of the number of block features in the physical feature sequence; In response to the actual length of the physical feature sequence being greater than the length standard value, segments outside the range of the length standard value are deleted, and the physical feature sequence is updated; In response to the actual length of the physical feature sequence being less than the length standard value, missing block features are filled in, and the physical feature sequence is updated; Obtain the Boolean mask matrix, the representation formula of which includes: ； in, It is a Boolean mask matrix; A sorting index for the position of block features in the physical feature sequence, used to distinguish between true block features and filled features. This corresponds to the true segmented features. Corresponding characterization of filling features; The actual length of the physical feature sequence before padding; The filled physical feature sequence is aggregated into global features based on the Boolean mask matrix.

5. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools as described in claim 1, characterized in that, The characterization formula for the physically inspired signal sequence includes: ； in, for The sequence of physical heuristic signals for time-series frames; for The stability margin sequence of time-series frames; The mean of the stability margin series; Standard deviation; Mapping for the Sigmoid activation function.

6. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools according to claim 1, characterized in that, The steps of feature encoding the skeleton sequence and combining it with the physical heuristic signal sequence to obtain high-risk fall features include: The skeleton sequence is mapped into a high-dimensional feature vector using a temporal convolutional network (TCN). Using the physical heuristic signal sequence as a soft mask, the high-dimensional feature vector is dynamically weighted to obtain gated features. The representation formula of the gated features includes: ； in, This is a gating feature; It is a high-dimensional feature vector; for The sequence of physical heuristic signals for time-series frames; The physical heuristic signal whose signal increase rate is greater than the rate threshold corresponding to the dangerous state is selected as the high-risk fall feature by the gating feature.

7. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools as described in claim 1, characterized in that, The fall prediction method also includes updating the high-risk fall characteristics: Determine the capacity of the experience pool for high-risk fall features and the maximum lifecycle threshold for the high-risk fall features; In response to the existence of new high-risk fall features waiting to be stored, the aging time of the high-risk fall features in the experience pool is obtained; Delete the high-risk fall features whose aging time is greater than the maximum lifespan threshold, and store the new high-risk fall features.

8. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools according to claim 1, characterized in that, The steps for fusing the physical feature sequence and the high-risk fall features based on the cross-attention mechanism to obtain the fused features include: Perform layer normalization on the physical feature sequence to obtain the query vector; Perform layer normalization on the high-risk fall features to obtain key vectors and value vectors; Based on the query vector, the key vector, and the value vector, semantic relevance data of the physical feature sequence and the high-risk fall features are obtained through dot product attention. The representation formula of the semantic relevance data includes: ； in, This is semantically relevant data; For query vector; The key vector; It is a value vector; It is a normalized exponential function; For transpose; This is the scaling factor; The semantically relevant data is processed through residual connections and post-normalization operations to obtain the fused features.

9. The method for predicting falls in the elderly based on the fusion of physical constraints and experience pools according to claim 1, characterized in that, The steps of performing global pooling aggregation on the fused features, and classifying and predicting based on the pooling aggregation results to obtain the fall probability and early warning time include: The fused features after global average pooling are input into a multilayer perceptron (MLP) to obtain the fall probability. Determine the moment of impact between the human body and the ground, as well as the moment of fall warning; The advance warning time is obtained based on the impact time and the fall warning time, and the formula for representing the advance warning time includes: ； in, For the first A sample of falls Early warning time; For the first A sample of falls The corresponding moment of impact; For the first A sample of falls A fall warning moment; The sampling frame rate.

10. A fall prediction system for the elderly based on the fusion of physical constraints and experience pools, characterized in that, include: The preprocessing module (10) is used to obtain the position of human joint points in the frame image of the target video and arrange the temporally continuous skeleton sequence based on the time order. The physical module (20) is used to extract features from the skeleton sequence to obtain a physical feature sequence and a physical heuristic signal sequence. The experience pool module (30) is used to perform feature encoding on the skeleton sequence and combine it with the physical heuristic signal sequence to obtain high-risk fall features; A cross-attention module (40) is used to perform feature fusion on the physical feature sequence and the high-risk fall features based on the cross-attention mechanism to obtain fused features; The prediction module (50) is used to perform global pooling aggregation on the fused features, and to classify and predict them to obtain the probability of falling and the early warning time.

Images