A smart old-age fall prediction and alarm method and device based on human posture estimation

Abstract

This invention relates to the fields of computer vision and pattern recognition, and discloses a smart elderly care fall prediction and alarm method and device based on human pose estimation. The method includes: acquiring video streams to extract skeleton key points and confidence scores; masking low-confidence nodes and inputting them into a spatiotemporal graph convolutional network, using Lagrange kinematics equations as a penalty term to reconstruct the coordinates of occluded parts; merging skeleton key points and reconstructed parts to construct a human key point set and calculating two-dimensional approximate centroid coordinates, extracting the bottom point of the feet to generate a supporting polygon; extracting the instantaneous velocity of the approximate centroid and combining it with the natural frequency of an inverted pendulum to obtain extrapolated centroid coordinates; and outputting a prediction signal and triggering an alarm when the extrapolated centroid exceeds the boundary of the supporting polygon and the vertical falling velocity component exceeds a threshold. This application overcomes the defect of broken bottom-level data chains in high-occlusion scenarios, accurately filters false positives caused by actions, and significantly advances the fall warning threshold to the initial response window of instability.

Description

Technical Field

[0001] This application relates to the fields of computer vision and pattern recognition, specifically to a method and device for fall prediction and alarm based on human posture estimation in indoor smart elderly care scenarios with obstructed view and complex lighting environments. Background Technology

[0002] In elderly care facilities and home-based elderly care environments, the severity of injuries caused by falls is highly correlated with rescue response time and the amount of cushioning preparation at the moment of impact. Conventional vision-based human pose estimation techniques mainly rely on action classification networks built from video sequences, which are essentially pattern matching. Because they require complete video frames of the fall process to output a judgment result, these methods have significant lag and are post-event detections, failing to provide an earlier time window for intervention mechanisms.

[0003] To overcome the aforementioned lag, existing improvement schemes use the descent speed of key points on the human body for early warning. However, such schemes suffer from significant misjudgments and failure rates in indoor elderly care scenarios. First, when the person being cared for performs normal controlled actions such as bending over to pick up a dropped item or leaning forward to sit down, the descent speed of their upper body key points often triggers false alarms, leading to decreased usability. Second, furniture and facilities in indoor environments frequently obscure the lower limb key points of the person, causing the 2D pose extraction algorithm to lose the coordinates of the feet and knees. This breaks the subsequent chain of geometric feature calculations based on the whole-body skeleton, resulting in missed detections or suspension of the detection process. Summary of the Invention

[0004] To address the technical problems of existing visual fall detection methods, which easily lose key lower limb nodes in occluded environments and cannot effectively distinguish between active forward leaning movements and passive imbalance fall movements due to reliance on speed features, resulting in delayed fall detection and a high likelihood of false alarms or missed alarms, this application provides a smart elderly care fall prediction and alarm method and device based on human posture estimation.

[0005] The specific technical solution of this application is as follows:

[0006] A smart elderly care fall prediction and alarm method based on human posture estimation includes the following steps:

[0007] Acquire indoor surveillance video streams and extract key points of the human skeleton and corresponding confidence scores from each frame of the image;

[0008] The skeleton key points with confidence scores below a set threshold are masked and input into the spatiotemporal graph convolutional network module. The coordinates of the occluded parts are reconstructed by using the motion trajectories of connected nodes combined with the Lagrange kinematic equations as a penalty term.

[0009] The key points of the skeleton and the coordinates of the occluded parts are merged to construct a set of human body key points. The two-dimensional approximate centroid coordinates of the human body are obtained by weighted calculation based on the set of human body key points. The bottom key points of the feet are extracted from the set of human body key points and the ground support polygon is generated by convex hull algorithm.

[0010] The instantaneous velocity is calculated by extracting the two-dimensional approximate centroid coordinates of multiple consecutive frames, and the extrapolated centroid coordinates are obtained by combining them with the preset natural frequency of the inverted pendulum model.

[0011] Determine whether the extrapolated centroid coordinates on the horizontal axis exceed the boundary of the supporting polygon. If they do, and the vertical falling velocity component of the two-dimensional approximate centroid coordinates is greater than the set velocity threshold, output an irreversible fall prediction signal and trigger the alarm actuator.

[0012] In a preferred embodiment, skeleton keypoints with confidence scores below a set threshold are masked and input into a spatiotemporal graph convolutional network module. The reconstructing of the occluded part coordinates using the motion trajectories of connected nodes combined with the Lagrange kinematics equation as a penalty term includes: extracting the motion trajectory sequence of connected nodes from previous frames for skeleton keypoints with confidence scores below the set threshold; inputting the motion trajectory sequence and the masked skeleton features into the spatiotemporal graph convolutional network module for spatiotemporal feature extraction; and during the training of the spatiotemporal graph convolutional network module, constructing a physical loss function containing the Lagrange kinematics equation as a penalty term to constrain the predicted output space of the occluded part coordinates.

[0013] In a preferred embodiment, during the training of the spatiotemporal graph convolutional network module, a physical loss function containing the Lagrange kinematics equations is constructed as a penalty term. The predicted output space constraining the coordinates of the occluded part includes: establishing the kinetic energy and potential energy terms of the human body based on a multi-link rigid body model; calculating the difference between the kinetic energy and potential energy terms to obtain the Lagrange quantity; constructing the residual terms of the Lagrange kinematics equations based on the partial derivatives of the Lagrange quantity with respect to the generalized coordinates and generalized velocity; superimposing the residual terms into the basic prediction loss function of the spatiotemporal graph convolutional network module to generate the physical loss function; and updating the network weight parameters of the spatiotemporal graph convolutional network module based on the physical loss function using the backpropagation algorithm to output the coordinates of the occluded part.

[0014] In a preferred embodiment, the process of extracting the bottom foot key points from the human body key point set and generating the ground support polygon using a convex hull algorithm includes: extracting the coordinates of both ankle joints and both toes from the human body key point set as the bottom foot key points; mapping the coordinates of both ankle joints and both toes to a two-dimensional image plane; calculating the minimum convex polygon region enclosed by the coordinates of both ankle joints and both toes in the two-dimensional image plane using a convex hull algorithm; and configuring the minimum convex polygon region as the support polygon.

[0015] In a preferred embodiment, the extrapolation of the centroid coordinates is calculated by combining the preset natural frequency of the inverted pendulum model with the following steps: extracting the horizontal component of the instantaneous velocity; calculating the ratio of the horizontal component to the natural frequency of the inverted pendulum model to obtain the horizontal position extrapolation; and summing the horizontal position component of the two-dimensional approximate centroid coordinates with the horizontal position extrapolation to generate the extrapolated centroid coordinates.

[0016] In a preferred embodiment, before obtaining the horizontal position extrapolation by calculating the ratio of the horizontal component to the natural frequency of the inverted pendulum model, the method further includes the step of determining the natural frequency of the inverted pendulum model: calculating the Euclidean distance between the hip joint coordinates and the ankle joint coordinates in the human body key point set; configuring the Euclidean distance as the equivalent pendulum length of the inverted pendulum; obtaining the pre-configured gravitational acceleration constant; calculating the quotient of the gravitational acceleration constant and the equivalent pendulum length of the inverted pendulum; performing a square root operation on the quotient, and configuring the result as the natural frequency of the inverted pendulum model.

[0017] In a preferred embodiment, before acquiring the indoor surveillance video stream, the method further includes: acquiring multimodal image data through an infrared module and a visible light module; performing time synchronization registration and spatial alignment processing on the multimodal image data; and inputting the aligned data into an edge computing module to generate an indoor surveillance video stream.

[0018] In a preferred embodiment, outputting an irreversible fall prediction signal and triggering an alarm actuator includes: generating a fall warning event record; attaching a timestamp to the fall warning event record; and sending the irreversible fall prediction signal to the alarm actuator via a local communication bus to execute the corresponding physical action command.

[0019] An electronic device includes: a memory, a processor, and a computer program stored in the memory; when the processor executes the computer program, it implements the steps in the smart elderly care fall prediction and alarm method based on human posture estimation provided in this application.

[0020] A computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the smart elderly care fall prediction and alarm method based on human posture estimation provided in this application.

[0021] The technical solution provided in this application brings the following beneficial effects: By introducing dynamic stability assessment mechanisms such as extrapolation of the centroid, false positive alarms caused by actions such as controlled bending are accurately filtered out, overcoming the limitations of single kinematic velocity threshold judgment; by introducing a spatiotemporal graph convolution feature completion mechanism with physical equation constraints, the defect of the underlying data chain being broken due to pad occlusion in indoor environments is successfully eliminated; through deep collaborative image restoration and motion state recognition, the system advances the time threshold for fall judgment from after physical contact to between 300 and 500 milliseconds in the early stage of instability, gaining an extremely valuable response window for safety intervention. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall system structure provided in an embodiment of the present invention.

[0023] Figure 2 This is a flowchart of the method provided in an embodiment of the present invention.

[0024] Figure 3 This is a logical structure block diagram of the edge computing module provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0026] Figure 1 This is a schematic diagram of the overall system structure provided in this embodiment of the invention. In this embodiment, the fall prediction and alarm system is deployed in an indoor elderly care setting. The system's hardware platform mainly includes an image acquisition module 101, an edge computing module 102, an alarm actuator 103, and a power supply regulator module 104 that provides power to all the above modules. The modules are interconnected via a communication bus 105.

[0027] The image acquisition module 101 employs a visible light and infrared dual-mode camera with a global shutter mechanism. This camera integrates an 850nm infrared illumination array. By using near-infrared light with a wavelength of 850nm, it ensures sufficient illumination in low-light environments at night without producing glaring red light, thus avoiding disturbing the subject's sleep. The image acquisition module 101 has a photosensitive sensor that automatically cuts off the visible light channel and activates infrared imaging mode when the ambient illuminance is detected to be below 5 lux. The final video stream output to the bus has a frame rate of 30 frames per second and a high-definition resolution of 1920 x 1080 pixels.

[0028] The edge computing module 102 employs a micro-industrial computer hardware with an embedded dedicated neural network processor unit, achieving an independent computing power of 6 trillion operations per second. Due to privacy concerns related to the monitored objects, the edge computing module 102 directly acquires the video stream from the image acquisition module 101 via a real-time streaming protocol and the local communication bus 105, performing closed-loop parsing locally on the device. This avoids the privacy risks and network transmission delays associated with uploading the original video stream to the cloud.

[0029] The predicted signal processed by the edge computing module 102 is transmitted to the alarm actuator 103. The alarm actuator 103 includes a local audible and visual alarm and a corresponding relay drive circuit, used to generate a physical prompt upon receiving a high-level fall detection signal. It should be noted that the above hardware environment is merely the physical carrier supporting the implementation of this application; any terminal device with corresponding data acquisition and logic operation capabilities can be used as an alternative implementation.

[0030] Figure 2 This is a flowchart of a method provided in an embodiment of the present invention. Figure 2 As shown, the method of this application is mainly executed by the software mounted on the edge computing module 102, and specifically includes the following steps:

[0031] S201: Acquire and preprocess indoor surveillance video stream. Specifically, the image acquisition module 101 acquires multimodal image data through its internal infrared and visible light modules. Due to slight parallax in the physical positions of the dual-modal sensors, the system performs time synchronization registration and spatial alignment processing on the multimodal image data before inputting the aligned data into the edge computing module to eliminate imaging deviations in the dual-camera system.

[0032] Human skeleton key points are extracted (S202). After receiving the video stream, the edge computing module 102 uses a lightweight, high-resolution network structure to perform frame-by-frame computation on the video stream. The system extracts multiple skeleton key points of the human body that conform to the standard human key point data format specification from each frame image, and outputs a confidence score calculated by the algorithm model for each extracted key point.

[0033] Specifically, the system performs feature extraction logic, that is, obtains a feature set including node coordinates and confidence scores. In a preferred embodiment, the output logic of the keypoint features is expressed by the following formula:

[0034]

[0035] Where P represents the complete set of two-dimensional human key points extracted from a single frame; i represents the index number of the human key point, which in this embodiment uses the standard 17-point model, so the value range is from 1 to 17; xi This represents the coordinate value of the i-th node along the horizontal axis of the image pixel plane; y i This represents the coordinate value of the node in the direction perpendicular to the coordinate axis of the image pixel plane; c i This represents the confidence score of the model's prediction of the key point's location, ranging from 0 to 1. By utilizing this lightweight network, the system can meet the real-time prediction requirement of at least 30 frames per second while ensuring the accuracy of key point coordinate localization.

[0036] The coordinates of the occluded part S203 are reconstructed. This embodiment uses a combination of physics and algorithms to reconstruct the rationality of missing data. To filter out false detections or body parts completely obscured by furniture, the system sets a confidence score threshold. In this embodiment, the threshold is set to 0.5. The system iterates through each node in set P in real time, and for nodes with a confidence score lower than the set threshold (i.e., ... The key points of the skeleton are masked. Masking means preventing the low-confidence coordinates from being passed to subsequent modules, setting them to zero and marking them as pending completion. Subsequently, the system inputs the masked data into the spatiotemporal graph convolutional network module.

[0037] Combination Figure 3 The logical structure diagram of the edge computing module shown is divided into four cascaded pipelines. The pose extraction pipeline 301 corresponds to the extraction of human skeleton key points S202 mentioned above. The feature completion pipeline 302 corresponds to the graph convolution operation inside the reconstruction of the coordinates of the occluded part S203. When performing graph convolution, the feature completion pipeline 302 extracts the motion trajectory sequence of the connected nodes in the preceding 15 frames in the time dimension for the skeleton key points that are determined to be occluded. Subsequently, the system uses this motion trajectory sequence and the skeleton features after masking the current frame as joint inputs and feeds them into the spatiotemporal graph convolutional network for spatiotemporal feature extraction.

[0038] Specifically, to ensure that the reconstructed human joints do not violate biological anatomy, a physical loss function containing the Lagrange kinematics equations is constructed as a penalty term during the training of the spatiotemporal graph convolutional network module. Specifically, the system establishes kinetic and potential energy terms for the human body based on a multi-link rigid body model; then, it calculates the difference between the kinetic and potential energy terms to obtain the Lagrange quantity; further, it solves for the partial derivatives of the Lagrange quantity with respect to generalized coordinates and generalized velocities to construct the residual terms of the Lagrange kinematics equations. The system then superimposes this residual term onto the basic prediction loss function of the spatiotemporal graph convolutional network module to generate the physical loss function. Based on this physical loss function, the system updates the network weight parameters of the spatiotemporal graph convolutional network module using the backpropagation algorithm, causing the model to converge. In the application inference phase, the system reconstructs the coordinates (x') based on the output of the trained network.i ,y' i By embedding physical equation constraints into the network loss layer, the reconstructed limb coordinates are required to conform to the limit constraints of the joint's physical motion rotation radius, thus avoiding the limb misalignment and stretching errors often generated by purely data-driven algorithms.

[0039] Calculate the approximate two-dimensional centroid coordinates and the supporting polygon S204. Figure 3 The system uses pipeline 303 for dynamics calculation. It merges the original high-confidence skeleton keypoints with the reconstructed coordinates of the occluded parts output from S203 to construct a complete set of human body keypoints. Based on this set, the system extracts human body state features.

[0040] Specifically, the system executes the logic of two-dimensional approximate centroid calculation, which involves assigning corresponding mass weight coefficients to each part node in the human body keypoint set, extracting the two-dimensional coordinate values ​​corresponding to each part node, multiplying the two-dimensional coordinate values ​​by the mass weight coefficients, and then summing the results to obtain the two-dimensional approximate centroid coordinates. In a preferred embodiment, this calculation logic is implemented through the following formula:

[0041]

[0042] Among them, X com This represents the approximate centroid horizontal pixel coordinates of the human body on the two-dimensional image plane; x i w represents the horizontal pixel coordinates of the i-th joint after merging and correction; i This represents the pre-configured mass weight coefficients corresponding to each part of the body. Based on the statistical characteristics of human biomechanics, in this embodiment, the weight constant assigned to the torso nodes is set to 0.45, and the weights of the limb nodes decrease according to their hierarchical distribution. Similarly, the vertical component Y is calculated. com Together they form the two-dimensional approximate centroid coordinates X. com .

[0043] Meanwhile, the edge computing system extracts the coordinates of both ankle joints and both toes from the human body keypoint set as the bottom-level keypoints of the feet; and maps these four bottom-level points onto the two-dimensional image plane; subsequently, it calculates the smallest convex polygon region that these four coordinate points can enclose on the image plane using a geometric convex hull algorithm, and configures this smallest convex polygon region as the supporting polygon of the ground. It is worth emphasizing that, due to the physical completion effect of the pre-reconstructed occluded part coordinates S203, even if the object's feet are occluded by the bed, the system can still use the reconstructed coordinates to maintain the complete calculation of the supporting polygon, completely solving the persistent problem of algorithm hang-up caused by lower limb occlusion.

[0044] Calculate the extrapolated centroid coordinates S205. This step performs the fusion of the biodynamic model. The system extracts the two-dimensional approximate centroid coordinates of 5 consecutive frames and calculates its instantaneous velocity V in the image plane coordinate system using a first-order difference method. com To describe the tendency of the human body to become unstable, the system models the human body as an inverted pendulum and solves the problem by combining the natural frequency of the inverted pendulum model.

[0045] Specifically, before calculating the center-of-mass offset, the system determines the natural frequency. The system calculates the Euclidean distance between the hip joint coordinates and ankle joint coordinates in the extracted keypoint set and configures it as the equivalent pendulum length of an inverted pendulum. The system obtains a pre-configured gravitational acceleration constant and calculates the object's inherent vibration characteristics through the relationship between the gravitational constant and the equivalent pendulum length. In a preferred embodiment, this calculation logic is implemented using the following formula:

[0046]

[0047] in, denoted by , g represents the natural frequency of the inverted human pendulum model; g represents the pre-acquired gravitational acceleration constant; and L represents the estimated equivalent pendulum length of the inverted human pendulum.

[0048] Subsequently, the system executes extrapolation logic to extract the instantaneous velocity V. com Calculate the horizontal component and its relationship with the natural frequency of the inverted pendulum model. The ratio is used to obtain the horizontal position extrapolation, and the actual approximate centroid coordinates are finally summed with this extrapolation. In a preferred embodiment, the calculation logic for obtaining the extrapolated state is implemented through the following formula:

[0049]

[0050] Among them, X xcom X represents the horizontal coordinate of the extrapolated centroid obtained from the system prediction; com V represents the approximate centroid coordinates of this frame; com This is the instantaneous horizontal velocity vector obtained through multi-frame difference. Let be the natural frequency of the inverted pendulum. By introducing this formula, the algorithm elevates a single position state into a trend representation that includes velocity and inertia.

[0051] Perform multiple decision-making and output alarm S206. Figure 3 The warning judgment pipeline 304 in the system performs the core warning logic judgment: determining the extrapolated centroid coordinate X. xcom Whether the position value on the horizontal axis exceeds the boundary between the calculated two-dimensional approximate centroid coordinates and the supporting polygon generated by supporting polygon S204; and simultaneously determine the two-dimensional approximate centroid coordinates X.com The system checks whether the vertical falling velocity component exceeds a set velocity threshold. In this embodiment, the set velocity threshold is configured as 0.5 meters per second.

[0052] The system only determines a true loss of control when both of the above conditions are met simultaneously: the extrapolated center of gravity indicates that the object is losing its steady state and will detach from the foot support area, and the falling speed exceeds the critical point. At this time, the edge computing module outputs an irreversible fall prediction signal and triggers the alarm execution mechanism 103 within a delay of less than 20 milliseconds. In specific implementation, the processor generates a fall warning event record; adds a digital timestamp to the fall warning event record; and then sends a high-level signal or action command through the local communication bus 105 to trigger the relay drive circuit, causing the audible and visual alarm to activate. In an extended embodiment, the system also encapsulates the recorded message through a built-in network communication interface and remotely sends it to the care center terminal via the transmission control protocol / Internet protocol. By adopting this extrapolated center of gravity determination mechanism, when the monitored object performs a controlled bending motion, although there is a velocity displacement in the upper body, its velocity inertia is within a safe range, so that the calculated extrapolated center of gravity is always bound within the foot support polygon range, thereby accurately filtering out false positive alarms. This mechanism will advance the time threshold for detecting falls from the moment of impact to between 300 and 500 milliseconds in the early stages of instability.

[0053] Based on the same concept, this application also provides an electronic device and a computer-readable storage medium. This device is typically used as a field edge terminal in an elderly care environment. The electronic device includes a memory, a processor, and a computer program stored in the memory. The memory is configured to store multiple instruction codes and neural network weight model parameters for the processor to run. When the processor executes the computer program in the memory, it can call corresponding hardware sensors and other modules to implement all the steps in the smart elderly care fall prediction and alarm method based on human posture estimation provided in this application. It should be noted that the electronic device can also be a distributed processing system composed of multiple computing nodes connected via an intranet or bus; this application does not limit the physical form of the processor.

[0054] In summary, this application utilizes deep learning neural networks to capture spatiotemporal features of images and introduces dynamic physical equations to strongly constrain and complete occluded nodes, constructing a high-fidelity dynamic motion model. Based on this, the system calculates the relative boundary states of the extrapolated centroid, which includes inertial variables, and the ground-supporting polygon, completely abandoning the approach of simply relying on velocity detection or post-hoc shape matching. This not only solves the pain point of algorithm failure in high-occlusion scenarios but also distinguishes between normal bending and stooping movements and passive weightlessness falls, achieving a non-contact transient fall warning with an ultra-low false alarm rate.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A smart elderly care fall prediction and alarm method based on human posture estimation, characterized in that, include: Acquire indoor surveillance video streams and extract the skeletal key points of the human body and the corresponding confidence score c from each frame of the image. i ; For the confidence score c i The skeleton key points below the set threshold are masked and input into the spatiotemporal graph convolutional network module. The coordinates of the occluded parts are reconstructed by using the motion trajectory of the connected nodes combined with the Lagrange kinematic equation as a penalty term. The coordinates of the skeleton key points and the occluded parts are merged to construct a human body key point set. Based on the weighted calculation of the human body key point set, the two-dimensional approximate centroid coordinates X of the human body are obtained. com The bottom key points of the feet are extracted from the set of human body key points and the ground support polygon is generated by convex hull algorithm. Extract the two-dimensional approximate centroid coordinates X from multiple consecutive frames. com Calculate instantaneous velocity V com Combined with the pre-set natural frequency of the inverted pendulum model The extrapolated centroid coordinates X were calculated. xcom ; Determine the extrapolated centroid coordinates X xcom Does it extend beyond the boundary of the supporting polygon on the horizontal axis? If it does, and the two-dimensional approximate centroid coordinate X... com If the vertical falling velocity component exceeds the set velocity threshold, an irreversible fall prediction signal is output and the alarm actuator is triggered.

2. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 1, characterized in that, The confidence score c i The skeleton key points below a set threshold are masked and input into a spatiotemporal graph convolutional network module. The coordinates of the occluded parts are reconstructed using the motion trajectories of connected nodes combined with the Lagrange kinematic equations as penalty terms, including: Regarding the confidence score c i For the skeleton key points that are below the set threshold, extract the motion trajectory sequence of connected nodes from previous frames; The motion trajectory sequence and the skeleton features after masking are input into the spatiotemporal graph convolutional network module for spatiotemporal feature extraction. During the training of the spatiotemporal graph convolutional network module, a physical loss function containing the Lagrange kinematic equations is constructed as the penalty term to constrain the predicted output space of the occluded part coordinates.

3. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 2, characterized in that, During the training process of the spatiotemporal graph convolutional network module, a physical loss function containing the Lagrange kinematics equations is constructed as the penalty term, and the predicted output space constrained by the coordinates of the occluded part includes: The kinetic and potential energy terms of the human body are established based on a multi-link rigid body model. The Lagrangian quantity is obtained by calculating the difference between the kinetic energy term and the potential energy term; The residual terms of the Lagrange kinematic equations are constructed based on the partial derivatives of the Lagrange quantities with respect to the generalized coordinates and the generalized velocities. The residual terms are superimposed onto the basic prediction loss function of the spatiotemporal graph convolutional network module to generate the physical loss function; Based on the physical loss function, the network weight parameters of the spatiotemporal graph convolutional network module are updated using the backpropagation algorithm, and the coordinates of the occluded part are output.

4. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 1, characterized in that, The step of extracting the bottom foot key points from the human body key point set and generating the ground support polygon using the convex hull algorithm includes: The coordinates of the two ankle joints and the coordinates of the two toes are extracted from the set of human body key points as the bottom key points of the foot. Map the coordinates of the two ankle joints and the coordinates of the two toes to a two-dimensional image plane; The minimum convex polygon region enclosed by the coordinates of the two ankle joints and the coordinates of the two toes in the two-dimensional image plane is calculated using the convex hull algorithm. Configure the minimum convex polygon region as the supporting polygon.

5. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 1, characterized in that, The inverted pendulum model combined with the pre-set natural frequency The extrapolated centroid coordinates X were calculated. xcom include: Extract the instantaneous velocity V com The horizontal component; Calculate the horizontal component and the natural frequency of the inverted pendulum model. The ratio is used to obtain the horizontal position extrapolation. The two-dimensional approximate centroid coordinates X com The horizontal position component is summed with the horizontal position extrapolation to generate the extrapolated centroid coordinate X. xcom .

6. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 5, characterized in that, The calculation of the horizontal component and the natural frequency of the inverted pendulum model Before obtaining the horizontal position extrapolation value from the ratio, the method also includes determining the natural frequency of the inverted pendulum model. Specifically, it includes: Calculate the Euclidean distance between the hip joint coordinates and the ankle joint coordinates in the set of human body key points; Configure the Euclidean distance as the equivalent pendulum length of an inverted pendulum; Obtain the pre-configured gravitational acceleration constant; Calculate the quotient of the gravitational acceleration constant and the equivalent length of the inverted pendulum; Perform a square root operation on the quotient and set the result as the natural frequency of the inverted pendulum model. .

7. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 1, characterized in that, Before acquiring the indoor surveillance video stream, the following is also included: Multimodal image data is acquired through infrared and visible light modules; Perform temporal synchronization registration and spatial alignment processing on the multimodal image data; The aligned data is input into the edge computing module to generate the indoor monitoring video stream.

8. The smart elderly care fall prediction and alarm method based on human posture estimation according to claim 1, characterized in that, The mechanism that outputs an irreversible fall prediction signal and triggers an alarm includes: Generate a record of fall warning events; Add a timestamp to the record of the fall warning event; The predicted irreversible fall signal is sent to the alarm actuator via the local communication bus to execute the corresponding physical action command.

9. An electronic device, characterized in that, include: Memory, processor, and computer program stored in said memory; When the processor executes the computer program, it implements the steps in the smart elderly care fall prediction and alarm method based on human posture estimation as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the smart elderly care fall prediction and alarm method based on human posture estimation as described in any one of claims 1 to 8.

Images