A method, device and medium for fall detection based on long short-term memory network and rule engine

Abstract

This invention discloses an age-friendly fall detection method, device, and medium based on a long short-term memory network and a rule engine. The method includes: acquiring a video stream and constructing a feature matrix; inputting the model to output a preliminary fall probability; identifying suspected events and extracting physical features; determining the fall based on rules of falling speed and aspect ratio; and dynamically adjusting and distributing the parameter set. This application can accurately intercept actions that are easily confused by the elderly, reducing the system's false alarm rate, and can improve the reliability and maintenance agility of the detection system by hot-updating the recognition boundary through the distribution of configuration text.

Description

Technical Field

[0001] This application relates to the field of computer vision and artificial intelligence application technology, belonging to the G06V category, and specifically refers to an age-friendly fall detection method, device and medium based on a long short-term memory network model and physical rule verification engine applied in the scenarios of smart elderly care platforms and intelligent medical companion robots. Background Technology

[0002] In smart elderly care institutions or home care environments, real-time monitoring of the activity status of the elderly is a core means of ensuring their safety. Non-contact fall detection technology based on computer vision dominates this field. Due to the decline in physiological function in the elderly, their daily high-frequency movements easily exhibit spatial displacement characteristics and temporal changes similar to real falls. Existing fall detection methods generally use end-to-end deep learning networks for video sequence action classification. However, a single deep learning model is a black-box architecture, lacking explicit constraints on the common sense of human physical movement. If it cannot accurately distinguish between the elderly's self-controlled age-appropriate high-frequency movements and real fall events, the system will push a large number of false alarms, leading to alert fatigue among medical staff. Furthermore, full retraining mechanisms are time-consuming and labor-intensive, and updates to new model parameters may cause a decline in the accuracy of detection baselines, resulting in poor iterative agility. Summary of the Invention

[0003] This application provides an age-friendly fall detection method, device, and medium based on long short-term memory networks and rule engines.

[0004] This application provides an age-friendly fall detection method based on a long short-term memory network and a rule engine, including: acquiring a video stream, extracting the coordinate sequence of key points of the human skeleton of the target object in the corresponding consecutive frames of the time window, and constructing a feature matrix by normalizing the distance between the center of the shoulders and the center of the hips.

[0005] The feature matrix is ​​input into a pre-trained long short-term memory network model, which outputs an initial fall probability.

[0006] If the initial probability of falling is greater than the preset confidence threshold, the time window is marked as a suspected event and the rule judgment logic is triggered to extract physical features including the aspect ratio of the bounding box and the falling speed of the center of gravity.

[0007] Determine whether the physical feature satisfies the condition that the centroid falling speed is lower than a preset falling speed threshold and the bounding box aspect ratio is greater than a preset aspect ratio threshold. If the condition is met, the suspected event is determined to be a normal action.

[0008] Otherwise, output a fall alarm;

[0009] Based on the historical test set, a search algorithm is used to dynamically adjust the parameter set, which includes the falling speed threshold and the aspect ratio threshold, and the updated parameter set is sent to the rule determination logic.

[0010] Furthermore, the step of extracting the coordinate sequence of key points of the human skeleton of the target object in the corresponding consecutive frames of the time window, and constructing a feature matrix by normalizing the distance between the center of the shoulders and the center of the hips, includes: locating the target object in the current frame and generating a bounding box;

[0011] Extract the two-dimensional pixel coordinates of key points including the head, shoulders, elbows, wrists, hips, knees, and ankles within the bounding box;

[0012] Calculate the center point coordinates of the shoulders and the center point coordinates of the hips, and use the Euclidean distance between the center point coordinates of the shoulders and the center point coordinates of the hips as a reference scale;

[0013] The two-dimensional pixel coordinates of all the key points are normalized by division relative to the reference scale to obtain a relative coordinate sequence, which is then assembled into the feature matrix according to the spatial distribution.

[0014] Furthermore, the step of performing a division normalization operation on the two-dimensional pixel coordinates of all the key points relative to the reference scale to obtain a relative coordinate sequence and assembling it into the feature matrix according to the spatial distribution is configured as follows: selecting the center point coordinates of the two hips as the origin of the coordinate system, performing a translation operation on each of the key points to obtain the relative reference coordinates;

[0015] Calculate a first ratio of the horizontal relative coordinates of each key point to the reference scale, and a second ratio of the vertical relative coordinates of each key point to the reference scale;

[0016] The first ratio and the second ratio are concatenated in a fixed order according to the key point type to form a one-dimensional feature vector of the current frame;

[0017] According to the chronological order of consecutive frames within the time window, the one-dimensional feature vectors corresponding to each frame are stacked row by row to generate the two-dimensional feature matrix, wherein the number of rows of the feature matrix is ​​equal to the number of frames contained in the time window, and the number of columns is equal to twice the number of corresponding key points.

[0018] Furthermore, the extraction of physical features including the aspect ratio of the bounding box and the falling velocity of the center of mass includes: extracting the start frame and the end frame from the time window corresponding to the suspected event;

[0019] Obtain the initial bounding box corresponding to the start frame and the end bounding box corresponding to the end frame;

[0020] Calculate the ratio of the height to the width of the ending bounding box, and use the ratio as the aspect ratio of the bounding box;

[0021] Calculate the initial centroid longitudinal coordinate of the target object in the starting frame and the ending centroid longitudinal coordinate in the ending frame. Divide the difference between the ending centroid longitudinal coordinate and the initial centroid longitudinal coordinate by the time length of the time window to obtain the centroid falling velocity.

[0022] Furthermore, after obtaining the initial bounding box corresponding to the starting frame and the ending bounding box corresponding to the ending frame, the extraction of physical features including the aspect ratio of the bounding box and the centroid falling speed further includes: performing Gaussian smoothing on the edge pixels of the initial bounding box and the ending bounding box to generate smooth boundaries;

[0023] The Kalman filter algorithm is used to perform state estimation and filtering calculations on the bounding box parameters and centroid coordinates of each frame in the continuous frame sequence, and to obtain the smoothed centroid trajectory and the filtered bounding box sequence.

[0024] Traverse the height information of each data point in the smoothed centroid trajectory, and extract the first peak longitudinal coordinate and the second valley longitudinal coordinate corresponding to the highest point position;

[0025] Based on the extracted first peak longitudinal coordinate and second valley longitudinal coordinate, the initially extracted baseline data is replaced, and the initial centroid longitudinal coordinate and the final centroid longitudinal coordinate are re-established.

[0026] Furthermore, the step of inputting the feature matrix into a pre-trained long short-term memory network model and outputting a preliminary fall probability includes: inputting the feature matrix into the multi-layer gating structure of the long short-term memory network model sequentially according to time steps;

[0027] Extract the hidden layer state vector at the last moment, and input the hidden layer state vector into the fully connected layer for dimensionality reduction.

[0028] The dimensionality-reduced features are mapped to a continuous numerical range using a logistic regression function to obtain a confidence score that represents the probability of a fall event, and this confidence score is used as the initial fall probability.

[0029] Furthermore, the step of dynamically adjusting the parameter set containing the falling speed threshold and the aspect ratio threshold using a search algorithm based on the historical test set, and sending the updated parameter set to the rule determination logic, includes: collecting negative sample video data that cause false triggering in the online operating environment, and constructing an offline baseline library as the historical test set;

[0030] Construct a two-dimensional parametric grid search space consisting of multiple sets of alternative fall velocity thresholds and alternative aspect ratio thresholds;

[0031] Traverse each set of candidate parameter combinations in the two-dimensional parameter grid search space, and perform offline simulation matching tests on the historical test set;

[0032] The simulated false alarm rate and simulated false alarm rate corresponding to each set of candidate parameter combinations are statistically analyzed, and the optimal parameter combination that meets the preset evaluation conditions is selected as the updated parameter set.

[0033] Furthermore, the step of statistically analyzing the simulated false positive rate and simulated false negative rate corresponding to each set of candidate parameter combinations, and selecting the optimal parameter combination that meets the preset evaluation conditions as the updated parameter set, is configured as follows: establishing an evaluation function, wherein the evaluation function takes the simulated false positive rate and the simulated false negative rate as input variables and assigns corresponding first weight coefficients and second weight coefficients to perform weighted summation to calculate the comprehensive penalty score;

[0034] Among all the alternative parameter combinations that satisfy the requirement that the simulated false alarm rate is lower than the preset safety threshold, the combination with the lowest comprehensive penalty score is selected as the optimal parameter combination.

[0035] The optimal parameter combination is encapsulated into a standard format configuration text;

[0036] The standard format configuration text is pushed to the service node where the rule judgment logic is running online via hot update through the microservice configuration center, so that the service node can reload and apply the falling speed threshold and the aspect ratio threshold without downtime.

[0037] This application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned age-friendly fall detection method based on a long short-term memory network and a rule engine.

[0038] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned age-friendly fall detection method based on a long short-term memory network and a rule engine.

[0039] This application's embodiments employ a serial two-level decision-making architecture. A deep learning model provides high-sensitivity, high-confidence initial screening to locate suspected events, while a rule engine provides physical common-sense decision-making for interception. Based on the physical characteristics of spatiotemporal skeleton points, normal age-appropriate behaviors of the elderly can be effectively separated. By introducing an evaluation-driven development mechanism to construct a closed loop, the blocking boundary can be hot-updated without retraining the neural network. This solution ensures that the response time meets real-time monitoring requirements without significantly increasing computational overhead, while simultaneously reducing the false alarm rate caused by non-fall-like actions, thus improving the overall maintenance agility of the smart elderly care system. Attached Figure Description

[0040] Figure 1 This is a block diagram of the logical structure of the age-friendly fall detection system provided in the embodiments of this application.

[0041] Figure 2 This is a flowchart of the age-friendly fall detection method provided in the embodiments of this application.

[0042] Figure 3 This is a flowchart of a local method for extracting physical features provided in an embodiment of this application.

[0043] Figure 4 This is a block diagram of the logical structure of the electronic device provided in the embodiments of this application.

[0044] Explanation of reference numerals in the attached figures

[0045] In the diagram: 101-Perception and Acquisition Layer, 102-Inference and Calculation Layer, 103-Decision Filtering Layer, 104-Evaluation and Optimization Layer, 105-Video Data Decoding and Distribution Module, 106-Internal Communication Bus, 107-Alarm Message Queue System, 401-Processor, 402-Memory, 403-Communication Interface. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0047] In one embodiment, the system as a whole adopts a distributed topology design based on a microservice architecture. For example... Figure 1As shown, the age-friendly fall detection system is divided into a perception and acquisition layer 101, an inference and computation layer 102, a decision filtering layer 103, and an evaluation and optimization layer 104. Data interaction between these layers is achieved through bridging infrastructure. The perception and acquisition layer 101 is responsible for capturing visual data from the front-end environment. This layer is equipped with visual sensors from intelligent medical companion robots or wall-mounted fixed monitoring equipment. The video stream captured by the perception and acquisition layer 101 is pushed to the back-end via the video data decoding and distribution module 105. The inference and computation layer 102 deploys video parsing microservices and model prediction microservices, responsible for parsing images and running deep learning inference operations. The decision filtering layer 103 independently deploys a rule engine microservice instance to execute business logic judgments. The evaluation and optimization layer 104 is deployed in a background offline cluster for running test-driven verification tasks. Data packets flow between microservice components via an internal communication bus 106. The internal communication bus 106 uses a remote procedure call framework or a hypertext transfer protocol. When the decision filtering layer 103 confirms a genuine abnormal event, it sends an alarm command to the alarm message queue system 107. The alarm message queue system 107, acting as an asynchronous communication hub, distributes and pushes the event to the caregiver's mobile terminal application and the hospital's alarm display screen. This service decoupling logical topology ensures that front-end perception, high-load computing power inference, and back-end lightweight rule judgment do not block each other, maintaining the overall concurrent processing performance of the system.

[0048] In the operating environment of the above system architecture, such as Figure 2 As shown, this embodiment provides an age-friendly fall detection method based on a long short-term memory network and a rule engine. This method is mainly executed collaboratively by the processors and corresponding software algorithm modules in the inference computing layer 102 and the decision filtering layer 103. The method includes the following processing steps:

[0049] Step S201: Acquire the video stream, extract the coordinate sequence of human skeleton key points of the target object in the corresponding consecutive frames of the time window, and construct the feature matrix by normalizing the distance between the center of the shoulders and the center of the hips.

[0050] The video data decoding and distribution module 105 acquires the video stream captured by the perception acquisition layer 101 in real time. In this embodiment, the acquisition frame rate of the video stream is set to 15 frames per second. To construct the temporal sequence for motion analysis, continuous frame segments of length equal to a time window are extracted from the video stream. The time window length is defined as... A frame corresponds to a video length of 2 seconds in physical time. For each frame within a time window, an object detection algorithm is used to locate the target object in the current frame, and a bounding box is generated outside the target object. Within the bounding box, a pose estimation network is used to extract the 2D pixel coordinates of key points of human skeletal parts. The number of extracted skeletal key points is configured to be 18 or 25, and their data format follows the human skeleton annotation specifications of public context target datasets. The extracted key point locations include, but are not limited to, the head, both shoulders, both elbows, both wrists, both hips, both knees, and both ankles.

[0051] Because the physical distance between the surveillance camera and the target object changes dynamically, the pixel ratio of the target object in the image is subject to perspective scaling. To eliminate the interference caused by this pixel scale change, normalization processing is performed. The center point coordinates of both shoulders and both hips are calculated separately. The Euclidean distance between the center point coordinates of the shoulders and the center point coordinates of the hips is used as the reference scale for the current frame. The center point coordinates of the hips are selected as the origin of the coordinate system. An origin translation operation is performed on all extracted keypoints to obtain the relative reference coordinates of each keypoint. The first ratio of the horizontal relative reference coordinate of each keypoint to the reference scale, and the second ratio of the vertical relative reference coordinate of each keypoint to the reference scale are calculated. This series of division normalization operations is expressed as: the feature relative value equals the pixel displacement value divided by the shoulder-hip Euclidean distance.

[0052] The first ratio and the second ratio are concatenated one-dimensionally according to a pre-defined fixed order of key point types to form the frame-level feature vector of the current single frame image, denoted by the symbol F. t Representation. Based on the chronological order of consecutive frames within the time window, T frame-level feature vectors F are... t Stacking the rows generates a two-dimensional feature matrix. The number of columns in this feature matrix is ​​twice the number of corresponding keypoints, and the number of rows is equal to the number of frames T contained in the time window. The complete expression for the feature matrix is:

[0053]

[0054] In the matrix expression, the symbol F1 represents the one-dimensional normalized feature vector extracted from the first frame, F T This represents the one-dimensional normalized feature vector extracted in frame T. By extracting skeletal keypoints and constructing a two-dimensional matrix, redundant pixel information from the background environment is removed, while retaining the core kinematic parameters representing human posture and spatiotemporal displacement, thus reducing the number of parameters and computational overhead of subsequent deep learning networks.

[0055] Step S202: Input the feature matrix into the pre-trained long short-term memory network model and output the initial fall probability.

[0056] The constructed feature matrix is ​​sequentially input into the Long Short-Term Memory (LSTM) network model deployed in the inference computation layer 102, step by step. This model contains a multi-layered gating structure, utilizing the cyclical collaborative work of forget gates, input gates, and output gates to remember and pass on contextual dependencies in a continuous sequence of time frames. After forward propagation inference, the hidden layer state vector output from the last time step is extracted. This vector condenses the action evolution information within the entire time window T. The hidden layer state vector is input into a fully connected layer for dimensionality reduction, compressing the high-dimensional hidden feature vector into low-dimensional features through matrix multiplication. The dimensionality-reduced features are mapped to a continuous numerical range using a normalized exponential function or a sigmoid function. This numerical range is limited to 0 to 1, yielding a confidence score representing the probability of a fall. This confidence score is used as the initial fall probability for the current time window, denoted by the symbol P. fall express.

[0057] Step S203: If the initial probability of falling is greater than the preset confidence threshold, the time window is marked as a suspected event and the rule judgment logic is triggered to extract physical features including the aspect ratio of the bounding box and the falling speed of the center of mass.

[0058] Initial probability of falling P fall The output constitutes the initial screening mechanism for the black-box sequence model. The system will initially determine the fall probability P. fall Compared with the preset confidence threshold Numerical comparisons are performed. In this embodiment, the confidence threshold... The value is set to 0.65. The initial screening criteria for suspected fall events are stated as: determine whether the logical equation is satisfied. If the condition is met, the current processing time window is labeled as a suspected event, and a wake-up trigger command is sent to the rule engine microservice in the decision filtering layer 103 via the communication bus. If the condition is not met, the current sequence is discarded, and the system waits for the input of the next time window to save computing resources.

[0059] When the rule-based decision logic is triggered, the physical dimension difference between the state before the action occurs and the stable state of the action is calculated. The start and end frames of the action change are extracted from the time window corresponding to the suspected event. The initial bounding box corresponding to the start frame and the end bounding box corresponding to the end frame are obtained. The ratio of the height to the width of the end bounding box is calculated and used as the aspect ratio of the bounding box, denoted by the symbol R. The basic definition is expressed as follows: , where H is the height of the closing bounding box and W is the width of the closing bounding box.

[0060] The centroid coordinates of the torso are extracted using data from skeletal keypoints. The initial centroid longitudinal coordinates of the target object at the start frame and the final centroid longitudinal coordinates at the end frame are obtained. The absolute value of the difference between the final and initial centroid longitudinal coordinates is taken to obtain the absolute drop displacement in the image's vertical coordinate system. The absolute drop displacement is then removed based on the duration of the event. The instantaneous longitudinal velocity of the center of mass is calculated and denoted by V. center express.

[0061] By introducing an initial screening and wake-up mechanism, the system avoids performing complex physical displacement extraction calculations on every frame of a long video stream. Secondary feature extraction is only triggered in regions where the model provides high confidence.

[0062] Furthermore, to eliminate keyframe localization errors caused by detection algorithm jitter or object occlusion, a trajectory correction mechanism is introduced during the extraction of physical features. For example... Figure 3 As shown, this local extraction method includes the following subdivided steps:

[0063] Step S301: Perform Gaussian smoothing on the edge pixels of the initial bounding box and the ending bounding box to generate smooth boundaries.

[0064] Step S302: The Kalman filter algorithm is used to perform state estimation and filtering calculations on the bounding box parameters and centroid coordinates corresponding to each frame in the continuous frame sequence, obtaining the smoothed centroid trajectory and the filtered bounding box sequence. The Kalman filter uses the predicted state from the previous moment and the measured state from the current moment for residual iteration. The system state vector is defined as a set containing the centroid's longitudinal position and velocity. Based on the physical law that an object approximately conforms to inertial motion in a short time, noise caused by occlusion-induced positional jumps is filtered out.

[0065] Step S303: Traverse the height information of each data point in the smoothed centroid trajectory, and extract the first peak longitudinal coordinate and the second valley longitudinal coordinate of the corresponding highest point position.

[0066] Step S304: Based on the extracted first peak vertical coordinate and second valley vertical coordinate, replace the reference data corresponding to the first and last frames of the initially captured video, and re-establish the initial centroid vertical coordinate and the end centroid vertical coordinate for subsequent calculations.

[0067] By using filtering and peak-valley extreme value extraction techniques, we ensure that the displacement parameters involved in the fall velocity calculation reflect the physical boundary where the fall occurred, thereby improving the confidence level of the rule-based judgment.

[0068] In one implementation, the extracted physical feature includes the duration of low-level stillness. This feature is represented as... The calculation logic is as follows: Starting from the end frame of the stable state after the action ends, a sliding time window is opened backward to continuously monitor the variance change of the centroid trajectory in the vertical displacement after smoothing. When the variance value remains within the minimum threshold range and the centroid is located in the lower half of the image, the duration of this state is accumulated as the low-order static duration. This duration represents the continuous physical state of the target object after falling, losing the ability to move, and remaining on the ground.

[0069] Step S204: Determine whether the physical features meet the conditions that the falling speed of the center of mass is lower than the preset falling speed threshold and the aspect ratio of the bounding box is greater than the preset aspect ratio threshold. If the conditions are met, the suspected event is determined to be a normal action.

[0070] Otherwise, output a fall alarm.

[0071] After acquiring the physical features, the decision filtering layer 103 loads the absolute veto rule defined within the rule engine framework. The logical representation of this anti-bending / squatting rule is: if ( )and( And (R>R) limit If the condition is false, the decision state will be triggered as a false alarm.

[0072] Among them, V limit R represents the falling speed threshold. limit This represents the aspect ratio threshold. The physical meaning of this logical judgment is as follows: the preceding model considers the motion characteristics of the current action to be similar to the fall height, but as long as the human body's center of mass is observed to descend slowly and the aspect ratio of the bounding box remains at a large value, the system forcibly intercepts the event using white-box physics rules and determines it to be a normal autonomous action suitable for aging.

[0073] Once all constraint nodes in the rule engine determine that the request should be allowed, the decision filtering layer 103 generates an anomaly alarm event and pushes it to external devices. Based on a two-level detection architecture that combines static and dynamic methods, the false alarm rate in complex scenarios is reduced to less than or equal to 1% under the computing power constraint of ensuring an overall detection response time of less than or equal to 2.5 seconds.

[0074] Step S205: Based on the historical test set, use a search algorithm to dynamically adjust the parameter set including the falling speed threshold and the aspect ratio threshold, and send the updated parameter set to the rule judgment logic.

[0075] This step is performed by the evaluation and optimization layer 104. During long-term operation, the smart elderly care platform accumulates log data. When caregivers confirm alarm events on-site via mobile terminals and report them as false alarms, the feature data corresponding to the video clips that triggered the false alarms are automatically broken down, labeled, and archived. Negative sample video data that caused false triggers in the online operating environment, as well as some verified real fall positive sample data, are collected to construct an offline baseline library, which serves as the system's historical test set.

[0076] To address the decision parameters required by the rule engine, a two-dimensional parameter grid search space is constructed, consisting of multiple sets of alternative fall velocity thresholds and alternative aspect ratio thresholds. The online rule engine search parameter set is defined as follows: The evaluation and optimization layer 104 initiates testing tasks at a preset fixed period. It traverses each candidate parameter combination in the two-dimensional parameter grid search space, performing baseline replay simulation matching tests on the historical test set. After the test, it statistically analyzes and outputs the simulated false positive rate and simulated false negative rate of each candidate parameter combination on the historical test set. The optimal parameter combination that meets the preset evaluation conditions is selected as the updated parameter set. Through the evaluation-driven development mechanism, the threshold adjustment task is performed offline in the background, replacing the high-cost retraining of the deep neural network.

[0077] In one embodiment, the logic for selecting the optimal parameter combination and distributing the configuration is limited. To balance the system's safety baseline and anti-interference capability, the evaluation and optimization layer 104 establishes an evaluation function. This evaluation function takes the simulated false alarm rate and simulated false alarm rate as two independent input variables, assigns corresponding first and second weight coefficients to each, and performs a weighted summation to calculate the comprehensive penalty score for each set of parameters. False alarms will have fatal consequences. The system excludes all parameter combinations that cause the simulated false alarm rate to exceed the preset safety threshold, ensuring the bottom line of life safety. Among the remaining candidate parameter combinations that meet the safety threshold, the set with the lowest comprehensive penalty score calculated by the evaluation function is selected as the optimal parameter combination.

[0078] The evaluation and optimization layer 104 encapsulates the optimal parameter combination into a configuration text based on a standardized format. This configuration text is then pushed to the online decision filtering layer 103 service node via a hot update through the microservice configuration center. This configuration text guides the service node to reload and apply the new fall speed threshold and aspect ratio threshold without downtime. Through dynamic parameter evolution and a seamless hot deployment mechanism, the system achieves targeted dynamic adjustment of boundary thresholds, ensuring the availability of the algorithm model in diverse and adaptable aging-in-place environments.

[0079] In one embodiment, such as Figure 4As shown, an electronic device is provided. This electronic device, serving as the hardware entity for running a detection method, is deployed in a cloud server node or edge computing box. The electronic device includes a processor 401, a memory 402, and a communication interface 403. The memory 402 stores an executable computer program, including a software instruction package for skeletal point extraction, long short-term memory network inference, and rule engine verification logic. When executing the instruction set, the processor 401 obtains a video stream from the communication interface 403, schedules underlying computing resources to complete feature matrix generation, forward propagation calculation, and false alarm filtering, realizing an age-appropriate fall detection method based on long short-term memory networks and a rule engine. Through the cooperation of a specific hardware environment and underlying microservice logic code, high-dimensional image data is transformed into safety alarm instructions, ensuring the operation and alarm functions of the electronic device.

[0080] In one embodiment, a computer-readable storage medium is provided. This computer-readable storage medium can be a non-volatile storage device capable of permanently storing data. A computer program is programmed or stored on it. When the program is retrieved and executed by the processor of an electronic device, it drives the electronic device to implement the data processing flow of the age-appropriate fall detection method. By embedding the judgment logic in the storage medium, the detection algorithm can be quickly copied and integrated into smart terminals or elderly care robots, improving its engineering and promotion value.

[0081] It should be noted that the embodiments described above are merely preferred solutions of this application. Those skilled in the art should understand that, without departing from the design principles of this application, equivalent substitutions can be made to local structures and parameters such as the granularity of microservice splitting, the number of neural network model layers, or the types of smoothing filtering algorithms, depending on the application requirements of specific engineering projects. Such substitutions can also achieve the goal of high-precision, low-false-positive detection and should all be included within the scope of protection of this application.

Claims

1. An age-friendly fall detection method based on long short-term memory networks and a rule engine, characterized in that, include: The video stream is acquired, and the coordinate sequence of human skeleton key points of the target object in the corresponding consecutive frames of the time window is extracted. The feature matrix is ​​constructed by normalizing the distance between the center of the shoulders and the center of the hips. The feature matrix is ​​input into a pre-trained long short-term memory network model, which outputs an initial fall probability. If the initial probability of falling is greater than the preset confidence threshold, the time window is marked as a suspected event and the rule judgment logic is triggered to extract physical features including the aspect ratio of the bounding box and the falling speed of the center of gravity. Determine whether the physical feature satisfies the condition that the centroid falling speed is lower than a preset falling speed threshold and the bounding box aspect ratio is greater than a preset aspect ratio threshold. If the condition is met, the suspected event is determined to be a normal action. Otherwise, output a fall alarm; Based on the historical test set, a search algorithm is used to dynamically adjust the parameter set, which includes the falling speed threshold and the aspect ratio threshold, and the updated parameter set is sent to the rule determination logic.

2. The method according to claim 1, characterized in that, The step of extracting the coordinate sequence of key points of the human skeleton of the target object in the corresponding consecutive frames of the time window, and constructing a feature matrix by normalizing the distance between the center of the shoulders and the center of the hips, includes: locating the target object in the current frame and generating a bounding box; Extract the two-dimensional pixel coordinates of key points including the head, shoulders, elbows, wrists, hips, knees, and ankles within the bounding box; Calculate the center point coordinates of the shoulders and the center point coordinates of the hips, and use the Euclidean distance between the center point coordinates of the shoulders and the center point coordinates of the hips as a reference scale; The two-dimensional pixel coordinates of all the key points are normalized by division relative to the reference scale to obtain a relative coordinate sequence, which is then assembled into the feature matrix according to the spatial distribution.

3. The method according to claim 2, characterized in that, The step of performing a division normalization operation on the two-dimensional pixel coordinates of all the key points relative to the reference scale to obtain a relative coordinate sequence and assembling it into the feature matrix according to the spatial distribution is configured as follows: selecting the center point coordinates of the two hips as the origin of the coordinate system, performing a translation operation on each of the key points to obtain the relative reference coordinates; Calculate a first ratio of the horizontal relative coordinates of each key point to the reference scale, and a second ratio of the vertical relative coordinates of each key point to the reference scale; The first ratio and the second ratio are concatenated in a fixed order according to the key point type to form a one-dimensional feature vector of the current frame; According to the chronological order of consecutive frames within the time window, the one-dimensional feature vectors corresponding to each frame are stacked row by row to generate the two-dimensional feature matrix, wherein the number of rows of the feature matrix is ​​equal to the number of frames contained in the time window, and the number of columns is equal to twice the number of corresponding key points.

4. The method according to claim 1, characterized in that, The extraction of physical features including the aspect ratio of the bounding box and the falling velocity of the center of mass includes: extracting the start frame and the end frame from the time window corresponding to the suspected event; Obtain the initial bounding box corresponding to the start frame and the end bounding box corresponding to the end frame; Calculate the ratio of the height to the width of the ending bounding box, and use the ratio as the aspect ratio of the bounding box; Calculate the initial centroid longitudinal coordinate of the target object in the starting frame and the ending centroid longitudinal coordinate in the ending frame. Divide the difference between the ending centroid longitudinal coordinate and the initial centroid longitudinal coordinate by the time length of the time window to obtain the centroid falling velocity.

5. The method according to claim 4, characterized in that, After obtaining the initial bounding box corresponding to the start frame and the end bounding box corresponding to the end frame, the extraction of physical features including the aspect ratio of the bounding box and the centroid falling speed further includes: performing Gaussian smoothing on the edge pixels of the initial bounding box and the end bounding box to generate smooth boundaries; The Kalman filter algorithm is used to perform state estimation and filtering calculations on the bounding box parameters and centroid coordinates of each frame in the continuous frame sequence, and to obtain the smoothed centroid trajectory and the filtered bounding box sequence. Traverse the height information of each data point in the smoothed centroid trajectory, and extract the first peak longitudinal coordinate and the second valley longitudinal coordinate corresponding to the highest point position; Based on the extracted first peak longitudinal coordinate and second valley longitudinal coordinate, the initially extracted baseline data is replaced, and the initial centroid longitudinal coordinate and the final centroid longitudinal coordinate are re-established.

6. The method according to claim 1, characterized in that, The step of inputting the feature matrix into a pre-trained long short-term memory network model and outputting a preliminary fall probability includes: inputting the feature matrix into the multi-layer gating structure of the long short-term memory network model step by step; Extract the hidden layer state vector at the last moment, and input the hidden layer state vector into the fully connected layer for dimensionality reduction. The dimensionality-reduced features are mapped to a continuous numerical range using a logistic regression function to obtain a confidence score that represents the probability of a fall event, and this confidence score is used as the initial fall probability.

7. The method according to claim 1, characterized in that, The step of dynamically adjusting the parameter set containing the falling speed threshold and the aspect ratio threshold using a search algorithm based on the historical test set, and sending the updated parameter set to the rule determination logic, includes: collecting negative sample video data that cause false triggering in the online operating environment, and constructing an offline baseline library as the historical test set; Construct a two-dimensional parametric grid search space consisting of multiple sets of alternative fall velocity thresholds and alternative aspect ratio thresholds; Traverse each set of candidate parameter combinations in the two-dimensional parameter grid search space, and perform offline simulation matching tests on the historical test set; The simulated false alarm rate and simulated false alarm rate corresponding to each set of candidate parameter combinations are statistically analyzed, and the optimal parameter combination that meets the preset evaluation conditions is selected as the updated parameter set.

8. The method according to claim 7, characterized in that, The method of statistically analyzing the simulated false alarm rate and simulated false negative rate for each set of candidate parameter combinations, and selecting the best parameter combination that meets the preset evaluation conditions as the updated parameter set, is configured as follows: establishing an evaluation function, wherein the evaluation function takes the simulated false alarm rate and simulated false negative rate as input variables and assigns corresponding first weight coefficients and second weight coefficients to perform weighted summation to calculate the comprehensive penalty score; Among all the alternative parameter combinations that satisfy the requirement that the simulated false alarm rate is lower than the preset safety threshold, the combination with the lowest comprehensive penalty score is selected as the optimal parameter combination. The optimal parameter combination is encapsulated into a standard format configuration text; The standard format configuration text is pushed to the service node where the rule judgment logic is running online via hot update through the microservice configuration center, so that the service node can reload and apply the falling speed threshold and the aspect ratio threshold without downtime.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images