Fall detection method based on multilayer screening and key point prediction

Abstract

The invention discloses a tumble detection method based on multilayer screening and key point prediction. The method comprises the following steps: constructing a daily behavior data set; establishing an identification fall detection model; outputting the contents of the detection frames in e frames before and after the identified frame with the tag name being the tumble in parallel, and screening out a key frame set by using the image difference between continuous frames; generating human body key point information of a central frame and adjacent frames thereof; calculating the moving speed of the vertical coordinate of the human body mass center point; acquiring the maximum height of a shoulder joint point on the right side of the image in the 3D key point graph relative to the ground when the human body squats; and if the moving speed is greater than a preset moving speed threshold value and the vertical coordinates of the shoulder joint points, on the right side of the image, of all the frames in the period are greater than the maximum height, it is determined that falling may occur. The tumble detection and recognition precision is high, the requirements of robustness, accuracy and real-time performance of a detection system can be met, and meanwhile the method can be directly deployed on existing mobile intelligent equipment.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a fall detection method based on multi-layer screening and key point prediction. Background Art

[0002] With the intensification of the aging population and the increase in the number of elderly people living alone, human fall detection has become an important means to ensure the safety of the elderly, people living alone and patients. This technology aims to detect fall events in a timely manner, ensure that rescue measures can be taken quickly in emergency situations, and reduce the harm caused by falls. For the elderly, falls not only cause immediate injuries, but may also cause long-term health problems, such as sequelae such as mobility difficulties left after fractures, and even threaten life. Therefore, human fall detection technology has important application potential in the fields of medical health, elderly care and smart home. At present, fall detection technology mainly includes three solutions based on wearable devices, environmental sensors and intelligent video analysis, which judge fall events by monitoring human movement, environmental changes or image information. However, existing technologies still face many challenges, such as insufficient accuracy and difficulty in effectively distinguishing daily activities from real fall events; for wearable devices, their portability and comfort also affect the user experience.

[0003] The application of detection technology in behavior detection and fall detection has gradually shifted from traditional feature extraction-based methods to deep learning methods, especially the use of convolutional neural networks (CNN) and long short-term memory networks (LSTM). These deep learning models can more effectively handle behavior recognition tasks in complex environments, especially in multi-person and multi-action scenarios. In addition, fusion detection technology combining multimodal data has gradually become a research hotspot to improve the accuracy and robustness of detection.

[0004] The traditional Mosaic data augmentation algorithm scales four images in equal proportions and stitches them into a new image. In this process, since the scaling is globally uniform, it may cause too much information loss during the shrinking of small objects in the image, making it difficult for the model to capture its key features. In addition, large objects may occupy too much space after uniform scaling, limiting the expression of other image information.

[0005] The article "Design and Implementation of Fall Detection System for Elderly Patients" published in China Medical Equipment (2024, 21(02), 157-161) uses the Real-time Streaming Protocol (RTSP), combined with YOLOv5 and Kalman algorithm, and adopts Vue and Flask technologies to design a fall detection system for elderly patients. The article "Fall Behavior Detection Algorithm Based on YOLOv5s" published in China Science and Technology (2023, 18(07), 786-792) improves the basic residual block, adds a mixed domain attention mechanism to the backbone network, and introduces a bidirectional feature pyramid structure to the neck to enhance the network detection accuracy while ensuring the amount of computation, providing an effective idea for implementing fall detection on monitoring devices with limited computing power. The above research on fall detection mainly focuses on a single scenario. This article aims to achieve high-precision detection and real-time detection in complex situations. In addition, due to problems such as insufficient computing power, there are still many problems in directly deploying existing models on smart devices. Summary of the invention

[0006] The present invention discloses a fall detection method based on multi-layer screening and key point prediction, aiming to overcome the shortcomings of existing fall detection technology. The fall detection method of the present invention has strong practicability and few limitations, and can meet the requirements of robustness, accuracy and real-time performance of the detection system. At the same time, it can be directly deployed on existing mobile smart devices to reduce hardware costs.

[0007] A fall detection method based on multi-layer screening and key point prediction, the method comprising the following steps:

[0008] S1: Collect images and videos of people falling, lying, standing, and sitting, and construct a daily behavior dataset after processing using adaptive Mosaic data enhancement;

[0009] S2: A fall detection model is established based on the YOLOv8 neural network, and the fall detection model is trained using the daily behavior dataset;

[0010] S3: Use the trained fall detection model to identify the input image; output the detection frame contents in the frames before and after the frame with the identified label name of fall in parallel, and use the image difference between consecutive frames to filter out the key frame set; for the key frame set, introduce the MediaPipe algorithm to generate the human body key point information of the central frame, and calculate the human body key point positions of the adjacent frames of the central frame;

[0011] S4: Based on the positions of the key points of the human body in adjacent frames, the moving speed s of the vertical coordinate of the center of mass of the human body is calculated; the maximum height H of the shoulder joint point on the right side of the image relative to the ground in the 3D key point map when the human body squats is collected th; When the movement speed s is greater than the preset movement speed threshold within the preset time period, and the ordinates of the shoulder joint points on the right side of the image in all frames during the period are greater than the maximum height H th , it is determined that a fall may have occurred.

[0012] Step S1 further comprises the following steps:

[0013] S11, collects images of people falling, lying, standing, and sitting;

[0014] S12, using adaptive Mosaic enhancement algorithm to automatically combine multiple images into one image according to the preset ratio, and automatically adjust the scaling ratio according to the area of ​​the target object in each image, so that small target objects occupy enough space after stitching, and large target objects are correspondingly reduced; the mosaic image A after stitching Mosaic It is expressed as:

[0015]

[0016] In the formula, Concat represents the image concatenation operation, represents the kth image after dynamic scaling, α k It is used to determine the relative proportion and area occupied by each image when stitching. k for The adjustment factor of the width and height after scaling;

[0017] In the mosaic process, the position-aware clipping strategy of the target object is introduced to ensure that the target object will not be clipped, and the transition function is used to smooth the pixel values ​​at the mosaic boundary; specifically, let (m k ,n k ) is image I k The center coordinates of the target object in the image are W. k ×G k , calculate the cropping area R k The expression is:

[0018]

[0019] At the splicing boundary, a transition function is used to smooth the pixel values; specifically, a Gaussian function is used to smooth the edge area J border After smoothing the pixel values, the edge area J is obtained. s for:

[0020]

[0021] Among them, (m,n) is the coordinate of a pixel in the image, indicating the position in two-dimensional space, (mb ,n b ) is the center point of the boundary area, and σ is used to control the smoothing range;

[0022] S13, use Labelimg software to annotate the spliced ​​images in YOLO format to construct a daily behavior dataset; the annotated categories include falling, standing, walking, and squatting.

[0023] Furthermore, the adjustment coefficient α k The calculation formula is:

[0024]

[0025] Among them, A target,k is the area of ​​the target object in the kth image, is the sum of the areas of all target objects in the four images, and j represents the index of the areas of different target objects in the four images, which are specifically numbered by image.

[0026] Furthermore, step S2 further includes the following steps:

[0027] FasterNet network is used as the backbone part of YOLOv8 neural network. BiFormer attention mechanism is integrated after the backbone network SPFF to remove redundant background features. MPDIoU loss function is introduced in the detection head of YOLOv8 neural network to improve the accuracy and efficiency of bounding box regression in target detection. A fall detection model is constructed.

[0028] Furthermore, in step S3, the process of selecting a key frame set by using the image difference between consecutive frames includes:

[0029] Assume that the frame sequence of parallel output is F = {f1,f2,...,f N}, where N is the total number of frames;

[0030] Define the image difference measurement function as Δ(f t ,f t+1 ), whose expression is:

[0031]

[0032] In the formula, f t (x,y) represents the tth frame image f t The color pixel value at pixel location (x,y);

[0033] The frame with the largest change before and after is selected using the following formula:

[0034] S={f t |Δ(ft ,f t+1 )|>τ,t=t0-e,...,t0+e}

[0035] Among them, t0 is the central frame where the fall label is detected, the set S represents the selected key frame set; τ represents the threshold value, which is used to judge whether the inter-frame change is significant. t ,f t+1 ) is greater than τ, the frame is selected into the set S.

[0036] Furthermore, in step S3, for the key frame set, the MediaPipe algorithm is introduced to generate the human key point information of the central frame, and the process of calculating the human key point positions of the adjacent frames of the central frame includes the following steps:

[0037] Assume frame The center frame is used to obtain the 3D key point position set of the center frame through MediaPipe. Where z is the number of key points of the human body;

[0038] For the adjacent frames of the center frame t0 and The following formula is used to optimize the search for the set of 3D key point positions of adjacent frames using the key point positions of the center frame as the initial position:

[0039]

[0040] Among them, the function Optimize() is based on similarity measurement and local gradient descent to adjust the initial key point position;

[0041] The displacement vectors of the key points at time c and c+1 are defined as:

[0042] v t (c) = p t (c+1)-p t (c), c = 1, 2, ..., z;

[0043] The expected position of the key point in the next frame is estimated through the linear prediction model and the second-order prediction model based on acceleration:

[0044] p t (c+2)=p t (c+1)+v t (c);

[0045] p t (c+2)=p t (c+1)+2v t (c)-v t (c-1);

[0046] The estimated result is used as the initial value for key point detection.

[0047] Step S4 further comprises:

[0048] The Blaze-Pose lightweight convolutional neural network architecture in the Mediapipe framework is used to extract the coordinates of 33 key points of the human body from the video image file to form a 99-dimensional vector, and the 3D coordinates are inferred using the visual key point detection function in Mediapipe. At the same time, the coordinates of the three joints of the nose, right shoulder and left hip are selected, and the average of these three coordinates is calculated as the coordinate of the center of mass of the human body, and the moving speed s of the vertical coordinate of the center of mass is calculated every 10 frames:

[0049]

[0050] Among them, y i,10 Represents the ordinate of the i-th joint point at the 10th frame, y i,1 represents the ordinate of the i-th joint point in the first frame, and Δt represents the time interval between adjacent frames;

[0051] The maximum height H of the right shoulder joint relative to the ground in the 3D key point map when the human body squats is collected th ; satisfy s≥s th After that, the y coordinate of the right shoulder joint point is obtained once every 15 frames. 11 , if within the preset time y 11 Always greater than H th , it is determined that a fall may have occurred.

[0052] Furthermore, the maximum height H th The calculation process includes the following steps:

[0053] The Euclidean distance q between the 11th joint point and the 23rd joint point is calculated using the following formula:

[0054]

[0055] The expression between L and q is constructed as:

[0056] L(q p )=ωq p +b;

[0057] where q p represents the pth value of the Euclidean distance q, and L represents the distance between the person and the camera;

[0058] Construct the optimal formula:

[0059]

[0060] in m is used to represent the number of samples used in the fall detection task;

[0061] The optimal value of coefficient ω and b is solved by the least squares estimation method * With b *

[0062]

[0063] Find L and H th The mapping relationship between them determines H th .

[0064] The beneficial effects of the present invention are:

[0065] The fall detection method based on multi-layer screening and key point prediction of the present invention has high judgment accuracy for fall behavior, strong practicality, and small limitations. It can meet the requirements of robustness, accuracy and real-time performance of the detection system, and can be directly deployed on existing mobile smart devices to reduce hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of a fall detection method based on multi-layer screening and key point prediction according to an embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of the improved YOLOV8 network structure of the present invention.

[0068] Figure 3 This is a diagram of the test results of the present invention. DETAILED DESCRIPTION

[0069] The following examples will enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.

[0070] The present invention discloses a fall detection method based on multi-layer screening and key point prediction, the method comprising the following steps:

[0071] S1: Collect images and videos of people falling, lying, standing, and sitting, and construct a daily behavior dataset after processing using adaptive Mosaic data enhancement;

[0072] S2: A fall detection model is established based on the YOLOv8 neural network, and the fall detection model is trained using the daily behavior dataset;

[0073] S3: Use the trained fall detection model to identify the input image; output the detection frame contents in the frames before and after the frame with the identified label name of fall in parallel, and use the image difference between consecutive frames to filter out the key frame set; for the key frame set, introduce the MediaPipe algorithm to generate the human body key point information of the central frame, and calculate the human body key point positions of the adjacent frames of the central frame;

[0074] S4: Based on the positions of the key points of the human body in adjacent frames, the moving speed s of the vertical coordinate of the center of mass of the human body is calculated; the maximum height H of the shoulder joint point on the right side of the image relative to the ground in the 3D key point map when the human body squats is collected th ; When the movement speed s is greater than the preset movement speed threshold within the preset time period, and the ordinates of the shoulder joint points on the right side of the image in all frames during the period are greater than the maximum height H th , it is determined that a fall may have occurred.

[0075] See also Figure 1 The fall detection method of the present invention specifically comprises the following steps:

[0076] S1: Collect images and videos of people falling, lying, standing, and sitting, and construct a sample set of behaviors after processing using adaptive Mosaic data enhancement. Step S1 specifically includes:

[0077] Adaptive Mosaic enhancement algorithm is used to automatically combine multiple images into one image according to a certain ratio, so that the model can be applied not only to large scenes but also to small scenes indoor target recognition; the adaptive Mosaic enhancement process is as follows:

[0078] A dynamic scaling strategy based on the size of the target object is used to improve the traditional Mosaic data enhancement algorithm. According to the area of ​​the target object in each image, its scaling ratio is automatically adjusted so that small target objects can occupy enough space after stitching, while large target objects are appropriately reduced. The scaling factor α is adjusted according to the area ratio occupied by the target object in each image. k , as shown in formula (1):

[0079]

[0080] Among them, A target,k is the area of ​​the target object in the kth image, is the sum of the areas of all target objects in the four images;

[0081] Mosaic image A after stitching Mosaic It is expressed as:

[0082]

[0083] Among them, Concat represents the image concatenation operation. represents the kth image after dynamic scaling, α k Determines the relative proportion and area occupied by each image when stitching, α k for The adjustment factor of the width and height after scaling;

[0084] In the mosaic stitching process, random cropping will cause the target object to be partially truncated, resulting in information loss at the border of the stitched image, affecting the continuity of the image and the model's learning of local information. Therefore, a position-aware cropping strategy for the target object is introduced to ensure that the target object will not be cropped; at the stitching boundary, a transition function is used to smooth the pixel values ​​to reduce stitching discontinuity;

[0085] Assume (m k ,n k ) is image I k The center coordinates of the target object in the image are W. k ×G k , calculate the cropping area R k The expression is:

[0086]

[0087] Further to the edge area J border The pixel color is smoothed by Gaussian function in the edge area J s :

[0088]

[0089] Among them, (m b ,n b ) is the center point of the boundary area, and σ is the control smoothing range.

[0090] In this embodiment, images of daily behaviors (falling, standing, squatting, walking) are crawled from the network search engine (Baidu, Google), and after the image data is integrated, a sample set is constructed, and the adaptive Mosaic data enhancement is performed. This paper adopts the adaptive Mosaic enhancement algorithm to automatically combine multiple images into one image according to a certain ratio, so that the model can be applied not only to large scenes but also to target recognition in small indoor scenes. Finally, a data set of 10,000 images is obtained, and the training set, validation set and test set required by the improved YOL0v8 model in this paper are divided according to the ratio of 8:1:1, that is, 8,000 images are training sets, 1,000 images are validation sets, and 1,000 images are test sets. Labelimg software is used to annotate the prepared data set in YOLO format. In addition to the required positive samples of falling and standing, some labels that are easy to misidentify are added, such as walking and squatting, and finally saved as a txt file.

[0091] S2: Based on the YOLOv8 neural network, an improved YOLOv8 neural network for fall detection is established. The YOLOv8 network structure is divided into three parts: backbone network, feature pyramid network (FPN), and detection head; this embodiment improves the YOLOv8 network structure. First, the FasterNet network is used in the Backbone part of YOLOv8 to maintain the original reasoning speed while reducing the computational complexity, and the Biformer attention mechanism is integrated after the backbone network SPFF to remove redundant background features; finally, the MPDIoU loss function is introduced in the YOLOV8 neural network to improve the accuracy and efficiency of bounding box regression in target detection, and improve the model's detection ability for different human behaviors.

[0092] refer to Figure 2 In this embodiment, the existing YOLOV8 neural network backbone network is replaced with the FasterNet structure, and the PConv and point convolution (PWConv) layers in FasterNet are used to effectively aggregate information from all channels and improve computational efficiency. Based on the existing YOLOV8, a dynamic sparse attention mechanism BiFormer is introduced between the Blackbone and Neck ends to reduce computational and memory requirements while maintaining model performance. The MPDIoU loss function is introduced, which provides a clear gradient direction. On the premise of considering the IoU itself, the partial derivative of the IoU relative to the bounding box coordinates and size is also considered, so that the model can more accurately predict the position and size of the target.

[0093] S3: In order to improve the detection speed, a frame generation link is added to output the detection box contents in the previous and next e frames with the label named Fall Down in parallel; and a processing method based on multi-level screening and key point prediction is designed.

[0094] Furthermore, the specific steps of S3 are:

[0095] In the original detection link, a frame generation link is added, and the detection frame contents in the front and back e frames with the label name Fall Down are output in parallel, thereby improving the detection speed.

[0096] In order to ensure accurate and fast drawing of human key points on the exported front and back e-frame images, a processing method based on multi-level screening and key point prediction is designed. The method includes frame selection optimization, key point position initialization and motion prediction correction.

[0097] Assume the frame sequence is F = {f1,f2,…,f N}, where N is the total number of frames. The key frames are selected by calculating the image difference between consecutive frames. The image difference measurement function is defined as Δ(f t ,f t+1 ), whose expression is:

[0098]

[0099] f t (x,y) represents the tth frame image f t The color pixel value at the pixel position (x, y); select the frame with the largest change before and after to reduce the processing of useless frames. The formula is as follows:

[0100] S={f t |Δ(f t ,f t+1 )|>τ,t=t0-e,...,t0+e} (6)

[0101] Among them, t0 is the central frame where the "Fall Down" label is detected, the set S represents the filtered key frame set, and τ represents the threshold value, which is used to judge whether the change between frames is significant. t ,f t+1 ) is greater than τ, the frame will be selected into the set S; assuming that the frame Center frame, the 3D key point position set of this frame has been obtained through MediaPipe Where z is the number of key points of the human body;

[0102] For the adjacent frames of the center frame t0 and Use the key point positions of the center frame as the initial positions for optimized search:

[0103]

[0104] Among them, the function Optimize() quickly adjusts the initial key point position based on similarity measurement and local gradient descent, which greatly reduces the amount of calculation;

[0105] Considering the continuity of human motion, in order to further reduce the amount of calculation, inter-frame motion prediction is used to accelerate 3D key point detection; by estimating the motion trajectory of the key points of the previous and next frames, the position of the key points of the next frame can be predicted; the displacement vector of the key point at time c and c+1 is defined as:

[0106] v t (c) = p t (c+1)-p t (c),c=1,2,...,z (9)

[0107] The expected position of the key point in the next frame is estimated through the linear prediction model and the second-order prediction model based on acceleration:

[0108] p t (c+2)=p t (c+1)+v t (c) (10)

[0109] p t (c+2)=p t (c+1)+2v t (c)-v t (c-1) (11)

[0110] The prediction results are used as the initial value for key point detection, which further reduces the calculation time.

[0111] S4: In order to further improve the detection accuracy and speed, the MediaPipe algorithm is introduced in the detection stage to generate human key point information. After the improved YOLOV8 model preliminarily detects the state of falling, the detection box content with the label name Fall Down in the frame image is output in parallel, and the MediaPipe algorithm is introduced at the same time to further determine whether the person is in a falling state.

[0112] In step S4, in the original detection link, the MediaPipe algorithm is used to calculate the acceleration and 3D position between the frames before and after the key point to determine whether it is in a falling state, thereby improving the detection accuracy;

[0113] The Blaze-Pose lightweight convolutional neural network architecture in the Mediapipe framework is used to extract the coordinates of 33 key points of the human body from the video image file to form a 99-dimensional vector, and the 3D coordinates are inferred using the visual detection key point (Landmarks) function in Mediapipe. The coordinates of the three joint points 0 (nose), 11 (right shoulder), and 24 (left hip) are selected, and the average value of their coordinates is calculated as the coordinates of the center of mass of the human body, and the moving speed s of the vertical coordinate of the center of mass is calculated every 10 frames, as shown in formula (12):

[0114]

[0115] Where:y i,10 Represents the ordinate of the i-th joint point at the 10th frame, y i,1 represents the ordinate of the i-th joint point in the first frame, and Δt represents the time interval between adjacent frames. It should be noted that although the mediapipe framework can extract 33 key points, considering the need for mobile terminal deployment, the lightweight requirement is proposed, and the present invention actually only selects 3 of them.

[0116] Since only based on s≥s th When judging a fall, misjudgment may occur. For example, resting in bed, squatting to pick up items, etc. may be misjudged as a fall. Therefore, it is necessary to further collect the maximum height H of the 11th joint relative to the ground in the 3D key point map when the human body squats. th ; satisfy s≥s th Then get y once every 15 frames 11 , if within a certain period of time y 11 Always greater than H th , it means that a fall may have occurred; when the relative distance between the human body and the camera changes, the height y of the joint point in the image i It is different, H th It also keeps changing. When the human body is close to the camera, H th The value of H is larger. As the distance between the person and the camera gradually increases, th will gradually decrease, which will also lead to different H th Therefore, it is necessary to determine the distance between the person and the camera, and then determine H at this distance th ; When the person is close to the camera, the Euclidean distance q between the 11th joint point and the 23rd joint point is large. As the person and the camera get farther and farther away, q will gradually decrease. Therefore, it can be used to determine the distance L between the person and the camera. The calculation formula of q is as follows:

[0117]

[0118] q, H thThe linear regression relationship model between L and L is shown in formula (14), and the expression between L and q is:

[0119] L(q p )=ωq p +b (14)

[0120] Where p represents the pth value, ω and the optimal value of b ω * With b * Solve it by the least squares estimation method, as shown in formula (15):

[0121]

[0122] The optimal formula is as follows:

[0123]

[0124]

[0125] in Similarly, we can get L and H th The relationship between th .

[0126] The fall detection method of this embodiment includes the following steps: collecting images and videos of people falling, lying, standing, and sitting, and using an adaptive Mosaic enhancement algorithm to automatically adjust the image scaling ratio according to the area of ​​the human body in the label frame in each image, so that small target objects can occupy more than 25% of the space after splicing, while large target objects are appropriately reduced; introducing a position-aware cropping strategy for target objects to ensure that the target objects will not be cropped; at the splicing boundary, using a transition function to smooth the pixel values ​​to improve the continuity of image splicing, thereby increasing the number of samples in the image data set and establishing daily behavior The YOLOv8 network structure is improved. First, the FasterNet network is used in the Backbone part of the original YOLOv8 to maintain the original reasoning speed while reducing the computational complexity, and the Biformer attention mechanism is integrated after the backbone network SPFF to remove redundant background features. Then, the MPDIoU loss function is introduced to improve the model's detection ability for different human behaviors. Based on the above improvements, the detection model trained by the YOLOv8 model is used for key point detection. In order to reduce the computational cost, the frame generation link is further added, and the detection frame content of the previous and next frames with the label "Fall Down" is output in parallel. A processing method based on multi-layer screening and key point prediction is adopted to improve the detection process and improve the accuracy of fall detection. Then, the MediaPipe algorithm is used to generate the key point information of the human body in the detection frame, and the acceleration and 3D position between the previous and next frames of the key point of the human body are calculated to increase the judgment accuracy of the fall behavior.

[0127] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A fall detection method based on multi-layer screening and key point prediction, characterized in that: The method comprises the following steps: S1: Collect images and videos of people falling, lying, standing, and sitting, and construct a daily behavior dataset after processing using adaptive Mosaic data enhancement; S2: A fall detection model is established based on the YOLOv8 neural network, and the fall detection model is trained using the daily behavior dataset; S3: Use the trained fall detection model to identify the input image; output the detection frame contents in the frames before and after the frame with the identified label name of fall in parallel, and use the image difference between consecutive frames to filter out the key frame set; for the key frame set, introduce the MediaPipe algorithm to generate the human body key point information of the central frame, and calculate the human body key point positions of the adjacent frames of the central frame; S4: Based on the positions of the key points of the human body in adjacent frames, the moving speed s of the vertical coordinate of the center of mass of the human body is calculated; the maximum height H of the shoulder joint point on the right side of the image relative to the ground in the 3D key point map when the human body squats is collected th ; When the movement speed s is greater than the preset movement speed threshold within the preset time period, and the ordinates of the shoulder joint points on the right side of the image in all frames during the period are greater than the maximum height H th , it is determined that a fall may have occurred.

2. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: Step S1 further comprises the following steps: S11, collects images of people falling, lying, standing, and sitting; S12, using adaptive Mosaic enhancement algorithm to automatically combine multiple images into one image according to the preset ratio, and automatically adjust the scaling ratio according to the area of ​​the target object in each image, so that small target objects occupy enough space after stitching, and large target objects are correspondingly reduced; the mosaic image A after stitching Mosaic It is expressed as: In the formula, Concat represents the image concatenation operation, represents the kth image after dynamic scaling, α k It is used to determine the relative proportion and area occupied by each image when stitching. k for The adjustment factor of the width and height after scaling; In the mosaic process, the position-aware clipping strategy of the target object is introduced to ensure that the target object will not be clipped, and the transition function is used to smooth the pixel values ​​at the mosaic boundary; specifically, let (m k ,n k ) is image I k The center coordinates of the target object in the image are W. k ×G k , calculate the cropping area R k The expression is: At the splicing boundary, a transition function is used to smooth the pixel values; specifically, a Gaussian function is used to smooth the edge area J border After smoothing the pixel values, the edge area J is obtained. s for: Among them, (m,n) is the coordinate of a pixel in the image, indicating the position in two-dimensional space, (m b ,n b ) is the center point of the boundary area, and σ is used to control the smoothing range; S13, use Labelimg software to annotate the spliced ​​images in YOLO format to construct a daily behavior dataset; the annotated categories include falling, standing, walking, and squatting.

3. The fall detection method based on multi-layer screening and key point prediction according to claim 2, characterized in that: The adjustment coefficient α k The calculation formula is: Among them, A target,k is the area of ​​the target object in the kth image, is the sum of the areas of all target objects in the four images, and j represents the index of the areas of different target objects in the four images, which are specifically numbered by image.

4. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: Step S2 further comprises the following steps: FasterNet network is used as the backbone part of YOLOv8 neural network. BiFormer attention mechanism is integrated after the backbone network SPFF to remove redundant background features. MPDIoU loss function is introduced in the detection head of YOLOv8 neural network to improve the accuracy and efficiency of bounding box regression in target detection. A fall detection model is constructed.

5. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: In step S3, the process of selecting a key frame set by using the image difference between consecutive frames includes: Assume that the frame sequence of parallel output is F = {f1,f2,...,f N }, where N is the total number of frames; Define the image difference measurement function as Δ(f t ,f t+1 ), whose expression is: In the formula, f t (x,y) represents the tth frame image f t The color pixel value at the pixel location (x,y); The frame with the largest change before and after is selected using the following formula: S={f t |Δ(f t ,f t+1 )|〉τ,t=t0-e,...,t0+e} Among them, t0 is the central frame where the fall label is detected, the set S represents the selected key frame set; τ represents the threshold value, which is used to judge whether the inter-frame change is significant. t ,f t+1 ) is greater than τ, the frame is selected into the set S.

6. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: In step S3, for the key frame set, the MediaPipe algorithm is introduced to generate the human key point information of the central frame, and the process of calculating the human key point positions of the adjacent frames of the central frame includes the following steps: Assume that frame f t0 The center frame is used to obtain the 3D key point position set of the center frame through MediaPipe. Where z is the number of key points of the human body; For the adjacent frames of the center frame t0 and The following formula is used to optimize the search for the set of 3D key point positions of adjacent frames using the key point positions of the center frame as the initial position: Among them, the function Optimize() is based on similarity measurement and local gradient descent to adjust the initial key point position; The displacement vectors of the key points at time c and c+1 are defined as: v t (c)=p t (c+1)-p t (c),c=1,2,...,z; The expected position of the key point in the next frame is estimated through the linear prediction model and the second-order prediction model based on acceleration: p t (c+2)=p t (c+1)+v t (c); p t (c+2)=p t (c+1)+2v t (c)-v t (c-1); The estimated result is used as the initial value for key point detection.

7. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: Step S4 further comprises: The Blaze-Pose lightweight convolutional neural network architecture in the Mediapipe framework is used to extract the coordinates of 33 key points of the human body from the video image file to form a 99-dimensional vector, and the 3D coordinates are inferred using the visual key point detection function in Mediapipe. At the same time, the coordinates of the three joints of the nose, right shoulder and left hip are selected, and the average of these three coordinates is calculated as the coordinate of the center of mass of the human body, and the moving speed s of the vertical coordinate of the center of mass is calculated every 10 frames: Among them, y i,10 Represents the ordinate of the i-th joint point at the 10th frame, y i,1 represents the ordinate of the i-th joint point in the first frame, and Δt represents the time interval between adjacent frames; The maximum height H of the right shoulder joint relative to the ground in the 3D key point map when the human body squats is collected th ; satisfy s≥s th After that, the y coordinate of the right shoulder joint point is obtained once every 15 frames. 11 , if within the preset time y 11 Always greater than H th , it is determined that a fall may have occurred.

8. The fall detection method based on multi-layer screening and key point prediction according to claim 1, characterized in that: The maximum height H th The calculation process includes the following steps: The Euclidean distance q between the 11th joint point and the 23rd joint point is calculated using the following formula: The expression between L and q is constructed as: L(q p )=ωq p +b; where q p represents the pth value of the Euclidean distance q, and L represents the distance between the person and the camera; Construct the optimal formula: in m is used to represent the number of samples used in the fall detection task; The optimal value of coefficient ω and b is solved by the least squares estimation method * With b * Find L and H th The mapping relationship between them determines H th .

Images