A fall detection method and system

Abstract

The application discloses a fall detection method and system, and relates to the technical field of fall detection.The application comprises the following steps: obtaining a fall data set; inputting the fall data set into an improved YOLOv8 model for training to obtain a fall detection model; and inputting a to-be-identified fall image into the fall detection model to obtain a fall detection result of the to-be-identified fall image.The application improves the existing YOLOv8, aims to fully utilize a large amount of fall image data, automatically learn and identify the postures of personnel in the fall image, and thus realize efficient, accurate and low-cost fall detection.

Description

Technical Field

[0001] This invention relates to the field of fall detection technology, and more specifically to a fall detection method and system. Background Technology

[0002] Accidental falls have gradually become a primary risk threatening the safety of the elderly. Against this backdrop, fall detection technology is particularly important, providing crucial protection for the health and safety of senior citizens. Currently, common fall detection methods include sensors, cameras, and artificial intelligence. However, traditional fall detection methods suffer from a series of problems, such as insufficient detection accuracy, high false alarm rates, significant latency, and over-reliance on human intervention. These problems collectively limit the ability of traditional methods to detect falls in a timely and accurate manner. Moreover, these methods are difficult to adapt to diverse scenarios and environments, thus limiting their effective application in real life. To overcome these limitations, there is an urgent need for further optimization and innovation of fall detection algorithms, especially in terms of lightweight design, to enable practical industrial deployment. Furthermore, improving detection speed and accuracy is also one of the key issues that needs to be addressed in this field. Summary of the Invention

[0003] In view of this, the present invention provides a fall detection method and system to address safety issues among the elderly. This invention improves upon the existing YOLOv8 framework, aiming to achieve efficient, accurate, and low-cost fall detection by fully utilizing a large amount of fall image data to automatically learn and recognize the postures of people in fall images.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A fall detection method includes the following steps:

[0006] Step 1: Collect a large amount of image data containing fall scenarios, covering different environments and lighting conditions. This data will constitute the fall dataset, providing a foundation for subsequent model training.

[0007] Step 2: Input the fall dataset into the improved YOLOv8 model for training. The improved YOLOv8 model is adjusted and optimized for fall behavior based on the original model to improve detection accuracy and speed in fall scenarios. Through repeated iterative training, the model parameters are optimized to ensure its robustness in different scenarios, ultimately resulting in a fall detection model.

[0008] Step 3: When a fall event needs to be detected, the fall image to be identified is input into the improved YOLOv8 model trained in Step 2. The model will analyze the image in real time, locating and identifying the position and posture of the fall. Finally, it outputs information about the possible fall events in the image.

[0009] Optionally, in step 1, the fall dataset includes several fall images and the posture type of the person in each fall image.

[0010] Optionally, in step 2, the improved YOLOv8 model is trained using the fall image as input and the posture type corresponding to the person in the fall image as output.

[0011] Optionally, in step 2, the improved YOLOv8 model is based on the traditional YOLOv8 model and incorporates the following three improvements:

[0012] The backbone network of YOLOv8 is replaced with the FasterNext module, making the network more lightweight by reducing GFLOPs and computational cost.

[0013] In YOLOv8's neck network, dynamic deformable convolutions are introduced to replace convolutional blocks used for downsampling, in order to better adapt to various changes in human posture and perform better when dealing with irregular shapes of falling targets.

[0014] The bounding box regression loss function of YOLOv8 has been improved by introducing WIoU to replace the original CIoU, making the model pay more attention to examples of average quality and reducing the harm caused by low-quality examples.

[0015] Optionally, the C2f module in the original YOLOv8 model backbone network consisted of Bottleneck, which involved redundant computation and increased the number of model parameters. The lightweight and high-precision FasterNext module is used to reduce the number of model parameters while enhancing the model's feature processing capabilities. The Fasternet_Block module in the Fasternet module first extracts spatial features from a portion of the input channels (e.g., one-quarter) using PConv, keeping the remaining channels unchanged. Then, it uses 1×1 ordinary convolutions to process the unaffected input channels and the channels after the PConv convolution, generating redundant feature maps with less computational cost. Assuming h and w are the height and width of the input data, and c is the number of input channels, c... p FLOPs represents the number of input channels, k is the kernel size, and the calculation process for standard convolution and partial convolution is as follows.

[0016] FLOPs1 = h × w × k 2 ×c 2 ;

[0017]

[0018]

[0019] The memory access requirements for standard convolution and partial convolution are calculated as follows.

[0020]

[0021] h×w×2c+k 2 ×c 2 ≈h×w×2c p ;

[0022] Assume c p If we take it as one-quarter of c, then PConv's PLOPs are one-sixteenth of those of a standard convolution, and its memory access is one-quarter of that of a regular convolution, which can effectively reduce the computational resources required by the model.

[0023] Optionally, a dynamically deformable convolution is introduced into the YOLOv8 neck network to replace the convolutional blocks used for downsampling. The key feature is that the dynamically deformable convolution adds a displacement variable to the position of each sampling point in the convolution kernel, enabling random sampling near the current position, thus moving beyond the limitations of ordinary convolution kernels sampling at regular grid points. The dynamically deformable convolution also adaptively learns the convolution kernel weight parameters, overcoming the drawback of traditional static convolution where all samples share the same convolution kernel parameters. Therefore, the dynamically deformable convolution can effectively handle the problem of large-target deformation during human falls.

[0024] In dynamically deformable convolutions, the shape change of the convolution kernel is learned by the network based on deformation offset. First, a regular convolution kernel S is used to sample the input feature map X, and then a weighted summation operation is performed using a weight matrix M. The size of the convolution kernel determines the range of the receptive field. For example, a 3×3 convolution kernel has:

[0025] S={(0,0), (0,1), (0,2),…, (2,1), (2,2)};

[0026] For each pixel on the output mapping feature map It can be represented as:

[0027]

[0028] in: This represents the offset of each sampling point.

[0029] At this point, the sampling point is located after the irregular offset. Up, and offset These are usually floating-point numbers. Therefore, bilinear interpolation is used to obtain the pixel values.

[0030] X(p)=∑ q G(p,q)·X(q);

[0031] in: This represents any position after the offset; q enumerates the four surrounding integer coordinates; G(•,•) are the weights corresponding to the four integer coordinates, G(•,•) is two-dimensional, consisting of two one-dimensional kernels:

[0032] G(q,p)=g(q x ,p x )·g(q y ,p y );

[0033] Where: g(a,b)=max(0,1-|ab|).

[0034] To demonstrate that the kernel of a dynamically deformable convolution is dynamic, we first present the output feature map of a regular convolution:

[0035] y = σ(W•x);

[0036] Where σ is the activation function, W is the convolution kernel weight matrix, and y is the output feature map. In contrast, the output feature map of dynamically deformable convolution is:

[0037]

[0038] Where: n is the number of weighting coefficients, α n These are weight coefficients with learnable parameters. It is the output feature map.

[0039] Dynamically deformable convolution dynamically adjusts the weights of the convolution kernel by linearly combining different weight matrices according to the corresponding weight coefficients before performing the convolution operation.

[0040] Optionally, a dynamic non-monotonic focusing mechanism is introduced, and Wise-IoU (WIoU) is designed. Its formula is as follows:

[0041]

[0042]

[0043] Among them, b and b gt ρ represents the centroids of the predicted bounding box and the ground truth bounding box, respectively; ρ represents the Euclidean distance between the two centroids; d represents the diagonal distance of the smallest enclosed region containing both the predicted and ground truth bounding boxes; the superscript * indicates that the smallest bounding box is separated from the computation graph; β represents the outlier, used to define the quality of the anchor box; α and δ are two hyperparameters, set to 1.9 and 3, respectively.

[0044] A fall detection system, comprising:

[0045] Dataset Acquisition Module: Used to acquire fall datasets;

[0046] Fall detection model training module: The fall dataset is input into the improved YOLOv8 model for training to obtain the fall detection model;

[0047] Fall detection result module: used to input the fall image to be identified into the fall detection model to obtain the fall detection result of the fall image to be identified.

[0048] As can be seen from the above technical solution, compared with the prior art, the present invention provides a fall detection method and system, which has the following advantages:

[0049] This invention replaces the backbone network of YOLOv8 with the FasterNext module, making the network more lightweight by reducing GFLOPs and computational cost. Then, dynamic deformable convolutions are introduced into the neck network of YOLOv8 to replace the convolutional blocks used for downsampling, so as to better adapt to various changes in human posture and perform better when dealing with irregular shapes of the target of a fall. Finally, the bounding box regression loss function of YOLOv8 is improved by introducing WIoU to replace the original CIoU, so that the model pays more attention to the examples of normal quality and reduces the harm caused by low-quality examples.

[0050] This invention optimizes YOLOv8 and focuses on fall detection, thereby achieving faster detection speed and more accurate detection results while reducing costs. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0053] Figure 2(a) is a schematic diagram of the structure of the traditional YOLOv8 model;

[0054] Figure 2(b) is a schematic diagram of the structure of the improved YOLOv8 model (FDW-YOLO) of the present invention, wherein the components are put together to indicate that the two components are in a data flow relationship.

[0055] Figure 3 This is a schematic diagram of the Fasternet module in this invention;

[0056] Figure 4 This is a comparison diagram of the Faster_Block module and the Ghostnet module in this invention.

[0057] Figure 5 This is a schematic diagram of the dynamically deformable convolution structure in this invention;

[0058] Figure 6 Example image of the fall dataset in Example 2;

[0059] Figure 7 This is an example diagram showing the detection results of the FDW-YOLO of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] YOLOv8 is the latest version of the YOLO object detection and image segmentation model developed by Ultralytics. YOLOv8 is a cutting-edge, state-of-the-art (SOTA) model that builds on the success of previous YOLO models and introduces new features and improvements to further enhance performance and flexibility. The model is implemented using the PyTorch framework, which can be easily deployed and optimized on different hardware platforms. YOLOv8 integrates the CSPNet and YOLOv7 ELAH structures in its design, forming the C2f module. (1) There are four C2f layers in the backbone network, with stacking times of 3, 6, 6, and 3, respectively. This structure is lightweight compared to YOLOv5. The backbone network continues the SPPF module of YOLOv5, which can effectively capture features at different scales and achieve the fusion of local and global features; (2) In the neck network, YOLOv8 adopts the PAN-FPN structure, which constructs a multi-scale feature pyramid and realizes the beneficial transmission of cross-layer information, so that the network can better perceive the features of targets at different scales; (3) In the detection head part, YOLOv8 has made important optimizations. Instead of using the previous coupled detection head, it introduces the concept of a decoupled head, using two parallel branches to extract category features and position features respectively, and then performs calculations. It is worth noting that YOLOv8 uses the idea of ​​Anchor-Free, so that the model does not need to rely on predefined anchor boxes and can directly predict the position and size of the target. This mode not only simplifies the model architecture, but also avoids the complex process of anchor box generation and screening, making the model more concise and efficient. In summary, YOLOv8 is an efficient and accurate target detection algorithm, which is suitable for the fall detection task in this invention.

[0062] This invention employs the FasterNext module. The original C2f module in the YOLOv8 model backbone network consisted of Bottleneck, which involved redundant computation and increased the number of model parameters. Using the lightweight and high-precision FasterNext module reduces the number of model parameters while enhancing the model's feature processing capabilities. The Fasternet_Block module in the Fasternet module first extracts spatial features from a portion of the input channels (e.g., one-quarter) using PConv, while keeping the remaining channels unchanged. Then, it processes the unaffected input channels and the channels convolved by PConv using a 1×1 ordinary convolution.

[0063] This invention employs dynamically deformable convolution, which learns the ability to deform spatial geometry by introducing a positional offset at each sampling point of the convolution kernel. This offset allows the convolution kernel to randomly sample near its current position, breaking the limitation of sampling regular grid points in traditional convolution. Dynamically deformable convolution also adaptively learns the kernel weight parameters, overcoming the drawback of traditional static convolution where all samples share the same kernel parameters.

[0064] This invention introduces WIoU to replace the original CIoU, making the model pay more attention to examples of normal quality and reducing the harm caused by low-quality examples.

[0065] Example 1

[0066] This embodiment discloses a fall detection method; see [link to documentation]. Figure 1 This includes the following steps:

[0067] Step 1: Obtain the fall dataset.

[0068] The fall dataset includes several fall images and the type of person's posture in each image. The experimental dataset used in this invention comes from three open-source fall datasets: UR Fall Detection Dataset, Fall detection Dataset, and Multiple cameras fall dataset. These datasets contain images of people falling and similar falling actions under different indoor scenes, lighting intensities, and viewpoints. A total of 4395 images were selected, with 3559 images used for training, 396 for validation, and 440 for testing. Labeling software was used to annotate the images, and based on the different characteristics of the falling actions, the actions of the person were divided into four categories: standing, sitting, stooping, and falling.

[0069] Step 2: Based on the aforementioned fall dataset, using fall images as input and the type corresponding to the person's posture in the fall images as output, train the improved YOLOv8 model to obtain the fall detection model.

[0070] The improved YOLOv8 model is based on the traditional YOLOv8 model (see Figure 2(a)) and the following three improvements are made to obtain the FDW-YOLO model (see Figure 2(b)):

[0071] (1) The FasterNext module was used to improve the YOLOv8 backbone network. This made the network more lightweight by reducing GFLOPs and computational cost. See the FasterNext module structure for details. Figure 3 .

[0072] The C2f module in the backbone network of the YOLOv8 model consists of Bottleneck, which involves redundant computation and increases the number of model parameters. To reduce the number of model parameters while enhancing the model's feature processing capabilities, a lightweight and high-precision FasterNext module is used.

[0073] The main difference between FasterNext and C2f modules lies in the Bottleneck module and FasterNet_Block. The Bottleneck module consists of two standard convolutions, which deepen the network and improve its feature extraction capabilities by stacking convolutions, but the model has high complexity. FasterNet_Block mainly consists of partial convolutions (PConv) and two 1×1 ordinary convolutions.

[0074] (2) Due to the large amount of redundancy in the intermediate feature maps generated by traditional convolution, FasterNext introduces a new module: FasterNet_Block. Compared to traditional convolution, this module can generate redundant feature maps with less computational cost. First, spatial features are extracted from a portion of the input channels (e.g., one-quarter) using PConv, while keeping the remaining channels unchanged. Then, 1×1 ordinary convolutions are used to process the unaffected input channels and the channels after PConv convolution.

[0075] Assume h and w are the height and width of the input data, and c is the number of input channels. p is the number of input channels, k is the kernel size, and the FLOPs calculation process for standard convolution and partial convolution is shown below.

[0076] FLOPs1 = h × w × k 2 ×c 2 ;

[0077]

[0078]

[0079] The memory access calculation process for standard convolution and partial convolution is shown below.

[0080]

[0081] h×w×2c+k 2 ×c 2 ≈h×w×2c p ;

[0082] Assume c pIf we take it as one-quarter of c, then PConv's FLOPs are one-sixteenth of those of a standard convolution, and its memory access is one-quarter of that of a regular convolution, which can effectively reduce the computational resources required by the model.

[0083] Compared to the Ghost module, which also utilizes redundant feature maps, the Ghost module first applies regular convolutions on all input channels to generate feature maps (a portion of the output channels, e.g., half). PConv takes into account input redundancy along the channel dimension, choosing to operate on a portion of the channels (e.g., a quarter), while the Ghost module ignores this. Furthermore, the Ghost module applies "cheap" operations (e.g., depthwise convolutions) to the intrinsic feature maps to generate the remaining portion of the output channels, but these "cheap" operations are not truly cheap in terms of memory access; depthwise convolutions have low floating-point operations per second. The FasterNet_Block module, on the other hand, chooses to operate using fast and efficient 1×1 convolutions. See [link to FasterNet_Block module and Ghost module for differences] Figure 4 .

[0084] (3) Dynamically deformable convolution is introduced into the neck network of YOLOv8. Dynamically deformable convolution achieves random sampling near the current position by adding a displacement variable to the position of each sampling point in the convolution kernel, thus no longer being limited to sampling at regular grid points by ordinary convolution kernels. Dynamically deformable convolution also adaptively learns the convolution kernel weight parameters, overcoming the disadvantage of traditional static convolution where all samples share the same convolution kernel parameters. See the structural diagram of the dynamically deformable convolution module. Figure 5 .

[0085] (4) A dynamic non-monotonic focusing mechanism is introduced, and Wise-IoU (WIoU) is designed. Its formula is as follows:

[0086]

[0087]

[0088] Among them, b and b gt ρ represents the centroids of the predicted bounding box and the ground truth bounding box, respectively; ρ represents the Euclidean distance between the two centroids; d represents the diagonal distance of the smallest enclosed region containing both the predicted and ground truth bounding boxes; the superscript * indicates that the smallest bounding box is separated from the computation graph; β represents the outlier, used to define the quality of the anchor box; α and δ are two hyperparameters, set to 1.9 and 3, respectively.

[0089] In this embodiment, YOLOv8 is an open-source, single-stage object detection algorithm developed by Ultralytics.

[0090] Step 3: Input the image of the fall to be identified into the fall detection model to obtain the fall detection result.

[0091] This embodiment also discloses a fall detection system, including:

[0092] Dataset Acquisition Module: Used to acquire fall datasets;

[0093] Fall detection model training module: The fall dataset is input into the improved YOLOv8 model for training to obtain the fall detection model;

[0094] Fall detection result module: used to input the fall image to be identified into the fall detection model to obtain the fall detection result of the fall image to be identified.

[0095] Example 2

[0096] This embodiment demonstrates the beneficial effects of the method of the present invention by performing target detection on a fall image through a specific experiment.

[0097] Experimental Platform: This experiment was conducted on a Linux operating system, using PyCharm Community as the integrated development environment (IDE) and implementing the model framework in Python. The main hardware configuration was: Ubuntu 22.04 64-bit operating system, Intel Core i5-13600KF CPU, GeForce RTX 2080ti (24GB) GPU, and 32GB RAM. The deep learning development environment consisted of: PyCharm 2023.1.2, Python 3.9, CUDA 12.1, cuDNN 8.9.0, and PyTorch 2.0.1.

[0098] Dataset: The images in this dataset are re-annotated manually based on three publicly available datasets: URFall Detection Dataset (URFD), Fall detectionDataset, and Multiple cameras fall dataset (MCFD). The dataset contains four pose types: fall, sit, stoop, and stand. Figure 6 Example images selected from the aforementioned fall dataset.

[0099] Experimental Setup: To verify the effectiveness of this invention, all experiments were conducted from scratch. The parameters were set as follows: a total of 300 epochs, a batch size of 64, an initial learning rate of 0.01, an Adam optimizer, and a weight decay of 0.0005. The first 3 epochs were the warmup phase, during which the learning rate decayed from 0.1 to 0.01 over three learning cycles. Mosica enhancement was disabled in the last 10 epochs. Specific parameter settings are shown in Table 1.

[0100] Table 1 Experimental parameter settings

[0101] Maximum number of training epochs 300 Learning rate (starting) 0.01 Batch size 64 Weight decay 0.0005 Optimizer Adam

[0102] Experimental Results: This experiment compared the performance of the YOLOv8 algorithm after incorporating the improved model, and also compared it with the results of YOLOv5. Four metrics were considered: parameter count, GFLOPs, mAP@.5, and mAP@.95. The model performance and complexity were compared. The comparison revealed that the proposed OEF-YOLO model improves detection accuracy with only a slight increase in parameters, and GFLOPs are also reduced. See Table 2 for details. The final detection results are shown below. Figure 7 .

[0103] Table 2 Comparison of OEF-YOLOv8 with various detector versions

[0104]

[0105] Meanwhile, this embodiment implements each of the three steps on the YOLOv8 detector. The YOLOv8 model size used in the experiment is the lightest n, and the experimental results are shown in Table 3.

[0106] Table 3 OEF-YOLOv8 Ablation Experiment

[0107] YOLOV8 3 8.1 96.4 72.5 YOLOV8+FasterNext 2.6 7.5 97.6 75.0 YOLOV8+FasterNext+DDConv 4.1 7.4 97.8 75.2 YOLOV8+FasterNext+DDConv+WIoU 4.1 7.3 97.9 74.3

[0108] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fall detection method, characterized in that, Includes the following steps: Obtain the fall dataset; The fall dataset is input into the improved YOLOv8 model for training to obtain the fall detection model; The fall detection model is input into the image of the fall to be identified to obtain the fall detection result of the image of the fall to be identified.

2. The fall detection method according to claim 1, characterized in that, The fall dataset includes several fall images and the types of human poses in the fall images.

3. The fall detection method according to claim 1, characterized in that, The improved YOLOv8 model is based on the traditional YOLOv8 model and incorporates the following improvements: The backbone network of YOLOv8 was replaced with the FasterNext module; In the neck network of YOLOv8, a dynamically deformable convolution is introduced to replace the convolutional block used for downsampling; The bounding box regression loss function of YOLOv8 is improved by introducing WIoU to replace the original CIoU.

4. The fall detection method according to claim 3, characterized in that, The Fasternet module's Fasternet_Block extracts spatial features from a portion of the input channels using PConv while keeping the remaining channels unchanged; it then processes the input channels and the channels resulting from the PConv convolution using 1×1 convolutions, where h and w are the height and width of the input data, and c is the number of input channels. p Here, k represents the number of input channels, and k is the kernel size. The FLOPs calculation process for standard convolution and partial convolution is as follows: FLOPs1=h×w×k 2 ×c 2 ; The memory access calculation process for standard convolution and partial convolution is as follows: h×w×2c+k 2 ×c 2 ≈h×w×2c p 。 5. A fall detection method according to claim 3, characterized in that, In dynamically deformable convolution, the shape change of the convolution kernel is based on network learning with deformation offset. The input feature map X is sampled using a regular convolution kernel S, and then a weighted summation operation is performed using a weight matrix M.

6. The fall detection method according to claim 3, characterized in that, The WIoU is introduced to replace the original CIoU, as shown in the following formula: Among them, b and b gt ρ represents the centroids of the predicted bounding box and the ground truth bounding box, respectively; ρ represents the Euclidean distance between the two centroids; d represents the diagonal distance of the smallest enclosed region containing both the predicted and ground truth bounding boxes; the superscript * indicates that the smallest bounding box is separated from the computation graph; β represents the outlier; α and δ are two hyperparameters.

7. A fall detection system, characterized in that, include: Dataset Acquisition Module: Used to acquire fall datasets; Fall detection model training module: The fall dataset is input into the improved YOLOv8 model for training to obtain the fall detection model; Fall detection result module: used to input the fall image to be identified into the fall detection model to obtain the fall detection result of the fall image to be identified.

Images