Fall event identification method, device and medium

Abstract

The present disclosure relates to a fall event recognition method, device and medium, and belongs to the field of computer technology. The method includes: using a sensor to collect an image of a target location; using a target detection model to recognize the image to obtain a bounding box of a human body and a bounding box of a bed; calculating the intersection and union ratio according to the bounding box of the human body and the bounding box of the bed; judging whether the intersection and union ratio is less than the intersection and union ratio threshold; if the intersection and union ratio is less than the intersection and union ratio threshold, determining that the human body is in an active state; using the GhostNet‑OpenPose posture estimation network to recognize the skeleton key points of the human body, and outputting the coordinates of each skeleton key point; based on the coordinates of each skeleton key point, calculating the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint; inputting the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into the decision tree fall classification algorithm to identify fall events. The present disclosure can accurately identify fall events.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a fall event recognition method, device, and medium. Background Art

[0002] Falls are a common and serious emergency for empty-nest elderly people. Falls and fainting in this population are often difficult to detect in time. Some elderly people choose nursing homes, but caregivers have limited attention spans and are prone to overlooking falls. Furthermore, fainting and falls in hospital wards often go undetected, leading to incalculable consequences.

[0003] Currently, there are two methods for fall event recognition:

[0004] (1) Fall event recognition method based on environmental sensors: This method mainly obtains real-time information about the human body by setting up floor pressure sensors, radar, acoustic sensors, etc. in the self-monitoring area. It integrates multiple sets of data to analyze the human body's movement patterns, thereby detecting abnormal movement behavior and determining the occurrence of fall behavior. Although this method is minimally invasive, it is sensitive to noise. Environmental sensors have blind spots, high configuration costs, and poor accuracy.

[0005] (2) Wearable device-based fall event recognition method: The sensor built into the transmission device collects human motion data, compares the human motion data with the threshold, and determines whether a fall event has occurred. This method has a high false alarm rate and requires the device to be worn at all times. If you forget to wear it, it will not be possible to identify whether a fall event has occurred. Battery life is also a pain point of this method.

[0006] (3) Fall recognition method based on visual information: This method obtains human motion information from monitoring equipment and analyzes the collected videos and images to determine the possibility of a fall. This algorithm does not require any equipment to be carried, has a good user experience and a high detection accuracy. Compared with fall recognition based on wearable devices, fall recognition based on visual information has less interference and higher accuracy and robustness. Fall feature extraction based on this method is generally divided into traditional methods and deep learning methods. Among them, the feature extraction method based on deep learning can achieve end-to-end learning. Before learning, there is no need to extract image edges, colors, spatial positions and other information, and useful features can be automatically extracted from the raw data through a deep network structure. In terms of fall recognition, deep learning can realize automatic feature extraction and analysis of monitoring videos, reduce the burden of manual monitoring, save labor costs, and extract rich multidimensional features, including posture, movement, environment and other information, which helps to more comprehensively identify fall events. With the continuous development of deep learning in the field of human posture recognition, accurate extraction of key points of human skeleton through convolutional neural networks is conducive to effective analysis of human posture. However, most of the currently proposed algorithms with high accuracy based on this method have complex network structures and cannot meet real-time requirements. Fall behavior recognition based on surveillance video streams is different from other image analysis tasks. The recognition process must be sufficiently accurate and rapid, and the algorithm must account for issues such as blurry, distorted, or noisy images in ordinary security monitoring under poor lighting conditions. Furthermore, the application scenarios of common fall recognition methods are relatively limited. When a person is in an environment with furniture for reclining and resting, a reclining posture can easily be confused with a fall, resulting in false alarms. Summary of the Invention

[0007] The present disclosure provides a fall event recognition method, device, and medium to solve the above-mentioned technical problems.

[0008] According to a first aspect of the present disclosure, a fall event recognition method is provided, the method comprising: using a sensor to collect an image of a target place, wherein the target place contains a human body and a bed; recognizing the image with the aid of a target detection model to obtain a bounding box of the human body and a bounding box of the bed, wherein, when constructing the target detection model, a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n; calculating an intersection-over-union (IoU) based on the bounding box of the human body and the bounding box of the bed; judging whether the IoU is less than an IoU threshold; if the IoU is less than the IoU threshold, judging that the human body is in an active state; using a GhostNet-OpenPose posture estimation network to identify skeletal key points of the human body, and outputting the coordinates of each skeletal key point, wherein, when constructing the GhostNet-OpenPose, the feature extraction part is replaced by GhostNet with VGG-19, and OpenPos e. A multi-branch iterative mechanism retains only an initial stage and a refinement stage, merges some convolutional structures to calculate key point heat maps and partial affinity fields, and uses the COCO dataset as a training set; based on the coordinates of each skeletal key point, calculates the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint; inputs the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into a decision tree fall classification algorithm to identify fall events, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeletal key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeletal key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

[0009] In some embodiments, before the image is recognized with the aid of a target detection model, the method further includes: reading the image frame by frame; adjusting each frame of the image to a preset size; and performing histogram equalization on the adjusted image to complete preprocessing of the image.

[0010] In some embodiments, if the IoU is greater than or equal to an IoU threshold, it is determined that the human body is resting in bed.

[0011] In some embodiments, after inputting the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into the decision tree fall classification algorithm to identify the fall event, it also includes: broadcasting a "fall" alarm message with the help of an alarm system.

[0012] According to a second aspect of the present disclosure, a fall event recognition device is provided, comprising: an image acquisition module for using a sensor to acquire an image of a target place, wherein the target place contains a human body and a bed; an image recognition module for recognizing the image with the aid of a target detection model to obtain a bounding box of the human body and a bounding box of the bed, wherein, when constructing the target detection model, a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n; an intersection-in-union (IoU) calculation module for calculating an IoU based on the bounding box of the human body and the bounding box of the bed; an IoU judgment module for judging whether the IoU is less than an IoU threshold; an active state judgment module for judging that the human body is in an active state if the IoU is less than an IoU threshold; a skeletal key point recognition module for identifying skeletal key points of the human body using a GhostNet-OpenPose posture estimation network and outputting the coordinates of each skeletal key point, wherein, when constructing the GhostNet-OpenPose, the feature extraction part is extracted with GhostNet. t replaces VGG-19, and the OpenPose multi-branch iterative mechanism only retains the initial stage and one refinement stage, merges some convolution structures to calculate the key point heat map and part of the affinity field, and uses the COCO dataset as the training set; a real-time joint angle calculation module is used to calculate the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint based on the coordinates of each skeletal key point; a fall event recognition module is used to input the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into a decision tree fall classification algorithm to identify fall events, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeletal key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeletal key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

[0013] According to a third aspect of the present disclosure, a fall event recognition device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the above-mentioned fall event recognition method based on instructions stored in the memory.

[0014] According to a fourth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the above-mentioned fall event recognition method is implemented.

[0015] Beneficial effects of the present disclosure:

[0016] (1) The histogram equalization technology was introduced in the image preprocessing stage, which enhanced the contrast of the image, improved the visibility of image details, and effectively improved the image quality, thereby increasing the recognition accuracy of the bed and the human body by nearly three percentage points respectively.

[0017] (2) The introduction of the CBAM module combines the channel attention mechanism and the spatial attention mechanism. The CBAM module can compress the spatial dimension without changing the channel dimension, better focus on the location information of the target, and accurately generate the bounding box of the human body and the bounding box of the bed.

[0018] (3) The original OpenPose key point detection network is optimized, and Long GhoseNet is used as the backbone feature extraction network, which includes technologies such as SE attention module and depthwise separable convolution, reducing the number of floating-point operations from 8.85GFLOPs to 6.28GFLOPs, effectively reducing the number of parameters and computational complexity of the model, enabling real-time recognition, and improving the recognition accuracy on the COCO key point recognition dataset by 9.5%, making it suitable for deployment and application in resource-constrained environments.

[0019] (4) We selected multiple features that best represent abnormal posture changes in the human body and adopted a decision tree model that can effectively solve the classification problem. Experimental evaluation showed that the model achieved an overall accuracy rate of 95.46% in identifying fall events, making the response to fall events more reliable. At the same time, it also achieved a good balance between precision and recall, with the calculated values ​​of the two being 94.87% and 93.54% respectively, which can take into account the recognition of both positive and negative samples.

[0020] As mentioned above, the technical means provided by the present disclosure can be applied to specific indoor places containing beds and human bodies. As a technical means for auxiliary care, it can improve the quality of life of the elderly and specific groups of people and reduce the occurrence of accidental injuries. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings.

[0023] Figure 1 is a flowchart illustrating a method for identifying a fall event according to some embodiments of the present disclosure.

[0024] Figure 2 is an original picture showing a target to be identified according to some embodiments of the present disclosure;

[0025] Figure 32 is a diagram showing the effect of YOLOv8n target recognition combined with the attention mechanism according to some embodiments of the present disclosure.

[0026] Figure 4 is a structural diagram illustrating a GhostNet-OpenPose pose estimation network according to some embodiments of the present disclosure.

[0027] Figure 5 1 is one of the diagrams showing the GhostNet-OpenPose recognition effect according to some embodiments of the present disclosure.

[0028] Figure 6 This is a second diagram showing the GhostNet-OpenPose recognition effect according to some embodiments of the present disclosure.

[0029] Figure 7 3 is a diagram showing the GhostNet-OpenPose recognition effect according to some embodiments of the present disclosure.

[0030] Figure 8 Schematic diagram showing human body activity states according to some embodiments of the present disclosure.

[0031] Figure 9 Schematic diagram showing a human body falling according to some embodiments of the present disclosure.

[0032] Figure 10 Schematic diagram showing a human body in a resting state according to some embodiments of the present disclosure.

[0033] Figure 11 is a block diagram illustrating a fall event recognition apparatus according to some embodiments of the present disclosure.

[0034] Figure 12 is a block diagram illustrating a fall event recognition device according to some other embodiments of the present disclosure.

[0035] Figure 13 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0036] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0037] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0038] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0039] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0040] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0041] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0042] With the aging of my country's population, the reality of aging and the challenges of providing for the elderly have become increasingly pressing. The social and family issues arising from this issue are particularly thorny. Falls are a common and serious emergency for empty-nest elderly people, and falls and fainting in this population are often difficult to detect in a timely manner. Some elderly people choose to live in nursing homes, but caregivers have limited resources and are prone to overlooking falls. Furthermore, fainting and falls in hospital wards often go undetected, leading to incalculable consequences.

[0043] There are two approaches to fall event detection: Environmental sensor-based fall event detection primarily uses floor pressure sensors, radar, and acoustic sensors within the monitoring area to obtain real-time human motion information. This data is then integrated to analyze human motion patterns, identifying abnormal movement behavior and determining a fall. While this approach is minimally invasive, it is sensitive to noise. Environmental sensors have blind spots, high deployment costs, and poor accuracy. Wearable device-based fall event detection uses sensors embedded in a transmitter to collect human motion data, comparing it to a threshold to determine if a fall has occurred. This approach has a high false alarm rate and requires the device to be worn at all times. If the device is forgotten, a fall cannot be detected. Battery life is also a drawback of this approach. Visual information-based fall detection: This approach obtains human motion information from monitoring devices and analyzes the captured video and images to determine the likelihood of a fall. This algorithm requires no equipment, offers a good user experience, and offers high detection accuracy. Compared to wearable device-based fall detection, visual information-based fall detection is less intrusive and offers higher accuracy and robustness. Fall feature extraction based on this method is generally categorized into traditional methods and deep learning approaches. Deep learning-based feature extraction methods achieve end-to-end learning, eliminating the need to extract image information such as edges, color, and spatial position prior to learning. Instead, they automatically extract useful features from raw data through deep network structures. In fall detection, deep learning can automatically extract and analyze features from surveillance videos, alleviating the burden of manual monitoring and saving labor costs. It also extracts rich multidimensional features, including posture, movement, and environmental information, facilitating more comprehensive fall identification. With the continuous advancement of deep learning in the field of human posture recognition, the accurate extraction of key points of the human skeleton using convolutional neural networks facilitates effective posture analysis. However, most of the currently proposed algorithms with high accuracy based on this method have complex network structures and cannot meet real-time requirements. Fall detection based on surveillance video streams differs from other image analysis tasks in that the recognition process must be sufficiently accurate and rapid, and the algorithm must account for issues such as blur, distortion, and noise that occur in standard security surveillance footage under low lighting conditions. In addition, the application scenarios of the current common fall recognition methods are relatively limited. When there is furniture for lying down and resting in the environment where a person is located, the person's lying posture can easily be confused with falling behavior, resulting in false alarms.

[0044] In light of this, the present disclosure proposes a fall event recognition method, device, and medium. Histogram equalization (CBA) technology is introduced during the image preprocessing stage to enhance image contrast, improve the visibility of image details, and effectively improve image quality, thereby increasing the recognition accuracy of beds and people by nearly three percentage points. The introduction of the CBAM module combines the channel attention mechanism and the spatial attention mechanism. The CBAM module compresses the spatial dimension without changing the channel dimension, better focusing on the target's position information and accurately generating bounding boxes for the person and bed. The original OpenPose keypoint detection network is optimized, using DragonGhoseNet as the backbone feature extraction network. This incorporates techniques such as the SE attention module and depthwise separable convolution, reducing the number of floating-point operations from 8.85 GFLOPs to 6.28 GFLOPs, effectively reducing the model's parameter count and computational complexity. This enables real-time recognition and improves recognition accuracy by 9.5% on the COCO keypoint recognition dataset, making it suitable for deployment and application in resource-constrained environments. Multiple features that best represent abnormal posture changes in the human body are selected, and a decision tree model is employed to effectively address classification problems. Experimental evaluations have shown that the model achieves an overall accuracy rate of 95.46% for fall event recognition, making it more reliable in responding to fall events. It also achieves a good balance between precision and recall, with calculated values ​​of 94.87% and 93.54% respectively, allowing for the identification of both positive and negative samples. As described above, the technical means provided by this disclosure can be applied to specific indoor locations containing beds and human bodies. As a technical means of assisting caregivers, it can improve the quality of life of the elderly and specific groups of people and reduce the occurrence of accidental injuries.

[0045] Figure 1 FIG. 1 is a flow chart illustrating a method for identifying a fall event according to some embodiments of the present disclosure. Figure 1 As shown, the fall event recognition method includes steps 110 to 180.

[0046] In step 110 , an image of a target location is collected using a sensor, wherein a human body and a bed are located in the target location.

[0047] In some embodiments, the method further includes: reading the image frame by frame; adjusting each frame of the image to a preset size; and performing histogram equalization on the adjusted image to complete preprocessing of the image.

[0048] In step 120, the image is recognized by means of a target detection model to obtain a bounding box of the human body and a bounding box of the bed. When constructing the target detection model, a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n. Figure 2 and Figure 3 shown.

[0049] In step 130 , an intersection-over-union (IoU) is calculated based on the bounding box of the human body and the bounding box of the bed.

[0050] In step 140 , it is determined whether the IoU is less than an IoU threshold.

[0051] In some embodiments, if the IoU is greater than or equal to an IoU threshold, it is determined that the human body is resting in bed.

[0052] In step 150 , if the IoU is less than an IoU threshold, it is determined that the human body is in an active state.

[0053] In step 160, the GhostNet-OpenPose pose estimation network is used to identify the skeleton key points of the human body, and the coordinates of each skeleton key point are output. When constructing the GhostNet-OpenPose, the feature extraction part is replaced by VGG-19 with GhostNet, the OpenPose multi-branch iterative mechanism only retains the initial stage and one refinement stage, and some convolution structures are merged to calculate the key point heat map and some affinity fields. The COCO dataset is used as the training set.

[0054] In step 170 , based on the coordinates of each skeletal key point, the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint are calculated.

[0055] In step 180, the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint are input into a decision tree fall classification algorithm to identify fall events, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, and the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeleton key point output by the GhostNet-OpenPose pose estimation network, and the historical data of the coordinates of the skeleton key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

[0056] In some embodiments, the method further includes: broadcasting a "fall" alarm message via an alarm system.

[0057] This method uses an object detection network with an attention mechanism to identify the boundaries of indoor human bodies and beds, and explores the optimal intersection-over-union ratio judgment threshold for human activity areas through experiments. Figure 8 、 Figure 9 、 Figure 10 As shown in the figure, the algorithm can measure the overlap between the human body and the bed by calculating the intersection of the two target detection frames and then judge the current activity state of the subject. Figure 8 and Figure 9 The overlap shown does not reach the set threshold, so it is in the active area. Figure 10 The overlap shown exceeds the set threshold and is in the rest area. Figure 4 As shown in , the improved lightweight posture estimation network is used to identify the coordinates of the key points of the human skeleton in the active area, as shown in Figure 5 、 Figure 6 and Figure 7 As shown, the training set used for training the lightweight posture estimation network is stated as follows: the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeleton key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeleton key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model. If and only if the person is in the activity area and the classifier determines that it is a fall posture, as shown Figure 9 As shown, it is determined that the person has fallen and a "fall" alarm message is sent.

[0058] Histogram equalization processing enhances image contrast and highlights picture details.

[0059] The intersection-over-union ratio threshold value may be set to 0.35. The embodiments of the present disclosure are not limited here.

[0060] The advantages of the present invention are as follows: (1) The histogram equalization processing technology is introduced in the image preprocessing stage, which enhances the image contrast, improves the visibility of image details, and effectively improves the image quality. The same batch of images are preprocessed with and without histogram equalization, and then the two groups of images are sent to the trained YOLOv8-n bed and human target detection network respectively. The histogram equalization processing can increase the average accuracy of bed and human target recognition in the image by nearly 3 percentage points respectively. Experiments show that the introduction of this method makes the image easier to process and analyze.

[0061] (2) The introduction of the CBAM module combines the channel attention mechanism and the spatial attention mechanism. This module can compress the spatial dimension without changing the channel dimension, better focus on the location information of the target, and accurately generate the bounding boxes of the bed and person targets. The attention mechanism module used in this study was inserted between the Backbone and Neck of YOLOv8-n. The network was trained using the COCO dataset, focusing only on the two target categories of Bed and Person. The training used the original weights for transfer learning. The training results show that the introduction of the CBAM module significantly improved the network's target recognition accuracy.

[0062] (3) The original OpenPose key point recognition network is optimized and GhostNet is used as the backbone feature extraction network, which includes technologies such as SE attention module and depthwise separable convolution, reducing the number of floating-point operations from 8.85GFLOPs to 6.28GFLOPs, effectively reducing the number of parameters and computational complexity of the model, enabling real-time recognition, and improving the recognition accuracy on the COCO key point recognition dataset by 9.5%, making it suitable for deployment and application in resource-constrained environments.

[0063] (4) We selected multiple features that best represent abnormal posture changes in the human body and adopted a decision tree model that can effectively solve the classification problem. Experimental evaluation showed that the model can accurately predict fall behavior overall, with a recognition accuracy of 95.46%, making it more reliable in responding to fall events. At the same time, it also achieved a good balance between precision and recall, with the calculated values ​​of the two being 94.87% and 93.54% respectively, and can take into account the recognition of both positive and negative samples.

[0064] The embodiments of the present disclosure have a wide range of applications and the method is highly versatile. It can be applied to indoor places containing beds and people, such as wards, bedrooms, and nursing homes. As auxiliary care, it can improve the quality of life of the elderly and specific groups of people and reduce the occurrence of accidental injuries.

[0065] Figure 11 FIG. 1 is a block diagram illustrating a fall event recognition apparatus according to some embodiments of the present disclosure. Figure 11 As shown, the fall event recognition device 1100 includes an image acquisition module 1110, an image recognition module 1120, an intersection-over-union calculation module 1130, an intersection-over-union judgment module 1140, an activity state judgment module 1150, a skeleton key point recognition module 1160, a real-time joint angle calculation module 1170, and a fall event recognition module 1180.

[0066] The image acquisition module 1110 is configured to use a sensor to acquire an image of a target location, wherein the target location contains a human body and a bed;

[0067] An image recognition module 1120 is configured to recognize the image using an object detection model to obtain a bounding box of a human body and a bounding box of a bed. When constructing the object detection model, a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n.

[0068] The intersection-over-union (IoU) calculation module 1130 is configured to calculate an IoU based on the bounding box of the human body and the bounding box of the bed;

[0069] The IoU judging module 1140 is configured to judge whether the IoU is less than an IoU threshold;

[0070] An activity state determination module 1150 is configured to determine that the human body is in an activity state if the intersection-over-union ratio is less than an intersection-over-union ratio threshold;

[0071] A skeletal keypoint identification module 1160 is configured to use the GhostNet-OpenPose pose estimation network to identify skeletal keypoints of the human body and output the coordinates of each skeletal keypoint. When constructing the GhostNet-OpenPose, the feature extraction portion is replaced by VGG-19 with GhostNet, the OpenPose multi-branch iterative mechanism retains only the initial stage and one refinement stage, and some convolutional structures are merged to calculate keypoint heat maps and some affinity fields. The COCO dataset is used as the training set;

[0072] The real-time joint angle calculation module 1170 is configured to calculate the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint based on the coordinates of each skeletal key point;

[0073] The fall event recognition module 1180 is configured to input the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into a decision tree fall classification algorithm to identify a fall event, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, and the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeleton key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeleton key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

[0074] In the device of the disclosed embodiment, histogram equalization technology is introduced during the image preprocessing stage, enhancing image contrast and improving the visibility of image details, effectively improving image quality and thereby increasing the recognition accuracy of beds and people by nearly three percentage points, respectively. The introduction of the CBAM module combines the channel attention mechanism with the spatial attention mechanism. The CBAM module compresses the spatial dimension without changing the channel dimension, better focusing on the target's positional information and accurately generating bounding boxes for the person and bed. The original OpenPose keypoint detection network is optimized, with Long GhoseNet serving as the backbone feature extraction network. This incorporates techniques such as the SE attention module and depthwise separable convolution, reducing the number of floating-point operations from 8.85 GFLOPs to 6.28 GFLOPs, effectively reducing the model's parameter count and computational complexity, enabling real-time recognition. The recognition accuracy on the COCO keypoint recognition dataset increased by 9.5%, making it suitable for deployment and application in resource-constrained environments. Multiple features that best represent abnormal posture variations in the human body are selected, and a decision tree model is employed to effectively address classification problems. Experimental evaluations have shown that the model achieves an overall accuracy rate of 95.46% for fall event recognition, making it more reliable in responding to fall events. It also achieves a good balance between precision and recall, with calculated values ​​of 94.87% and 93.54% respectively, allowing for the identification of both positive and negative samples. As described above, the technical means provided by this disclosure can be applied to specific indoor locations containing beds and human bodies. As a technical means of assisting caregivers, it can improve the quality of life of the elderly and specific groups of people and reduce the occurrence of accidental injuries.

[0075] Figure 12 FIG. 1 is a block diagram illustrating a fall event recognition device according to other embodiments of the present disclosure. Figure 12 As shown, the fall event recognition device 1200 includes a memory 1210 and a processor 1220 coupled to the memory 1210. The memory 1210 is used to store instructions for executing the corresponding embodiments of the fall event recognition method. The processor 1220 is configured to execute the fall event recognition method according to any of the embodiments of the present disclosure based on the instructions stored in the memory 1210.

[0076] Figure 13 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. Figure 13 As shown, computer system 1300 may be implemented as a general-purpose computing device and includes a memory 1310, a processor 1320, and a bus 1330 that connects various system components.

[0077] The memory 1310 may include, for example, system memory, non-volatile storage media, and the like. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for executing at least one embodiment of the fall event identification method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0078] Processor 1220 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, or discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the image acquisition module, image recognition module, intersection-over-union (IoU) calculation module, IoU determination module, activity state determination module, skeletal key point recognition module, real-time joint angle calculation module, and fall event recognition module, can be implemented by a central processing unit (CPU) executing instructions in memory that execute corresponding steps, or by dedicated circuits that execute corresponding steps.

[0079] The bus 1330 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0080] Computer system 1300 may also include input / output interfaces 1340, a network interface 1350, a storage interface 1360, and the like. These interfaces 1340, 1350, and 1360, as well as memory 1310 and processor 1320, may be connected via bus 1330. Input / output interfaces 1340 provide connection interfaces for input / output devices such as a display, mouse, and keyboard. Network interface 1350 provides connection interfaces for various networked devices. Storage interface 1360 provides connection interfaces for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0081] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0082] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0083] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to operate in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0084] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0085] In this disclosure, histogram equalization technology is introduced in the image preprocessing stage to enhance image contrast, improve the visibility of image details, and effectively improve image quality, thereby increasing the recognition accuracy of beds and people by nearly three percentage points respectively. The introduction of the CBAM module combines the channel attention mechanism and the spatial attention mechanism. The CBAM module can compress the spatial dimension without changing the channel dimension, better focus on the location information of the target, and accurately generate the bounding boxes of the person and the bed. The original OpenPose key point detection network is optimized, and Long GhoseNet is used as the backbone feature extraction network. It includes technologies such as SE attention module and depthwise separable convolution, which reduces the number of floating-point operations from 8.85GFLOPs to 6.28GFLOPs, effectively reducing the number of model parameters and computational complexity, enabling real-time recognition. The recognition accuracy on the COCO key point recognition dataset is improved by 9.5%, making it suitable for deployment and application in resource-constrained environments. Multiple features that best represent abnormal posture changes of the human body are selected, and a decision tree model that can effectively solve classification problems is adopted. Experimental evaluations have shown that the model achieves an overall accuracy rate of 95.46% for fall event recognition, making it more reliable in responding to fall events. It also achieves a good balance between precision and recall, with calculated values ​​of 94.87% and 93.54% respectively, allowing for the identification of both positive and negative samples. As described above, the technical means provided by this disclosure can be applied to specific indoor locations containing beds and human bodies. As a technical means of assisting caregivers, it can improve the quality of life of the elderly and specific groups of people and reduce the occurrence of accidental injuries.

[0086] Thus far, the fall event identification method, apparatus, and medium according to the present disclosure have been described in detail. To avoid obscuring the concepts of the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will fully understand how to implement the technical solutions disclosed herein.

[0087] Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art will understand that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A fall event recognition method, characterized in that: The method comprises: Using a sensor to collect an image of a target location, wherein a human body and a bed are located in the target location; The image is recognized by means of a target detection model to obtain a bounding box of a human body and a bounding box of a bed, wherein a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n when constructing the target detection model; Calculating an intersection-over-union ratio according to the bounding box of the human body and the bounding box of the bed; Determine whether the intersection-over-union ratio is less than an intersection-over-union ratio threshold; If the intersection-over-union ratio is less than the intersection-over-union ratio threshold, it is determined that the human body is in an active state; The GhostNet-OpenPose posture estimation network is used to identify the skeleton key points of the human body and output the coordinates of each skeleton key point. When constructing the GhostNet-OpenPose, the feature extraction part is replaced by VGG-19 with GhostNet, the OpenPose multi-branch iterative mechanism only retains the initial stage and a refinement stage, and some convolution structures are merged to calculate the key point heat map and some affinity fields, and the COCO dataset is used as the training set; Based on the coordinates of each bone key point, the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint are calculated; The real-time joint angle of the elbow joint and the real-time joint angle of the knee joint are input into a decision tree fall classification algorithm to identify fall events, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeleton key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeleton key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

2. The fall event recognition method according to claim 1, characterized in that: Before the image is recognized by means of the target detection model, the method further includes: Reading the image frame by frame; Resize each frame to the preset size; The adjusted image is subjected to histogram equalization processing to complete the preprocessing of the image.

3. The fall event recognition method according to claim 1 or 2, characterized in that: If the intersection-and-union ratio is greater than or equal to the intersection-and-union ratio threshold, it is determined that the human body is resting in bed.

4. The fall event recognition method according to claim 1 or 2, characterized in that: After inputting the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into the decision tree fall classification algorithm to identify the fall event, it also includes: broadcasting a "fall" alarm message with the help of an alarm system.

5. A fall event recognition device, characterized in that: include: An image acquisition module, used for acquiring an image of a target location using a sensor, wherein the target location contains a human body and a bed; An image recognition module, used to recognize the image with the help of a target detection model to obtain a bounding box of a human body and a bounding box of a bed, wherein when constructing the target detection model, a CBAM module combining a channel attention mechanism and a spatial attention mechanism is added to YOLOv8n; An intersection-and-union ratio calculation module, used to calculate the intersection-and-union ratio according to the bounding box of the human body and the bounding box of the bed; An I / O ratio judgment module, used to judge whether the I / O ratio is less than an I / O ratio threshold; An activity state determination module, configured to determine that the human body is in an activity state if the intersection-over-union ratio is less than an intersection-over-union ratio threshold; A skeleton key point recognition module is used to identify the skeleton key points of the human body using the GhostNet-OpenPose posture estimation network and output the coordinates of each skeleton key point. When constructing the GhostNet-OpenPose, the feature extraction part replaces VGG-19 with GhostNet, the OpenPose multi-branch iterative mechanism only retains the initial stage and one refinement stage, merges some convolution structures to calculate the key point heat map and some affinity fields, and uses the COCO data set as a training set; A real-time joint angle calculation module is used to calculate the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint based on the coordinates of each bone key point; A fall event recognition module is used to input the real-time joint angle of the elbow joint and the real-time joint angle of the knee joint into a decision tree fall classification algorithm to identify a fall event, wherein the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the knee joint are used as training sets to train the decision tree fall classification algorithm, the historical data of the joint angle of the elbow joint and the historical data of the joint angle of the elbow joint are calculated based on the historical data of the coordinates of each skeleton key point output by the GhostNet-OpenPose posture estimation network, and the historical data of the coordinates of the skeleton key point are calculated based on the historical data of the aspect ratio of the bounding box of the human body output by the target detection model.

6. A fall event recognition device, characterized in that: include: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the fall event recognition method according to any one of claims 1 to 4 based on instructions stored in the memory.

7. A computer storable medium, characterized in that: Computer program instructions are stored thereon, and when the instructions are executed by a processor, the fall event recognition method as described in any one of claims 1 to 4 is implemented.

Images