Fall detection method and device, electronic equipment and computer storage medium

CN119028018APending Publication Date: 2024-11-26BEIJING JINGDONG TUOXIAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411147166.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing fall detection technology based on deep learning has problems such as missed or misdetection in human detection accuracy and human condition detection. The model has low accuracy in human detection and area detection due to image rotation.

Method used

Adaptive rotary convolution module (LARC module) and lightweight convolution module (CGhost module) are combined to extract image features of different orientations through rotation and convolution processing, enhance the model's ability to detect human posture and state, and pass through the channel Attention and spatial attention mechanisms adjust the area weight of the feature map to improve detection accuracy and accuracy.

Benefits of technology

Improve the accuracy and accuracy of fall detection, reduce missed detection and missed detection, enhance the model's detection ability of different angles and environments, and is suitable for embedded devices, achieving higher detection generalization capabilities and real-time monitoring. application.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028018A_ABST
    Figure CN119028018A_ABST
Patent Text Reader

Abstract

The invention provides a fall detection method and device, electronic equipment and a storage medium, and relates to the field of image processing technologies, intelligent monitoring and intelligent old-age care. The image feature extraction method disclosed by the invention comprises the following steps: obtaining features in different orientations in an image through rotation and convolution processing on the image; obtaining a target feature map according to the features of different orientations; determining image features according to the target feature map; and determining that a fall event occurs according to the image features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the fields of image processing, smart monitoring and smart elderly care technology, and in particular to a fall detection method, device, electronic device and computer storage medium. Background Art

[0002] As the population ages, falls have become a major threat to human life expectancy. Therefore, developing an efficient and accurate fall detection system is of great significance for preventing falls in the elderly and providing timely assistance. In addition, considering the high cost of manual care and the possibility that falls may occur in various environments, an automated fall detection system can be used as an effective auxiliary tool to reduce the workload of caregivers and improve care efficiency.

[0003] Fall detection technology based on deep learning mainly uses computer vision and deep learning algorithms to analyze video streams or images to achieve real-time detection of fall events. These technologies usually use efficient target detection algorithms, such as the YOLO (You Only Look Once) series, to identify fall behaviors by analyzing human posture and movement patterns. These systems can be applied to a variety of scenarios, including home monitoring of the elderly, hospital ward monitoring, and public place monitoring to prevent and respond to fall events in a timely manner. Summary of the invention

[0004] One object of the present disclosure is to improve the precision and accuracy of fall detection.

[0005] According to one aspect of some embodiments of the present disclosure, a fall detection method is proposed, including: obtaining features of different orientations in an image by rotating and convolving the image; obtaining a target feature map based on the features of different orientations; determining image features based on the target feature map; and determining the occurrence of a fall event based on the image features.

[0006] In some embodiments, obtaining a target feature map according to features includes: obtaining a target feature map through linear convolution according to features in different orientations.

[0007] In some embodiments, obtaining a target feature map according to features of different orientations includes: obtaining a first feature map through linear convolution according to features of different orientations; obtaining a second feature map through nonlinear convolution according to features of different orientations; merging the first feature map and the second feature map, and obtaining a target feature map through linear convolution.

[0008] In some embodiments, determining image features based on the target feature map includes: adjusting weights of regions in the feature map to obtain image features.

[0009] In some embodiments, adjusting the weight of a region in a feature map to obtain image features includes: using a channel attention and a spatial attention mechanism to synchronously process the feature map to determine the weight of the region in the feature map; and determining the image features based on the weight and the feature map.

[0010] In some embodiments, the number of feature dimensions of the target feature map is greater than one.

[0011] In some embodiments, determining image features according to a target feature map includes: separately processing image feature maps of different feature sizes to obtain image features of corresponding sizes.

[0012] In some embodiments, determining the occurrence of a fall event based on the image features includes: determining a predicted bounding box based on the image features; and determining the occurrence of a fall event based on changes in the predicted bounding box.

[0013] In some embodiments, the method further includes: when a predicted bounding box belonging to an object of the target category is not detected in the current frame image, determining a predicted bounding box of the current frame image according to a predicted bounding box position in a historical image.

[0014] In some embodiments, determining the occurrence of a fall event based on changes in the predicted border includes: determining a state change of the target category object based on changes in at least one feature of the ratio, width, and height of the predicted border of adjacent frame images.

[0015] In some embodiments, extracting a predicted border based on an image is implemented based on a YOLOv8 model.

[0016] According to one aspect of some embodiments of the present disclosure, an image feature extraction device is proposed, comprising: a rotational convolution module, configured to obtain features of different orientations in an image by rotating and convolving the image; a feature map acquisition module, configured to obtain a target feature map based on features of different orientations; a feature determination module, configured to determine image features based on the target feature map; and a fall event determination unit, configured to determine the occurrence of a fall event based on the image features.

[0017] In some embodiments, the fall detection device runs an image feature extraction model based on the YOLOv8 model.

[0018] In some embodiments, the device meets at least one of the following conditions: the rotational convolution model is a convolution module of a YOLOv8 model; the feature map acquisition module is the backbone of the YOLOv8 model; and the feature determination module is located at the head of the YOLOv8 model.

[0019] According to one aspect of some embodiments of the present disclosure, an image feature extraction device is proposed, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the methods mentioned above based on instructions stored in the memory.

[0020] According to one aspect of some embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, any one of the methods mentioned above is implemented.

[0021] According to one aspect of some embodiments of the present disclosure, a computer program product is provided, including a computer program or instructions, and when the computer program or instructions are executed by a processor, any one of the methods mentioned above is implemented.

[0022] According to one aspect of some embodiments of the present disclosure, an electronic device is provided, comprising: the fall detection device mentioned above; and an alarm device, configured to issue an alarm when the fall detection device determines that a fall event has occurred. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute improper limitations on the present disclosure.

[0024] Figure 1 Flowcharts of some embodiments of the fall detection method disclosed herein.

[0025] Figure 2 Flowcharts of other embodiments of the fall detection method disclosed herein.

[0026] Figure 3 Schematic diagram of a model of some embodiments of the fall detection method disclosed in the present invention.

[0027] Figure 4 Schematic diagram of some embodiments of the adaptive rotation convolution module of the model of the fall detection method disclosed in the present invention.

[0028] Figure 5 Schematic diagram of some embodiments of an improved Ghost lightweight module of the model of the fall detection method disclosed in the present invention.

[0029] Figure 6 Schematic diagram of some embodiments of the collaborative attention module of the model of the fall detection method disclosed in the present invention.

[0030] Figure 7 Schematic diagrams of some embodiments of the fall detection device disclosed herein.

[0031] Figure 8Schematic diagrams of other embodiments of the fall detection device disclosed in the present invention.

[0032] Fig. 9 Schematic diagrams of some further embodiments of the fall detection device disclosed herein.

[0033] Fig.10 Schematic diagrams of some embodiments of the electronic device disclosed herein. DETAILED DESCRIPTION

[0034] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.

[0035] In the related art, there are many methods for fall detection.

[0036] Millimeter-wave radar based on Transformer and multi-scale convolutional neural network performs fall detection, performs short-time Fourier feature extraction (fall and non-fall behavior) for each type of radar signal; establishes training sets and data sets for the extracted spectrum features and loads them; constructs a multi-scale convolutional neural network model based on Transformer, trains a multi-scale convolutional neural network classification model based on Transformer, and completes classification and recognition. Compared with the fall behavior classification method using the traditional training classification framework, this method has higher accuracy and stronger robustness.

[0037] The fall prevention and detection method based on the agile development of machine learning algorithms collects the acceleration and heart rate data of users; adds the historical heart rate data to the cloud-based exercise attenuation warning model, and obtains the attenuation rate of the heart rate data based on the predicted heart rate data and historical data. If the attenuation rate is higher than the preset attenuation threshold, it is determined that the current state is attenuated; the user's acceleration data is used as the input of the fall detection algorithm model, and the fall detection algorithm model is used to predict the user's current state results. At the same time, the data is recycled through this mechanism for the training of the fall detection algorithm model. This method is flexible in application scenarios and low in cost; it can train the fall prediction model in a targeted manner based on the differences in physiological functions between individual users, and the fall prediction is more targeted.

[0038] Although deep learning-based fall detection technology has made significant progress, there are still some shortcomings and challenges. For example, the detection accuracy of people and human body status is not high, and there are cases of missed detection or false detection. The inventors found that part of the reason for this problem is that the monitoring sensors will rotate the direction of the collected images, and the rotation angle is uncertain, which affects the accuracy of the model in detecting the human body and the area occupied by the human body.

[0039] In response to the above problems, the present invention proposes an image feature extraction method that can adaptively extract features of objects in different orientations in an image, thereby providing a more accurate and comprehensive data basis for subsequent feature-based recognition of targets and their postures, thereby improving the precision and accuracy of detection.

[0040] The flowcharts of some embodiments of the fall detection method disclosed in the present invention are as follows: Figure 1 As shown, steps S11-S15 are included. In some embodiments, the fall detection method can be implemented based on a real-time object detector (such as a YOLOv8 model). For example, based on the modules of the YOLOv8 model and their connection relationships, some modules are replaced to produce additional technical effects without affecting the original functions.

[0041] In step S11 (or Figure 2 In step 210), the features of the image in different directions are obtained by rotating and convolving the image. In some embodiments, the image can be rotated at different angles during the convolution operation, and the feature extraction of different directions can be realized by using the rotation convolution kernel.

[0042] Features of objects in different orientations can be extracted by setting an adaptive convolution kernel. For example, based on YOLOv8, the Conv (convolution) module in its model structure can be replaced with a convolution module using an adaptive convolution kernel, thereby achieving the extraction of features in different orientations without changing the YOLOv8 architecture. In some embodiments, the improved structure of the image feature extraction method based on the YOLOv8 model can be as follows: Figure 3 As shown in , the Conv module is replaced by the LARC module 31 which can extract high-quality features of objects with different orientations. Figure 3 As shown in , the YOLOv8 model also includes the SPPF (Spatial PyramidPooling -Fast, spatial pyramid pooling - fast) module and the UnSample (deconvolution) module.

[0043] The inventors proposed a LARC (Lightweight Adaptive Rotational Convolution) module to extract high-quality features of objects in different orientations through adaptive convolution kernels. The structure of the LARC module can be as follows: Figure 4 As shown in , based on the Conv module, a rotation convolution kernel is added, and feature extraction in different directions of the image is achieved through adaptive rotation convolution. In addition, compared with the Conv module, the structure of the LARC module is lightweight, which is easy to deploy in embedded devices and is conducive to promotion and application.

[0044] In step S13, a target feature map is obtained according to the features of different orientations. In some embodiments, the C2f module of the YOLOv8 model can be used to obtain the target feature map.

[0045] In some embodiments, a target feature map is obtained by linear convolution according to features of different orientations. Considering that linear convolution has low computational pressure and high efficiency, the use of linear convolution operation is conducive to quickly generating more feature maps, thereby improving the receptive field and feature diversity of the network while keeping the number of parameters and the amount of computation low. Such a processing method can improve the generalization ability of image detection and reduce the dependence on training samples.

[0046] In some embodiments, based on the YOLOv8 model, its backbone module C2f can be replaced with an improved Ghost (ghost convolution) lightweight module (hereinafter referred to as CGhost module), such as Figure 3 As shown in , the target feature map is obtained, and the LARC-CGhost module in the figure is the LARC module and the CGhost module. This method reduces the computational burden by replacing modules without changing the YOLOv8 infrastructure, increases the number of feature maps, and is conducive to rapid deployment and implementation. The lightweight feature makes the method disclosed in this disclosure suitable for deployment in embedded devices, which is conducive to promotion and application.

[0047] For example, the network structure of the CGhost module can be as follows Figure 5 As shown in the figure, it includes Input module, GhostConv module, DWGhost (Depthwise Convolution) module, and Output module. By introducing residuals, the network converges faster during training and improves the generalization ability of the model. The CGhost module generates more feature maps through cheap linear convolution operations, which are spliced ​​with the feature maps of nonlinear convolution, effectively increasing the receptive field and feature diversity of the network, while keeping the parameters and computational complexity low, improving the processing speed of the model on embedded devices.

[0048] In some embodiments, the operation of obtaining the target feature map can be as follows: Figure 2 As shown in steps 231-233.

[0049] In step 231, a first feature map is obtained through linear convolution according to features in different orientations.

[0050] In step 232, a second feature map is obtained through nonlinear convolution according to features in different orientations.

[0051] In step 233, the first feature map and the second feature map are merged and processed by linear convolution to obtain a target feature map. For example, the first feature map and the second feature map are merged by a Concat module, and then processed by linear convolution again, such as by a CGhost module again, to obtain a target feature map.

[0052] Based on the method in the embodiment shown above, more feature information can be mined and more feature maps can be generated, further improving the receptive field and feature diversity of the network.

[0053] By combining multiple modules in parallel and serially, target feature maps of different sizes can be obtained in different modules and serial levels, so that the number of feature sizes of the target feature map is greater than 1, providing more information for further data processing.

[0054] In step S15, the image features are determined according to the target feature map. In some embodiments, the processing method in the YOLOv8 network architecture can be used, for example, the target feature map is processed based on its backbone C2f module to obtain the image features. Alternatively, the CGhost module mentioned above can be used to replace the C2f module to process the target feature map to obtain the image features. In some embodiments, the number of channels for obtaining the image features matches the number of sizes of the target feature map, and the image features of the corresponding sizes are obtained by processing the target feature maps of each size separately.

[0055] In addition, while analyzing the image features from the feature map, the weights of the regions in the feature map can be adjusted to obtain the image features.

[0056] For example, you can use Figure 6 The network structure shown in ( Figure 3 CAM (Collaborative Attention Module) in the 1 、F 2 Generate channel attention and spatial attention synchronously, use channel attention and spatial attention mechanism to synchronously process feature maps, determine the weight of the region in the feature map, and determine the image features based on the weight and feature map. This method can reduce the dependence of the spatial weight map on the channel, make the network pay more attention to the high-weight areas in the feature map, and improve the network's ability to process images.

[0057] In some embodiments, Figure 3 As shown in , the CAM module can be located at the head of the YOLOv8 network architecture, which is conducive to rapid deployment and improves subsequent detection accuracy.

[0058] In step S17, it is determined whether a fall event has occurred based on the image features. In some embodiments, a predicted bounding box may be first determined based on the image features, and then whether a fall event has occurred may be determined based on the predicted bounding box. For example, an object of a target category (such as a human body) may be determined based on the image features, and a predicted bounding box of the object may be extracted. Further, whether a fall event has occurred may be determined based on changes in the shape, size, etc. of the predicted bounding box.

[0059] In some embodiments, the predicted bounding box and the probability of belonging to the human category are first determined based on the image features. The detection method can adopt any method of obtaining a detection box of a target based on features in the related art, such as the Detect module used in YOLO.

[0060] In some embodiments, the position of the detection frame in the undetected image frame can also be determined in combination with the detection results of the historical frame images. For example, it is first determined whether the predicted frame belonging to the target category object (such as a human body) is detected in the current frame image. If the predicted frame belonging to the target category object is not detected in the current frame image, the predicted frame of the current frame image is determined according to the predicted frame position in the historical image. For example, the following formula can be used to determine the predicted frame in the current frame image:

[0061]

[0062]

[0063] in, and Respectively represent the coordinates of the upper left corner and lower right corner of the regression box of the current frame, and Respectively represent the upper left corner and lower right corner coordinates of the regression box in the previous frame image, Represents the coordinate difference of the first two frames, where is the coordinate difference of the upper left corner coordinates of the first two frames, is the coordinate difference of the lower right corner coordinates of the previous two frames; similarly, and Respectively represent the coordinates of the upper left corner and lower right corner of the regression box in the second frame image. represents the coordinate difference between the second frame image and the third frame image, where is the coordinate difference between the upper left corner coordinates of the second frame image and the third frame image. is the coordinate difference between the coordinates of the lower right corner of the second frame image and the third frame image. By using this method, the problem of inaccurate detection during rapid target movement can be solved, and the accuracy of detection is further improved.

[0064] Based on the method in the embodiment shown above, it is possible to further obtain a detection frame of the target based on the extracted image features, providing a more intuitive and easier to process data basis for performing subsequent judgments based on the detection frame.

[0065] Furthermore, whether a fall event occurs is determined based on the predicted border. For example, based on the change in at least one feature of the ratio, width, and height of the predicted border of adjacent frame images, the change in the state of the human body is determined, such as changing from a standing or sitting position to a prone or lying position.

[0066] In some embodiments, the information of the predicted border obtained above can be input into an MLP (multi-layer perceptron), and the MLP determines whether a fall event occurs in the image based on the features of the border, such as the normalized values ​​of the ratio, width, and height. For example,

[0067]

[0068]

[0069]

[0070] in and are the height and width of the border respectively, and are the height and width of the input image, respectively. Represents the maximum height of the bounding box, the above RBB is the aspect ratio of the border, WN is the ratio of the width of the border to the image width, and HND is the ratio of the height of the non-border area in the image to the image height.

[0071] Based on the above feature analysis, a fall state queue is obtained. In some embodiments, further, the maximum length of the queue is set to 2, and the queue stores the fall state of the human body in the current video segment. Due to the setting of the queue length, that is, the queue only stores the fall state information of the most recent two frames. In this way, the system can take into account the changes in the human body state in a short period of time, so as to more accurately determine whether a fall has occurred.

[0072] Based on the method in the embodiment shown above, features of different directions can be extracted from the image, overcoming the problem of feature omission and false detection caused by the image shooting angle, providing a more comprehensive and accurate data basis for subsequent target detection and selection, and then timely judging whether a fall event has occurred, so as to facilitate response, such as activating protective measures, or issuing alarm information, etc., thereby improving the comprehensiveness and accuracy of fall detection and the reliability of safety protection.

[0073] In some embodiments, in the fall detection method of the present disclosure, the machine learning model used, for example Figure 3 The model shown in can be trained in advance. For example, a human image dataset is collected to train the machine learning model. In some embodiments, the original dataset image can be rotated to enhance the network's detection accuracy at different angles.

[0074] For example, the COCO (Common Objects in Context) dataset is used as a training sample. The COCO dataset is preprocessed so that the model can better adapt to the needs of human detection in actual scenes. First, all images containing human bodies in the original COCO dataset are filtered, and other non-human category labels in the image are removed to ensure that the model only focuses on the human body during training. Next, a rotation operation is applied to the filtered images to simulate different human postures that may be encountered in the real world by changing the orientation of the image. Specifically, the image is rotated by specific angles, such as 90 degrees and 270 degrees, to cover different directional changes. After the rotation operation, the coordinates of the human body bounding box in the image need to be updated according to the following formula

[0075]

[0076] in, represents the coordinates of the regression box in the original image, represents the coordinates of the regression box in the rotated image, is the rotation angle. This ensures the accuracy of the coordinate information so that the model can correctly recognize the rotated human body.

[0077] In addition, in order to enhance the adaptability of the model to various environmental changes, data enhancement is performed on the filtered and rotated images, including brightness adjustment, adding slight noise, and contrast adjustment. These operations help improve the performance of the model under different lighting and noise conditions. Through these preprocessing steps, the COCO dataset is converted into a dataset that is more suitable for training fall detection models, enabling the model to learn to accurately detect human bodies in a variety of complex situations, thereby improving the overall performance and accuracy of the fall detection system.

[0078] After the training is completed, the trained model is used to detect the image frames in the video, and the detected human body boundary box information is output. The model's processing of the image frames can be as shown in the image feature extraction method in any of the above embodiments.

[0079] By testing the image feature extraction method proposed in the present invention on a fall dataset (LE2I dataset), it was found that the method proposed in the present invention achieved an accuracy of 87.10%, a sensitivity of 88.00%, and a specificity of 85.49%, which proved the effectiveness of the method for fall detection in a natural state and is suitable for real-time monitoring applications on embedded mobile devices.

[0080] Schematic diagrams of some embodiments of the fall detection device 70 of the present disclosure are as follows Figure 7 In some embodiments, the fall detection device runs an image feature extraction model based on the YOLOv8 model.

[0081] The rotational convolution module 701 can obtain features of different orientations in the image by rotating and convolving the image. In some embodiments, the image can be rotated at different angles during the convolution operation, and the feature extraction of different orientations can be achieved by using the rotational convolution kernel. In some embodiments, the rotational convolution module 701 can execute the method in any embodiment of step S11 above.

[0082] The feature map acquisition module 702 can acquire the target feature map according to the features of different orientations. In some embodiments, the C2f module of the YOLOv8 model can be used to obtain the target feature map. In some embodiments, the target feature map is acquired by linear convolution according to the features of different orientations. Considering that the computational pressure of linear convolution is low and the efficiency is high, the use of linear convolution operation is conducive to the rapid generation of more feature maps, thereby improving the receptive field and feature diversity of the network, while maintaining a low level of parameter quantity and computational complexity. Such a processing method can improve the generalization ability of image detection and reduce the dependence on training samples. In some embodiments, the feature map acquisition module 502 can execute the method in any embodiment of step S13 above.

[0083] The feature determination module 703 can determine the image features according to the target feature map. In some embodiments, the processing method in the YOLOv8 network architecture can be adopted, for example, the target feature map is processed based on its backbone C2f module to obtain image features. Or the CGhost module mentioned above can be used to replace the C2f module to process the target feature map to obtain image features. In some embodiments, the number of channels for obtaining image features matches the number of sizes of the target feature map, and the image features of the corresponding size are obtained by processing the target feature map of each size separately. In some embodiments, the feature determination module 703 can execute the method in any embodiment of step S15 above.

[0084] The fall event determination unit 704 can determine whether a fall event occurs according to the image features acquired by the feature determination module 703. In some embodiments, the fall event determination unit 704 can determine whether a fall event occurs based on the method in any embodiment of step S17 above.

[0085] Such a fall detection device can extract features in different directions from the image, overcome the problem of feature omission and false detection caused by the problem of image shooting angle, provide a more comprehensive and accurate data basis for subsequent target detection and selection, improve the accuracy and reliability of detection, and on this basis, promptly judge whether a fall event has occurred, so as to facilitate response, such as activating protective measures, or issuing alarm information, etc., thereby improving the comprehensiveness and accuracy of fall detection and the reliability of safety protection.

[0086] In some embodiments, the fall detection device may meet at least one of the following conditions: the rotation convolution model is a convolution module of the YOLOv8 model; the feature map acquisition module is the backbone of the YOLOv8 model; and the feature determination module is located at the head of the YOLOv8 model. Such a fall detection device can be implemented based on the YOLOv8 model, which can take into account the advantages of the YOLOv8 model and improve deployment efficiency.

[0087] The structural schematic diagram of an embodiment of the image feature extraction device disclosed in the present invention is as follows Figure 8 As shown. The image feature extraction device includes a memory 801 and a processor 802. Wherein: the memory 801 can be a disk, a flash memory or any other non-volatile storage medium. The memory is used to store the instructions in the corresponding embodiment of the image feature extraction method described above. The processor 802 is coupled to the memory 801 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 802 is used to execute the instructions stored in the memory, which can improve the detection precision and accuracy.

[0088] In one embodiment, it is also possible to Fig. 9 As shown, the fall detection device 900 includes a memory 901 and a processor 902. The processor 902 is coupled to the memory 901 via a BUS bus 903. The image feature extraction device 900 can also be connected to an external storage device 905 via a storage interface 904 to call external data, and can also be connected to a network or another computer system (not shown) via a network interface 906. No further detailed description will be given here.

[0089] In this embodiment, the detection precision and accuracy can be improved by storing data instructions in a memory and processing the instructions through a processor.

[0090] In another embodiment, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method in the corresponding embodiment of the image feature extraction method. It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] Schematic diagrams of some embodiments of the electronic device disclosed herein are as follows Fig.10 shown.

[0092] The fall detection device 1000 may be any one of the ones mentioned above.

[0093] The alarm device 1010 can issue an alarm when the fall detection device determines that a human body falls. In some embodiments, the alarm device 1010 can trigger the start of protective measures.

[0094] In some embodiments, the electronic device may be a wearable device, or may be mounted near the image detection device.

[0095] The electronic device shown above can extract features in different directions from the image, overcome the problem of feature omission and false detection caused by the problem of image shooting angle, provide a more comprehensive and accurate data basis for subsequent target detection and selection, and improve the accuracy and reliability of detection.

[0096] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0100] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0101] It should be noted that the terms "first", "second", etc. in the specification, claims, and drawings of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present disclosure rather than to limit it. Although the present disclosure has been described in detail with reference to the preferred embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present disclosure, which should be included in the scope of the technical solution for protection of the present disclosure.

Claims

1. A fall detection method, include: By rotating and convolving the image, the features of different orientations in the image are obtained; According to the features of the different orientations, a target feature map is obtained; Determining image features according to the target feature map; A fall event is determined according to the image features.

2. The fall detection method according to claim 1, in, The acquiring the target feature map according to the features includes: acquiring the target feature map through linear convolution according to the features in different orientations.

3. The fall detection method according to claim 1, in, The acquiring the target feature map according to the features of the different orientations comprises: According to the features in different orientations, a first feature map is obtained by linear convolution; According to the features of the different orientations, a second feature map is obtained by nonlinear convolution; The first feature map and the second feature map are combined and processed by linear convolution to obtain the target feature map.

4. The fall detection method according to claim 1, in, Determining the image features according to the target feature map includes: adjusting the weights of the regions in the feature map to obtain the image features.

5. The fall detection method according to claim 4, in, The adjusting the weight of the region in the feature map to obtain the image feature comprises: The feature map is processed synchronously using a channel attention and a spatial attention mechanism to determine a weight of a region in the feature map; The image features are determined according to the weights and the feature map.

6. The fall detection method according to claim 1, in, The number of feature dimensions of the target feature map is greater than 1.

7. The fall detection method according to claim 6, in, Determining the image feature according to the target feature map comprises: The image feature maps of different feature sizes are processed respectively to obtain the image features of corresponding sizes.

8. The image feature extraction method according to claim 1, in, Determining the occurrence of a fall event according to the image features includes: Determining a predicted border according to the image features; The occurrence of a fall event is determined according to a change in the predicted border.

9. The fall detection method according to claim 1 or 8, further comprising: include: When the predicted bounding box belonging to the target category object is not detected in the current frame image, the predicted bounding box of the current frame image is determined according to the predicted bounding box position in the historical image.

10. The fall detection method according to claim 8, in, Determining the occurrence of a fall event according to the change of the predicted border includes: The occurrence of a fall event is determined based on a change in at least one feature of the ratio, width, and height of the predicted borders of adjacent frame images.

11. The fall detection method according to any one of claims 1 to 8, in, Extracting predicted borders from images is implemented based on the YOLOv8 model.

12. A fall detection device, include: The rotation convolution module is configured to obtain features of different orientations in the image by rotating and convolving the image; A feature map acquisition module is configured to acquire a target feature map according to the features of the different orientations; A feature determination module is configured to determine image features according to the target feature map; A fall event determination unit is configured to determine the occurrence of a fall event according to the image features.

13. The fall detection device according to claim 12, in, The fall detection device runs an image feature extraction model based on the YOLOv8 model.

14. The fall detection device according to claim 13, in, The device meets at least one of the following requirements: The rotational convolution model is a convolution module of the YOLOv8 model; The feature map acquisition module is the backbone of the YOLOv8 model; The feature determination module is located at the head of the YOLOv8 model.

15. A fall detection device, include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method according to any one of claims 1 to 11 based on instructions stored in the memory.

16. A non-transitory computer-readable storage medium having computer program instructions stored thereon, which implement the method of any one of claims 1 to 11 when executed by a processor.

17. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

18. An electronic device, include: The fall detection device according to any one of claims 12 to 15; and The alarm device is configured to issue an alarm when the fall detection device determines that a fall event has occurred.