Human fall detection method and system

Abstract

The present invention provides a method for detecting human falls, comprising: obtaining an image to be detected; processing the image based on a detection model to obtain human contour feature data in the image; obtaining human shape data based on the human contour feature data; determining human motion state parameters based on the human shape data; and determining a detection result based on the human motion state parameters. This method improves the computational efficiency and speed of fall detection, making it more suitable for real-time detection in everyday scenarios.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for detecting a human fall. Background Art

[0002] With the continuous advancement of technology and medical care, and the rising average age, the aging population has become a problem that cannot be ignored in today's society. At the same time, how to detect elderly people's falls in a timely manner and provide assistance has become an urgent need for many families with migrant workers.

[0003] Currently, there are two main methods for detecting falls on the market. One is sensor-based detection, which requires multiple sensors deployed on the human body or in the surrounding environment to determine whether a fall has occurred based on parameters such as acceleration, sound, and vibration. Sensor-based methods are affected by sensor type, placement, location, and battery life, and can hinder the body's free movement. Multiple sensors are also costly and have low system detection accuracy.

[0004] The detection method based on machine vision detects and tracks real-time surveillance video stream data. It first needs to extract the locations of multiple key skeleton points of the detection target in the video, and then connect the key skeleton points in a certain order to determine whether the current human posture is in a fall state. When there are multiple human targets in the surveillance image, they need to be calculated and detected one by one. Not only is the calculation complexity high and it is difficult to adapt to detection in complex backgrounds, but also because the body is highly occluded by each other during a fall and the posture is complex and uncontrolled, detection is more difficult. Summary of the Invention

[0005] The purpose of the present invention is to provide a human fall detection method and system, in order to solve the technical problems existing in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A human fall detection method includes: acquiring an image to be detected;

[0008] Based on the processing of the image to be detected by the detection model, human body contour feature data in the image to be detected is obtained; human body shape data is obtained based on the human body contour feature data; human body motion state parameters are determined based on the human body shape data; and a detection result is determined based on the human body motion state parameters.

[0009] In some embodiments, the detection model processes the image to be detected by: extracting human body contours from multiple frames of the image to be detected; determining the target part and intercepting the image of the target part to obtain a target image; and performing binarization and contour extraction on the intercepted target image.

[0010] In some embodiments, the obtaining of human body shape data based on the human body contour feature data includes: establishing a structural element, scanning the fuzzy contour images of the human body falling and comparison in turn and performing an "AND" operation, dilating the image, and connecting the broken and discontinuous boundary pixels in the extracted contour; scanning the image again and performing an "AND" operation, corroding the image, making the extracted edges smoother and maintaining the original human body size contour approximately unchanged, and filling the pixels inside the contour to obtain the falling and comparison human shape data.

[0011] In some embodiments, determining the human motion state parameters based on the human body shape data includes: according to the fall and comparison human shape data, counting the vertical distance between the highest position and the lowest position, which are targets h1 and h2 respectively; counting the total number of pixels of the human body shape, which are human body surface areas S1 and S2 respectively; when a fall occurs, uniformly measuring the change parameters of targets of different distances and body shapes by calculating the ratio of the change amount and the fall state; processing the change parameters per unit time to calculate the change speed of height and surface area respectively, and obtaining the human motion state parameters by weighted summation; calculating the human motion state parameter E, and judging the state of the current detection target according to the value of E; wherein the calculation method of the motion state parameter is as follows:

[0012]

[0013] Among them, Δt is the extraction time interval between the fall and comparison images corresponding to the video stream data, and T1 and T2 are the weight parameters of the human body height change and surface area change in the two images, respectively.

[0014] In some embodiments, determining the detection result based on the human motion state parameter includes:

[0015] Set the threshold T fall , according to the motion state parameter E of the human body, it is judged whether there is a fall in the image of the current screen: when E is less than 0, the height and surface area of the human body are in an increasing trend, that is, the target currently being detected is in a motion state from a lower height to a higher height, which is not a fall; when E is greater than 0 and less than the threshold T fall When E is greater than the threshold T, the current detection target has a tendency to decrease in height but moves slowly, which does not mean it is falling. fall When the height and surface area of the detected target decrease rapidly, it is determined that a fall has occurred.

[0016] In some embodiments, the training data of the detection model is obtained as follows:

[0017] We shoot grayscale videos of people falling under multiple lighting conditions and background complexities. In addition to different falling images, the people in the videos also include at least one image of bending over, walking, and running. We randomly capture three frames of images per second from the captured video. After manually annotating the data of falling and non-falling states based on the current image, we randomly extract the images in proportion according to different environments and postures to create training and test datasets.

[0018] In some embodiments, the structure and training of the detection model are as follows:

[0019] YOLOv5 is used as the preliminary detection model, and the training data set is input into the preliminary detection model for training. In the preliminary detection model, YOLOv5 adopts a mosaic data augmentation method that combines at least one of random cropping, scaling, and color space adjustment to achieve recognition of targets of different sizes by constructing a feature pyramid. During training, the difference between the predicted box and the labeled data is calculated and iterated to obtain the anchor box size suitable for the current data set. When training the preliminary detection model, the training conditions are met.

[0020] At the same time, the present invention also discloses a human fall detection system, comprising:

[0021] The first acquisition module is used to acquire the image to be detected; the detection module is used to process the image to be detected based on the detection model to obtain the human body contour feature data in the image to be detected; the second acquisition module is used to obtain human body shape data based on the human body contour feature data; the first determination module is used to determine the human body motion state parameters based on the human body shape data; the second determination module is used to determine the detection result based on the human body motion state parameters.

[0022] At the same time, the present invention also discloses a human fall detection device, which includes a processor and a memory; the memory is used to store instructions, and when the instructions are executed by the processor, the device causes the device to implement any of the above-mentioned human fall detection methods.

[0023] At the same time, the present invention also discloses a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer runs any of the above-mentioned human fall detection methods.

[0024] Beneficial effects

[0025] Compared with the prior art, the present invention has the following significant advantages:

[0026] By combining machine vision with motion state parameter calculation, the computational efficiency and speed of fall detection are improved compared to deep learning methods based on human posture judgment, making it more suitable for real-time detection in real-life scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of a human fall detection system involved in this embodiment;

[0028] Figure 2 1 is a flow chart of the human fall detection method involved in this embodiment;

[0029] Figure 3 1 is a flow chart of a human fall detection method according to this embodiment;

[0030] Figure 4 It is a schematic diagram of the model training process involved in this embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] On the contrary, this application covers any alternatives, modifications, equivalents, and solutions made within the spirit and scope of this application as defined by the claims. Furthermore, to facilitate a better understanding of this application, certain specific details are described in detail below in the detailed description of this application. Those skilled in the art will be able to fully understand this application without these details.

[0033] The following will be combined Figure 1-4 A human fall detection method and system according to an embodiment of the present application are described in detail. It should be noted that the following embodiments are only used to explain the present application and do not constitute a limitation of the present application.

[0034] Example 1

[0035] like Figure 1 As shown, a human fall detection system 100 includes:

[0036] The first acquisition module 110 is used to acquire the image to be detected. In some embodiments, the first acquisition module 110 can be implemented by a camera, monitor, or other photographing or recording device.

[0037] The detection module 120 is used to process the image to be detected based on the detection model to obtain human body contour feature data in the image to be detected.

[0038] like Figure 4As shown, in some embodiments, the detection model is implemented based on a trained neural network model. Specifically, to adapt the model to detection under different conditions, multiple grayscale videos of people falling are captured under different lighting conditions and background complexities. In addition to different falling situations, the human body in the video should also include various postures such as bending, walking, and running. Three frames of images are randomly captured from the captured video every second. After manually annotating the data of the falling and non-falling states based on the current image, the training and test datasets are randomly sampled in proportion according to different environments and postures.

[0039] In some embodiments, YOLOv5 can be used as a preliminary detector, and the training data set can be input into the network for training. In the network, YOLOv5 adopts the Mosaic data enhancement method of splicing by random cropping, scaling, color space adjustment, etc., and realizes the recognition of targets of different sizes by constructing a feature pyramid. During training, the difference between the predicted box and the real target is calculated and iterated to obtain the anchor box size suitable for the current data set. At the same time, the use of convolutional layers instead of fully connected layers greatly increases the detection speed of the network. When training the network, if the model error of the training result does not converge or the detection error in the test data set is large, adjust the parameters and re-input the data for training until the model can detect the human fall situation and position under different conditions to a certain extent.

[0040] To extract human contours, the model feeds video stream data into the model. Once the model detects a fall, it extracts the contours of the person by reading the preceding and following frames of the current frame, as well as the corresponding frame from one second ago and its preceding and following frames. Part of the image is then captured based on the detected position. If multiple objects in the same image represent falls, the model extracts and captures them separately based on the target positions provided by the network.

[0041] For the grayscale value I(x, y) at the (x, y) position in the intercepted image, the contour extraction process is as follows:

[0042] I'(x, y)=|I(x, y)-I1(x, y)|I(x, y)-I2(x, y)| (1)

[0043] Wherein, I1(x, y) and I2(x, y) are the grayscale values of the image of the previous frame and the next frame at the position (x, y) of the current frame respectively.

[0044] Since the position of stationary objects does not change, methods based on pixel changes between previous and next frames can effectively remove surrounding environmental information and eliminate the interference and impact of different backgrounds on target detection. When a person moves or falls, their posture will undergo subtle changes over a short period of time. By calculating these small displacement changes and combining them with image capture of key areas, interference from other moving objects or people can be eliminated. After binarization of the image, the blurred outlines of the person falling and the contrast can be extracted separately.

[0045] The second acquisition module 130 is configured to acquire human body shape data based on the human body contour feature data.

[0046] When a fall occurs, since each part of the human body does not move at a uniform speed as a whole, but the hands, head and other parts move faster, while the waist, hips and other parts move relatively slowly, the human body contour extracted in step 2 is not a smooth, continuous curve and requires further morphological closing operation.

[0047] A structuring element is created. The blurred outline images of the fallen and contrasted figures are scanned sequentially and then subjected to an AND operation. This dilates the image and connects any discontinuous border pixels within the extracted outline. The image is then scanned again and subjected to an AND operation to erode the image, smoothing the extracted edges while maintaining a near-identical outline of the original figure. The pixels within the outline are then filled to obtain the shape of the fallen and contrasted figure.

[0048] The first determining module 140 is configured to determine a human motion state parameter based on the human body shape data.

[0049] In some embodiments, based on the aforementioned fall and comparison human shape data, the vertical distance between the highest position and the lowest position can be counted as the target h1, h2 and the sum of the number of pixels of the human body shape is the human body surface area S1, S2. When a fall occurs, the involuntary fall of the human body will cause the height of the human body to drop rapidly and the overall surface area to shrink. By calculating the ratio of the change amount to the fall state, a unified measurement of the target change parameters for different distances and body shapes can be achieved. Subsequently, the change parameters are processed per unit time to calculate the change speed of height and surface area respectively, and the weighted sum is used to obtain the human body motion state parameters. Calculate the human body motion state parameter E, and judge the state of the current detection target according to the size of the E value. The calculation method of the motion state parameter is as follows:

[0050]

[0051] Among them, Δt is the extraction time interval between the fall and comparison images corresponding to the video stream data, and T1 and T2 are the weight parameters of the human body height change and surface area change in the two images, respectively.

[0052] The second determining module 150 is configured to determine a detection result based on the human motion state parameter.

[0053] The specific determination of the detection result may include first setting a threshold T fall , based on the size of the human motion state parameter E calculated by the first determination module 140, it is determined whether there is a fall in the image on the current screen. When E is less than 0, the height and surface area of the human body are on an increasing trend, proving that the currently detected target is in a state of motion from a lower height to a higher height and is not a fall; when E is greater than 0 and less than the threshold T fall When E is greater than the threshold T, the current detection target has a tendency to decrease in height but moves slowly, which may be bending or squatting voluntarily, and is also not a fall. fall When the height and surface area of the detected target decrease rapidly, it can be determined that a fall has occurred, and an alarm or rescue action should be taken in time.

[0054] Example 2

[0055] like Figure 2 、 3 A method for detecting a human fall is shown, and the method includes the following specific process 200:

[0056] Step 210: Acquire an image to be detected.

[0057] Step 220 : obtaining human body contour feature data in the image to be detected based on processing of the image to be detected by the detection model.

[0058] Specifically, step 220 includes:

[0059] Step 221: fall pre-detection and location.

[0060] The detection model processes the image to be detected by passing the monitoring video stream data as input to the YOLOv5 neural network. When the network detects a fall, it records the current time and the position coordinates of the detection box where the human fall is detected in the entire video image.

[0061] Step 222: extract the human body contour.

[0062] According to the time of the fall, the previous and next frames are extracted, as well as the comparison image of the previous second and the previous and next frames. The image areas with the same size as the detection frame at the corresponding position coordinates are intercepted in sequence. The human body contours in the fall image and the comparison image are extracted respectively:

[0063] I'(x, y)=|I(x, y)-I1(x, y)|+|I(x, y)-I2(x, y)| (1)

[0064] In formula (1), I(x, y) is the grayscale value of the pixel at the position (x, y) in the image, and I1(x, y) and I2(x, y) are the grayscale values of the pixel at the same position in the previous and next frame images, respectively.

[0065] Then, the human body contour images of the extracted image and the comparison image are binarized:

[0066]

[0067] In formula (2), T is the grayscale threshold.

[0068] Step 230: Acquire human body shape data based on the human body contour feature data.

[0069] According to the fall contour map and the comparison contour map obtained in step 220, morphological closing operations are performed in sequence. First, the image is expanded, and a 3×3 structural element is established. The structural element is convolved with a pixel point in the image and its surrounding neighborhood points to calculate the maximum value of the local grayscale. When the maximum value is greater than 0, the pixels of the current window are filled according to the template. After the sliding window traverses all pixels and the expansion is completed, a 3×3 structural element is re-established to expand the image. After convolving with the pixel point and its neighborhood pixels, the local minimum value is calculated. When the minimum value is less than 255, the current window pixels are filled according to the current template, and the sliding window traverses the entire image.

[0070] Finally, the center pixels of the outline are filled, using the center pixels of the fallen and comparison images as seed points. The search area is continuously expanded and included in the connected domain until the outline pixels are touched. The grayscale value of the pixels in the connected domain is assigned to 255, and the shapes of the fallen person and the comparison person are obtained.

[0071] Step 240, determining the human body motion state parameters based on the human body shape data.

[0072] According to the human body shape image extracted in step 230, the vertical distance h1, h2 between the statistical highest pixel point and the lowest pixel point of the human body shape in the fall and comparison images and the number of pixels contained in the entire human body shape are the human body surface area parameters S1, S2, and the human body motion state parameter E is calculated:

[0073]

[0074] Wherein, Δt is the time interval from the detection of the fall to the extraction of the contrast image corresponding to the video stream, and T1 and T2 are the pre-set weight parameters of height change and surface area change, respectively.

[0075] Step 250: Determine a detection result based on the human body motion state parameters.

[0076] The human motion state parameter E calculated in step 240 and the preset threshold value T fall , determine whether the target person has fallen at the current time and location. When the motion state parameter is lower than the threshold, it proves that the target person is stationary or moving slowly, and there is no fall. The detection should continue to the next frame of the image. When the motion state parameter is higher than the threshold, it means that the target person is moving rapidly towards the bottom surface. It can be determined that a fall has occurred, and relevant warnings and rescue are issued according to the time and location of the fall.

[0077] At the same time, the present invention also discloses a human fall detection device, which includes a processor and a memory; the memory is used to store instructions, and when the instructions are executed by the processor, the device causes the device to implement any of the above-mentioned human fall detection methods.

[0078] At the same time, the present invention also discloses a computer-readable storage medium, which stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer runs any of the above-mentioned human fall detection methods.

[0079] In summary, the technical solution of the present invention adopts a method that combines machine vision with motion state parameter calculation to solve the problems of large computing space and long computing time in fall detection based on human posture detection methods, thereby improving detection efficiency and meeting real-time detection requirements.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting a human fall, characterized in that: include: Obtain the image to be detected; Processing the image to be detected based on the detection model, obtaining human body contour feature data in the image to be detected; Acquiring human body shape data based on the human body contour feature data; determining a human body motion state parameter based on the human body shape data; Determine a detection result based on the human body motion state parameter; The acquiring of human body shape data based on the human body contour feature data comprises: Establish a structural element, scan the blurred contour images of the falling human body and the contrast image in sequence, perform an "AND" operation on them, dilate the images, and connect the boundary pixels with breaks and discontinuities in the extracted contours; Scan the image again and perform an "AND" operation to erode the image, making the extracted edges smoother and maintaining the original human body size outline approximately unchanged. Fill the pixels inside the outline to obtain the falling and comparison human shape data; Determining the human body motion state parameters based on the human body shape data includes: According to the falling and comparison humanoid data, the vertical distance between the highest position and the lowest position is counted, which are the target 、 ; The sum of the number of pixels of the human body shape is the human body surface area 、 ; When a fall occurs, the change parameters of targets of different distances and body shapes are uniformly measured by calculating the ratio of the change amount to the fall state; The changing parameters are processed per unit time to calculate the changing speed of height and surface area respectively, and the weighted sum is used to obtain the human body motion state parameters; Calculate the motion parameters of the human body ,according to The value determines the state of the current detection target; the calculation method of the motion state parameter is as follows: in, The extraction time interval between the fall and comparison images corresponding to the video stream data, 、 are the weight parameters for the height change and surface area change of the human body in the two images respectively.

2. A human fall detection method according to claim 1, characterized in that: The detection model processes the image to be detected including: Extracting the human body contour from multiple frames of the image to be detected; Determine the target part and intercept the image of the target part to obtain the target image; The captured target image is binarized and its contours extracted.

3. A human fall detection method according to claim 1, characterized in that: Determining the detection result based on the human body motion state parameter includes: Setting thresholds , according to the motion state parameters of the human body Determine whether there is a fall in the image on the current screen: when When it is less than 0, the height and surface area of the human body are on an increasing trend, that is, the target currently being detected is in a state of moving from a lower height to a higher height, and is not a fall situation; when Greater than 0 and less than the threshold When the current detection target has a tendency to descend in height but moves slowly, it does not belong to a fall situation; When E is greater than the threshold When the height and surface area of the detected target decrease rapidly, it is determined that a fall has occurred.

4. A human fall detection method according to any one of claims 1 to 3, characterized in that: The training data of the detection model is obtained as follows: Shoot multiple grayscale videos of people falling under different lighting conditions and background complexity. In addition to different falling images, the videos also include at least one image of the person bending over, walking, or running. Three frames of images are randomly captured from the captured video every second. After manually annotating the data of the falling and non-falling states based on the current image, the training and test datasets are randomly sampled in proportion according to different environments and postures.

5. A human fall detection method according to claim 4, characterized in that: The structure and training of the detection model are as follows: Use YOLOv5 as the preliminary detection model and input the training data set into the preliminary detection model for training; In the preliminary detection model, YOLOv5 uses a mosaic data augmentation method that combines at least one of random cropping, scaling, and color space adjustment to recognize objects of different sizes by constructing a feature pyramid. During training, the difference between the predicted box and the labeled data is calculated and iterated to obtain the anchor box size suitable for the current dataset; When training the preliminary detection model, until the training conditions are met.

6. A human fall detection system, characterized in that: The method according to claim 1 is implemented as follows: A first acquisition module is used to acquire an image to be detected; A detection module, configured to process the image to be detected based on a detection model to obtain human body contour feature data in the image to be detected; A second acquisition module is used to acquire human body shape data based on the human body contour feature data; A first determining module, configured to determine a human body motion state parameter based on the human body shape data; The second determining module is used to determine the detection result based on the human body motion state parameter.

7. A human fall detection device, comprising a processor and a memory; the memory is used to store instructions, and when the instructions are executed by the processor, the device causes the human fall detection method as described in any one of claims 1 to 5 to be implemented.

8. A computer-readable storage medium storing computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the human fall detection method according to any one of claims 1 to 5.

Images