Fall detection method, device, equipment and storage medium

Abstract

The present invention discloses a fall detection method, device, equipment and storage medium, wherein the fall detection method comprises: obtaining a video image of a target area, and extracting a human body image set containing a target human body in the video image; performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate; based on the image state features, screening out the target human body image to be detected from the human body image set; based on a preset fall detection model, performing fall detection on the target human body image to obtain a fall detection result. The fall detection method disclosed by the present invention can solve the technical problems that the existing fall detection method has poor detection accuracy and is prone to misjudgment.

Description

Technical Field

[0001] The present invention belongs to the technical field of security monitoring, and in particular relates to a fall detection method, device, equipment and storage medium. Background Art

[0002] With the arrival of an aging society, young and middle-aged people are working outside. If the elderly have abnormal behaviors such as falls at home, if they cannot get effective help in time or inform their family in time, it will lead to serious consequences. Therefore, research on fall detection has attracted more and more attention.

[0003] Existing elderly care systems include those based on multiple sensors and those based on acoustics. Although these fall detection methods have simple algorithms, their detection accuracy is poor and misjudgment is prone to occur in practical applications. Summary of the invention

[0004] The main purpose of the present invention is to provide a fall detection method, device, equipment and storage medium, aiming to solve the technical problems that the existing fall detection methods have poor detection accuracy and are prone to misjudgment.

[0005] In order to achieve the purpose, the technical solution adopted by the present invention is as follows:

[0006] A fall detection method, the fall detection method comprising the following steps:

[0007] Acquire a video image of a target area, and extract a human body image set containing a target human body in the video image;

[0008] Performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate;

[0009] Based on the image state features, selecting a target human image to be detected from the human image set;

[0010] Based on a preset fall detection model, fall detection is performed on the target human body image to obtain a fall detection result.

[0011] Furthermore, the step of extracting a human body image set containing a target human body in the video image comprises:

[0012] Acquire a moving object image set containing moving objects in the video image by a preset background difference method;

[0013] The human body image set containing the target human body is screened out from the moving object image set by using a preset DPM algorithm.

[0014] Further, when the image state feature includes the centroid distance, the step of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes:

[0015] Extracting the coordinates of the centroid point and the bottom edge endpoint of each human body image in the human body image set;

[0016] Based on the extracted centroid coordinates and the bottom edge endpoint coordinates, the centroid distance of each human body image in the human body image set is calculated by a preset centroid distance calculation formula; wherein the centroid distance h refers to the distance from the centroid point O (x0, y0) to the line AB between the two bottom edge endpoints A (x1, y1) and B (x2, y2) in the rectangular human body image; the centroid distance calculation formula is:

[0017]

[0018] Further, when the image state feature includes the height change rate, the step of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes:

[0019] The height change rate of each human body image in the human body image set is calculated by a preset height change rate calculation formula; wherein the height change rate calculation formula is:

[0020]

[0021] Wherein, height is the real-time height of the target human body in the rectangular human body image, and heightavg is the average height of the target human body in the human body image set.

[0022] Furthermore, the step of selecting a target human image to be detected from the human image set based on the image state feature comprises:

[0023] Determine whether the aspect ratio of each of the human body images in the human body image set is greater than a first preset threshold, whether the centroid distance is less than a second preset threshold, and whether the height change rate is less than a third preset threshold;

[0024] If the aspect ratio of the human body image is greater than the first preset threshold and the center of mass distance is less than the second preset threshold, or, the aspect ratio is greater than the first preset threshold and the height change rate is less than the third preset threshold, or, the center of mass distance is less than the second preset threshold and the height change rate is less than the third preset threshold, then the human body image is determined as the target human body image.

[0025] Furthermore, before the step of performing fall detection on the target human body image based on a preset fall detection model to obtain a fall detection result, the method further includes:

[0026] Acquire labeled training sample data, wherein the training sample data includes a fall sample image and a non-fall sample image, the fall sample image has a label of "1", and the non-fall sample image has a label of "0";

[0027] Constructing a convolutional neural network model, wherein the convolutional neural network model includes a convolution layer, a pooling layer, a fully connected layer and a Softmax layer;

[0028] The training sample data is input into the convolutional neural network model for training, and the model parameters of the convolutional neural network are optimized by the stochastic gradient descent method to obtain the trained fall detection model.

[0029] Furthermore, the fall detection result includes a fall behavior event and a normal behavior event; after the step of performing fall detection on the target human body image based on a preset fall detection model and obtaining the fall detection result, the step further includes:

[0030] When the fall detection result is a fall behavior event, an early warning message is issued.

[0031] Correspondingly, the present invention further provides a fall detection device, the fall detection device comprising:

[0032] An extraction module, used to obtain a video image of a target area and extract a human body image set containing a target human body in the video image;

[0033] A feature detection module, used for performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate;

[0034] A screening module, used for screening a target human image to be detected from the human image set based on the image state feature;

[0035] The fall detection module is used to perform fall detection on the target human body image based on a preset fall detection model to obtain a fall detection result.

[0036] Correspondingly, the present invention also proposes a fall detection device, which includes a memory, a processor, and a fall detection program stored in the memory and executable on the processor, wherein the fall detection program is configured to implement the steps of the aforementioned fall detection method.

[0037] Correspondingly, the present invention further proposes a storage medium, on which a fall detection program is stored, and when the fall detection program is executed by a processor, the steps of the aforementioned fall detection method are implemented.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The fall detection method proposed by the present invention obtains a video image of a target area and extracts a human body image set containing a target human body in the video image; then performs state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate; then based on the image state features, the target human body image to be detected is screened out from the human body image set; finally, based on a preset fall detection model, fall detection is performed on the target human body image; in this way, the target human body image suspected of having a fall behavior is first screened out by organically integrating a plurality of image state features reflecting the human body activity state, and then the fall detection is performed on the target human body image suspected of having a fall behavior through the fall detection model, thereby not only improving the accuracy of human body fall detection, but also because when performing fall detection through the fall detection model, it is not necessary to detect each frame of the video image, nor is it necessary to detect all human body images containing the target human body, but only the screened target human body image suspected of having a fall behavior needs to be detected, so the computing pressure of the processor can be greatly reduced, thereby improving the efficiency of human body fall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0041] Figure 1 A schematic diagram of the structure of a fall detection device in a hardware operating environment according to an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of a fall detection method according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a human body image corresponding to a normal walking human body in one embodiment of the present invention;

[0044] Figure 4 A schematic diagram of a human body image corresponding to a human body falling down in one embodiment of the present invention;

[0045] Figure 5FIG. 4 is a schematic diagram of the structure of a fall detection device in one embodiment of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth in order to fully understand the present invention, and the embodiments described are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0049] like Figure 1 As shown, Figure 1 It is a structural diagram of a fall detection device in a hardware operating environment involved in an embodiment of the present invention.

[0050] The fall detection device of the embodiment of the present invention can be an image data processing device, such as a camera, a server, etc., or it can be a terminal device with image data acquisition and image data processing functions, such as a smart TV, a smart phone, a tablet computer and other terminal devices. The embodiment of the present invention does not impose specific restrictions on this.

[0051] like Figure 1As shown, the fall detection device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the fall detection device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0053] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a fall detection program.

[0054] In such Figure 1 In the fall detection device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 mainly includes an input unit such as a keyboard, and the keyboard includes a wireless keyboard and a wired keyboard, which is used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the fall detection program stored in the memory 1005, and perform the operations in the fall detection method in any of the following embodiments of the present invention.

[0055] Based on the above hardware structure, an embodiment of a fall detection method of the present invention is proposed.

[0056] Reference Figure 2 An embodiment of the present invention provides a fall detection method, which is applied to a fall detection device. The fall detection device may be a smart TV, a smart phone, a server, etc. For the convenience of description, the fall detection method of the embodiment of the present invention is described by taking a smart TV as a fall detection device as an example. Specifically, the fall detection method includes the following steps:

[0057] Step S11, acquiring a video image of a target area, and extracting a human body image set containing a target human body in the video image;

[0058] Step S12, performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate;

[0059] Step S13, based on the image state features, selecting a target human body image to be detected from the human body image set;

[0060] Step S14: performing fall detection on the target human body image based on a preset fall detection model to obtain a fall detection result.

[0061] In the above step S11, in a specific application scenario, a camera can be installed in the area to be monitored (such as a living room), and the smart TV can obtain the video image of the target area through the camera, wherein the target area refers to the shooting area that can be covered by the camera, and the camera and the smart TV can communicate through the existing wired or wireless methods. In this step, since the acquired video image may contain other moving objects in addition to the human body, it is necessary to extract the frames containing the human body from the video image to obtain a human body image set composed of human body images (i.e., images containing the human body). In a specific implementation method, the smart TV can perform denoising on the acquired video image according to the human body motion characteristics, remove the moving objects that do not meet the human body motion characteristics from the video image, and the remaining video frames containing only the human body are human body images, thereby obtaining a human body image set containing the target human body. In this way, after acquiring the video image of the target area, by extracting the human body image containing the target human body in the video image, on the one hand, the subsequent image processing pressure can be reduced, and on the other hand, the interference of non-human images can be avoided to affect the accuracy of subsequent fall detection.

[0062] In the above steps S12 to S13, since the extracted human image set may include human images reflecting human body falling behavior, and may also include human body images reflecting human body not falling behavior, therefore, in order to further reduce the subsequent image processing pressure and avoid the interference of human body images reflecting human body not falling behavior and affecting the accuracy of subsequent fall detection, this embodiment further processes the human body image set initially screened. Specifically, the inventors of the present application have found that compared with normal human walking, if the target human body in the target area falls, then the image height, image width and image center of mass position of the collected human body image will also change accordingly. For example, referring to Figure 3 and Figure 4Generally, when a human body falls, the human body will exhibit abnormal behaviors such as leaning over or squatting. Accordingly, the image height H of the corresponding human body image will become smaller, the image width T will become larger, and the centroid position O will decrease. Based on this, this embodiment uses the aspect ratio, centroid distance, and height change rate of the human body image as the screening basis to screen out human body images with suspected falling behaviors (i.e., target human body images) from the human body image set as the objects to be subsequently subjected to fall detection.

[0063] It should be noted here that the above aspect ratio refers to the ratio between the image width and the image height. Figure 3 and Figure 4 , assuming that the image width of a human body image is T and the image height is H, the aspect ratio of the human body image is T / H. Generally speaking, if the target human body in the target area has not fallen, the aspect ratio of the collected human body image will be less than 1. On the contrary, if the target human body in the target area has fallen, the aspect ratio of the collected human body image will be greater than 1. Therefore, if the aspect ratio of a human body image is detected to be greater than 1, the human body image is likely to be an image of a target human body with a fall.

[0064] Reference Figure 3 and Figure 4 The above-mentioned centroid distance h refers to the vertical distance between the centroid point O of the human body image and the bottom edge AB of the human body image. Generally speaking, if the target human body in the target area falls, the centroid distance of the collected human body images will become smaller. Therefore, if the centroid distance of a human body image is detected to be less than a certain value (this value can be obtained through experiments, for example, it can be 0.83~0.86, preferably 0.85), then the human body image is likely to be a target human body image with a falling behavior.

[0065] The above-mentioned height change rate refers to the ratio between the real-time height of the target human body in the current human body image and the average height of the target human body in all human body images (i.e., the human body image set). Generally speaking, if the target human body in the target area falls, the image height of the collected human body image will also change to a certain extent. Therefore, if the height change rate of a human body image is detected to be less than a certain threshold (the threshold can be obtained through experiments, for example, it can be 0.83~0.86, preferably 0.85), then the human body image is likely to be an image of the target human body with a falling behavior.

[0066] In the above step S14, the above fall detection model is obtained by training with a fall / non-fall sample data set, wherein the fall / non-fall sample data set may include a fall sample image based on a human body and a normal walking sample image. After training, the fall detection model can be used to perform fall detection on human body images, and output a fall detection result to characterize whether a person has fallen in the target area. In some specific implementations, a fall detection model for fall detection based on human body images can be constructed based on a convolutional neural network model. Among them, the classification function of the convolutional neural network model uses a softmax function, and the function formula is:

[0067] The Softmax layer of the convolutional neural network model can map the input data to real numbers between 0 and 1, and normalize them to ensure that the sum is 1. In practical applications, after a human image is input into the model, if the prediction value output by the model is less than the preset threshold, the detection result corresponding to the human image can be classified as a fall behavior event, and if the prediction value output by the model is greater than the preset threshold, the detection result corresponding to the human image can be classified as a normal behavior event.

[0068] The fall detection method proposed in this embodiment obtains a video image of a target area and extracts a human body image set containing a target human body in the video image; then performs state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate; then, based on the image state features, the target human body image to be detected is screened out from the human body image set; finally, based on a preset fall detection model, fall detection is performed on the target human body image; in this way, the target human body image suspected of having a fall behavior is first screened out by organically integrating a plurality of image state features reflecting the human body activity state, and then the fall detection is performed on the target human body image suspected of having a fall behavior through the fall detection model, thereby not only improving the accuracy of human body fall detection, but also because when performing fall detection through the fall detection model, it is not necessary to detect each frame of the video image, nor is it necessary to detect all human body images containing the target human body, but only the screened target human body image suspected of having a fall behavior needs to be detected, so the computing pressure of the processor can be greatly reduced, thereby improving the efficiency of human body fall detection.

[0069] Furthermore, in the above step S11, a human body image set containing a target human body in the video image may be extracted in the following manner:

[0070] Step S111, obtaining a moving object image set containing moving objects in the video image by a preset background difference method;

[0071] Step S112, filtering out a human body image set containing a target human body from the moving object image set by using a preset DPM algorithm.

[0072] In the above step S111, the background difference method is a moving target detection method that can detect moving objects from the background. Its basic idea is to subtract each current frame image from the background image stored in advance or acquired in real time, and calculate the area that deviates from the background by more than a certain threshold as the moving area. The algorithm is simple to implement, and the subtraction result directly gives the position, size, shape and other information of the target, which can provide a complete description of the moving target area. Especially for the case where the camera is stationary, the background difference method is the preferred method for realizing real-time detection and extraction of moving targets. Therefore, this step can effectively extract the moving object image containing the moving object in the video image through the background difference method, thereby obtaining a moving object image set composed of multiple moving object images.

[0073] In the above step S112, the DPM (Deformable Part Model) algorithm is a component-based detection method, which has strong robustness to the deformation of the target and is widely used in pedestrian detection. The DPM algorithm uses a root filter and several component filters to describe the structure of the target, describes the overall outline of the target through the root filter, and describes the various components of the target through several component filters. By calculating the root filter and component filter responses and the comprehensive score of the component model relative to the root model position offset, it is determined whether there is a target to be detected. For targets that are prone to deformation, this design makes target detection more accurate. The training process simplifies the problem of learning a deformable component model through latent support vector machine (Latent Support Vector Machine, Latent-SVM for short) data into a two-classification problem, thereby improving the overall detection performance of the system. Therefore, this step can effectively filter out the human body images containing pedestrians in the moving object image set through the DPM algorithm, thereby obtaining a human body image set consisting of multiple human body images.

[0074] Further, refer to Figures 2 to 4 When the image state feature includes the centroid distance, the step S12 of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes:

[0075] Step S121, extracting the coordinates of the centroid point and the bottom edge endpoint of each human body image in the human body image set;

[0076] Step S122, based on the extracted centroid coordinates and bottom edge endpoint coordinates, the centroid distance of each human body image in the human body image set is calculated by a preset centroid distance calculation formula; wherein the centroid distance h refers to the distance from the centroid point O (x0, y0) to the line AB between the two bottom edge endpoints A (x1, y1) and B (x2, y2) in the rectangular human body image; the centroid distance calculation formula is:

[0077]

[0078] In the above step S121, specifically, in order to facilitate the extraction of the centroid coordinates of the human body image, refer to Figure 3 and Figure 4 When extracting a human image from a video image, the human image can be intercepted as an image with a rectangular outline frame (i.e., the smallest rectangular frame circumscribing the human outline in the human image) by combining an existing image outline detection algorithm (such as a DMP algorithm, etc.). In this way, the intersection coordinates O of the two diagonals are the centroid coordinates. Of course, in some other implementations, the centroid coordinates of the human image can also be determined by other existing image centroid algorithms. As long as the centroid coordinates of the human image can be extracted, this embodiment does not impose specific restrictions on this. The coordinates of the two bottom edge endpoints A and B are the coordinates of the first pixel and the last pixel of the bottom edge.

[0079] In the above step S122, after obtaining the coordinates of the centroid point O and the coordinates of the two bottom edge endpoints A and B of a human body image, the centroid distance h of the human body image can be calculated by combining the above centroid distance calculation formula.

[0080] Further, refer to Figures 2 to 4 When the image state feature includes a height change rate, the step S12 of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes:

[0081] Step S123, calculating the height change rate of each human body image in the human body image set by using a preset height change rate calculation formula; wherein the height change rate calculation formula is:

[0082]

[0083] Where height is the real-time height of the target human body in the rectangular human body image, and heightavg is the average height of the target human body in the human body image set.

[0084] In this embodiment, specifically, for the convenience of calculation, refer to Figure 3 and Figure 4When extracting a human body image from a video image, the human body image can be intercepted as an image with a rectangular outline frame by combining an existing image contour detection algorithm (such as a DMP algorithm, etc.). In this way, the image height H of the human body image is the real-time height of the target human body, and the average image height of all human body images is the average height of the target human body in the human body image set. Then, the height change rate of each human body image can be calculated by combining the above-mentioned height change rate calculation formula.

[0085] Furthermore, the step S13 of selecting the target human body image to be detected from the human body image set based on the image state feature comprises:

[0086] Step S131, determining whether the aspect ratio of each human body image in the human body image set is greater than a first preset threshold, whether the centroid distance is less than a second preset threshold, and whether the height change rate is less than a third preset threshold;

[0087] If the aspect ratio of the human body image is greater than the first preset threshold and the center of mass distance is less than the second preset threshold, or the aspect ratio of the human body image is greater than the first preset threshold and the height change rate is less than the third preset threshold, or the center of mass distance of the human body image is less than the second preset threshold and the height change rate is less than the third preset threshold, then execute step S132 to determine the human body image as the target human body image.

[0088] In this embodiment, by combining at least two image state features to determine whether they meet predetermined conditions, target human images suspected of having fallen behaviors are screened out from the human image set, which is beneficial to improving the reliability of the target human images, and further beneficial to further improving the efficiency and accuracy of subsequent fall detection. Specifically, the first preset threshold value may be selected as 1, the second preset threshold value may be selected as 0.85, and the third preset threshold value may be selected as 0.85. If the aspect ratio of a human image is greater than 1 and the centroid distance is less than 0.85, the human image may be determined as a target human image for subsequent fall detection through a fall detection model; or, if the aspect ratio of a human image is greater than 1 and the height change rate is less than 0.85, the human image may be determined as a target human image for subsequent fall detection through a fall detection model; or, if the centroid distance of a human image is less than 0.85 and the height change rate is less than 0.85, the human image may be determined as a target human image for subsequent fall detection through a fall detection model.

[0089] Furthermore, before the step S14 of performing fall detection on the target human body image based on the preset fall detection model and obtaining the fall detection result, the following steps are further included:

[0090] Step S101, obtaining labeled training sample data, wherein the training sample data includes fall sample images and non-fall sample images, the label of the fall sample image is "1", and the label of the non-fall sample image is "0";

[0091] Step S102, constructing a convolutional neural network model, wherein the convolutional neural network model includes a convolutional layer, a pooling layer, a fully connected layer and a Softmax layer;

[0092] Step S103: input the training sample data into the convolutional neural network model for training, and optimize the model parameters of the convolutional neural network by the stochastic gradient descent method to obtain a trained fall detection model.

[0093] In the above step S101, specifically, the camera can be used to collect normal walking sample videos reflecting normal walking of a person and fall sample videos reflecting falling of a person, and then the background difference method combined with the DMP algorithm is used to extract human images containing human bodies from each sample video (of course, other mature pedestrian detection algorithms can also be used to extract human images containing pedestrians from video recordings), and then the obtained human images are divided into fall sample images and non-fall sample images and marked with corresponding sample labels, wherein the sample label of the fall sample image can be marked as "1", and the sample label of the non-fall sample image can be marked as "0".

[0094] In the above step S103, the convolution layer is used for feature nonlinear mapping, the pooling layer is used to reduce the data scale and improve the robustness, and the fully connected layer and the Softmax layer are used for classification and recognition.

[0095] In the above step S103, specifically, the parameters of the convolutional neural network training are the weight a and the bias b between the connection layers. The training process uses a stochastic gradient descent optimization method, which includes two stages: forward propagation and back propagation. The training process is specifically as follows:

[0096] The weight a is initialized by random initialization, and the bias b is initialized by fixed value initialization;

[0097] The training sample data T: {Xi, Yi} is input into the convolutional neural network, and the sample features are propagated forward through each layer, where Xi is the feature data of the training samples (fall sample images and non-fall sample images), and Yi is the sample label;

[0098] Calculate the error c between the sample label and the actual output;

[0099] The network weights are adjusted with the goal of minimizing the training error until the error c is less than the preset error allowable value, and the weight a and bias b are fixed. Thus, an optimized convolutional neural network is obtained, which can be used as a fall detection model to accurately detect falls in human images.

[0100] In this embodiment, based on the above steps, the preset fall detection model can be made more accurate, thereby further improving the accuracy of human fall detection.

[0101] Furthermore, the fall detection result includes a fall behavior event and a normal behavior event; after the above step S14 of performing fall detection on the target human image based on the preset fall detection model and obtaining the fall detection result, it also includes

[0102] Step S15: When the fall detection result is a fall behavior event, a warning message is issued.

[0103] In this embodiment, when the fall detection result output by the fall detection model is a fall behavior event, the smart TV can issue early warning information accordingly, such as sounding an alarm on the spot, issuing a voice prompt on the spot, sending a reminder message to the user terminal bound to the smart TV (bound through an App, etc.), etc., so that the fallen person can be discovered in time, so as to obtain effective help in time and reduce the risk of serious consequences.

[0104] Correspondingly, refer to Figure 5 , an embodiment of the present invention further provides a fall detection device, the fall detection device comprising:

[0105] An extraction module 11 is used to obtain a video image of a target area and extract a human body image set containing a target human body in the video image;

[0106] The feature detection module 12 is used to perform state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate;

[0107] A screening module 13, used for screening a target human body image to be detected from a human body image set based on image state features;

[0108] The fall detection module 14 is used to perform fall detection on the target human body image based on a preset fall detection model to obtain a fall detection result.

[0109] Furthermore, the extraction module 11 includes:

[0110] An acquisition unit, used for acquiring a moving object image set containing moving objects in a video image by a preset background difference method;

[0111] The screening unit is used to screen out a human body image set containing a target human body from a moving object image set by using a preset DPM algorithm.

[0112] Furthermore, the feature detection module 12 includes:

[0113] An extraction unit, used to extract the coordinates of the centroid point and the bottom edge endpoint of each human body image in the human body image set;

[0114] The first calculation unit is used to calculate the centroid distance of each human body image in the human body image set based on the extracted centroid coordinates and the bottom edge endpoint coordinates by a preset centroid distance calculation formula; wherein the centroid distance h refers to the distance from the centroid point O (x0, y0) to the line AB between the two bottom edge endpoints A (x1, y1) and B (x2, y2) in the rectangular human body image; the centroid distance calculation formula is:

[0115]

[0116] Furthermore, the feature detection module 12 further includes:

[0117] The second calculation unit is used to calculate the height change rate of each human body image in the human body image set by using a preset height change rate calculation formula; wherein the height change rate calculation formula is:

[0118]

[0119] Where height is the real-time height of the target human body in the rectangular human body image, and heightavg is the average height of the target human body in the human body image set.

[0120] Furthermore, the screening module 13 includes:

[0121] a judging unit, used to judge whether the aspect ratio of each human body image in the human body image set is greater than a first preset threshold, whether the centroid distance is less than a second preset threshold, and whether the height change rate is less than a third preset threshold;

[0122] The first determination unit is used to determine the human body image as a target human body image when the aspect ratio of the human body image is greater than a first preset threshold and the centroid distance is less than a second preset threshold.

[0123] The second determining unit is used to determine the human body image as a target human body image when the aspect ratio of the human body image is greater than a first preset threshold and the height change rate is less than a third preset threshold.

[0124] The third determination unit is used to determine the human body image as a target human body image when the centroid distance of the human body image is less than the second preset threshold and the height change rate is less than the third preset threshold.

[0125] Furthermore, the above-mentioned fall detection device further includes:

[0126] An acquisition module, used to acquire labeled training sample data, wherein the training sample data includes a fall sample image and a non-fall sample image, the label of the fall sample image is "1", and the label of the non-fall sample image is "0";

[0127] A building module is used to build a convolutional neural network model, wherein the convolutional neural network model includes a convolution layer, a pooling layer, a fully connected layer and a Softmax layer;

[0128] The training module is used to input the training sample data into the convolutional neural network model for training, and optimize the model parameters of the convolutional neural network through the stochastic gradient descent method to obtain a trained fall detection model.

[0129] Furthermore, the fall detection result includes a fall behavior event and a normal behavior event; the above-mentioned fall detection device also includes:

[0130] The early warning module is used to issue an early warning message when the fall detection result is a fall behavior event.

[0131] It should be noted that, regarding the fall detection device in the above embodiment, the specific manner in which each module or unit performs the operation has been described in detail in the embodiment of the method, and those skilled in the art can understand that it will not be repeated here.

[0132] Correspondingly, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium, on which a fall detection program is stored. When the fall detection program is executed by a processor, the steps of the fall detection method in any of the above embodiments are implemented.

[0133] In this embodiment, the above-mentioned storage medium may include but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card and other media that can store program code.

[0134] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0135] It should be noted that other contents of the fall detection method, device, equipment and storage medium disclosed in the present invention can be found in the prior art and will not be described in detail here.

[0136] The above description is only an optional embodiment of the present invention and does not limit the present invention in any form. Therefore, any modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A fall detection method, characterized in that: The fall detection method comprises the following steps: Acquire a video image of a target area, and extract a human body image set containing a target human body in the video image; Performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate; Based on the image state features, selecting a target human image to be detected from the human image set; Based on a preset fall detection model, performing fall detection on the target human body image to obtain a fall detection result; When the image state feature includes the centroid distance, the step of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes: Extracting the coordinates of the centroid point and the bottom edge endpoint of each human body image in the human body image set; Based on the extracted centroid coordinates and the bottom edge endpoint coordinates, the centroid distance of each human body image in the human body image set is calculated by a preset centroid distance calculation formula; wherein the centroid distance h refers to the distance from the centroid point O (x0, y0) to the line AB between the two bottom edge endpoints A (x1, y1) and B (x2, y2) in the rectangular human body image; the centroid distance calculation formula is: When the image state feature includes the height change rate, the step of performing state feature detection on the human body image set to obtain the image state feature of each human body image in the human body image set includes: The height change rate of each human body image in the human body image set is calculated by a preset height change rate calculation formula; wherein the height change rate calculation formula is: Wherein, height is the real-time height of the target human body in the rectangular human body image, and heightavg is the average height of the target human body in the human body image set.

2. The fall detection method according to claim 1, characterized in that: The step of extracting a human body image set containing a target human body from the video image comprises: Acquire a moving object image set containing moving objects in the video image by a preset background difference method; The human body image set containing the target human body is screened out from the moving object image set by using a preset DPM algorithm.

3. The fall detection method according to claim 1, characterized in that: The step of selecting a target human image to be detected from the human image set based on the image state feature comprises: Determine whether the aspect ratio of each of the human body images in the human body image set is greater than a first preset threshold, whether the centroid distance is less than a second preset threshold, and whether the height change rate is less than a third preset threshold; If the aspect ratio of the human body image is greater than the first preset threshold and the center of mass distance is less than the second preset threshold, or, the aspect ratio is greater than the first preset threshold and the height change rate is less than the third preset threshold, or, the center of mass distance is less than the second preset threshold and the height change rate is less than the third preset threshold, then the human body image is determined as the target human body image.

4. The fall detection method according to any one of claims 1 to 3, characterized in that: Before the step of performing fall detection on the target human body image based on the preset fall detection model to obtain the fall detection result, the method further includes: Acquire labeled training sample data, wherein the training sample data includes a fall sample image and a non-fall sample image, the label of the fall sample image is "1", and the label of the non-fall sample image is "0"; Constructing a convolutional neural network model, wherein the convolutional neural network model includes a convolution layer, a pooling layer, a fully connected layer and a Softmax layer; The training sample data is input into the convolutional neural network model for training, and the model parameters of the convolutional neural network are optimized by the stochastic gradient descent method to obtain the trained fall detection model.

5. The fall detection method according to any one of claims 1 to 3, characterized in that: The fall detection result includes a fall behavior event and a normal behavior event; after the step of performing fall detection on the target human body image based on a preset fall detection model to obtain the fall detection result, the step further includes: When the fall detection result is a fall behavior event, an early warning message is issued.

6. A fall detection device, characterized in that: include: An extraction module, used to obtain a video image of a target area and extract a human body image set containing a target human body in the video image; A feature detection module, used for performing state feature detection on the human body image set to obtain image state features of each human body image in the human body image set; wherein the image state features include aspect ratio, centroid distance and height change rate; A screening module, used for screening a target human image to be detected from the human image set based on the image state feature; A fall detection module, used to perform fall detection on the target human body image based on a preset fall detection model to obtain a fall detection result; The feature detection module is further used for: Extracting the coordinates of the centroid point and the bottom edge endpoint of each human body image in the human body image set; Based on the extracted centroid coordinates and the bottom edge endpoint coordinates, the centroid distance of each human body image in the human body image set is calculated by a preset centroid distance calculation formula; wherein the centroid distance h refers to the distance from the centroid point O (x0, y0) to the line AB between the two bottom edge endpoints A (x1, y1) and B (x2, y2) in the rectangular human body image; the centroid distance calculation formula is: The feature detection module is further used for: The height change rate of each human body image in the human body image set is calculated by a preset height change rate calculation formula; wherein the height change rate calculation formula is: Wherein, height is the real-time height of the target human body in the rectangular human body image, and heightavg is the average height of the target human body in the human body image set.

7. A fall detection device, characterized in that: The invention comprises a memory, a processor and a fall detection program stored in the memory and executable on the processor, wherein the fall detection program is configured to implement the steps of the fall detection method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a fall detection program, which, when executed by the processor, implements the steps of the fall detection method according to any one of claims 1 to 5.

Images