Fall detection method based on convolutional neural network and multi-discriminant features

Abstract

The present invention proposes a fall detection method based on convolutional neural network (CNN) and multi-discriminant features. First, the algorithm uses CNN to extract the coordinates of human joints and skeleton information in the video sequence; secondly, the joint coordinates are secondary processed with the help of the transformation relationship between points and vectors, and the angles between the human spine, left calf, right calf and the ground are calculated as a multi-feature extraction structure to collect rich fall information; finally, the comparison results of the angle and the threshold and the changes in the height-to-width ratio of the human calibration frame are comprehensively analyzed to realize the fall detection function. In addition, the present invention designs an IoT system framework for processing video sequences in the cloud to alleviate the problem of insufficient computing power of user terminals. Compared with traditional fall detection algorithms, the present invention has higher accuracy and better universality.

Description

Technical Field

[0001] The invention relates to image processing technology, in particular to a fall detection method based on convolutional neural network and multiple discriminant features. Background Art

[0002] Research results from the World Health Organization show that falling has become a major safety hazard for people over 65 years old, and 30% of the elderly fall at least once a year. If they cannot get timely help after falling, it is very easy to cause secondary injuries or even endanger their lives. Therefore, fall detection has become a research hotspot for scholars at home and abroad. Existing fall detection algorithms can be divided into two categories: non-visual and visual. Non-visual methods rely on the advantages of cheap, portable, and easy-to-deploy sensor equipment to embed sensors into wearable devices or install them in indoor environments, such as accelerometers, gyroscopes, vibration sensors, pressure sensors, etc., to obtain the movement information of the elderly and monitor their status, and have achieved certain results. However, due to factors such as the need to wear the device for a long time, limited detection range, and external vibration interference, the detection accuracy is low and it is difficult to promote on a large scale.

[0003] The vision-based method can not only collect rich human posture information, but also extract and track features in the video stream to detect falls, effectively making up for the shortcomings of non-visual methods. Based on the visual method, the present invention combines convolutional neural networks and designs a multi-discriminant feature structure to process video sequences frame by frame, extract and track the coordinates of human joints and skeleton information, and avoid the loss of feature information of the detection target. Unlike the traditional method that only extracts single feature information as the basis for fall discrimination, this method performs secondary processing on the joint coordinates to generate three vectors representing the position and direction of the human spine and left and right calves. By deeply fusing the comparison results of the angle between the vectors and the threshold, as well as the change in the aspect ratio of the human calibration frame, a multi-discriminant feature structure is constructed to achieve accurate fall detection function. At the same time, the cloud server is fully utilized to improve the real-time performance of the detection. Finally, the present invention effectively solves the shortcomings of insufficient feature extraction, poor generalization of the discrimination basis, and insufficient terminal computing power in the traditional visual method. Summary of the invention

[0004] The present invention proposes a fall detection method based on convolutional neural network and multiple discriminant features, which can realize accurate human fall detection function.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0006] A fall detection method based on convolutional neural network and multi-discriminant features comprises the following steps:

[0007] Step 1) Feature extraction: Use the VGG-19 network to process the original video sequence frame by frame, and input the feature map obtained after processing the original video sequence into the OpenPose model to obtain the coordinates of the human joint points and the human skeleton posture information;

[0008] Step 2) Fall detection: Based on the human skeleton posture information, the obtained human joint coordinates are connected to obtain three key feature vectors. The set aspect ratio of the human body calibration frame and the three key feature vectors and their angles with the horizontal direction are used as discriminant features and compared with the threshold to achieve human fall detection and discrimination.

[0009] The feature extraction comprises the following steps:

[0010] Step 1.1: Use the first 10 layers of the VGG-19 network to process the original video sequence frame by frame to generate feature maps as input to the OpenPose model;

[0011] Step 1.2: The OpenPose model processes the feature map in two stages. The first stage is used to annotate the confidence map of the joint points: predict the coordinates of the joint points based on the feature map; the second stage is used to generate a partial affinity domain vector field: connect the predicted joint point coordinates in pairs to form a limb vector, that is, predict the vector map between the joint point pairs as the human skeleton posture information;

[0012] Step 1.3: Iterate step 1.2 until the set number of iterations is reached to obtain the coordinates of the human body joint points and the human body skeleton posture information.

[0013] A loss function is added after each stage, and the loss function is spatially weighted to solve the problem that the data set cannot be fully labeled. i The loss function of the partial affinity field prediction stage is And the loss function of the joint point confidence map stage Respectively expressed as:

[0014]

[0015]

[0016] in, is the real partial affinity field of any point p in the image, C is the limb type, For any point p in the image at stage t i The generated partial affinity field, is the true confidence map of the location of any point p in the image, J is the joint point type, For any point p in the image at stage t iThe generated confidence map, W(p) is a binary mask. When the annotation at pixel p is missing, W(p) = 0, otherwise, W(p) = 1.

[0017] The first stage is used to mark the confidence map of the joint points. Specifically, if there is only one person in the feature map, each joint point of the person generates a position confidence map. If there are multiple people in the feature map, each visible joint point j of the person k corresponds to a peak value, indicating the most accurate prediction of the true joint point position; the confidence map of any point p in the feature map It is expressed as:

[0018]

[0019] where x j,k ∈R 2 is the true position of the joint point j of character k in the feature map, R is the size of the feature map, and σ is used to control the diffusion degree of the peak, that is, the probability distribution of the corresponding joint point, and then obtain the score of each predicted joint point. By performing non-maximum suppression, a set of discrete part candidate positions are obtained, that is:

[0020]

[0021] in, Represents the set of j types of joint points.

[0022] The second stage is used to generate a partial affinity domain vector field, specifically: each candidate joint point is arbitrarily combined in pairs to form a candidate limb vector, and the line integral of the partial affinity domain vector field is calculated to detect the reliability of the connection, that is, the integral value E:

[0023]

[0024] Among them, u is the uniform sampling between, p(u) is the corresponding sampling value, L c (p(u)) is the unit vector pointing from joint point j1 to joint point j2;

[0025] The set of all candidate joint points is:

[0026]

[0027] in, represents the mth candidate position of the jth part; at the same time, it determines the possible limb connection between each two parts, which is defined as Indicates joint points and Whether it is connected, if connected, the value is 1, otherwise it is 0, and the candidate limbs are connected according to the bipartite graph matching method that two edges cannot share a node, and finally the edge with the maximum integral value is found as the selected limb to characterize the position of the limb:

[0028]

[0029]

[0030]

[0031] Among them, E mn is the integral value of the partial affinity field of the mth joint point in class j1 and the nth joint point in class j2, E c is the set of C-type limbs, W c For all possible connected C-type limbs, Q j1 is the set of joint points of type j1, Q j2 It is the set of j2 type joint points.

[0032] In the second stage, a total of 25 joints are detected and labeled with numbers 0 to 24, where number 0 represents the nose, number 1 represents the neck, numbers 5 and 2 represent the left and right shoulders respectively, numbers 6 and 3 represent the left and right elbows respectively, numbers 7 and 4 represent the left and right wrists respectively, number 8 represents the center of gravity, numbers 12 and 9 represent the left and right hips respectively, numbers 13 and 10 represent the left and right knees respectively, numbers 14 and 11 represent the left and right ankles respectively, numbers 16 and 15 represent the left and right eyes respectively, numbers 18 and 17 represent the left and right ears respectively, numbers 19 and 21 represent the left and right big toes respectively, numbers 20 and 22 represent the left and right little toes respectively, numbers 24 and 23 represent the left and right heels respectively, and the coordinates of each joint point are defined as: nose (x0, y0), neck (x1, y1), ..., right heel (x 23 ,y 23 ), left heel (x 24 ,y 24 ).

[0033] The fall determination comprises the following steps:

[0034] Step 2.1: Generate corresponding feature vectors by connecting adjacent joint points, which respectively represent the positions and directions of the human spine, left calf, and right calf;

[0035] Step 2.2: Construct a human body calibration frame;

[0036] Step 2.3: By comparing the feature vector and the human body calibration frame with the threshold respectively, it is determined whether the detected target has fallen.

[0037] The step 2.1 is specifically as follows: suppose the characteristic vectors of the human spine, left calf, and right calf are Use the neck (x1, y1), center of mass (x8, y8), right knee (x 10 ,y 10 )、Right ankle(x 11 ,y 11 )、Left knee(x 13 ,y 13 )、Left ankle(x 14 ,y 14 )’s joint point coordinates are expressed as:

[0038]

[0039]

[0040]

[0041] use It represents the unit vector of the x-axis parallel to the ground, with the right direction of the feature map as the positive direction, denoted as but and The corresponding angle is expressed by the formula:

[0042]

[0043] Among them, θ1 represents the angle between the human spine and the ground; θ2 represents the angle between the left calf and the ground; θ3 represents the angle between the right calf and the ground.

[0044] The step 2.2 is specifically as follows: using X max , Y max , X min , Y min Construct a human body calibration frame, where X max , Y max , X min , Y min The minimum coordinate value of the joint point of the character in each frame image (X min ,Y min ) and the point with the maximum coordinate value (X max ,Y max ), X max -X min Indicates the width of the human body calibration box, Y max -Y min Represents the length of the human body calibration frame, and the aspect ratio of the human body calibration frame is:

[0045]

[0046] The step 2.3 is specifically as follows:

[0047] The threshold range of the tilt angle is set to 0°<α<θ i <β<180° and α<β, the length-width ratio threshold of the calibration frame is set to J;

[0048] The judgment results are divided into falling state and normal state, where:

[0049] (1) Falling state

[0050] When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, and R<J, a fall occurs;

[0051] (2) Normal state

[0052] When θ1, θ2, and θ3 are all greater than α and less than β, the character is in a standing state;

[0053] When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and θ2 and θ3 both satisfy greater than α and less than β, the character is in a bent state;

[0054] When θ1 satisfies greater than α and less than β, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, the character is in a squatting or sitting position;

[0055] When the character is in normal state, R>J is satisfied.

[0056] The present invention has the following beneficial effects and advantages:

[0057] 1. The method of the present invention can process the video sequence frame by frame to extract and track the coordinates of the human body joints and skeleton information, thus avoiding the loss of feature information of the detection target and ensuring effectiveness.

[0058] 2. The method of the present invention designs a multi-feature extraction structure that uses the angle between the human spine and the ground, the angle between the left and right calves and the ground, and the length-to-width ratio of the human body calibration frame as fall discrimination features, thereby solving the problem of poor generalization of the discrimination basis in traditional methods.

[0059] 3. The method of the present invention designs a fall detection IoT system framework, solves the problem of insufficient computing power of terminal devices through the powerful computing power of cloud servers, and completes real-time fall detection in multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of the fall detection method;

[0061] Figure 2 It is a flow chart of the fall detection method based on IoT system;

[0062] Figure 3 Implement a schematic for fall detection. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation method of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0065] like Figure 1 and Figure 3 As shown, the method of the present invention mainly includes two parts:

[0066] (1) Feature extraction;

[0067] (2) fall identification;

[0068] The specific implementation process is described as follows.

[0069] Step 1: Feature extraction.

[0070] Features include human joint coordinates and skeleton posture information. That is, the OpenPose model is used to process the original video sequence frame by frame, and the VGG-19 network structure is used to process the feature map. The feature map is processed in two branches: part position reliability prediction and part affinity field association, to complete the prediction of character joint coordinates and the extraction of human skeleton information.

[0071] Step 1.1: Calculate the loss function.

[0072] In a bottom-up manner, the joints are detected first, then they are associated with individuals, the original image is input, initialized and fine-tuned by the first 10 layers of VGG-19, and a set of feature maps are generated as the input of the first stage. There are two branches in the first stage of the iteration. Branch 1 is responsible for predicting the heat map of the joint position, that is, the approximate position of the joint; Branch 2 is responsible for predicting the vector map of the relationship between the joint pairs, that is, the correct connection of adjacent joints. At the same time, a loss function is applied after each iteration stage, and the loss function is spatially weighted to solve the practical problem that some data sets cannot be fully labeled. For example, in stage t iThe loss function of the partial affinity field prediction branch And the loss function of the joint point confidence branch It is expressed as:

[0073]

[0074]

[0075] in, is the real partial affinity field of any point p in the image, C is the limb type, For any point p in the image at stage t i The generated partial affinity field. is the true confidence map of the location of any point p in the image, J is the joint point type, For any point p in the image at stage t i The generated confidence map has one confidence map for each joint point. W(p) is a binary mask. When the annotation at pixel p is missing, W(p) = 0, otherwise, W(p) = 1.

[0076] Step 1.2: Annotate the confidence map of the 2D joint points, that is, each specific part of the body can be represented by the pixel coordinates in the image. Ideally, if there is a person in the image, a part position confidence map will be generated for each joint point of the person. If there are multiple people in the image, each visible joint point j of person k corresponds to a peak value to represent the most accurate predicted true position. The confidence map at any point p in the image is It can be expressed as:

[0077]

[0078] x j,k ∈R 2 is the real position of joint point j of person k in the image, R is the size of the image, and σ is used to control the diffusion degree of the peak value, that is, the probability distribution of the corresponding joint point. Then the score of each predicted joint point is obtained, and the accuracy of selecting body joint points is guaranteed by performing non-maximum suppression, that is:

[0079]

[0080] in Represents the set of j types of joint points.

[0081] Step 1.3: Partial affinity field generation. By performing non-maximum suppression on the confidence map in step 1.2, a set of discrete candidate part positions are obtained. The candidate joint points can be combined arbitrarily to generate a large number of candidate limbs. However, there are often a large number of incorrect connections in these limbs, so the partial affinity field vector field is used to find the optimal matching result, that is, the accurate limb position. The reliability of the connection is detected by calculating the size of the line integral of the corresponding partial affinity field along the line segment connecting the candidate part positions, that is, the integral value E:

[0082]

[0083] Among them, u is the The sampling is uniform between L and p(u), and p(u) is the corresponding sampling value. c (p(u)) is the unit vector pointing from joint point j1 to joint point j2.

[0084] The set of all candidate joint points is:

[0085]

[0086] in, represents the mth candidate position of the jth part. At the same time, the possible limb connection between each two parts is determined, which is defined as Indicates joint points and Whether it is connected, if connected, the value is 1, otherwise it is 0. And according to the bipartite graph matching method that two edges cannot share a node, the candidate limbs are connected. The ultimate goal is to find the edge with the maximum weight as the selected limb:

[0087]

[0088]

[0089]

[0090] Among them, E c is the weight, E mn is the integral value of the partial affinity field of the mth joint point in class j1 and the nth joint point in class j2, E c is the set of C-type limbs, W c For all possible connected C-type limbs, Q j1 is the set of joint points of type j1, Q j1is the set of joint points of type j2. Formula (1.8) and formula (1.9) restrict that no two edges can share joint points, that is, no two limbs of the same type share the same joint point. Therefore, the correct connection of each type of limb can be obtained by selecting the maximum value of E, and then the limbs that share the same joint points are connected to the human skeleton. At the same time, a total of 25 joint points are detected, and each joint point is labeled with a number from 0 to 24 for easy description. Among them, number 0 represents the nose, number 1 represents the neck, numbers 5 and 2 represent the left and right shoulders, numbers 6 and 3 represent the left and right elbows, numbers 7 and 4 represent the left and right wrists, number 8 represents the center of gravity, numbers 12 and 9 represent the left and right hips, numbers 13 and 10 represent the left and right knees, numbers 14 and 11 represent the left and right ankles, numbers 16 and 15 represent the left and right eyes, numbers 18 and 17 represent the left and right ears, numbers 19 and 21 represent the left and right big toes, numbers 20 and 22 represent the left and right little toes, numbers 24 and 23 represent the left and right heels, and the coordinates of each joint point are defined as: nose (x0, y0), neck (x1, y1), ..., right heel (x 23 ,y 23 ), left heel (x 24 ,y 24 ).

[0091] Step 2: Fall detection.

[0092] The present invention utilizes the fact that the information contained in the human torso during a fall is richer than that of a single joint point, connects the extracted coordinates of the human joint points, and constructs a multi-discriminant feature extraction structure composed of the angle between the human spine and the horizontal direction, the angle between the left and right calves and the horizontal direction, and the aspect ratio of the human calibration frame to complete the task of human fall detection.

[0093] Step 2.1: Calculate the inclination angles of the human spine and left and right calves.

[0094] According to the characteristic that the human trunk contains more information than a single joint point during a fall, the present invention generates corresponding feature vectors by connecting adjacent joint points to represent the positions and directions of the human spine, left calf, and right calf respectively. The joint points involved in the present invention are the neck, the center of gravity, the right knee, the right ankle, the left knee, and the left ankle, and are represented by the coordinates (x1, y1), (x8, y8), (x 10 ,y 10 )、(x 11 ,y 11 )、(x 13 ,y 13 )、(x 14 ,y 14 )express.

[0095] Therefore, the eigenvector can be obtained by the following formula

[0096]

[0097]

[0098]

[0099] in, Represents the human spine vector; represents the left calf vector; represents the right calf vector. represents the unit vector of the x-axis and is recorded as Because it is always parallel to the ground, it can be used to represent the direction vector of the real ground. and The corresponding angle can be expressed by the formula:

[0100]

[0101] Among them, θ1 represents the angle between the human spine and the ground; θ2 represents the angle between the left calf and the ground; θ3 represents the angle between the right calf and the ground.

[0102] Step 2.2: Calculate the height-to-width ratio of the human body calibration frame

[0103] Based on step 2.1, considering that the length-to-width ratio of the human body changes during the fall, this method uses X max , Y max , X min , Y min Construct a human body calibration frame, where X max , Y max , X min , Y min is the point with the minimum coordinate value of the joint point of the character in each frame of the image processed by the algorithm (X min ,Y min ) and the point with the maximum coordinate value (X max ,Y max ), as another feature to determine whether a person falls. max -X min Represents the width of the human body calibration box, Y max -Y min Represents the length of the human body calibration frame, and the aspect ratio of the human body calibration frame is:

[0104]

[0105] During the fall, θ1, θ2, θ3 and R are constantly changing, so it is necessary to determine the threshold range when the person falls. In order to make the method universal, the threshold range of the inclination angle is first set to 0°<α<θi <β<180° and α<β, the length-to-width ratio threshold of the calibration box is set to J.

[0106] Step 2.3: By fusing the fall discrimination features extracted in steps 2.1 and 2.2, the comparison results of parameters θ1, θ2, θ3 and R with the corresponding thresholds can be used to determine whether the detected target has fallen.

[0107] The judgment results can be divided into falling state and normal state, where:

[0108] (1) Falling state

[0109] When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, and R<J, a fall occurs;

[0110] (2) Normal state

[0111] When θ1, θ2, and θ3 are all greater than α and less than β, the character is in a standing state;

[0112] When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and θ2 and θ3 both satisfy greater than α and less than β, the character is in a bent state;

[0113] When θ1 satisfies greater than α and less than β, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, the character is in a squatting or sitting position;

[0114] When the character is in normal state, R>J is satisfied.

[0115] like Figure 2 As shown, the detection method of the present invention is embedded in a cloud server, utilizing the computing power of the cloud server to achieve the purpose of simultaneous detection of multiple scenes.

[0116] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fall detection method based on convolutional neural network and multi-discriminant features, characterized in that: The following steps are involved: Step 1) Feature extraction: Use the VGG-19 network to process the original video sequence frame by frame, and input the feature map obtained after processing the original video sequence into the OpenPose model to obtain the coordinates of the human joint points and the human skeleton posture information; Step 2) Fall discrimination: Based on the human skeleton posture information, the obtained human joint point coordinates are connected to obtain three key feature vectors, and the set human body calibration frame aspect ratio and the angles between the three key feature vectors and the horizontal direction are used as discriminant features. The above three key feature vectors are compared with the threshold to realize human fall detection and discrimination; The fall determination comprises the following steps: Step 2.1: Generate three key feature vectors by connecting adjacent joint points, which represent the positions and directions of the human spine, left calf, and right calf respectively; Step 2.2: Construct a human body calibration frame; Step 2.3: By comparing the three key feature vectors and the human body calibration frame with the threshold, it is determined whether the detected target has fallen; The step 2.1 is specifically as follows: suppose the characteristic vectors of the human spine, left calf, and right calf are Use the neck (x1, y1), center of mass (x8, y8), right knee (x 10 ,y 10 )、Right ankle(x 11 ,y 11 )、Left knee(x 13 ,y 13 )、Left ankle(x 14 ,y 14 )’s joint point coordinates are expressed as: use It represents the unit vector of the x-axis parallel to the ground, with the right direction of the feature map as the positive direction, denoted as but and The corresponding angle is expressed by the formula: Among them, θ1 represents the angle between the human spine and the ground; θ2 represents the angle between the left calf and the ground; θ3 represents the angle between the right calf and the ground; The step 2.3 is specifically as follows: The threshold range of the tilt angle is set to 0°<α<θ i <β<180° and α<β, the length-to-width ratio threshold of the calibration frame is set to J; The judgment results are divided into falling state and normal state, where: (1) Falling state When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, and R<J, a fall occurs; (2) Normal state When θ1, θ2, and θ3 are all greater than α and less than β, the character is in a standing state; When θ1 satisfies greater than 0° and less than α, or greater than β and less than 180°, and θ2 and θ3 both satisfy greater than α and less than β, the character is in a bent state; When θ1 satisfies greater than α and less than β, and at least one of θ2 or θ3 satisfies greater than 0° and less than α, or greater than β and less than 180°, the character is in a squatting or sitting position; When the character is in normal state, R>J is satisfied.

2. A fall detection method based on convolutional neural network and multi-discriminant features according to claim 1, characterized in that: The feature extraction comprises the following steps: Step 1.1: Use the first 10 layers of the VGG-19 network to process the original video sequence frame by frame to generate feature maps as input to the OpenPose model; Step 1.2: The OpenPose model processes the feature map in two stages. The first stage is used to annotate the confidence map of the joint points: predict the coordinates of the joint points based on the feature map; the second stage is used to generate a partial affinity domain vector field: connect the predicted joint point coordinates in pairs to form a limb vector, that is, predict the vector map between the joint point pairs as the human skeleton posture information; Step 1.3: Iterate step 1.2 until the set number of iterations is reached to obtain the coordinates of the human body joint points and the human body skeleton posture information.

3. A fall detection method based on convolutional neural network and multi-discriminant features according to claim 2, characterized in that: A loss function is added after each stage, and the loss function is spatially weighted to solve the problem that the data set cannot be fully labeled. i The loss function of the partial affinity domain vector field prediction stage is And the loss function of the joint point confidence map stage Respectively expressed as: in, is the real partial affinity field of any point p in the image, C is the limb type, For any point p in the image at stage t i The generated partial affinity field, is the true confidence map of the location of any point p in the image, J is the joint point type, For any point p in the image at stage t i The generated confidence map, W(p) is a binary mask. When the annotation at pixel p is missing, W(p) = 0, otherwise, W(p) = 1.

4. The fall detection method based on convolutional neural network and multi-discriminant features according to claim 2, characterized in that: The first stage is used to mark the confidence map of the joint points. Specifically, if there is only one person in the feature map, each joint point of the person generates a position confidence map. If there are multiple people in the feature map, each visible joint point j of the person k corresponds to a peak value, indicating the most accurate prediction of the true joint point position; the confidence map of any point p in the feature map It is expressed as: where x j,k ∈R 2 is the true position of the joint point j of character k in the feature map, R is the size of the feature map, and σ is used to control the diffusion degree of the peak, that is, the probability distribution of the corresponding joint point, and then obtain the score of each predicted joint point. By performing non-maximum suppression, a set of discrete part candidate positions are obtained, that is: in, Represents a set of j types of joint points.

5. The fall detection method based on convolutional neural network and multi-discriminant features according to claim 2, characterized in that: The second stage is used to generate a partial affinity domain vector field, specifically: each candidate joint point is arbitrarily combined in pairs to form a candidate limb vector, and the line integral of the partial affinity domain vector field is calculated to detect the reliability of the connection, that is, the integral value E: Among them, u is the uniform sampling between, p(u) is the corresponding sampling value, L c (p(u)) is the unit vector pointing from joint point j1 to joint point j2; The set of all candidate joint points is: in, represents the mth candidate position of the jth part; at the same time, it determines the possible limb connection between each two parts, which is defined as Represents candidate joint points and Whether it is connected, if connected, the value is 1, otherwise it is 0, and the candidate limbs are connected according to the bipartite graph matching method that two edges cannot share a node, and finally the edge with the maximum integral value is found as the selected limb: Among them, E mn is the integral value of the partial affinity field of the mth joint point in class j1 and the nth joint point in class j2, E c is the set of C-type limbs, W c For all possible connected C-type limbs, is the set of j1-type joint points, It is the set of j2 type joint points.

6. A fall detection method based on convolutional neural network and multi-discriminant features according to claim 5, characterized in that: In the second stage, a total of 25 joints are detected and labeled with numbers 0 to 24, where number 0 represents the nose, number 1 represents the neck, numbers 5 and 2 represent the left and right shoulders respectively, numbers 6 and 3 represent the left and right elbows respectively, numbers 7 and 4 represent the left and right wrists respectively, number 8 represents the center of gravity, numbers 12 and 9 represent the left and right hips respectively, numbers 13 and 10 represent the left and right knees respectively, numbers 14 and 11 represent the left and right ankles respectively, numbers 16 and 15 represent the left and right eyes respectively, numbers 18 and 17 represent the left and right ears respectively, numbers 19 and 21 represent the left and right big toes respectively, numbers 20 and 22 represent the left and right little toes respectively, numbers 24 and 23 represent the left and right heels respectively, and the coordinates of each joint point are defined as: nose (x0, y0), neck (x1, y1), ..., right heel (x 23 ,y 23 ), left heel (x 24 ,y 24 ).

7. A fall detection method based on convolutional neural network and multi-discriminant features according to claim 1, characterized in that: The step 2.2 is specifically as follows: using X max , Y max , X min , Y min Construct a human body calibration frame, where X max , Y max , X min , Y min The minimum coordinate value of the joint point of the character in each frame image (X min ,Y min ) and the point with the maximum coordinate value (X max ,Y max ), X max -X min Indicates the width of the human body calibration box, Y max -Y min Represents the length of the human body calibration frame, and the aspect ratio of the human body calibration frame is:

Images