Pedestrian Fall Detection Method Based on Bidirectional Decay Loss

Abstract

A pedestrian fall detection method based on bidirectional attenuation loss. First, pedestrian images are collected, and then the fall state of pedestrians in the pedestrian images is detected. The detection of pedestrian images is carried out using a pedestrian fall detection network. The construction method of this detection network is as follows: taking the YOLOX object detection network as the basic network, adding an angular dimension to the detection head to rotate the detection box, and using the bidirectional attenuation loss function as the regression loss function; the backbone network and the feature transfer structure continue to use the structure of YOLOX; the method for designing the bidirectional attenuation loss function is to first convert two rotated rectangular boxes into Gaussian distributions in the detected human detection box, and use the result of Gaussian multiplication to measure the rotation angle deviation; then, combined with the vector formula, the deviation of the offset distance and the deviation of the offset angle of the offset are reflected by the length of the projection vector. The detection efficiency of this method is significantly higher than that of existing detection methods, making the detection of pedestrian falls feasible.

Description

Technical Field

[0001] The present invention relates to the technical field of human behavior recognition, and specifically relates to a pedestrian fall detection method based on a convolutional neural network applicable to complex monitoring scenarios. Technical Background

[0002] Human action recognition has always been one of the research hotspots in computer vision, and has broad application prospects in many fields such as intelligent monitoring, motion analysis, and human-computer interaction. Its purpose is to extract the motion information in the video and identify the main action types that appear in the video.

[0003] Nowadays, with the aggravation of population aging, the proportion of the elderly population is increasing rapidly. At the same time, existing research shows that the frequency of falls increases with age, and the fatality rate is second only to traffic injuries, becoming the second largest accidental injury death. Therefore, the detection task of such abnormal behaviors has become one of the hot research topics in computer vision.

[0004] In the prior art, Chinese patent application No. 109887238A discloses a fall detection system based on vision and artificial intelligence. Its principle is to collect the physiological and posture data of the human body, and obtain a comprehensive fall risk probability value through algorithms such as human body posture, center of gravity detection algorithm, breathing detection algorithm, and environmental object recognition algorithm. This method is applicable to care places such as hospitals and is not applicable to the detection of pedestrian falls in public places.

[0005] Chinese patent application No. 115331283A discloses a detection system and its detection method for detecting a person's fall in a living space. Its algorithm for identifying a fall includes screenshot of a video stream, minimum image interception of human body key points of each person in the image, posture recognition of single-person human body key points, recognition of chairs in the living space, and recognition of the ground in the living space, and forms a logic for determining human fall, and finally detects the fall state. This technology uses a 2D human pose estimation method, intercepts the minimum range of a single person, and determines whether to fall based on the relative positions of human joint points. Its computational complexity is large and it is not applicable to the detection of pedestrian fall states. Summary of the Invention

[0006] In order to solve the problem of efficient detection of pedestrians falling on the road surface, the present invention uses a convolutional neural network and improves the network, starting from the offset distance and offset angle of the pedestrian detection frame to detect pedestrian falls.

[0007] Specifically, the present invention is a pedestrian fall detection method based on bidirectional decay loss. First, pedestrian images are collected, and then the fall state of pedestrians in the pedestrian images is detected. The detection of pedestrian images is carried out using a pedestrian fall detection network. The construction method of this detection network is as follows: <omitted for simplicity as it's not clear what's required here without further context, but you can add it back if needed> <omitted for simplicity as it's not clear what's required here without further context, but you can add it back if needed>

[0008] Taking the YOLOX object detection network as the basic network, adding an angular dimension to its detection head to rotate the detection box, and using a bidirectional decay loss function as the regression loss function; the backbone network and the feature transfer structure continue to use the structure of YOLOX;

[0009] The method for designing the bidirectional decay loss function is as follows: first, in the detected human detection box, convert two rotated rectangular boxes into Gaussian distributions, and use the result of Gaussian multiplication to measure the rotation angle deviation; then, in combination with the vector formula, reflect the deviation distance deviation and deviation angle deviation of the offset amount through the length of the projection vector; the specific steps are as follows:

[0010] 1) Convert the two rotated rectangular boxes B gt (x, y, h, w, θ) and B pre (x, y, h, w, θ into Gaussian distributions G gt (μ, ∑) and G pre (μ, Σ):

[0011] ∑ = RΛR T , μ = (x, y) T (1)

[0012]

[0013] In the formula, x and y are the coordinates of the center point of the corresponding rotated rectangular box in the figure, h and w are its height and width respectively, and θ represents its rotation angle. R represents the rotation matrix, and Λ represents the diagonal matrix of eigenvalues.

[0014] 2) First, regard the two Gaussian distributions as having the same center point, and perform Gaussian multiplication to obtain the Gaussian distribution of the intersection region;

[0015] G KF (μ, ∑) = G gt (μ, ∑)G pre (μ, ∑) (3)

[0016] 3) Convert the Gaussian distribution G KF of the intersection region back to the rotated rectangular box B KF , then calculate the area S B of the rotated rectangular box, and calculate the approximate intersection over union loss IoU_loss through the area;

[0017]

[0018]

[0019] IoU_loss = 1 - IoU (6)

[0020] In the formula, the area S of the rectangular boxB It can be calculated through the covariance formula. Here, n represents the dimension, and 2 is taken here. They respectively correspond to the areas of the rotated rectangular boxes in the intersection region, the true value, and the predicted value.

[0021] 4) For the obtained rotated rectangular box region, the attenuation coefficient for attenuation based on the center point offset is α

[0022] Loss = α × IoU_Loss (7)

[0023] The attenuation ratio is directly related to the center point offset, and the composition of the attenuation coefficient is as follows:

[0024]

[0025] In the formula, and are respectively: the vector connecting the center point of the true value to the center point of the prediction box The projection lengths in the two directions of the long side and the short side of the true value respectively serve as the independent variables in the attenuation coefficient; λ is a hyperparameter; S h and S w are respectively scaling factors, and their values are the farthest attenuation distances of the prediction box in the long side and short side directions of the true value.

[0026] Preferably, the YOLOX_l object detection network is used as the basic network. and values are respectively obtained through the projection formula, and the hyperparameter λ takes the value of 1.75.

[0027] The scaling factor S h and S w The calculation process is as follows:

[0028]

[0029]

[0030]

[0031] In the formula, w gt and h gt respectively represent the width and height of the true box, and w pre and h pre respectively represent the width and height of the prediction box.

[0032] The present invention designs a new loss function to comprehensively reflect the offset distance and offset angle of the detection box, and realizes the detection of the pedestrian fall state on a rotated detection network improved based on the YOLOX network. The detection efficiency of this method is significantly higher than that of the existing detection methods, making the detection of pedestrian falls feasible. Description of the Drawings

[0033] Figure 1 is a schematic diagram of the two-way attenuation loss calculation process;

[0034] Figure 2a and Figure 2b is a schematic diagram of the composition of the attenuation coefficient, where

[0035] Figure 2a represents the projection vector, Figure 2b represents the farthest attenuation distance;

[0036] Figure 3a and Figure 3b are the images of the actual skewed IoU and the approximation changing with the offset of the center point, where

[0037] Figure 3a represents the actual change of the skewed IoU, Figure 3b represents the two-way attenuation loss;

[0038] Figure 4 is the structure diagram of the pedestrian fall detection network;

[0039] Figure 5a and Figure 5b are the detection effect diagrams. Detailed Implementation Manner

[0040] The following further describes the present case in combination with the drawings and the detailed implementation manner.

[0041] The present invention uses a neural network to identify and detect pedestrian images, and improves the neural network. The new loss function is used to comprehensively reflect the offset distance and offset angle of the predicted value relative to the true label, and realizes the detection of the pedestrian fall state on a rotation detection network improved based on the YOLOX network. This loss calculation method abandons the angle prediction that is prone to inconsistent problems, and instead uses a Gaussian model for prediction, while taking into account the problems caused by the offset.

[0042] Step 1: Design a two-way attenuation loss function

[0043] First, convert it into a Gaussian distribution in the detected human detection box, and use the result of Gaussian multiplication to measure the rotation angle deviation.

[0044] After that, in combination with the vector formula, the offset distance deviation and offset angle deviation of the offset amount are reflected by the length of the projection vector.

[0045] The specific process of the algorithm is as Figure 1 .

[0046] 1) The two rotation rectangles B of the true value and the predicted value gt(x, y, h, w, θ) and B pre (x, y, h, w, θ) is transformed into a Gaussian distribution G gt (μ, ∑) and G pre (μ, ∑):

[0047] ∑ = RΛR T , μ = (x, y) T (1)

[0048]

[0049] In the formula, x and y are the coordinates of the center point of the corresponding rotated rectangle in the figure, h and w are its height and width respectively, and θ represents its rotation angle. R represents the rotation matrix, and Λ represents the diagonal matrix of eigenvalues.

[0050] 2) First, consider the two Gaussian distributions as having the same center point, and perform Gaussian multiplication to obtain the Gaussian distribution of the intersection region;

[0051] G KF (μ, ∑) = G gt (μ, ∑) G pre (μ, ∑) (3)

[0052] 3) Convert the Gaussian distribution G of the intersection region KF back to the rotated rectangle B KF , and then calculate the area S of the rotated rectangle B , and calculate the approximate intersection over union loss IoU_loss through the area calculation;

[0053]

[0054]

[0055] IoU_loss = 1 - IoU (6)

[0056] In the formula, the rectangle area S B can be calculated through the covariance formula, n represents the dimension, and here it is taken as 2. They correspond to the areas of the rotated rectangles of the intersection region, the ground truth, and the prediction respectively.

[0057] 4) Decay the obtained rotated rectangle region according to the center point offset, and the decay coefficient is α.

[0058] Loss = α × IoU_Loss (7)

[0059] The decay ratio is directly related to the center point offset, and the specific composition of the decay coefficient is as follows:

[0060]

[0061] In the formula, and are respectively the projection lengths of the vector connecting the center point of the true value to the center point of the prediction box in the two directions of the long side and the short side of the true value, serving as the independent variables in the attenuation coefficient, as Figure 2a shown. and can be obtained through the projection formula respectively, and the hyperparameter λ takes the value of 1.75.

[0062] Due to the diverse and unpredictable postures of pedestrians, the aspect ratios of the detection boxes are inconsistent. Therefore, the present invention introduces the scaling factors S h and S w , whose function is to design the attenuation amplitude of the loss for the length and width of the specific target. Their values are respectively the farthest attenuation distances of the prediction box in the long side and short side directions of the true value (the center point offset distance when the skewed IoU just decays to 0, as Figure 2b shown).

[0063] The specific calculation process is as follows:

[0064]

[0065]

[0066]

[0067] In the formula, w gt and h gt respectively represent the width and height of the true box, and w pre and h pre respectively represent the width and height of the prediction box.

[0068] Figure 3a and Figure 3b are respectively the approximate skewed IoU and the actual skewed IoU varying with the center point offset for the calculation of the bidirectional attenuation loss. The attenuation amplitude and attenuation distance of the bidirectional attenuation loss function in each direction conform to the variation law of the actual skewed IoU, and due to the characteristics of the exponential function, the non - overlapping situation of the rectangular boxes can be calculated.

[0069] Step 2: Build a pedestrian fall detection network:

[0070] The YOLOX model has four different versions, including YOLOX_s, YOLOX_m, YOLOX_l, and YOLOX_x. Its feature extraction backbone consists of the CSPDarknet53 structure and the SPP layer, and the feature map results are fused, stacked, and passed to its unique decoupled detection head through the PANet feature fusion structure. YOLOX is the first one-stage detection model in the YOLO series based on anchor-free, which enables YOLOX to achieve better performance and stronger transferability on the basis of light weight.

[0071] The present invention uses the YOLOX_l object detection network as the basic network and makes adaptive modifications for rotation detection: including adding an angle dimension to the detection head to rotate the detection box, and using the bidirectional attenuation loss as the regression loss function. The backbone network and the feature transfer structure continue to use the structure of YOLOX. The network structure is as Figure 4 shown.

[0072] Step 3: Train the network:

[0073] The hardware configuration used for training adopts the Intel(R) Xeon(R) Silver 4210R processor and two NVIDIA 2080Ti graphics cards. The software environment is Ubuntu 18.04, and the algorithm implementation is completed based on the Python 3.7 and PyTorch deep learning frameworks.

[0074] In the preprocessing stage of the dataset, for the convenience of training, all images are cropped to a fixed size of 640×640 pixels, and the images containing targets will be retained as the input samples for training the network. To avoid affecting the detection of targets at the edges, a 200-pixel padding is set during cutting. Before feeding into the network for training, Mosaic augmentation and MixUp augmentation are used as the augmentation strategies for the network, and data augmentation is turned off in the last few rounds of training. In the experiment, dual-GPU parallel training is adopted, the batch size is 4, the network depth is 1.33, and the network width is 1.25. The initial learning rate for model training is 0.01, the optimization method is stochastic gradient descent, the momentum is 0.9, and the weight decay is 0.05. The hyperparameter λ in the bidirectional attenuation loss is set to 2.

[0075] The detection effect is shown as Figure 5a and Figure 5b .

Claims

1. A pedestrian fall detection method based on bidirectional attenuation loss. First, pedestrian images are collected, and then the fall state of pedestrians in the pedestrian images is detected. Its characteristics are The detection of pedestrian images is carried out using a pedestrian fall detection network. The construction method of this detection network is as follows: Taking the YOLOX object detection network as the basic network, adding an angle dimension to the detection head to rotate the detection box, and using a bidirectional attenuation loss function as the regression loss function; the backbone network and the feature transfer structure continue to use the structure of YOLOX; The method for designing the bidirectional attenuation loss function is to first convert two rotated rectangular boxes into Gaussian distributions in the detected human detection box, and use the result of Gaussian multiplication to measure the rotation angle deviation; Then, in combination with the vector formula, the deviation of the offset distance and the deviation of the offset angle of the offset are reflected by the length of the projection vector; the specific steps include: 1) Convert the two rotated rectangular boxes B of the true value and the predicted value gt (x, y, h, w, θ) and B pre (x, y, h, w, θ) into Gaussian distributions G gt (μ, ∑) and G pre (μ, ∑): ∑=RΛR T ,μ=(x,y) T (1) In the formula, x and y are the coordinates of the center points of the corresponding rotated rectangular boxes in the figure, h and w are their heights and widths respectively, and θ represents the rotation angle; R represents the rotation matrix, and Λ represents the diagonal matrix of eigenvalues; 2) First, regard the two Gaussian distributions as having the same center point, and perform Gaussian multiplication to obtain the Gaussian distribution of the intersection area; G KF (μ, ∑) = G gt (μ, ∑)G pre (μ, ∑) (3) 3) Convert the Gaussian distribution G of the intersection region KF back to the rotated rectangle B KF , and then calculate the area S of the rotated rectangle B , and calculate the approximate intersection over union loss IoU_loss through the area calculation; IoU_loss = 1 - IoU (6) In the formula, the area S of the rectangular box B is calculated by the covariance formula, where n represents the dimension, and here n is taken as 2; correspond to the areas of the rotated rectangular boxes of the intersection region, the true value, and the predicted value respectively; 4) For the obtained rotated rectangular box area, the attenuation coefficient for attenuation according to the center point offset is α Loss = α × IoU_Loss (7) The attenuation ratio is directly related to the center point offset. The composition of the attenuation coefficient is as follows: In the formula, and are respectively: the vector of the line connecting the center point of the true value to the center point of the prediction box are respectively the projection lengths in the two directions of the long side and the short side of the true value, and they are used as the independent variables in the attenuation coefficient; λ is a hyperparameter; S h and S w are respectively the scaling factors, and their values are respectively the farthest attenuation distances of the prediction box in the long side and short side directions of the true value.

2. The pedestrian fall detection method based on bidirectional attenuation loss according to claim 1, wherein Use the YOLOX_1 object detection network as the basic network.

3. The pedestrian fall detection method based on bidirectional attenuation loss according to claim 1, characterized in that and The values of are obtained by the projection formula respectively, and the hyperparameter λ is taken as 1.

75.

4. The pedestrian fall detection method based on bidirectional attenuation loss according to claim 1, wherein the scaling factor S h and S w The calculation process is as follows: In the formula, w gt and h gt respectively represent the width and height of the ground truth box, and w pre and h pre respectively represent the width and height of the predicted box.

5. The pedestrian fall detection method based on bidirectional attenuation loss according to claim 1, characterized in that Train the constructed detection network: In the preprocessing stage of the dataset, for the convenience of training, all images are cropped to a fixed size of 640×640 pixels, and the images containing the targets are used as the input samples for training the network; when cropping, set an overlapping padding of 200 pixels; Before the input samples are fed into the detection network for training, use Mosaic augmentation and MixUp augmentation as the augmentation strategies for the detection network, and turn off data augmentation in the last few rounds of training; During training, the batchsize is 4, the network depth is 1.33, and the network width is 1.25; the initial learning rate for training is 0.01, the optimization method is stochastic gradient descent, the momentum is 0.9, and the weight decay is 0.05; the hyperparameter λ in the bidirectional attenuation loss is set to 2.

Images