Establishment method and application of personnel tumble detection model in dense crowd environment

Abstract

The invention relates to an establishment method and application of a personnel tumble detection model in a dense crowd environment. Comprising the following steps: S1, constructing a data set, and obtaining personnel falling pictures with a personnel dense condition as the data set; s2, an SPPFCSPC module, a CBAM attention mechanism and an EIOU-Soft-NMS module are introduced on the basis of the YOLOv5 model, and an FIDC-YOLOV5 model, namely, a fall detection model, is obtained; wherein the SPPFCSPC module is used for improving the feature extraction performance of a model and reducing the parameter quantity of the model; wherein the CBAM attention mechanism is used for improving the perception ability of the model; wherein the EIOU-Soft-NMS module is used for realizing detection of falling of people under the condition that shielding exists in a crowd; s3, training the tumble detection model by using the data set obtained in the step S1; and S4, testing and evaluating the fall detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of fall detection, and in particular to a method for establishing a person fall detection model in a densely populated environment and an application thereof, which is applied to fall detection in public places with densely populated people. Background Art

[0002] Accidental falls pose a major challenge to public safety, especially in crowded places. Falls are likely to lead to stampedes. To solve this problem, the development of a fall detection model is crucial. Existing fall detection systems are usually developed based on environmental sensors and wearable sensors. The detection methods of these systems are often used to detect a specific group of people, such as the elderly. If they are deployed in crowded public places, they require huge deployment and computing costs.

[0003] Traditional detection methods are divided into environmental sensors and wearable sensors. Environmental sensors detect possible falls by installing monitoring devices in people's residences and public places to collect data on pressure, vibration, and sound. However, these sensors are usually installed in specific areas of residential or public places and may provide limited coverage because they can only identify falls of people within a certain range. In addition, such detection systems usually lack the ability to identify environmental information surrounding the fall, which limits the accuracy and reliability of the detection. Wearable sensor devices require users to insist on wearing specific devices, however, this may not be an easy thing for users to do.

[0004] Therefore, fall detection based on computer vision is a better choice, which provides a low-cost solution to the fall detection problem. Although many fall detection models have been proposed, most of them are developed for vulnerable groups such as the elderly, and the application scenarios are often at home or in places where the elderly live. There are relatively few studies on public places with dense crowds. Summary of the invention

[0005] The purpose of the present invention is to provide a method for establishing a fall detection model for a dense crowd environment and its application in response to the problems mentioned in the background technology, to establish a fall detection model in a dense crowd, to adopt a new pooling structure SPPFCSPC to replace the SPPF pooling structure of the original network, to improve the feature extraction performance while reducing the number of parameters; to introduce an attention module CBAM, combined with the C3 module, the attention mechanism CBAM improves the perception ability of the model through channel attention and spatial attention, thereby effectively extracting different features of people falling and standing without increasing the complexity of the network; to solve the problems of missed detection and false detection in dense crowd detection in the presence of occlusion by introducing the EIOU-Soft-NMS module.

[0006] In order to achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] The method for establishing a fall detection model in a dense crowd environment includes the following steps:

[0008] S1. Build a data set and obtain pictures of people falling in crowded places as the data set;

[0009] S2. Based on the YOLOv5 model, the SPPFCSPC module, CBAM attention mechanism and EIOU-Soft-NMS module are introduced to obtain the FIDC-YOLOV5 model, i.e., the fall detection model;

[0010] Among them, the SPPFCSPC module is used to improve the feature extraction performance of the model and reduce the number of model parameters;

[0011] Among them, the CBAM attention mechanism is used to improve the perception ability of the model;

[0012] Among them, the EIOU-Soft-NMS module is used to detect people falling in the presence of occlusion in the crowd;

[0013] S3, using the data set obtained in step S1 to train the fall detection model;

[0014] S4. Test and evaluate the fall detection model.

[0015] YOLOv5 is a real-time target detection algorithm model based on deep learning.

[0016] Furthermore, in step S2, the SPPFCSPC pooling structure is used to replace the original SPPF pooling structure; the SPPFCSPC pooling structure combines the grouped convolution block with the classic SPP method, and has several grouped convolution blocks and three serial maximum pooling layers with the same receptive field; the input feature maps are grouped by grouped convolution, and independent convolution operations are performed in each group; by distributing the calculations in different groups, the computational load and the number of parameters are reduced.

[0017] Furthermore, the grouped convolution blocks have 6, including 1×1 and 3×3 convolutions.

[0018] Furthermore, in step S2, the CBAM attention mechanism is integrated into the C3 module to obtain the CBAMC3 module, which improves the perception ability of the model through channel attention and spatial attention, thereby effectively extracting different features of people falling and standing without increasing the complexity of the network;

[0019] The CBAM attention mechanism consists of two key parts, namely the channel attention module CAM and the spatial attention module SAM, which are respectively embedded in different layers of the convolutional neural network (CNN) to enhance feature representation;

[0020] The channel attention module CAM is used to enhance the feature expression of each channel;

[0021] The spatial attention module SAM is used to emphasize the importance of different positions in the image.

[0022] Furthermore, the method steps of the channel attention module CAM to realize feature expression are as follows:

[0023] For the input feature map F:

[0024] First, perform global maximum pooling MaxPool and global average pooling AvgPool operations on each channel to calculate the maximum eigenvalue on each channel and the average eigenvalue Generate two feature vectors containing the number of channels, which represent the global maximum feature and average feature of each channel respectively;

[0025] Secondly, the feature vectors after global maximum pooling and average pooling are input into a shared fully connected layer to learn the attention weight of each channel. The shared fully connected layer includes a multi-layer perceptron MLP. Through learning, the network can adaptively determine the channels that are more important for the current task.

[0026] Again, intersect the global maximum eigenvector and the average eigenvector to obtain the final attention weight vector;

[0027] Subsequently, in order to ensure that the attention weight is between 0 and 1, the final attention weight vector is activated by the sigmoid function to generate the channel attention weight Mc(F). The generation formula of the channel attention weight is formula (1);

[0028]

[0029] Where σ represents the Sigmoid activation function, W0∈R C / r×C , W1∈R C×C / r , the weights W0 and W1 of MLP are shared; in deep learning, r is the reduction ratio, C is Channels, which means the number of channels; R represents the batch size, which is the number of images;

[0030] Finally, the above channel attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted feature map F', which is calculated as formula (2), thereby emphasizing the channels that are helpful to the current task and suppressing irrelevant channels;

[0031] F'=Mc(F)×F (2)

[0032] Furthermore, the spatial attention module SAM emphasizes the importance of different positions in the image, and its specific implementation steps are as follows:

[0033] For the input feature map F', the input feature map F' is the product of the channel attention weight Mc(F) and the original feature map F;

[0034] First, the maximum pooling and average pooling operations are performed on the channel dimension respectively, and the channel information of the feature map is aggregated by using these two pooling operations to generate two 2D maps, namely and To generate features of different context scales;

[0035] Secondly, the features after maximum pooling and average pooling are concatenated in the channel dimension to obtain a feature map containing multi-scale context information;

[0036] Thirdly, the feature map containing multi-scale context information is processed through a convolutional layer to generate spatial attention weights;

[0037] Subsequently, the Sigmoid activation function is applied to the generated spatial attention weights to limit the weights between 0 and 1, and the calculation formula is (3);

[0038]

[0039] Among them, σ represents the sigmoid function; f 7×7 represents a convolution operation with a filter size of 7×7; MaxPool() represents a global maximum pooling operation; and AvgPool() represents a global average pooling operation;

[0040] Finally, the obtained spatial attention weight Ms(F') is applied to the input feature map F', and the features of each spatial position are weighted to obtain the attention-weighted feature map F', which is calculated as formula (4); thereby highlighting the important image areas and reducing the influence of unimportant areas;

[0041] F”=Ms(F')F' (4)

[0042] Furthermore, the CBAM attention mechanism multiplies the output features of the channel attention module and the spatial attention module element by element to obtain the final attention-enhanced features; the attention-enhanced features are used as input to subsequent network layers to suppress noise and irrelevant information while retaining key information.

[0043] Furthermore, the CBAM attention mechanism first integrates the channel dimension and then the spatial dimension in the CBAM module; specifically, by connecting the channel attention module and the spatial attention module in series, the feature map F is enhanced twice to obtain the final output feature map F', and the processing formula (5) is:

[0044] F'=Mc(F)F

[0045] F”=Ms(F')F' (5)

[0046] Furthermore, in step S2, the EIOU-Soft-NMS module is introduced. Specifically, the IOU function in the Soft-NMS module is replaced with the EIOU loss function to construct the EIOU-Soft-NMS module, which can effectively reduce missed detections and improve detection accuracy.

[0047] Furthermore, the EIOU-Soft-NMS module includes two key parts, namely, the EIOU loss function and the Soft-NMS non-maximum suppression module;

[0048] The Soft-NMS non-maximum suppression module is used to suppress redundant boxes during detection. It introduces a fractional attenuation function to handle overlapping detection boxes. The specific formula is as follows:

[0049]

[0050] Among them, S i represents the score of each bounding box, M represents the bounding box with the highest score at the moment, and B i Indicates one of the remaining bounding boxes. When detecting an object, multiple prediction boxes will be generated first, and the most suitable box will be obtained through screening. This refers to the process of screening using the Soft-NMS non-maximum suppression module, in addition to the highest scoring bounding box, there is one remaining bounding box; D represents the final prediction box set after being suppressed by the Soft-NMS non-maximum suppression module; σ is a coefficient, which can be selected to be 0.3;

[0051] The EIOU loss function is a loss function widely used in target detection tasks, which is mainly used to evaluate the degree of overlap between target detection results and actual annotation boxes, and is used as an optimization target in the model training process to improve the performance of the model.

[0052] The formula of the EIOU loss function is as follows:

[0053]

[0054] The EIOU loss function consists of overlap loss (IOU), center distance loss (dis) and width-height loss (asp); where ρ represents the Euclidean distance between two center points. b represents the center point of the prediction box; b gt represents the center point of the actual box; C represents the diagonal distance of the minimum enclosing area that can contain both the predicted box and the actual box, h gt represents the height of the actual box; h represents the height of the predicted box; w g t represents the width of the actual box; w represents the width of the predicted box; C w Indicates the width of the minimum bounding box covering the predicted box and the actual box; C h Indicates the height of the minimum bounding box covering the predicted box and the actual box.

[0055] Furthermore, the IOU function in the Soft-NMS non-maximum suppression module is replaced with the EIOU loss function. The EIOU-Soft-NMS module shows excellent performance in fall detection in crowded situations, and can accurately identify falling people even in dense crowd occlusion.

[0056] Furthermore, the specific method of using the data set to train the generated fall detection model is to set the number of training times to 200 times, input the pictures in the data set into the FIDC-YOLOV5 network for training, and obtain the trained FIDC-YOLOV5 model, that is, the final fall detection model.

[0057] Furthermore, the fall detection model is tested and evaluated, and the specific method steps include:

[0058] First, the fall detection model is evaluated, including:

[0059] Ablation experiment, used to verify the rationality of module selection;

[0060] F1 curve evaluation and PR curve evaluation;

[0061] The specific method of the ablation experiment is to construct models with different module combinations, such as a model with only pooling structure optimization; train the constructed models with the same data set; compare the performance parameters after training, including precision (P), recall (R), and mean average precision (mAP); and conclude that the combination containing the SPPFCSPC pooling structure, the CBAM attention module, and the EIOU-Soft-NMS module has the best performance (see Table 1).

[0062] The F1 curve and PR curve of the FIDC-YOLOV5 model are evaluated. The evaluation results show that the F1 curve of the FIDC-YOLOV5 model is very spacious and the top is close to 1, which means that the model can perform well in precision and completeness and can perform well in a wide confidence interval; the PR curve of the FIDC-YOLOV5 model can perform well in completeness while maintaining accuracy, and the area of ​​the PR curve is close to 1, which reflects the good performance of the model.

[0063] The dataset constructed by the present invention is used to train several other mainstream target detection models, and the parameters of the several models obtained after training are compared and evaluated with the FIDC-YOLOV5 model proposed by the present invention; the parameters include precision P, recall rate R, F1 score and mean average precision mAP; in order to evaluate the detection effect more intuitively, the detection effect diagrams of different models are compared and evaluated;

[0064] The data set uses images of people falling down in a crowded environment.

[0065] Beneficial effects:

[0066] The present invention fills the current research gap in fall detection models in the presence of dense crowds. Taking into account the problem of occlusion in densely populated situations, the YOLOV5 model is improved, and a fall detection model FIDC-YOLOV5 in dense crowds is proposed. In real public places, the threat of trampling caused by falls when there are dense crowds is ubiquitous. The present invention provides a model for detecting falls of people in dense crowds. Compared with other fall detection models, this model can provide more accurate and comprehensive detection of falls of people in a crowd, help improve the safety of public places, and assist staff in handling fall incidents faster. Compared with traditional fall detection methods, fall detection based on computer vision is a better choice, providing a low-cost, high-precision solution to the problem of fall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The present invention will be further described below in conjunction with the accompanying drawings. The accompanying drawings are only used to illustrate preferred embodiments and are not to be considered as limiting the present application.

[0068] Figure 1 This is a network structure diagram of the FIDC-YOLOV5 fall detection model in Example 1 of the present invention;

[0069] Figure 2 It is a structural diagram of the SPPFCSPC pooling module in Example 2 of the present invention;

[0070] Figure 3It is the CBAMC3 module structure diagram in Example 2 of the present invention (i.e., the C3 structure diagram with the CBAM attention mechanism module added);

[0071] Figure 4 This is a structural diagram of the CAM channel attention module in Example 3 of the present invention;

[0072] Figure 5 This is a structural diagram of the SAM spatial attention module in Example 4 of the present invention;

[0073] Figure 6 This is a structural diagram of the CBAM attention mechanism module in Example 5 of the present invention;

[0074] Figure 7 It is an F1 curve graph of the FIDC-YOLOV5 fall detection model in Example 8 of the present invention;

[0075] Figure 8 It is a PR curve diagram of the FIDC-YOLOV5 fall detection model in Example 8 of the present invention;

[0076] Fig. 9 This is a detection effect diagram of the FIDC-YOLOV5 model in Example 8 of the present invention;

[0077] Fig.10 This is the detection effect diagram using the YOLOV8 model;

[0078] Fig.11 This is the detection effect diagram using the YOLOV7 model;

[0079] Fig.12 This is the detection effect diagram using the YOLOV6 model;

[0080] Fig.13 This is the detection effect diagram using the YOLOV5 model;

[0081] Fig.14 The present invention is a flow chart of a method for establishing a human fall detection model in a densely populated environment. DETAILED DESCRIPTION

[0082] The following will be combined with the attached Figure 1 to Figure 14 The embodiments of the technical solution of the present application are described in detail. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the protection scope of the present application.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the term "including" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0084] Refer to the attached Fig.14 A method for establishing a fall detection model for a dense crowd environment includes the following steps:

[0085] The method for establishing a fall detection model in a dense crowd environment includes the following steps:

[0086] S1. Build a data set and obtain pictures of people falling in crowded places as the data set;

[0087] S2. Based on the YOLOv5 model, the SPPFCSPC module, CBAM attention mechanism and EIOU-Soft-NMS module are introduced to obtain the FIDC-YOLOV5 model, i.e., the fall detection model;

[0088] S3, using the data set obtained in step S1 to train the fall detection model;

[0089] S4. Test and evaluate the fall detection model.

[0090] Embodiment 1:

[0091] The step S1 constructs a data set and obtains pictures of people falling in crowded situations as a data set; the data set includes a training set and a verification set, the training set is used to train the model, and the verification set is used to verify the reliability of the model; the training set and the verification set are used in each training, and the evaluation data obtained from the verification set will be used for the next training.

[0092] In this embodiment, the dataset consists of 2286 labeled images, with a total of 16116 person annotations; including 1885 training set images and 401 validation set images; 1464 photos of people falling with crowd occlusion are selected from 7379 photos in the data-fall-down dataset, and 422 photos of dense crowds selected from the WiderPerson dataset form the training set.

[0093] Embodiment 2:

[0094] The network structure of the FIDC-YOLOV5 model in this embodiment is as follows Figure 1 shown.

[0095] In step S2, the SPPFCSPC pooling structure is used to replace the original SPPF pooling structure, which improves the feature extraction performance of the model and reduces the number of model parameters. Figure 2 At the same time, the CBAM attention mechanism is integrated into the C3 module to obtain the CBAMC3 module, as shown in Figure 3 As shown in the figure, the perception ability of the model is improved by channel attention and spatial attention, so that different features of people falling and standing can be effectively extracted without increasing the complexity of the network. In order to improve the detection accuracy in dense crowds, the original NMS module is replaced by the EIOU-Soft-NMS module, which realizes the detection of people falling in the presence of occlusion in the crowd.

[0096] The SPPFCSPC pooling structure combines the grouped convolution block with the classic SPP method. It combines multiple grouped convolution blocks (including 1×1 and 3×3 convolutions) and three serial maximum pooling layers with the same receptive field. Grouped convolution divides the input feature map into several groups (for example, four groups) and performs independent convolution operations within each group. This reduces the computational load and the number of parameters by distributing the calculations across different groups.

[0097] The CBAM attention mechanism consists of two key parts, namely the channel attention module CAM (such as Figure 4 ) and the spatial attention module SAM (as shown Figure 5 These two modules can be embedded into different layers of a convolutional neural network (CNN) to enhance feature representation.

[0098] Embodiment 3:

[0099] The goal of the channel attention module CAM is to enhance the feature expression of each channel. In the channel attention module, for the input feature map F:

[0100] First, perform global maximum pooling and global average pooling operations on each channel, calculate the maximum eigenvalue and average eigenvalue on each channel, and generate two vectors containing channel numbers, which represent the global maximum feature and average feature of each channel respectively;

[0101] Then, the feature vector after global maximum pooling and average pooling is input into a shared fully connected layer; this fully connected layer is used to learn the attention weights of each channel. Through learning, the network can adaptively decide which channels are more important for the current task;

[0102] Again, intersect the global maximum eigenvector and the average eigenvector to obtain the final attention weight vector;

[0103] Subsequently, to ensure that the attention weights are between 0 and 1, a Sigmoid activation function is applied to generate channel attention weights Mc(F), as shown in equation (1). These weights will be applied to each channel of the original feature map.

[0104]

[0105] Finally, using the obtained attention weights, multiply them with each channel of the original feature map to obtain the attention-weighted feature map F', as shown in formula (2); this will emphasize channels that are helpful for the current task and suppress irrelevant channels.

[0106] F'=Mc(F)×F (2)

[0107] Embodiment 4:

[0108] The goal of the spatial attention module (SAM) is to emphasize the importance of different positions in the image. For the input feature map F', the input feature map F' is the product of the channel attention weight Mc(F) and the original feature map F:

[0109] First, the maximum pooling and average pooling operations are performed on the channel dimension to generate features of different context scales;

[0110] Secondly, the features after maximum pooling and average pooling are concatenated in the channel dimension to obtain a feature map containing multi-scale context information;

[0111] Again, the feature map is processed through a convolutional layer to generate spatial attention weights;

[0112] Subsequently, similar to the channel attention module, a Sigmoid activation function is applied to the generated spatial attention weights to constrain the weights between 0 and 1, as shown in Equation (3);

[0113]

[0114] Finally, the obtained spatial attention weight Ms(F') is applied to the input feature map F' to weight the features of each spatial position and obtain the attention-weighted feature map F', as shown in equation (4); thereby highlighting important image areas and reducing the influence of unimportant areas.

[0115] F”=Ms(F')F' (4)

[0116] Embodiment 5:

[0117] The CBAM attention mechanism of this embodiment is as follows Figure 6As shown. The attention module CBAM multiplies the output features of the channel attention module and the spatial attention module element by element to obtain the final attention-enhanced feature. This enhanced feature will be used as the input of the subsequent network layer to suppress noise and irrelevant information while retaining key information. In the CBAM module, the channel dimension is integrated first, and then the spatial dimension is integrated, and the model effect is better. CBAM connects the channel attention module and the spatial attention module in series, and performs two enhancements on the feature map F to obtain the final output feature map F".

[0118] The whole process is as follows

[0119] F'=Mc(F)F

[0120] F”=Ms(F')F' (5)

[0121] Embodiment 6:

[0122] In step S2, the EIOU-Soft-NMS module is introduced. Specifically, the IOU function in the Soft-NMS module is replaced with the EIOU loss function to construct the EIOU-Soft-NMS module, which effectively reduces missed detections and improves detection accuracy.

[0123] In target detection, the loss function is directly related to whether the detection effect is accurate, among which the IOU loss function is currently mainly used in the field of target detection. IOU stands for Intersection over Union, which is an evaluation indicator widely used in the field of target detection. Its main function is to measure the degree of overlap between the predicted bounding box and the actual bounding box, thereby evaluating the accuracy of the target detection model. The calculation of the IOU loss function is based on the ratio of the intersection area of ​​the predicted bounding box and the actual bounding box to the union area. Specifically, it calculates the ratio of the area of ​​the intersection of two bounding boxes to the area of ​​their union. IOU is expressed by the following formula (5).

[0124]

[0125] Based on the IOU function, researchers have proposed GIOU, DIOU and CIOU after multiple iterations. GIOU (Generalized Intersection over Union) is an extension of IOU, which takes into account the overlap and relative position between bounding boxes and can handle non-overlapping situations; it introduces an additional penalty term to better evaluate the relationship between the target box and the actual box. DIOU (Distance Intersection over Union) is based on GIOU. DIOU introduces the distance information between the center points and emphasizes the distance between the centers of the bounding boxes to better optimize the position and scale; CIOU (Complete Intersection over Union) is a further extension of DIOU, which takes into account the aspect ratio of the bounding box and the center point distance, and comprehensively considers the position, size and overlap to provide a more comprehensive evaluation.

[0126] Yolov5 uses the CIOU function by default. Although CIOU takes into account the overlapping area, center point distance and aspect ratio of bounding box regression, its formula only reflects the difference in aspect ratio, rather than the actual difference between the aspect ratio and its confidence, which sometimes hinders the effective optimization of the model. To address this problem, some scholars separated the aspect ratio based on CIOU and proposed EIOU. The EIOU function formula is as follows:

[0127]

[0128] The loss function consists of overlap loss, center distance loss, and width and height loss. ρ represents the Euclidean distance between two center points. C represents the diagonal distance of the minimum enclosing area that can contain both the predicted box and the actual box. b and b gt Represent the center points of the predicted box and the actual box respectively. gt and h represent the heights of the actual box and the predicted box respectively. gt and w, respectively represent the width of the actual box and the predicted box. w and C h It is the width and height of the minimum bounding box covering two boxes (predicted box and actual box). NMS (Non-Maximum Suppression) is a widely used technology in the field of computer vision, which aims to solve the problem of overlapping target boxes in target detection tasks based on deep learning methods.

[0129] In the classic NMS algorithm, IOU is the only factor considered. However, these traditional NMS methods may have problems when dealing with dense objects: if an object appears in the overlapping area of ​​another object, that is, when two target boxes are close, the box with a lower score will be deleted because of the large overlapping area. The NMS formula is as follows.

[0130]

[0131] In the formula, S i represents the score of each bounding box, where M specifically refers to the bounding box with the highest current score, and B i It represents one of the remaining bounding boxes. A threshold N is set t , when the intersection-over-union (IOU) of a bounding box with the highest-scoring bounding box is greater than this threshold, the score of the bounding box will be directly set to 0. However, this processing method may cause the target bounding box that should have been detected to be deleted, thereby causing the problem of missed detection.

[0132] Soft-NMS handles overlapping detection boxes by introducing a score decay function. When the IOU of a detection box and the highest scoring box exceeds a certain threshold, instead of setting its score to zero directly, its score is reduced according to its IOU value. This can retain more useful detection boxes while reducing redundancy. The formula is as follows:

[0133]

[0134] In the formula, set D refers to the final prediction box set after Soft-NMS suppression, and the σ coefficient is generally taken as 0.3.

[0135] When detecting dense crowds, missed detection caused by occlusion is a serious problem. For this reason, the present invention replaces the IOU function in Soft-NMS with the EIOU function and constructs an EIOU-Soft-NMS module, which effectively reduces missed detections and improves detection accuracy.

[0136] Embodiment 7:

[0137] In step S3, the data set is trained using the FIDC-YOLOV5 model. In this embodiment, all experiments are conducted using the PyTorch deep learning framework on a win11 system. The python version is 3.7, the cuda version is 11.3, and the torch version is 1.12. The computer is equipped with an AMD Ryzen 77735H with Radeon Graphics@3.20GHz CPU and an RTX 4060 (8GB) GPU. The SGD optimizer is used during training, and the training hyperparameters are set as follows: learning rate, momentum, weight decay, number of trainings, and batch size are 0.01, 0.937, 0.005, 200, and 8, respectively.

[0138] Embodiment 8:

[0139] The step S4 is to evaluate the trained model:

[0140] Precision (P), recall (R), F1 score, and mean average precision (mAP) are important indicators for evaluating network performance. Precision P (precision) indicates the proportion of objects correctly predicted by the model among all detected objects, while recall R (Recall) can also be considered as the recall rate, which indicates the number of predicted pairs in all positive classes. The F1 score is the harmonic mean of precision and recall. The mean average precision (mAP) is the average of the AP values ​​of different categories, reflecting the overall performance of the model. These evaluation indicators can be calculated using the corresponding equations. Precision (P), recall (R), F1 score, and mean average precision (mAP) are calculated using the following formulas:

[0141]

[0142] TP (Truth Positive): predicted positive class, actual positive class, that is, the number of correct predictions;

[0143] FP (False Positive): predicted positive class, actual negative class, that is, the number of errors in predicting the positive class;

[0144] FN (False Negative): predicted negative class, actual positive class, that is, the number of errors in predicting negative class;

[0145] P (Precision) can also be considered as the precision rate, that is, the number of correct positive classes among all predicted positive classes.

[0146]

[0147] Generally speaking, P is negatively correlated with R. The F1 score takes into account both the precision and recall of the classification model.

[0148]

[0149] The APs are:

[0150]

[0151] Among the commonly used mAP indicators, mAP@[0.5] and mAP@[0.5:0.95] are the most common. mAP@0.5 represents the mAP calculated when the IOU threshold is 0.5. mAP@0.5:0.95 represents the average mAP calculated under multiple IOU thresholds (from 0.5 to 0.95, with a step size of 0.05). This evaluation method evaluates the performance of the model under multiple different IOU thresholds, providing a more comprehensive evaluation.

[0152] In this embodiment, an improved FIDC-YOLOV5 model is used to conduct experiments in a data set; the F1 curve ( Figure 7 As shown) and PR curve ( Figure 8 The F1 curve uses confidence (indicating the degree of certainty of the model about the detected target) as the horizontal axis and the F1 score as the vertical axis. Figure 7 As shown, we can see that the F1 curve of the model is very wide and the top is close to 1, which means that the model can perform well in precision and recall, and can perform well in a wider confidence interval.

[0153] The PR curve reflects the relationship between precision and recall, with recall as the horizontal axis and accuracy as the vertical axis. Figure 8 As shown in the figure, the curve of the model can also do a good job of recall while maintaining accuracy, and the curve area is close to 1, which reflects the good performance of the model.

[0154] In order to further verify its effectiveness, an ablation experiment is also set up to explore the impact of each module of FIDC-YOLOV5 on the model performance. See Table 1 for details.

[0155] Table 1 Results of ablation experiments

[0156]

[0157] As shown in Table 1, after replacing the original SPPF pooling structure with the SPPFSFPC pooling structure, mAP@[0.5:0.95] increased by 3.5%, which means that the SPPFSFPC pooling structure achieves better feature fusion and improves the accuracy of the network. After adding the CBAM attention mechanism, mAP@[0.5] and mAP@[0.5:0.95] increased by 0.2 and 1.8% respectively. This shows that the model can better extract the features of people falling after adding the attention mechanism. After replacing the original NMS module with EIOU-Soft-NMS, mAP@[0.5] and mAP@[0.5:0.95] increased by 0.6 and 1.6% respectively. This means that the EIOU-Soft-NMS module we designed can better detect the occurrence of people falling in the face of dense crowds, and has achieved excellent accuracy improvement. It is difficult for the combination of other modules to achieve the best effect, which proves the rationality of our module selection.

[0158] We compared FIDC-YOLOV5 with several popular target detection models, including YOLOv5s, YOLOv6, YOLOv7-tiny, and YOLOv8n. The experimental results are shown in Table 2. According to Table 2, FIDC-YOLOV5 achieved the highest F1 score and mAP@[0.5] average precision, especially 85.5% average precision on mAP@[0.5:0.95]. Compared with YOLOv6, F1, mAP@[0.5] and mAP@[0.5:0.95] increased by 0.14, 12.7% and 33.7% respectively. Compared with YOLOv7-tiny, F1, mAP@[0.5] and mAP@[0.5:0.95] increased by 0.02, 2.6% and 17% respectively. Compared with YOLOv8n, F1, mAP@[0.5] and mAP@[0.5:0.95] are improved by 0.02%, 1.2% and 5.9% respectively. In addition, compared with the base model YOLOv5s, F1, mAP@[0.5] and mAP@[0.5:0.95] are improved by 0.01%, 0.8% and 6.9% respectively. This shows that our model achieves excellent detection performance.

[0159] Table 2 Comparison of experimental results of different target detection models

[0160]

[0161] In order to more intuitively compare the actual applications of different target detection models, several detection effect pictures on the validation set are selected for comparison. Fig. 9 It is not difficult to see that the FIDC-YOLOV5 model can perform well in precision and completeness and has a very high confidence level. Fig.10The YOLOV8 model had missed detections and false detections, identifying the bag on the football field as a fallen person, and failing to correctly identify many fallen people. In addition, the confidence level in identifying the fallen people was low. Fig.11 YOLOV7 identified a half-squatting staff member as a fallen person, resulting in a false detection. Fig.12 YOLOV6 has the problem of identifying both the staff and the fallen person as the fallen person, and the confidence level is also low. Fig.13 The YOLOV5 model missed a person who fell on the football field, and the recognition confidence in crowded places was lower than that of the FIDC-YOLOV5 model.

[0162] Therefore, the advantages of the present invention are: a wide range of application environments: it can still accurately identify in dense crowds and is not affected by environmental complexity. High accuracy: through advanced algorithm models, it can effectively distinguish between normal activities and falls, reducing false alarm rates. Scalability: it can be integrated with other security systems to achieve more comprehensive security management.

Claims

1. A method for establishing a fall detection model in a dense crowd environment, characterized in that: The steps include: S1. Build a data set and obtain pictures of people falling in crowded places as the data set; S2. Based on the YOLOv5 model, the SPPFCSPC module, CBAM attention mechanism and EIOU-Soft-NMS module are introduced to obtain the FIDC-YOLOV5 model, i.e., the fall detection model; Among them, the SPPFCSPC module is used to improve the feature extraction performance of the model and reduce the number of model parameters; Among them, the CBAM attention mechanism is used to improve the perception ability of the model; Among them, the EIOU-Soft-NMS module is used to detect people falling in the presence of occlusion in the crowd; S3, using the data set obtained in step S1 to train the fall detection model; S4. Test and evaluate the fall detection model.

2. The method for establishing a fall detection model in a dense crowd environment according to claim 1, characterized in that: In step S2, the SPPFCSPC pooling structure is used to replace the original SPPF pooling structure; the SPPFCSPC pooling structure combines the grouped convolution block with the classic SPP method, and has several grouped convolution blocks and three serial maximum pooling layers with the same receptive field; the input feature maps are grouped by grouped convolution, and independent convolution operations are performed in each group; by distributing the calculations in different groups, the computational load and the number of parameters are reduced.

3. The method for establishing a fall detection model in a dense crowd environment according to claim 2, characterized in that: The grouped convolution blocks have 6, including 1×1 and 3×3 convolutions.

4. The method for establishing a fall detection model in a dense crowd environment according to claim 1, characterized in that: In step S2, the CBAM attention mechanism is integrated into the C3 module to obtain the CBAMC3 module, which improves the perception ability of the model through channel attention and spatial attention, so as to effectively extract different features of people falling and standing without increasing the complexity of the network; The CBAM attention mechanism consists of two key parts, namely the channel attention module CAM and the spatial attention module SAM, which are respectively embedded in different layers of the convolutional neural network CNN to enhance feature representation; The channel attention module CAM is used to enhance the feature expression of each channel; The spatial attention module SAM is used to emphasize the importance of different positions in the image.

5. The method for establishing a fall detection model in a dense crowd environment according to claim 4, characterized in that: The method steps of implementing feature expression by the channel attention module CAM are as follows: For the input feature map F: First, perform global maximum pooling MaxPool and global average pooling AvgPool operations on each channel to calculate the maximum eigenvalue on each channel and the average eigenvalue Generate two feature vectors containing the number of channels, which represent the global maximum feature and average feature of each channel respectively; Secondly, the feature vectors after global maximum pooling and average pooling are input into a shared fully connected layer to learn the attention weight of each channel. The shared fully connected layer includes a multi-layer perceptron MLP. Through learning, the network can adaptively determine the channels that are more important for the current task. Again, intersect the global maximum eigenvector and the average eigenvector to obtain the final attention weight vector; Subsequently, in order to ensure that the attention weight is between 0 and 1, the final attention weight vector is activated by the sigmoid function to generate the channel attention weight Mc(F). The generation formula of the channel attention weight is formula (1); Where σ represents the Sigmoid activation function, W0∈R C / r×C , W1∈R C×C / r , the weights W0 and W1 of MLP are shared; r is reduction ratio, which means the reduction ratio; C is Channels, which means the number of channels; R represents the batch size, which is the number of images; Finally, the above channel attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted feature map F', which is calculated as formula (2), thereby emphasizing the channels that are helpful to the current task and suppressing irrelevant channels; F' = Mc(F) x F (2).

6. The method for establishing a fall detection model in a dense crowd environment according to claim 5, characterized in that: The spatial attention module SAM emphasizes the importance of different positions in the image, and its specific implementation steps are as follows: For the input feature map F', the input feature map F' is the product of the channel attention weight Mc(F) and the original feature map F; First, the maximum pooling and average pooling operations are performed on the channel dimension respectively, and the channel information of the feature map is aggregated by using these two pooling operations to generate two 2D maps, namely and To generate features of different context scales; Secondly, the features after maximum pooling and average pooling are concatenated in the channel dimension to obtain a feature map containing multi-scale context information; Thirdly, the feature map containing multi-scale context information is processed through a convolutional layer to generate spatial attention weights; Subsequently, the Sigmoid activation function is applied to the generated spatial attention weights to limit the weights between 0 and 1, and the calculation formula is (3); Among them, σ represents the sigmoid function; f 7×7 represents a convolution operation with a filter size of 7×7; MaxPool() represents a global maximum pooling operation; and AvgPool() represents a global average pooling operation; Finally, the obtained spatial attention weight Ms(F') is applied to the input feature map, and the features of each spatial position are weighted to obtain the attention-weighted feature map F'', which is calculated as formula (4); thereby highlighting the important image areas and reducing the influence of unimportant areas; F" = Ms(F')F' (4).

7. The method for establishing a fall detection model in a dense crowd environment according to claim 5, characterized in that: The CBAM attention mechanism multiplies the output features of the channel attention module and the spatial attention module element by element to obtain the final attention-enhanced features; the attention-enhanced features are used as input to subsequent network layers to suppress noise and irrelevant information while retaining key information; The CBAM attention mechanism first integrates the channel dimension and then the spatial dimension in the CBAM module. Specifically, by connecting the channel attention module and the spatial attention module in series, the feature map F is enhanced twice. The processing formula (5) is: F'=Mc(F)F F" = Ms(F')F' (5).

8. The method for establishing a fall detection model in a dense crowd environment according to claim 7, characterized in that: In step S2, the EIOU-Soft-NMS module is introduced. Specifically, the IOU function in the Soft-NMS module is replaced with the EIOU loss function to construct the EIOU-Soft-NMS module, which effectively reduces missed detections and improves detection accuracy. The EIOU-Soft-NMS module includes two key parts, namely the EIOU loss function and the Soft-NMS non-maximum suppression module; The Soft-NMS non-maximum suppression module is used to suppress redundant boxes during detection. It introduces a fractional attenuation function to handle overlapping detection boxes. The specific formula is as follows: Among them, S i represents the score of each bounding box, M represents the bounding box with the highest score at the moment, and B i Indicates one of the remaining bounding boxes. When detecting an object, multiple prediction boxes will be generated first, and the most suitable box will be obtained through screening. In the process of screening using the Soft-NMS non-maximum suppression module, in addition to the highest scoring bounding box, there is one remaining bounding box. D represents the final set of prediction boxes after being suppressed by the Soft-NMS non-maximum suppression module. σ is a coefficient, which can be selected to be 0.

3. The EIOU loss function is a loss function widely used in target detection tasks, which is mainly used to evaluate the degree of overlap between target detection results and actual annotation boxes, and is used as an optimization target in the model training process to improve the performance of the model. The formula of the EIOU loss function is as follows: L EIOU =L IOU +L dis +L asp The loss function is composed of overlap loss IOU, center distance loss dis and width and height loss asp; where ρ represents the Euclidean distance between two center points; b represents the center point of the prediction box; b gt represents the center point of the actual box; C represents the diagonal distance of the minimum enclosing area that can contain both the predicted box and the actual box, h gt represents the height of the actual box; h represents the height of the predicted box; w gt represents the width of the actual box; w represents the width of the predicted box; C w Indicates the width of the minimum bounding box covering the predicted box and the actual box; C h Indicates the height of the minimum bounding box covering the predicted box and the actual box.

9. The method for establishing a fall detection model in a dense crowd environment according to claim 1, characterized in that: The specific method of using the data set to train the generated fall detection model is to set the number of training times to 200, input the pictures in the data set into the FIDC-YOLOV5 network for training, and obtain the trained FIDC-YOLOV5 model, that is, the final fall detection model.

10. The method for establishing a fall detection model in a dense crowd environment according to claim 1, characterized in that: The fall detection model is tested and evaluated. The specific steps include: First, the fall detection model is evaluated, including: Ablation experiment, used to verify the rationality of module selection; F1 curve evaluation and PR curve evaluation; The specific method of the ablation experiment is to construct a model column with different module combinations; train the constructed models with the same data set; compare the performance parameters after training, including precision P, recall rate R, and mean average precision mAP; and conclude that the combination containing SPPFCSPC pooling structure, CBAM attention module and EIOU-Soft-NMS module has the best performance; The F1 curve and PR curve of the FIDC-YOLOV5 model are evaluated. The evaluation results show that the F1 curve of the FIDC-YOLOV5 model is very spacious and the top is close to 1, which means that the model can perform well in precision and completeness and can perform well in a wide confidence interval; the PR curve of the FIDC-YOLOV5 model can perform well in completeness while maintaining accuracy, and the area of ​​the PR curve is close to 1, which reflects the good performance of the model.

11. The model for detecting a person falling in a dense crowd environment according to any one of claims 1 to 10 is applied to detecting a person falling in a dense crowd environment, characterized in that: The data set is input into the fall detection model for training, and several other mainstream target detection models are trained at the same time. The parameters of the several models obtained after training are compared and evaluated with the FIDC-YOLOV5 fall detection model; the parameters include precision P, recall rate R, F1 score and mean average precision mAP; in order to evaluate the detection effect more intuitively, the detection effect diagrams of different models are compared and evaluated; The dataset uses images of people falling in a crowded environment; the other mainstream target detection models include YOLOv5s, YOLOv6, YOLOv7-tiny and YOLOv8n.

Images