A Campus Abnormal Behavior Detection Method Based on Improved PBAS Algorithm and YOLOv5

Abstract

A campus abnormal behavior detection method based on an improved PBAS algorithm and YOLOv5 includes the following steps: 1) Select the Yolov5 network for model training: Collect a large number of data samples, calibrate the samples, compare the models under different parameter settings, and select the final training parameters and model to prepare for subsequent abnormal behavior detection; 2) Improve the PBAS algorithm to extract the dynamic foreground: Use the improved PBAS algorithm to complete the extraction of the dynamic foreground in the video motion area, capture effective dynamic behaviors, filter out static and dynamic backgrounds, so as to shield the interference of environmental factors such as light and dynamic background on target detection; 3) The Yolov5 model detects abnormal behaviors: Use the processed video frames as the input of the YOLOv5 model for target detection, so as to determine whether there are dangerous behaviors among students. The detection results of the present invention are accurate, and the missed detection and false detection rates are relatively small, and it can well complete the detection of campus abnormal behaviors.

Description

Technical Field

[0001] The present invention relates to a method for detecting abnormal behaviors on campus. Background Art

[0002] In recent years, campus security has received unprecedented attention. There is an urgent need to use an intelligent video surveillance system in computer vision technology to detect abnormal behaviors on campus and alarm for abnormal behaviors. Detecting and screening key abnormal behaviors on campus can reduce the work tasks of monitoring personnel and has a positive effect and practical significance for the management of campus security work. For abnormal behavior detection, it can be regarded as consisting of two steps: the first step is foreground extraction, that is, abstracting a series of abnormal behavior characteristics in the scene and extracting valuable behavior characteristics among them. The second step is object detection, establishing a model for detecting abnormal behaviors. However, due to the complexity of the scene and the diversity of abnormal behaviors, how to efficiently extract descriptive and discriminative features of abnormal behaviors and express them effectively has become a difficulty and focus.

[0003] In the foreground extraction work, the present invention adopts the PBAS algorithm. The PBAS algorithm is an adaptive background segmentation algorithm based on a pixel model (Pixel-Based Adaptive Segmenter, PBAS) proposed by Hofmann et al. in 2012. The background model of this algorithm is constructed by collecting background samples, and the thresholds and update rates in the model are adaptively optimized, reducing the false detection rate generated by the background. The PBAS algorithm combines the advantages of the SACON algorithm and the VIBE algorithm and is improved on this basis. The main feature is to introduce the idea of cybernetics and a method for measuring the background complexity, so that the foreground judgment threshold and the background model update rate change adaptively with the change of the background complexity and are updated in a timely manner. Therefore, the PBAS algorithm has a good processing effect on real-time surveillance videos and has an adaptive background sample set update strategy, which can be better used in actual situations than the fixed update sample set strategy of the VIBE algorithm.

[0004] For the target detection task, the present invention adopts the YOLOv5 neural network. The YOLOv5 algorithm consists of three parts. The first part is the input end, which is composed of the part where the input size of the training picture is 608. The second part is the backbone network, which uses the CSPDarknet53 network to extract rich information features from the input image. The third part is the detection layer, which uses multiple scales for detection. After the feature pyramid network structure, a new path aggregation network structure with a bottom-up path is added to achieve the fusion of feature information at different scales. Then, predictions are made on the three generated feature maps. In addition, YOLOv5 continues to use the multi-scale detection structure of YOLOv4. After the backbone network extracts features, two upsamplings and three convolutions are performed to predict the categories and positions of large, medium, and small targets at different scales. YOLOv5 uses adaptive anchor box calculation to adaptively calculate the optimal anchor box values in the training set for different datasets. YOLOv5 is a detection algorithm with high precision and fast speed, and has good results on open-source datasets. However, due to the low clarity of the monitoring cameras used in some schools, its detection performance still needs to be improved in the task of abnormal behavior recognition.

[0005] In summary, the existing foreground extraction and target detection technologies have achieved rapid development. However, considering target recognition in a specific scenario such as a school, the results are often uncertain. Therefore, it is particularly important to select appropriate algorithms, models, or specific recognition strategies for different scenarios to improve the recognition efficiency. Since this method mainly focuses on abnormal behaviors in the campus, its content mainly includes common dangers or actions and behaviors that do not conform to campus norms occurring in the campus. Therefore, this method mainly selects three main abnormal behaviors for detection, namely students falling, fighting, and climbing. Summary of the Invention

[0006] The present invention aims to overcome the problems of low accuracy and large environmental influence of the existing campus abnormal detection technology, and at the same time take into account the real-time requirement, and provides a campus abnormal behavior detection method based on an improved PBAS algorithm and YOLOv5.

[0007] The present invention extracts the dynamic foreground of the image based on the improved PBAS algorithm, then uses the trained network model for target detection, and finally determines the abnormal behavior in the image, providing a reliable abnormal behavior detection method for the security system.

[0008] A campus abnormal behavior detection method based on an improved PBAS algorithm and YOLOv5 of the present invention includes the following steps:

[0009] 1) Select the Yolov5 network for model training: First, collect a large number of data samples according to three types of abnormal behaviors to make a dataset. Preprocess the dataset images, then calibrate the abnormal behavior areas of the dataset images. Finally, train the model and adjust the parameters repeatedly for many times. Set the network parameters by combining the synchronous training results and the network structure itself to finally obtain an abnormal behavior detection model;

[0010] 2) Improve the PBAS algorithm to extract the dynamic foreground: In the part of dynamic foreground extraction, by improving the way of updating the adaptive decision threshold of the PBAS algorithm, effectively extract the dynamic area of the video frame, so as to avoid interference from light and dynamic background during foreground extraction;

[0011] 3) Use the Yolov5 model to detect abnormal behaviors: After extracting the dynamic foreground from the video frame through the improved PBAS algorithm, use it as the input of the YOLOv5 model for abnormal behavior detection.

[0012] The technical concept of the present invention is: This technical route mainly aims at detecting abnormal behaviors in specific campus scenarios, which is mainly divided into the following 3 steps: 1. Collect a large number of data samples, calibrate the samples, compare the models under different parameter settings, and select the final training parameters and models to prepare for subsequent abnormal behavior detection. 2. Use the video frame as the input, use the improved PBAS algorithm to frame the dynamic foreground in the image, and filter out the static background to reduce the false detection of the static background. 3. Input the obtained dynamic foreground into the trained Yolov5 network model for abnormal behavior detection and finally frame the abnormal area.

[0013] The beneficial effects of the present invention are mainly manifested in: 1. The detection results are accurate, and the omission and false detection rates are relatively small; 2. It can well complete the detection of campus abnormal behaviors and greatly save labor costs. Description of the Drawings

[0014] Figure 1 is the flowchart of the present invention.

[0015] Figure 2 is the average loss function graph of the present invention.

[0016] Figure 3 is the flowchart of the PBAS algorithm of the present invention.

[0017] Figures 4(a) to 4(c) is the comparison graph of the effects before and after the improvement of the PBAS algorithm of the present invention. Among them, Fig. 4(a) is the original image, Fig. 4(b) is the effect diagram before improvement, and Fig. 4(c) is the effect diagram after improvement.

[0018] Figure 5 is the detection effect graph of campus abnormal behaviors of the present invention. Detailed Embodiments

[0019] The present invention will be further described below with reference to the accompanying drawings.

[0020] A campus abnormal behavior detection method based on an improved PBAS algorithm and YOLOv5 includes the following steps:

[0021] 1) Select the Yolov5 network for model training: Collect a large number of data samples, calibrate the samples, compare the models under different parameter settings, and select the final training parameters and model to prepare for subsequent abnormal behavior detection;

[0022] 2) Improve the PBAS algorithm to extract the dynamic foreground: Use the improved PBAS algorithm to complete the extraction of the dynamic foreground in the video motion area, capture effective dynamic behaviors, filter out static and dynamic backgrounds, so as to shield the interference of environmental factors such as light and dynamic background on target detection;

[0023] 3) The Yolov5 model detects abnormal behaviors: Use the processed video frames as the input of the YOLOv5 model for target detection, so as to determine whether students have dangerous behaviors.

[0024] Furthermore, in the step 1), select the Yolov5 network for model training: The specific steps are as follows,

[0025] 1.1) Dataset production: First, extract video frames from the three types of abnormal videos collected; then preprocess the dataset composed of video frames and pictures, strictly screen the pictures, filter out noise, enhance the images, etc.; finally, use the LabelImg plotting tool to frame the abnormal behavior areas in the preprocessed pictures and generate a txt file for model training.

[0026] 1.2) Model training: After repeatedly training the model and combining the training results with the structure of the YOLOv5 network itself, the relevant parameters are finally set as follows in this paper: the initial learning rate is 0.01, the learning rate decay weight is 0.0005, and the number of training iterations is 200 times. The loss function of YOLOv5 uses GIOULoss as the bounding box, and the value inferred by Box is the mean value of the GIoU loss function. The loss function is as Figure 2 shown, and it can be seen that the smaller the selected box is, the more accurate the detection result is.

[0027] In the step 2), improve the PBAS algorithm to extract the dynamic foreground: Screen the areas suspected of students' abnormal behaviors and combine Figure 3, first, use the improved PBAS algorithm to accurately extract the dynamic foreground, and then use the framed dynamic region as the input of step 3) for object detection to determine whether the student has dangerous behaviors. The specific steps are as follows:

[0028] 2.1) Input the video frame, and compare it with the current pixel through the background model of the image to determine the classification of the foreground and the background. The background model consists of N historical pixel values observed near the current video frame:

[0029] B(x i ) = B 1 (x i ),..., B k (x i );..., B N (x i ) (1)

[0030] Meanwhile, the PBAS algorithm determines whether the current pixel belongs to the background or the foreground by comparing the current frame I(x i ) with the background model B(x i ). Specifically, it compares the first N historical pixel values of a pixel point in the background model with the current pixel value. If the distance between the current value and at least #min of the historical values is less than the decision threshold R(x i ), then this point is determined as a foreground point; otherwise, it is determined as a background point.

[0031] 2.2) Improve the adaptive decision threshold update. According to the PBAS algorithm and step 2.1), the calculation formula for the foreground segmentation mask can be determined as:

[0032]

[0033] where F = 0 and F = 1 represent that this pixel point is a background point and a foreground point respectively, and dist(I(x i ), Bk(x i )) represents the distance between the current point and the background model. To solve the problem of mis-extracting the dynamic background together when extracting the dynamic foreground, a non-linear adaptive decision threshold update method is adopted. Using the non-linear relationship between the decision threshold R(x i ) and the background complexity , the sensitivity to the recognition of small-area dynamic backgrounds is improved, thus solving the problem of mis-extracting the dynamic background. The specific improvement steps are as follows:

[0034] 2.2.1) First, when the algorithm determines whether the pixel points in the moving area are foreground points or background points through the target box, calculate the area of the target box as S target (x i ), and at the same time, introduce a balance function R ban (xi ) and set it to a fixed absolute value.

[0035] 2.2.2) Then make R(x i ) and be redefined from the original linear relationship to a non-linear relationship, that is

[0036]

[0037] where R ban (x i ) is a fixed absolute value. Since the area of the target box in the dynamic background is usually small, that is, when S target (x i ), is smaller, the pixel point x i is more likely to be a background point, and the ratio of R ban (x i ) to S target (x i ) is larger. Therefore, compared with the decision threshold R(x i ) value in the original PBAS algorithm, the improved decision threshold R(x i ) value changes more. It can be seen from Equation (5) that when making a foreground or background decision, if the R(x i ) value is larger, the number of times the value is less than the decision threshold R(x i ) is greater than #min, then at this time the pixel point is judged as a background point, that is, it can better and quickly and effectively suppress the area where the background change is more complex. And when S target (x i ), exceeds a certain value, at this time the pixel points in the moving area are more likely to be foreground points, and the ratio of R ban (x i ) to S target (x i ) also tends to 0, and the decision threshold R(x i ) and the background complexity are still in a linear relationship. Therefore, the fact that the target box area S target (x i ) is too large will not affect the judgment of the moving foreground. According to the above analysis, the dynamic update method of the decision threshold R(x i ) can be redefined as:

[0038]

[0039] where B(x i ) is a state variable and R ban (x i ) is a fixed absolute value.

[0040] 2.3) Update the background model. If the current pixel point xi If it is determined to be a background point, the pixel point is used to randomly and uniformly replace the pixel points in the background sample, B k (x i ), k ∈ 1, …, N. On this basis, the sample points in the current pixel point sample set will be replaced with a probability of 1 / T(x i ), and each pixel point corresponds to a probability value 1 / T(x i ).

[0041] 2.4) Determine the threshold and update the learning rate. The background complexity is introduced to enable the background decision threshold to be adaptively updated. If a pixel point has a large difference from the background model, then this pixel point will be detected as a foreground point, and this pixel point will also update the background model with a relatively small probability. Therefore, when the PBAS algorithm establishes the background model, it also establishes an array D(x i ) to record the minimum similarity distance value, and the background complexity is the average value of the minimum similarity distances.

[0042]

[0043] Among them, is the average value of N 0 minimum distance matrix values, that is, the background complexity. According to the above analysis, when the background changes greatly, the background complexity is also larger. At this time, a larger decision threshold R(x i ) and T(x i ) are required. Therefore, the PBAS algorithm adaptively adjusts the discrimination threshold as shown in the following formula:

[0044]

[0045] Among them, R inde and R scale are both preset fixed values. At the same time, in order to reduce the impact on the established background model when misjudging pixel points, it is necessary to reduce the frequency of background update. Therefore, the update strategy of the learning rate T(x i ) can be expressed as:

[0046]

[0047] Among them, T inc and T dec are preset fixed values.

[0048] Figure 4 is a comparison diagram before and after processing by the PBAS algorithm. By extracting the dynamic foreground in the video motion area, the dynamic foreground and the static background can be effectively separated, thereby greatly reducing the impact of the video background on the recognition of abnormal behaviors and improving the accuracy of abnormal behavior detection.

[0049] In step 3), the Yolov5 model detects abnormal behaviors: after the dynamic foreground is extracted from the video frame through the improved PBAS algorithm, it is used as the input of the YOLOv5 model for abnormal behavior detection. The Mosaic data augmentation method is used at the input end of YOLOv5. Four pictures are randomly used as training data after scaling, cropping and splicing to enrich the picture background. At the same time, adaptive anchor box calculation and picture scaling are adopted to reduce the computational amount and improve the target detection speed. In addition, YOLOv5 uses GIoU_Loss as the loss function of the Boundingbox at the output end, and filters the target boxes through non-maximum suppression (NMS), thus effectively solving the problem of non-overlapping borders.

[0050] Figure 5 The result after the detection by this method shows that through the adaptive decision threshold update of the improved algorithm, the interference caused by the dynamic background is effectively suppressed, and thus the dynamic foreground of the video frame image is accurately extracted. Finally, the accurate recognition of abnormal behaviors is realized.

[0051] The content described in the embodiments of this specification is only a list of the implementation forms of the inventive concept and is only for illustrative purposes. The protection scope of the present invention should not be regarded as limited to the specific forms stated in this embodiment. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those of ordinary skill in the art according to the inventive concept of the present invention.

Claims

1. A campus abnormal behavior detection method based on the improved PBAS algorithm and YOLOv5, including the following steps: 1) Use the Yolov5 network for model training: Collect a large number of data samples, calibrate the samples, and compare the models under different parameter settings to select the final training parameters and model; 2) Improve the PBAS algorithm to extract the dynamic foreground: Use the improved PBAS algorithm to complete the dynamic foreground extraction of the video motion area and capture effective dynamic behaviors; 3) The Yolov5 model detects abnormal behaviors: Use the processed video frames as the input of the YOLOv5 model for object detection to determine whether students have dangerous behaviors; The specific content of step 2) includes: 2.1) Input the video frame, and compare it with the current pixel through the background model of the image to determine the classification of the foreground and background; where the background model is composed of N historical pixel values observed in the vicinity of the current video frame: B(x i ) = B 1 (x i ),..., B k (x i ),..., B N (x i ) (1) At the same time, the PBAS algorithm determines whether the current pixel belongs to the background or the foreground by comparing the current frame I(xi) with the background model B(xi). Specifically, it compares the first N historical pixel values of a pixel point in the background model with the current pixel value. If the distance between the current value and at least #min of the historical values is less than the decision threshold R(xi), then this point is determined as a foreground point; otherwise, it is determined as a background point; 2.2) Improve the adaptive decision threshold update. According to the PBAS algorithm and step 2.1), the foreground segmentation mask calculation formula can be determined as: where F = 0 and F = 1 represent that the pixel is a background pixel and a foreground pixel respectively, and dist(I(x i ), Bk(x i )) represents the distance between the current point and the background model; the specific improvement steps are as follows: 2.2.1) First, when the algorithm determines whether the pixel points in the motion area are foreground points or background points through the target box, calculate the area of the target box as S target (x i ), and at the same time reference a balance function R ban (x i ) and set it to a fixed absolute value; 2.2.2) Then, R(x i ) and which was originally a linear relationship is redefined as a non-linear relationship; that is Among them, R is defined ban (x i ) as a fixed absolute value; since the area of the target box in the dynamic background is usually small, that is, when S target (x i ) is smaller, the smaller the pixel point x i , the more likely it is to be a background point, and the larger the ratio of R ban (x i ) to S target (x i ). Therefore, compared with the decision threshold R(x i ) value in the original PBAS algorithm, the improved decision threshold R(x i ) value changes more. As can be seen from Equation (5), when making a foreground or background decision, if the value of R(x i ) is larger, the number of times the value is less than the decision threshold R(x i ) is greater than #min, then the pixel point is judged as a background point at this time; and when S target (x i ) exceeds a certain value, at this time the pixel points in the moving area are more likely to be foreground points, and the ratio of R ban (x i ) to S target (x i ) also tends to 0, and the decision threshold R(x i ) and the background complexity are still linearly related. Therefore, the excessive area of the target box S target (x i ) will not affect the judgment of the moving foreground; based on the above analysis, the dynamic update method of the decision threshold R(x i ) can be redefined as: where B(x i ) is a state variable and R ban (x i ) is a fixed absolute value; 2.3) Update the background model. If the current pixel point x i is determined to be a background point, then randomly and uniformly replace the pixel point in the background sample with this pixel point, B k (x i ), k ∈ 1, …, N; On this basis, the sample points in the current pixel point sample set will be replaced with a probability of 1 / T(x i ), and each pixel point corresponds to a probability value of 1 / T(x i ); 2.4) Determine the threshold and update the learning rate. Introduce background complexity to enable adaptive update of the background decision threshold. If a pixel has a large difference from the background model, then this pixel will be detected as a foreground point, and this pixel will also update the background model with a relatively small probability; Therefore, when the PBAS algorithm builds the background model, it also builds an array D(x i ) to record the minimum similarity distance value, and the background complexity is the average value of the minimum similarity distances; Among them, is the average value of N 0 minimum distance matrix values, that is, the background complexity; the PBAS algorithm adaptively adjusts the discrimination threshold as shown in the following formula: Among them, R inde and R scale are both preset fixed values; the update strategy of the learning rate T(x i ) is expressed as: where T inc and T dec are preset fixed values.

2. For a campus abnormal behavior detection method based on the improved PBAS algorithm and YOLOv5 as described in claim 1, the specific content of step 1) includes: 1.1) Dataset production: Extract video frames from the three types of abnormal behavior videos collected; Then preprocess the dataset images, screen the images, filter and denoise, and enhance the images; finally, use the LabelImg plotting tool to frame the abnormal behavior areas in the preprocessed images and generate txt files for model training; 1.2) Model training: After repeatedly training the model and combining the training results with the structure of the YOLOv5 network itself, set the following relevant parameters: the initial learning rate is 0.01, the learning rate decay weight is 0.0005, and the training iteration times are 200 times; The loss function of YOLOv5 uses GIOU Loss as the bounding box, and the value inferred by Box is the mean value of the GIoU loss function.

Images