Anomaly behavior detection method based on background decoupling

Abstract

The application discloses an abnormal behavior detection method based on background decoupling, comprising the following steps: step 1, image data collection; step 2, image data preprocessing; step 3, feature extraction or feature construction is carried out on the preprocessed data; step 4, selecting a suitable deep learning model and training; step 5, model evaluation: using evaluation indexes to evaluate the trained model, according to the evaluation result, adjusting the model parameters or replacing the model to improve the accuracy of detection; step 6, deployment and monitoring: deploying the trained model to the actual application scene, and monitoring and detecting the input data in real time. The application solves the problems that the existing detection method is prone to have a bias type, and the detection precision fluctuates greatly.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target behavior detection, and particularly relates to an abnormal behavior detection method based on background decoupling. BACKGROUND

[0002] Abnormal behavior detection refers to detecting behaviors deviating from normal behavior patterns in a specific scene. In the field of target detection, abnormal behavior detection can be regarded as a high-level image understanding operation, which extracts logical information from input image sequences and performs behavior modeling. There are generally two modeling ideas: one is to first learn a model of normal behavior and detect abnormalities based on the model; the other is to automatically learn normal and abnormal models by observing statistical characteristics of batch or online data. For a specific scene, human abnormal behavior is complex and diverse, and it is difficult to define. A simple solution is to divide human behavior into two categories, ignore the transition behavior, and divide it into normal behavior and abnormal behavior. Because for a specific scene such as a laboratory, the demand for abnormal behavior recognition may only be a simple classification problem such as whether to wear a work uniform and a work cap, so dividing into two categories can well solve the problem of defining abnormal behavior. At present, with the continuous improvement of the social security system, public cameras have formed a huge monitoring network. However, due to the large number of pedestrian images involved in the related video monitoring recordings, the traditional manual monitoring method is very low in efficiency, and using computers to process monitoring videos for abnormal behavior detection has very important practical significance.

[0003] With the continuous development of deep learning in the field of computer vision, scholars began to use deep learning for abnormal behavior detection. Yi et al. fused global features and local features while using convolutional sparse coding to classify abnormal behaviors, achieving relatively good detection effect. Jian et al. proposed 3DCNN, which captures features along the spatial and temporal dimensions through 3D convolution, thereby obtaining information between adjacent frames. Lim et al. proposed using additional features of different layers as context and attention mechanisms to capture multi-scale information of pictures to increase detection accuracy.

[0004] The existing abnormal behavior detection model has the following shortcomings:

[0005] (1) Shortcomings of multi-sample feature fusion algorithm

[0006] Existing multi-sample feature fusion methods involve stacking two or more samples together in different proportions and then using a model to learn global information from several samples on a single sample. However, the stacking of global information easily produces redundant features, which are highly coupled with each other. This coupling can lead to biased learning by the model, making it unable to learn specific features for abnormal behavior detection. The bias focuses on the messy information from multiple samples, resulting in extremely poor model performance.

[0007] (2) Disadvantages of data augmentation algorithms

[0008] Most data augmentation algorithms (such as Gaussian blur, Gaussian noise, and affine transformation) have achieved excellent results in image classification. However, in the field of anomaly detection, most data augmentation algorithms fail to improve model performance and instead lead to varying degrees of accuracy decline. The reason for this is that directly augmenting the entire image is a coarse-grained approach. The model fails to learn the coupling features between the background and foreground, tending instead to learn features from the entire image. Consequently, in actual detection, the model is highly susceptible to scene interference, resulting in significant fluctuations in anomaly detection accuracy. Summary of the Invention

[0009] To achieve the above objectives, this invention provides an abnormal behavior detection method based on background decoupling, which solves the problems of existing methods being prone to bias and having large fluctuations in detection accuracy.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is an abnormal behavior detection method based on background decoupling, comprising the following steps:

[0011] Step 1: Image data collection;

[0012] Step 2: Image data preprocessing;

[0013] Step 3: Perform feature extraction or feature construction on the preprocessed data;

[0014] Step 4: Select an appropriate deep learning model and train it;

[0015] Step 5: Model Evaluation: Use evaluation metrics to evaluate the trained model. Based on the evaluation results, adjust the model parameters or replace the model to improve the detection accuracy.

[0016] Step 6, Deployment and Monitoring: Deploy the trained model to the actual application scenario and monitor and detect the input data in real time.

[0017] Furthermore, step 1 specifically involves collecting data that needs to detect abnormal behavior, including real-time data, historical data, and simulated data.

[0018] Furthermore, the preprocessing in step 2 specifically includes cleaning, deduplication, standardization, and missing value imputation of the collected data.

[0019] Furthermore, the deep learning model in step 4 includes convolutional neural networks, recurrent neural networks, and autoencoders.

[0020] Furthermore, before training the deep learning model in step 4, the image data needs to be decoupled: each image data put into the model first undergoes a randomization strategy: a random number between 0 and 1 is set. If the randomly generated number is greater than 0.5, background decoupling is not performed; if the randomly generated number is less than or equal to 0.5, the corresponding image data is divided into N×N equal parts and classified into target image blocks and background image blocks; each classified image block is randomly used with one of four decoupling methods, and the number of image blocks applied by each decoupling method is less than or equal to 1 / 4 of the total number of image blocks.

[0021] Furthermore, the decoupling method includes a fine-grained background enhancement method based on data augmentation, specifically: replacing background image block regions by adaptively selecting affine transformation, perspective transformation, and tone transformation.

[0022] Furthermore, the decoupling method includes a fine-grained background fusion method based on multi-sample feature fusion, specifically: background information from other images is modularized in a fine-grained manner using random selection; then, the background information from other images is fused with the background information of the current decoupled image using the Mixup data augmentation method based on the principle of minimizing neighborhood risk, generating a new background image patch. The background image patch generation method is as follows: (x n ,y n )=λ(x i ,y i )+(1-λ)(x j ,y j ), where the ratio of image transparency λ ranges from 0 to 1, (x n ,y n ) is the new background image patch generated by interpolation, (x i ,y i ) and (x j ,y j These are the currently decoupled image and other randomly selected images, respectively.

[0023] Further, the decoupling method comprises a fine-grained background information weight reduction method based on an attention mechanism, specifically, the weight value of the original sample background image block is quickly calculated by using the attention mechanism, and then the corresponding weight value is multiplied in each background image block; the attention mechanism is: Attention=f(g(x),x), g(x) represents generating an attention value based on the input image x, the attention value corresponds to the attention degree of the judgment area, and f(g(x),x) represents processing the input image x based on the attention g(x).

[0024] Further, the decoupling method comprises a foreground image block enhancement method based on Gaussian noise, specifically, after the original image is divided into foreground image blocks and background image blocks, Gaussian noise is randomly added to the foreground image blocks, and the calculation formula is:

[0025]

[0026] Wherein, mu, delta 2 are parameters of the distribution, respectively, the expectation and variance of the Gaussian distribution, p(x) is the image output after Gaussian function calculation with Gaussian noise, x is the input image; delta represents the standard deviation, which is used to describe the dispersion degree of the normal distribution data distribution; e is the base number of natural logarithm.

[0027] Further, the evaluation index comprises accuracy, recall rate, F1 value and ROC curve.

[0028] The beneficial effects of the present application are:

[0029] (1) The abnormal behavior detection method based on the data augmentation fine-grained distribution mechanism provided by the present application firstly divides the scene into foreground and background, and divides the background area block into N*N equal intervals, then randomly enhances different background image blocks through data augmentation algorithms such as affine transformation, perspective transformation and hue transformation, and the fine-grained background information can effectively avoid the instability caused by light, angle change and occlusion in the actual scene, and greatly eliminates the interference of the background to the foreground in a single scene, effectively improving the anti-interference performance of the model detection. The data augmentation fine-grained distribution mechanism of the present application randomly performs data augmentation algorithm on each background image block, and a background sample independent of the foreground is obtained by randomly combining each data augmented background image block, and the texture and color information of the whole picture is also greatly enriched. The model trained under the background decoupled sample by using the method of the present application can solve the detection fluctuation problem caused by too large foreground and background coupling in a single scene, and since the processing is performed on the sample, the training consumption of the benchmark model is not increased, and the generalization performance of the model is also improved.

[0030] (2) This invention divides each sample into a foreground image block and a background image block. Then, a feature fusion strategy is used for the background image blocks between samples, and another feature fusion strategy is used for the foreground image blocks between samples. This enhances the independence of the foreground and background, greatly weakens the adverse effect of the background on foreground detection, and enriches the number of samples with fine granularity, thereby improving the robustness of the model. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of an abnormal behavior detection method based on background decoupling according to an embodiment of the present invention.

[0033] Figure 2 This is a decoupling flowchart of an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] This invention proposes an abnormal behavior detection method based on background decoupling, such as... Figure 1 As shown, the process includes steps such as data collection, data preprocessing, feature engineering, model selection, model training, model evaluation, deployment, and monitoring. The specific implementation process is as follows:

[0036] Step 1. Data Collection: Collect the data needed to detect abnormal behavior, which can be real-time data, historical data, or simulated data. In this embodiment, the data is collected through a surveillance camera, and the complete video is cut into frames per second as the detection dataset.

[0037] Step 2. Data Preprocessing: Perform preprocessing operations such as cleaning, deduplication, standardization, and missing value imputation on the collected data to facilitate subsequent analysis and modeling. For example, some images in the collected image data may be too blurry or grayscale, and these abnormal images will be removed.

[0038] Step 3. Feature Engineering: Extract or construct features from the preprocessed data. Since deep learning models typically do not require manual feature construction, this step usually transforms the raw data into a form that can be processed by the deep learning model, such as labeling images and generating corresponding label files.

[0039] Step 4. Model Selection: Choose an appropriate deep learning model. Common deep learning anomaly detection models include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Autoencoders. This embodiment uses the Yolov5 network based on convolutional neural networks, which is currently a well-performing single-stage detector in the target industrial field.

[0040] Step 5. Model Training: Train the selected model using the cleaned and preprocessed dataset. Before training, image data augmentation is performed. The decoupling method designed in this invention decouples the image data before it is fed into the model at this stage: each image data fed into the model is first subjected to a random strategy, which mainly determines whether to use background decoupling augmentation for the image data fed into the model at this time. The background decoupling method of this invention can generate a sample with more diverse backgrounds from the original sample. The background-decoupled dataset is then input into the model for training. The goal of the training is to find an optimal model that can best fit normal data and identify abnormal data.

[0041] Specifically, a randomization strategy is set before the image is fed into the model: a random number between 0 and 1 is generated using Python's `random` function. If the randomly generated number is greater than 0.5, no data augmentation (background decoupling) is performed; if the randomly generated number is less than or equal to 0.5, the training image data is divided into N×N equal parts, where N... 2 Image blocks can be categorized into target image blocks and background image blocks, with those not containing abnormal behavior targets belonging to the background image blocks. All image blocks are randomly subjected to the four image enhancement strategies proposed in this invention. For example, Gaussian noise is added to the first image block, the second image block is randomly replaced with an image block of the same size from other training samples, the color contrast of the third image block is increased, and so on. In this case, the target image block in the sample is located within background image blocks that have undergone different data enhancements. This background decoupling method is achieved through hybrid data enhancement, ultimately replacing the target in the original image sample with a completely new scene. This increases the diversity of samples during model training and guides the model to learn target features in various complex backgrounds, reducing the model's dependence on the background during detection. Its application scenarios include abnormal target detection and abnormal behavior detection, with a wide range of applicability.

[0042] The above image block classification principle is as follows: use bounding boxes with label information to determine the position of the target in the image. After segmentation, it can be known whether each segmented block intersects with the position of the target's bounding box. If there is an intersection, it is the target image block; otherwise, it is the background image block.

[0043] In this embodiment, to avoid the extreme case where N×N image blocks all undergo the same data augmentation, this embodiment constrains that the number of image blocks for which each method is applied is less than or equal to 1 / 4 of the total number of image blocks. Specifically, four numbers are initially set to represent the number of times each of the four methods is used on an image block, initially set to 0. For each image block for which one augmentation method is applied, the corresponding number is incremented by 1. If one of these numbers has reached N×N / 4, the remaining image blocks will not use the augmentation method corresponding to that number.

[0044] Step 6. Model Evaluation: Evaluate the trained model using evaluation metrics. Common evaluation metrics include precision, recall, F1 score, and ROC curve. Based on the evaluation results, adjust model parameters or replace the model to improve detection accuracy.

[0045] Step 7. Deployment and Monitoring: Deploy the trained model to real-world application scenarios and perform real-time monitoring and detection of the input data. For high-risk or high-value scenarios, it is necessary to establish automated monitoring and early warning systems to promptly detect and handle abnormal events.

[0046] The four image enhancement strategies proposed in this invention include: fine-grained background enhancement based on data augmentation, fine-grained background fusion based on multi-sample feature fusion, fine-grained background information weighting based on attention mechanism, and random recombination of background modules to construct decoupled samples. During general model training, training samples are decoupled to generate decoupled samples before being fed into the model for training. The specific decoupling process is as follows: Figure 2 As shown in the figure. The following will elaborate on the content proposed in this invention.

[0047] (1) Fine-grained background enhancement based on data augmentation

[0048] In the original training samples, the image is divided into N×N target image blocks and background image blocks. 2 Only no more than 1 / 4 of the background image patches can be adaptively selected using affine transformation, perspective transformation, and tone transformation to replace the background image patch regions in the decoupled sample. In this case, the selection and replacement regions of the data augmented background image patches are determined by random numbering. The purpose of data augmented background image patches is to increase the texture and color diversity of the background, so they cannot replace foreground object patches.

[0049] (2) Fine-grained background fusion based on multi-sample feature fusion

[0050] Multi-sample feature fusion fully utilizes information from other samples in the training set. It employs a random selection method to modularize the background information of other samples in a fine-grained manner, and then uses the Mixup data augmentation method, based on the principle of minimizing neighborhood risk, to fuse the background information of other samples with the background information of the currently decoupled sample. Mixup uses linear difference to obtain new sample data, where (x... n ,y n ) is the new background image patch generated by interpolation, (x i ,y i ) and (x j ,y j Given the current training sample and other randomly selected samples, the background image patch is generated as follows:

[0051] (x n ,y n )=λ(x i ,y i )+(1-λ)(x j ,y j ),

[0052] The transparency ratio λ ranges from 0 to 1. An image patch x·λ represents the sum of all pixels in that patch multiplied by λ. If λ is 0.1, the patch is essentially transparent, having 0.1 times the clarity of the initial patch. The number of generated background patches must not exceed 1 / 4 of the total number of background patches. These generated background patches replace the original training sample background information patches, but cannot replace foreground object information patches.

[0053] (3) Fine-grained background information weighting based on attention mechanism

[0054] In the field of abnormal behavior detection, the attention mechanism, simply put, is the process of quickly determining the importance of different regions in an image. This process can be simply described as:

[0055] Attention = f(g(x),x)

[0056] g(x) represents the attention value generated based on the input image x, which corresponds to the degree of attention paid to the discrimination region. f(g(x),x) represents the processing of the input image x based on the attention g(x). This invention utilizes an attention mechanism to quickly calculate the weight values ​​of the original sample background image patches, and then multiplies each background image patch by the corresponding weight value. Generally, the weight values ​​of the background image patches are very small, and multiplying the original background image patches by a small value leads to a reduction in the original information value. This reweighting operation can reduce the information features of the original background, ensuring that the model can more easily locate and identify the target region.

[0057] (4) Gaussian noise-based foreground image patch enhancement

[0058] Noise can be viewed as a random signal, possessing statistically significant characteristics. Power spectral density (the spectral distribution of power, PDF) is one such characteristic of noise, used to classify it. Gaussian noise refers to noise whose probability density function follows a Gaussian distribution (i.e., a normal distribution). If a noise's amplitude distribution follows a Gaussian distribution, and its power spectral density is uniformly distributed, it is called Gaussian white noise. The second moments of Gaussian white noise are uncorrelated, and the first moment is a constant, representing the temporal correlation between successive signals. The probability density function is shown below:

[0059]

[0060] Where μ, δ 2 Let μ be the parameter of the distribution, and σ be the expectation and variance of the Gaussian distribution, respectively. When these parameters have definite values, the image p(x) with Gaussian noise output after Gaussian function calculation is also determined, especially when μ = 0 and σ = 0. 2 When δ = 1, the distribution of the input image x is a standard normal distribution; δ represents the standard deviation, used to describe the dispersion of normally distributed data; e is the base of the natural logarithm. In this invention, after dividing the original samples into foreground and background image blocks, Gaussian noise is randomly added to the foreground image blocks. This added Gaussian noise makes the model training more generalizable and increases the robustness of the foreground image blocks against interference.

[0061] Table 1 shows the performance metrics of the augmentation method proposed in this invention compared to several mainstream augmentation methods. This embodiment uses Yolov5 as the baseline model, and the dataset used is an anomaly detection dataset collected by a certain enterprise, totaling 30,000 images. The differences between the proposed method and mainstream data augmentation methods are evaluated using four widely used metrics. Table 1 shows the comparison results of mAP (0.5), mAP (.5: 0.95), accuracy, and recall.

[0062] Table 1. Data Enhancement Performance Based on Different Indicators

[0063] As shown in Table 1, regarding the Mean Average Precision (@0.5), this invention improves by 6.6% compared to YOLOv5, 8.1% compared to Cutout, 15.5% compared to Mixup, 4.2% compared to Mosaic, and 5.3% compared to Copy-Paste. This invention's method, like Mosaic and Copy-Paste, involves scaling, thus achieving improvements of varying degrees. Furthermore, this invention allows the target to be placed in different scenes, enhancing the background independence of the model's detection, thus also achieving improvements compared to these two methods. The results demonstrate that this invention's method improves upon current mainstream methods in various metrics and is more suitable for anomaly detection under real-world surveillance cameras.

[0064] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An abnormal behavior detection method based on background decoupling, characterized in that, Includes the following steps: Step 1: Image data collection; Step 2: Image data preprocessing; Step 3: Perform feature extraction or feature construction on the preprocessed data; Step 4: Select an appropriate deep learning model and train it; Step 5: Model Evaluation: Use evaluation metrics to evaluate the trained model. Based on the evaluation results, adjust the model parameters or replace the model to improve the detection accuracy. Step 6, Deployment and Monitoring: Deploy the trained model to the actual application scenario and monitor and detect the input data in real time; Before training the deep learning model in step 4, the image data needs to be decoupled: each image data put into the model first undergoes a randomization strategy: a random number between 0 and 1 is set. If the randomly generated number is greater than 0.5, background decoupling is not performed; if the randomly generated number is less than or equal to 0.5, the corresponding image data is divided into N×N equal parts and classified into target image blocks and background image blocks; each classified image block is randomly subjected to one of four decoupling methods, and the number of image blocks applied to each decoupling method is less than or equal to 1 / 4 of the total number of image blocks; One of the decoupling methods includes a fine-grained background enhancement method based on data augmentation, specifically: replacing background image block regions by adaptively selecting affine transformation, perspective transformation, and tone transformation.

2. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, Step 1 specifically involves collecting data that needs to detect abnormal behavior, including real-time data, historical data, and simulation data.

3. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The preprocessing in step 2 specifically involves cleaning, deduplicating, standardizing, and filling in missing values ​​for the collected data.

4. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The deep learning models in step 4 include convolutional neural networks, recurrent neural networks, and autoencoders.

5. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The decoupling method includes a fine-grained background fusion method based on multi-sample feature fusion. Specifically, it involves randomly selecting background information from other images and modularizing it into fine-grained blocks. Then, it uses a data augmentation method called Mixup, based on the principle of minimizing neighborhood risk, to fuse the background information of the other images with the background information of the current decoupled image, generating a new background image block. The background image block generation method is as follows: The value of the image transparency ratio λ ranges from 0 to 1. It is a new background image patch generated by interpolation. and These are the currently decoupled image and other randomly selected images.

6. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The decoupling method includes a fine-grained background information weighting method based on an attention mechanism, specifically: the weight values ​​of the original sample background image blocks are quickly calculated using an attention mechanism, and then multiplied by the corresponding weight value in each background image block; the attention mechanism is as follows: The Indicates based on input image An attention value is generated, which corresponds to the degree of attention paid to the region to be judged. Indicates attention-based Processing input images .

7. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The decoupling method includes a foreground image patch enhancement method based on Gaussian noise. Specifically, after dividing the original image into foreground image patches and background image patches, Gaussian noise is randomly added to the foreground image patches. The calculation formula is as follows: in, Let be the parameters of the distribution, and be the expectation and variance of the Gaussian distribution, respectively. This is the image with Gaussian noise output after Gaussian function calculation. Input image; Standard deviation is used to describe the dispersion of normally distributed data. It is the base of the natural logarithm.

8. The abnormal behavior detection method based on background decoupling according to claim 1, characterized in that, The evaluation metrics include precision, recall, F1 score, and ROC curve.

Images