Abnormal behavior recognition method based on local intuitionistic fuzzy support vector machine

Abstract

The abnormal behavior recognition method based on the local intuitionistic fuzzy support vector machine (LIFSVM) includes the following steps: 1) extracting local category information of samples to obtain neighborhood information within and between sample classes; 2) calculating sample intra-class weights and inter-class weights based on imbalance, local sample information, and outlier factors; 3) constructing a noise-insensitive LIFSVM optimization model for the problem of imbalanced abnormal behavior recognition; 4) deducing the optimization problem and solving a system of linear equations to obtain the optimal solution of the model; 5) constructing non-parallel proximal hyperplanes for abnormal and normal behaviors, respectively, and thereby obtaining the optimal decision function of the LIFSVM model; and 6) predicting whether the behavior is dangerous. This invention can improve the recognition rate of abnormal behavior and more accurately complete the behavior recognition task.

Description

Technical Field

[0001] The present invention belongs to the field of abnormal behavior image recognition, and in particular to the problem of abnormal behavior recognition in unbalanced noise scenarios. Specifically, it relates to an abnormal behavior recognition method based on a Local Intuitionistic Fuzzy Support Vector Machine (LIFSVM). Background Art

[0002] Abnormal behavior recognition is to identify whether there are abnormal behaviors of people in the monitoring area. Human behavior recognition has a wide range of applications in various fields, including traffic safety, intelligent video surveillance, human-computer interaction and other real-life scenarios. For example, in real life, pedestrians often look down at their phones while crossing the zebra crossing. In addition, some people, for the sake of "convenience", cross the road quickly by dangerously climbing over the fence, such as Figure 1 As shown in the figure, these inappropriate behaviors pose a significant threat to traffic safety, so how to quickly and easily prevent them is a topic of considerable research. Furthermore, when elderly individuals in the family experience health hazards such as falls, serious tragedies can occur due to a lack of timely detection and medical attention. Therefore, there is an urgent need for high-performance intelligent human abnormal behavior recognition technology.

[0003] However, identifying abnormal human behavior faces two challenges. First, the probability of abnormal behavior is much lower than that of normal behavior. In a dataset, the number of abnormal behavior samples is far smaller than that of normal behavior samples. Therefore, identifying abnormal human behavior is a highly class-imbalanced problem. Most machine learning methods assume data balance and are not directly applicable to behavior recognition scenarios. With class imbalance, the disparity in the number of class samples can cause the classifier to favor the majority class, leading to misclassification of minority class samples. Minority classes often have fewer but more valuable samples, resulting in poor classifier performance. Second, during data collection, data samples can be noisy due to factors such as equipment failure, excessive or insufficient lighting, and obstructions. This noise / outlier problem can significantly negatively impact the learning of machine learning models, reducing their recognition performance. Therefore, accurately identifying abnormal human behavior is a very challenging task.

[0004] Existing machine learning-based human behavior recognition technology primarily consists of two stages: behavioral feature extraction and abnormal behavior identification. Specifically, behavioral features of people in the monitoring scene are first extracted. Then, a machine learning model learns and identifies abnormal behaviors based on these features. However, in the actual behavioral feature extraction process, the extracted behavioral features in the monitoring scene are often noisy, and data from different categories is often unbalanced. Therefore, in response to these two problems, scholars in this field have proposed corresponding abnormal behavior identification solutions in the following literature: [1] Batuwita R, Palade V. FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning[J]. IEEE Transactions on Fuzzy Systems, 2010, 18(3): 558-571. [2] Richhariya B, Tanveer MA robust fuzzy least squares twin support vector machine for classimbalance learning[J]. Applied Soft Computing, 2018, 71: 418-432. [3] Rezvani S, Wang X, Pourpanah F. Intuitionistic Fuzzy Twin Support Vector Machines[J]. IEEE Transactions on Fuzzy Systems, 2019, 27(11): 2140-2151. [4] Tanveer M, Ganaie MA, Bhattacharjee A. Intuitionistic Fuzzy Weighted Least Squares Twin SVMs[J].IEEE Transactions on Cybernetics, 2022:1-10.

[0005] Among them, the literature [1] Fuzzy Support Vector Machines for Class Imbalance Learning (FSVM-CIL) proposed fuzzy membership functions from three levels: the distance from the sample to the class center, the distance from the sample to the estimated hyperplane, and the distance from the sample to the true hyperplane to solve the noise and outlier problems, and multiplied the obtained weights by pre-set parameters according to the category to solve the class imbalance problem. The literature [2] A robust fuzzy least squares twin support vector machine for class imbalance learning (RFLSTSVM-CIL) proposed a new fuzzy membership function based on the imbalance rate IR to calculate the weight of the majority class samples, while setting the weights of the minority class to 1. The literature [3] Intuitionistic Fuzzy Twin Support Vector Machines (IFTSVM) uses the concept of intuitionistic fuzzy and proposes a new intuitionistic fuzzy membership function, which can not only reduce the impact of noise, but also distinguish noise from support vectors. In the optimization problem proposed in the Intuitionistic Fuzzy Weighted Least Squares Twin SVMs (IFWLSTSVM) [4], not only weighting of samples from the other class is performed, but also intra-class weighting of the current class is performed. Although the above fuzzy technology can improve the noise resistance of the support vector machine (SVM) model to a certain extent, the following challenges still exist:

[0006] 1) In FSVM-CIL, when calculating fuzzy membership, only intra-class information is considered, and the differences between sample categories are not fully considered. In addition, multiplying the class by a pre-set parameter to solve the class imbalance problem requires setting different parameters for different datasets, which makes it inconvenient to use.

[0007] 2) RFLSTSVM-CIL sets all minority class weights to 1 to amplify the influence of the minority class. However, this causes the model to be unable to correctly identify noise / outliers in the minority class, making it sensitive to noise / outliers in the minority class. Furthermore, it confuses the concepts of noise points and support vectors, easily misclassifying important support vectors as noise points and assigning them small weights.

[0008] 3) IFTSVM and IFWLSTSVM are methods based on Euclidean space distance, which will reduce the accuracy for non-Gaussian distributed data; and both papers do not consider the class imbalance problem, so they have great limitations. Summary of the Invention

[0009] The present invention aims to overcome the above-mentioned shortcomings of the prior art and proposes a method for identifying abnormal behavior based on a local intuitionistic fuzzy support vector machine. The details are as follows:

[0010] 1) The present invention uses the kNN (K nearest neighbor) method to extract the local category information of the sample to obtain the neighborhood information within and between the sample class. On this basis, the present invention proposes a quantile noise data detection algorithm based on intra-class neighborhood information. By calculating the distance between each sample and its nearest neighbor samples in the same category and averaging these distances to obtain the degree of deviation between the sample and the same category samples, the quantile of the deviation degree of all the same category samples is then calculated. If the deviation degree of the sample is closer to the upper outlier cutoff value of the same category, it means that the sample deviates more from the distribution center, that is, the sample is assigned a smaller intra-class weight. For samples with a deviation degree greater than the upper outlier cutoff point, they are detected as noise samples and assigned an intra-class weight of 0.

[0011] 2) In order to better detect noise data, the present invention will further introduce a category fuzzy information binary (μ, v) for each sample, where μ represents the sample membership and v represents the sample non-membership. In the intuitive fuzzy framework, in order to construct inter-class weights with local information, an inter-class correlation algorithm based on inter-class neighborhood information is proposed. The algorithm determines the inter-class correlation of samples by calculating the proportion of the number of samples of another class in the neighborhood of each sample. If the inter-class correlation of samples is greater, it means that the sample is more relevant to the other class, that is, the sample is given a smaller inter-class weight. At the same time, in order to alleviate the imbalance problem of abnormal behavior data, for the minority class, when constructing the inter-class correlation, the imbalance rate is introduced, that is, in theory, the number of minority class data and majority class data samples is balanced as much as possible. In order to protect the minority class samples in the overlap area, for the majority class, when constructing the inter-class weight, the imbalance rate is also introduced to reduce the inter-class weight of the majority class samples in the overlap area.

[0012] 3) In addition, in the model, in order to reduce the deviation of the outliers in the current class, the present invention proposes a new optimization problem based on intra-class and inter-class neighborhood information. This optimization problem no longer only adds inter-class weights to the samples of the other class, but also adds intra-class weights to the samples of the current class. The purpose is that when the model learns the proximal hyperplane, the proximal hyperplane should not only be far away from the samples of the other class, but also correctly distinguish between the normal samples and the abnormal samples of the current class, so that the normal samples are gathered around the proximal hyperplane as much as possible, while ignoring the abnormal samples. At the same time, in order to overcome the deficiency of the SVM model that is more inclined to the majority class, the present invention considers it from the perspective of the algorithm level, that is, when the model learns the proximal hyperplane corresponding to the minority class, it adds an imbalance rate to the current class sample to increase the penalty for misclassification of minority class samples, so that the proximal hyperplane finally learned reduces the misclassification of minority class samples.

[0013] To facilitate better understanding and description, this invention establishes a unified notation convention, in which scalars are represented by lowercase or uppercase letters without bold, such as x, X. Vectors are represented by lowercase letters with bold, such as x. Matrices are represented by uppercase letters with bold, such as X.

[0014] The abnormal behavior recognition method based on local intuitionistic fuzzy support vector machine proposed in this invention is suitable for application scenarios of identifying abnormal human behavior and dangerous actions. The overall process of this invention is as follows Figure 2 The specific steps are described as follows:

[0015] Step 1: Construction of abnormal behavior dataset. First, use OpenCV to capture the surveillance video stream in real time and sample to generate video frame images. Then, perform preprocessing such as denoising, grayscale conversion, and histogram equalization on the sampled images, and use the mediapipe open source module to extract the posture key points of the human body in the image. Then, use these key points to construct the posture feature x i , and the output y of the sample i Manually mark whether it is abnormal behavior (1 represents abnormal behavior, 2 represents normal behavior). Finally, normalize the data to obtain a supervised data set. in is the sample feature of dimension n, y i is the output corresponding to the sample, and m is the sample size. The dataset is divided into two non-repeated parts by category: the data sample matrix of the minority class of abnormal behavior (marked as 1) (abbreviated as: A) and the data sample matrix of the majority class of normal behavior (labeled as 2) (Abbreviation: B), where m1 and m2 are the number of samples in the minority class and the majority class respectively. Define the data class imbalance rate k is a parameter of the kNN nearest neighbor algorithm, which indicates the number of selected “nearest neighbors”. is the collection of samples of the sth category, s=1,2.

[0016] Step 2: Calculate the intra-class weights of the minority class and the majority class respectively according to the kNN algorithm and the quartile method, and record them as It represents the intra-class weight of the i-th sample in the s-th class. The intra-class weight process of the present invention is as follows: Figure 3 shown.

[0017] (2.1) By calculating the distance between each sample and its nearest neighbor samples in the same field, and averaging these distances, we can obtain the degree of deviation between the sample and the samples of the same category, which is recorded as Indicates the degree of intra-class deviation of the i-th sample in the s-th class.

[0018]

[0019] Among them, x i is the i-th sample in the s-th class, x m is x i kNN samples of .

[0020] (2.2) According to the category, the deviation degree of all samples of the same category is calculated, including the upper quartile, lower quartile and interquartile difference of each category, which are respectively recorded as and IQR (s) , s=1,2, represents the upper and lower quartiles and the interquartile range of the sth category.

[0021]

[0022] (2.3) According to the category, calculate the upper outlier cutoff point in different categories, denoted as t (s) , s=1,2, represents the upper outlier cutoff point of the sth class.

[0023]

[0024] (2.4) According to the properties of the upper outlier cutoff, the closer the deviation of a sample is to the upper outlier cutoff value of the same category, the further it deviates from the center of the distribution. In other words, the sample is assigned a smaller intra-class weight, but not less than 0. Furthermore, because the number of minority and majority class samples is unbalanced, and the smaller minority class samples have more important research value, they should be protected to the greatest extent possible. Based on the above analysis, the intra-class weight of the present invention is constructed as follows:

[0025] For minority class samples:

[0026] For majority class samples:

[0027] Step 3: According to the kNN method and the intuitionistic fuzzy method, the inter-class weights of the minority class and the majority class are calculated respectively. The inter-class weight process of the present invention is as follows: Figure 4 shown.

[0028] (3.1) Membership calculation: Use the same calculation method as in step 2 to calculate the membership value, which is recorded as It represents the membership of the i-th sample in the s-th class. Its meaning is the probability that the i-th sample belongs to the s-th class.

[0029] (3.2) Non-membership calculation: Combining the kNN method and the intuitionistic fuzzy method, the non-membership is redefined:

[0030] (3.2.1) Combine the information of the two types of data to calculate the inter-class correlation, denoted as represents the inter-class correlation of the i-th sample in the s-th class and is constructed as follows:

[0031]

[0032] in, and They represent the number of class 1 samples and the number of class 2 samples in the kNN of the i-th sample. Here, the inter-class correlation of the minority class The calculation of r is added because the number of samples of minority class data and majority class data differs by r times. Therefore, the data is simulated here to increase r times around each minority class sample. The purpose is to balance the number of minority class data and majority class data samples as much as possible in theory.

[0033] (3.2.2) Calculate the sample non-membership based on the membership and inter-class correlation, denoted as It represents the non-membership of the i-th sample in the s-th class. Its meaning is the probability that the i-th sample does not belong to the s-th class.

[0034]

[0035] (3.3) According to the membership and non-membership, the weight between sample classes is calculated and recorded as represents the inter-class weight of the i-th sample in the s-th class and is constructed as follows:

[0036]

[0037] The operation of formula (8) is to construct the inter-class weight of samples, which is as follows: When the non-membership value is 0, the inter-class weight is equal to the membership value. When the non-membership value is not less than the membership value, the inter-class weight is 0. In other cases, the inter-class weight is

[0038] (3.4) Inter-class weights for majority class samples Further corrections are as follows:

[0039]

[0040] Formula (9) indicates that in the overlap region, when the number of minority class samples in the kNN of the majority class samples is r times or greater than the number of majority class samples, the inter-class weight of the majority class samples is reduced by r times. Otherwise, it remains unchanged. The purpose is to protect the minority class samples as much as possible in the sample overlap region.

[0041] Step 4: The LIFSVM method proposed in this invention is similar to the twin support vector machine, which finds a proximal hyperplane f1(x) and f2(x) for the minority class and the majority class respectively, namely:

[0042]

[0043] Among them, f1(x) and f2(x) are the proximal hyperplanes of abnormal behavior and normal behavior respectively, and w 1，2 and b 1，2 are the weight and bias of the corresponding hyperplane, and x is the feature of the sample.

[0044] In order to realize empirical risk, LIFSVM considers the following two loss functions

[0045]

[0046] and

[0047]

[0048] Among them, q (1) ,q (2) is the intra-class weight of the minority and majority class samples, f (1) , f (2) is the class weight between the minority class and the majority class samples. c1, c2>0 is the penalty parameter.

[0049] However, if LIFSVM only focuses on minimizing the empirical risk, this may lead to overfitting in practice, and another difficulty will also arise, that is, singular points of the matrix may appear during its learning process. In order to overcome the above problems, the present invention introduces an additional regularization term ||w|| 2 +b 2 , a structural risk minimization version is proposed. Then, the original problem of LIFSVM is obtained

[0050]

[0051]

[0052] Where ξ is a non-negative slack variable; c i , i=1…4 is the penalty parameter.

[0053] For the convenience of expression, the present invention converts the vector form (13) (14) of the above samples into the following matrix form

[0054]

[0055]

[0056] Where, Equations (15) and (16) are optimization problems for finding the proximal hyperplane of abnormal and normal behavior, respectively. Q1 and Q2 are diagonal matrices, whose main diagonals are the intra-class weights of the minority class and the majority class (4)(5), and F1 and F2 are diagonal matrices, whose main diagonals are the inter-class weights of the minority class and the majority class (8)(9), defined as follows:

[0057]

[0058] They are used to control the influence of samples within or between classes; e1, e2 are unit vectors of appropriate dimensions.

[0059] The optimization problem (15) for the minority class is analyzed. Minimizing the first term of the objective function is to make the minority class samples closer to the first hyperplane, that is, the samples are concentrated as close to the hyperplane as possible. The second term of the objective function is the square loss function of the slack variable. The second constraint requires that the distance between the majority class samples and the first hyperplane is at least 1. The slack variable measures the error when the constraint is not met. In other words, minimizing the first two terms aims to maximize the separation of majority class samples from minority class samples. The last term is the regularization term, which is to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting. Similarly, the optimization problem (16) is similar to (15).

[0060] Step 5: Solve the two optimization problems in step 4.

[0061] (5.1) Taking the optimization problem (15) as an example, first substitute the equality constraint into the objective function, and we can get

[0062]

[0063] (5.2) Take the gradient of (18) with respect to w1 and b1, and set the gradient equal to 0, and we get

[0064]

[0065]

[0066] (5.3) According to equations (19) and (20), solve have to

[0067]

[0068] Wherein, H1=rQ1[A e1], G1=F2[B e2], and I represents the unit matrix.

[0069] (5.4) Similarly, solve have to

[0070]

[0071] Here, H2 = F1[A e1], G2 = Q2[B e2]. At this point, the training phase of the LIFSVM method is completed.

[0072] Step 6: Identify abnormal behavior (model prediction). After obtaining the optimal model parameters, construct an abnormal behavior prediction decision function. For a given video image to be detected, first use Opencv to capture the video stream in real time, and then use the mediapipe open source module to extract the key points of the human body posture. Finally, use these key points to generate the human body posture feature x. For a given feature x, calculate the distance from the feature x to the proximal hyperplane (10) of abnormal behavior and normal behavior to predict whether the scene is abnormal behavior. The decision function is as follows:

[0073]

[0074] Where |·| represents the absolute value. If the distance between feature x and the proximal hyperplane of abnormal behavior is closer than that of normal behavior, then (23) outputs 1, and the system determines that the current sample x is abnormal behavior and performs abnormal behavior recording or alarm. If the distance between feature x and the proximal hyperplane of normal behavior is closer than that of abnormal behavior, then (23) outputs 2, and the system determines that the current sample x is normal behavior.

[0075] This paper proposes the LIFSVM method for application scenarios such as dangerous fence climbing in imbalanced abnormal behavior recognition systems. When the dataset has characteristics such as noise / outliers and class imbalance learning, the LIFSVM method performs better. The present invention has the following advantages:

[0076] 1) Compared with FSVM-CIL and RFLSTSVM-CIL, LIFSVM rationally constructs and utilizes both intra-class and inter-class information in sample data, adding intra-class weights to samples of the current class and inter-class weights to samples of the other class. This enables the model to not only distinguish between noise and normal samples within the same class but also learn inter-class information, such as the imbalance rate between different classes. For abnormal behavior recognition data with noise and class imbalance, the LIFSVM approach achieves more accurate recognition.

[0077] 2) Compared with IFTSVM and IFWLSTSVM, in the intuitionistic fuzzy framework, LIFSVM uses the kNN method to extract the local category information of samples when constructing inter-class weights with local information, so that the model has superior recognition effect on non-Gaussian distribution data.

[0078] 3) In addition, in order to overcome the deficiency of the SVM model that it is more inclined to normal behavior, the present invention considers it from the perspective of the algorithm level. That is, when the model learns the proximal hyperplane corresponding to abnormal behavior, it adds an imbalance rate to the current class sample to increase the penalty for misclassification of minority class samples, so that the proximal hyperplane finally learned reduces the misclassification of abnormal behavior samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 Identify application scenarios for dangerous climbing over barriers.

[0080] Figure 2 It is the overall algorithm flow chart of the present invention.

[0081] Figure 3 This is a flow chart of the intra-class weight algorithm of the present invention.

[0082] Figure 4 This is a flow chart of the inter-class weight algorithm of the present invention. DETAILED DESCRIPTION

[0083] The following is a preferred embodiment of the present invention, which is described in detail with reference to the accompanying drawings. The embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The abnormal behavior recognition method based on the local intuitionistic fuzzy support vector machine has the following specific steps:

[0084] Step 1: Construction of abnormal behavior dataset. First, use OpenCV to capture the surveillance video stream in real time and sample to generate video frame images. Then, perform preprocessing such as denoising, grayscale conversion, and histogram equalization on the sampled images, and use the mediapipe open source module to extract the posture key points of the human body in the image. Then, use these key points to construct the posture feature x i , and the output y of the sample i Manually mark whether it is abnormal behavior (1 represents abnormal behavior, 2 represents normal behavior). Finally, normalize the data to obtain a supervised data set. in is the sample feature of dimension n, y i is the output corresponding to the sample, and m is the sample size. The dataset is divided into two non-repeated parts according to category: the data sample matrix of the minority class of abnormal behavior (labeled as 1) (abbreviated as: A) and the data sample matrix of the majority class of normal behavior (labeled as 2) (Abbreviation: B), where m1 and m2 are the number of samples in the minority class and the majority class respectively. Define the data class imbalance rate k is a parameter of the kNN nearest neighbor algorithm, which indicates the number of selected “nearest neighbors”. is the collection of samples of the sth category, s=1,2.

[0085] Step 2: Calculate the intra-class weights of the minority class and the majority class respectively according to the kNN algorithm and the quartile method, and record them as It represents the intra-class weight of the i-th sample in the s-th class. The intra-class weight process of the present invention is as follows: Figure 3 shown.

[0086] (2.1) By calculating the distance between each sample and its nearest neighbor samples in the same field, and averaging these distances, we can obtain the degree of deviation between the sample and the samples of the same category, which is recorded as Indicates the degree of intra-class deviation of the i-th sample in the s-th class.

[0087]

[0088] Among them, x i is the i-th sample in the s-th class, x m is x i kNN samples of .

[0089] (2.2) According to the category, the deviation degree of all samples of the same category is calculated, including the upper quartile, lower quartile and interquartile difference of each category, which are respectively recorded as and IQR (s) , s=1,2, represents the upper and lower quartiles and the interquartile range of the sth category.

[0090]

[0091] (2.3) According to the category, calculate the upper outlier cutoff point in different categories, denoted as t (s) , s=1,2, represents the upper outlier cutoff point of the sth class.

[0092]

[0093] (2.4) According to the properties of the upper outlier cutoff, the closer the deviation of a sample is to the upper outlier cutoff value of the same category, the further it deviates from the center of the distribution. In other words, the sample is assigned a smaller intra-class weight, but not less than 0. Furthermore, because the number of minority and majority class samples is unbalanced, and the smaller minority class samples have more important research value, they should be protected to the greatest extent possible. Based on the above analysis, the intra-class weight of the present invention is constructed as follows:

[0094] For minority class samples:

[0095] For majority class samples:

[0096] Step 3: According to the kNN method and the intuitionistic fuzzy method, the inter-class weights of the minority class and the majority class are calculated respectively. The inter-class weight process of the present invention is as follows: Figure 4 shown.

[0097] (3.1) Membership calculation: Use the same calculation method as in step 2 to calculate the membership value, which is recorded as It represents the membership of the i-th sample in the s-th class. Its meaning is the probability that the i-th sample belongs to the s-th class.

[0098] (3.2) Non-membership calculation: Combining the kNN method and the intuitionistic fuzzy method, the non-membership is redefined:

[0099] (3.2.1) Combine the information of the two types of data to calculate the inter-class correlation, denoted as represents the inter-class correlation of the i-th sample in the s-th class and is constructed as follows:

[0100]

[0101] in, and They represent the number of class 1 samples and the number of class 2 samples in the kNN of the i-th sample. Here, the inter-class correlation of the minority class The calculation of r is added because the number of samples of minority class data and majority class data differs by r times. Therefore, the data is simulated here to increase r times around each minority class sample. The purpose is to balance the number of minority class data and majority class data samples as much as possible in theory.

[0102] (3.2.2) Calculate the sample non-membership based on the membership and inter-class correlation, denoted as It represents the non-membership of the i-th sample in the s-th class. Its meaning is the probability that the i-th sample does not belong to the s-th class.

[0103]

[0104] (3.3) According to the membership and non-membership, the weight between sample classes is calculated and recorded as represents the inter-class weight of the i-th sample in the s-th class and is constructed as follows:

[0105]

[0106] The operation of formula (8) is to construct the inter-class weight of samples, which is as follows: When the non-membership value is 0, the inter-class weight is equal to the membership value. When the non-membership value is not less than the membership value, the inter-class weight is 0. In other cases, the inter-class weight is

[0107] (3.4) Inter-class weights for majority class samples Further corrections are as follows:

[0108]

[0109] Formula (9) indicates that in the overlap region, when the number of minority class samples in the kNN of the majority class samples is r times or greater than the number of majority class samples, the inter-class weight of the majority class samples is reduced by r times. Otherwise, it remains unchanged. The purpose is to protect the minority class samples as much as possible in the sample overlap region.

[0110] Step 4: The LIFSVM method proposed in this invention is similar to the twin support vector machine, which finds a proximal hyperplane f1(x) and f2(x) for the minority class and the majority class respectively, namely:

[0111]

[0112] Among them, f1(x) and f2(x) are the proximal hyperplanes of abnormal behavior and normal behavior respectively, and w 1，2 and b 1，2 are the weight and bias of the corresponding hyperplane, and x is the feature of the sample.

[0113] In order to realize empirical risk, LIFSVM considers the following two loss functions

[0114]

[0115] and

[0116]

[0117] Among them, q (1) ,q (2) is the intra-class weight of the minority and majority class samples, f (1) , f (2) is the class weight between the minority class and the majority class samples. c1, c2>0 is the penalty parameter.

[0118] However, if LIFSVM only focuses on minimizing the empirical risk, this may lead to overfitting in practice, and another difficulty will also arise, that is, singular points of the matrix may appear during its learning process. In order to overcome the above problems, the present invention introduces an additional regularization term ||w|| 2 +b 2 , a structural risk minimization version is proposed. Then, the original problem of LIFSVM is obtained

[0119]

[0120]

[0121] Where ξ is a non-negative slack variable; c i , i=1…4 is the penalty parameter.

[0122] For the convenience of expression, the present invention converts the vector form (13) (14) of the above samples into the following matrix form

[0123]

[0124]

[0125] Where, Equations (15) and (16) are optimization problems for finding the proximal hyperplane of abnormal and normal behavior, respectively. Q1 and Q2 are diagonal matrices, whose main diagonals are the intra-class weights of the minority class and the majority class (4)(5), and F1 and F2 are diagonal matrices, whose main diagonals are the inter-class weights of the minority class and the majority class (8)(9), defined as follows:

[0126]

[0127] They are used to control the influence of samples within or between classes; e1, e2 are unit vectors of appropriate dimensions.

[0128] The optimization problem (15) for the minority class is analyzed. Minimizing the first term of the objective function is to make the minority class samples closer to the first hyperplane, that is, the samples are concentrated as close to the hyperplane as possible. The second term of the objective function is the square loss function of the slack variable. The second constraint requires that the distance between the majority class samples and the first hyperplane is at least 1. The slack variable measures the error when the constraint is not met. In other words, minimizing the first two terms aims to maximize the separation of majority class samples from minority class samples. The last term is the regularization term, which is to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting. Similarly, the optimization problem (16) is similar to (15).

[0129] Step 5: Solve the two optimization problems in step 4.

[0130] (5.1) Taking the optimization problem (15) as an example, first substitute the equality constraint into the objective function, and we can get

[0131]

[0132] (5.2) Take the gradient of (18) with respect to w1 and b1, and set the gradient equal to 0, and we get

[0133]

[0134]

[0135] (5.3) According to equations (19) and (20), solve have to

[0136]

[0137] Wherein, H1=rQ1[A e1], G1=F2[B e2], and I represents the unit matrix.

[0138] (5.4) Similarly, solve have to

[0139]

[0140] Here, H2 = F1[A e1], G2 = Q2[B e2]. At this point, the training phase of the LIFSVM method is completed.

[0141] Step 6: Abnormal behavior identification stage (model prediction). After obtaining the optimal model parameters, construct an abnormal behavior prediction decision function. For a given video image to be detected, first use Opencv to capture the video stream in real time, and then use the mediapipe open source module to extract the key points of the human body posture. Finally, use these key points to generate the human body posture feature x. For a given feature x, calculate the distance between the feature x and the proximal hyperplane (10) of abnormal behavior and normal behavior to predict whether the scene is abnormal behavior. The decision function is as follows:

[0142]

[0143] Where |·| represents the absolute value. If the distance between feature x and the proximal hyperplane of abnormal behavior is closer than that of normal behavior, then (23) outputs 1, and the system determines that the current sample x is abnormal behavior and performs abnormal behavior recording or alarm. If the distance between feature x and the proximal hyperplane of normal behavior is closer than that of abnormal behavior, then (23) outputs 2, and the system determines that the current sample x is normal behavior.

[0144] The data set used in the present invention is the dangerous fence-climbing data in the detection of abnormal human behavior. It is captured from real-life fence-climbing videos, which include both dangerous fence-climbing behaviors and normal fence-passing behaviors. In order to compare the superiority of the method of the present invention in dealing with noise and unbalanced data, 0.4 disturbance noise was added to each data set, and the class imbalance rates of the data sets were r=4, 6, 8, and 10, respectively. In the experiment, in order to ensure the fairness of the distribution of experimental training sets and test sets, a ten-fold cross-validation method was used, and its mean was taken as the result, and the final result was the mean of the eleven ten-fold cross-validation results, in order to avoid the randomness brought about by the random division of data in the ten-fold cross-validation. Since the experiment involves class imbalance learning, the present invention uses G_mean as an indicator to measure the quality of the model. G_mean is calculated as follows:

[0145]

[0146] Among them, TP represents the number of positive samples predicted as positive, FN represents the number of positive samples predicted as negative, FP represents the number of negative samples predicted as positive, and TN represents the number of negative samples predicted as negative.

[0147] The present invention compares LIFSVM with double-boundary support vector machine (TBSVM), least squares double-boundary support vector machine (LSTBSVM), IFTSVM and three existing popular synthetic imbalanced data methods, namely synthetic minority class oversampling technique (SMOTE), adaptive synthetic sampling (ADASYN) and bounded synthetic minority class oversampling technique (BLSMOTE), and finally measures the advantages and disadvantages of the methods by average ranking.

[0148]

[0149] Table 1 Comparison between the method of the present invention and the existing support vector machine method on dangerous fence climbing data

[0150]

[0151] Table 2 Comparison of the method of the present invention and three existing synthetic unbalanced data methods on dangerous hurdle climbing data

[0152] As shown in Tables 1 and 2, IFTSVM only slightly outperforms the proposed LIFSVM method on the dataset with an imbalance ratio of 4. On the other three datasets, LIFSVM outperforms IFTSVM. The other methods all perform worse than LIFSVM on each dataset. Therefore, the above experiments demonstrate that the proposed LIFSVM method exhibits certain advantages in learning with noise / outliers and class imbalance.

[0153] The above are preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions conceived by a person skilled in the art within the technical scope disclosed by the present invention without creative effort are encompassed within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the scope of protection defined in the claims.

Claims

1. The abnormal behavior recognition method based on local intuitionistic fuzzy support vector machine includes the following steps: Step 1: Construction of abnormal behavior dataset. First, use OpenCV to capture the surveillance video stream in real time and sample it to generate video frame images. Then, perform denoising, grayscale conversion, and histogram equalization on the sampled images. Use the mediapipe open source module to extract key points of human posture in the images. These keypoints are then used to construct the pose features x i , and the output y of the sample i Manually mark whether it is abnormal behavior, 1 represents abnormal behavior, and 2 represents normal behavior; finally, normalize the data to obtain a supervised data set in is the sample feature of dimension n, y i is the output corresponding to the sample, m is the sample size; the data set is divided into two non-repeating parts by category: the minority class of abnormal behavior is the data sample matrix marked as 1 Abbreviated as: A, and the majority class of normal behavior is the data sample marked as 2 This matrix Abbreviation: B, where m1 and m2 are the number of samples in the minority class and the majority class respectively; define the data class imbalance rate Step 2: Calculate the intra-class weights of the minority class and the majority class respectively according to the kNN algorithm and the quartile method, and record them as represents the intra-class weight of the i-th sample in the s-th class; is the set of samples of the sth category, s = 1, 2; (2.1) By calculating the distance between each sample and its neighboring samples in the same neighborhood, and averaging these distances, we can obtain the degree of deviation between the sample and the samples of the same type, which is recorded as Indicates the degree of intra-class deviation of the i-th sample in the s-th class; in, x i is the i-th sample in the s-th class, x l is x i kNN samples; k is the parameter of the kNN nearest neighbor algorithm, which indicates the number of selected "nearest neighbors"; (2.2) According to the category, the deviation degree of all samples of the same category is calculated, including the upper quartile, lower quartile and interquartile difference of each category, which are respectively recorded as and IQR (s) , s = 1, 2, indicating the upper and lower quartiles and the interquartile range of the sth category; (2.3) According to the category, calculate the upper outlier cutoff point in different categories, denoted as t (s) , s=1,2, represents the upper outlier cutoff point of the sth class; (2.4) The intra-class weight is constructed as follows: For minority class samples: For majority class samples: Step 3: Calculate the inter-class weights of the minority class and the majority class respectively according to the kNN method and the intuitionistic fuzzy method; (3.1) Membership calculation: Use formula (4) and formula (5) to calculate the membership value, which is recorded as Represents the membership of the i-th sample in the s-th class; its meaning is the probability that the i-th sample belongs to the s-th class; (3.2) Non-membership calculation: Combining the kNN method and the intuitionistic fuzzy method, the non-membership is redefined: (3.2.1) Combine the information of the two types of data to calculate the inter-class correlation, denoted as represents the inter-class correlation of the i-th sample in the s-th class and is constructed as follows: in, and They represent the number of class 1 samples and the number of class 2 samples in the kNN of the i-th sample respectively; (3.2.2) Calculate the sample non-membership based on the membership and inter-class correlation, denoted as Represents the non-membership of the i-th sample in the s-th class; its meaning is the probability that the i-th sample does not belong to the s-th class; (3.3) According to the membership and non-membership, the weight between sample classes is calculated and recorded as represents the inter-class weight of the i-th sample in the s-th class and is constructed as follows: The operation of formula (8) is to construct the inter-class weight of samples, which is as follows: when the non-membership value is 0, the inter-class weight is equal to the membership value; when the non-membership value is not less than the membership value, the inter-class weight is 0; in other cases, the inter-class weight is (3.4) Inter-class weights for majority class samples Further corrections are as follows: Formula (9) indicates that in the overlap region, when the number of minority class samples in the kNN of the majority class samples is r times greater than the number of majority class samples, the inter-class weight of the majority class samples is reduced by r times; otherwise, it remains unchanged. Step 4: The local intuitionistic fuzzy support vector machine (LIFSVM) method is similar to the twin support vector machine, which finds a proximal hyperplane f1(x) and f2(x) for the minority class and the majority class respectively, namely: Among them, f1(x), f2(x) are the proximal hyperplanes of abnormal behavior and normal behavior, w 1，2 , b 1，2 are the weights and biases corresponding to the hyperplane, and x is the feature of the sample; In order to realize empirical risk, LIFSVM considers the following two loss functions and Among them, q (1) ,q (2) is the intra-class weight of the minority and majority class samples, f (1) , f (2) is the inter-class weight of minority class and majority class samples; c1, c2>0 is the penalty parameter; By introducing an additional regularization term ‖w‖ 2 +b 2 , a structural risk minimization version is proposed; then, the original problem of LIFSVM is obtained Where ξ is a non-negative slack variable; c i ,i=1…4 is the penalty parameter; For the convenience of expression, the vector form of the above samples (Equation (13) and Equation (14)) is converted into the following matrix form: Among them, Equations (15) and (16) are the optimization problems for finding the proximal hyperplane of abnormal behavior and normal behavior, respectively; Q1 and Q2 are diagonal matrices, whose main diagonals are the intra-class weights of the minority class and the majority class, Equations (4) and (5); F1 and F2 are diagonal matrices, whose main diagonals are the inter-class weights of the minority class and the majority class, Equations (8) and (9), which are defined as follows: Used to control the influence of samples within or between classes; e1, e2 are unit vectors of appropriate dimensions; The optimization problem of the minority class (Equation (15)) is analyzed. The first term of the objective function is minimized to make the distance between the minority class samples and the first hyperplane closer, that is, the samples are gathered as close to the hyperplane as possible. The second term of the objective function is the square loss function of the slack variable. The second constraint condition requires that the distance between the majority class samples and the first hyperplane is at least 1. The slack variable measures the error when the constraint condition is not met, that is, minimizing the first two terms, aiming to maximize the separation of the majority class samples from the minority class samples. The last term is the regularization term, in order to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting. Similarly, the optimization problem (16) is similar to (15). Step 5: Solve the two optimization problems in step 4; (5.1) Taking the optimization problem (15) as an example, first substitute the equality constraint into the objective function, and we can get (5.2) Take the gradient of (18) with respect to w1 and b1, and set the gradient equal to 0, and we get (5.3) According to equations (19) and (20), solve have to Where H1=rQ1[A e1], G1=F2[B e2], and I represents the identity matrix; (5.4) Similarly, solve have to Where H2 = F1[A e1], G2 = Q2[B e2]; at this point, the training phase of the LIFSVM method is completed; Step 6: Identify abnormal behavior; after obtaining the optimal model parameters, construct an abnormal behavior prediction decision function; for a given video image to be detected, first use Opencv to capture the video stream in real time, then use the mediapipe open source module to extract the key points of the human body posture; finally, use these key points to generate the human body posture feature x; for a given feature x, calculate the distance between the feature x and the proximal hyperplane (10) of abnormal behavior and normal behavior respectively to predict whether the scene is abnormal behavior. The decision function is as follows: Where |·| represents the absolute value. If the distance between feature x and the proximal hyperplane of abnormal behavior is closer than that of normal behavior, then the output of formula (23) is 1, and the system determines that the current sample x is abnormal behavior and performs abnormal behavior recording or alarm. If the distance between feature x and the proximal hyperplane of normal behavior is closer than that of abnormal behavior, then the output of formula (23) is 2, and the system determines that the current sample x is normal behavior.

Images