Big data artificial intelligence monitoring system

Abstract

The invention discloses a big data artificial intelligence monitoring system. The system comprises a data acquisition module for acquiring enterprise data in real time; the data processing module is used for preprocessing the enterprise data acquired by the data acquisition module; the data storage module is used for storing the preprocessed data and carrying out encryption protection on the stored data through a data encryption module; the data encryption module is used for encrypting the stored and transmitted data through key generation and data encryption; and the real-time monitoring module is used for constructing a GAN-SVM model to automatically identify an abnormal behavior mode by considering the access authority, the data sensitivity degree, the employee background, the demission tendency, the accident history, the social network relationship and the work performance of the employees, judging the abnormal behaviors of the employees according to the GAN-SVM model, sending early warning information to a security administrator when the abnormal behaviors are detected, and sending the early warning information to the security administrator. The automatic response rule is executed; and the user interaction module is responsible for providing an interaction interface and establishing a user feedback mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and in particular to a big data artificial intelligence monitoring system. Background Art

[0002] Big data is a new concept that emerged along with the explosive growth of information data and the rapid development of network computing technology. It has four characteristics: a huge data scale, rapid data flow, diverse data types, and a low value density. Artificial intelligence refers to a new technical science that studies, develops, and applies theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. With the accelerating digital transformation, all industries are actively exploring the paths and methods of digital transformation. As an important part of digital transformation, the big data artificial intelligence monitoring system is playing an increasingly important role. It can help enterprises achieve goals such as data-driven decision-making, improved operational efficiency, and cost reduction.

[0003] After retrieval, a Chinese invention patent with the patent number CN118282759A discloses a big data network real-time monitoring system and a monitoring method, belonging to the technical field of data monitoring. It includes: a data processing module for preprocessing the data in an enterprise; a data storage module for storing the processed data; a data encryption module including a key generation unit and a data encryption unit, where the key generation unit is used to generate the key required for encryption, and the data encryption unit is used to encrypt the data in the data storage module using the key; an internal threat detection module; and a real-time monitoring module. By setting up an internal threat detection module, it is possible to monitor the employees within the enterprise and avoid the internal employees of the enterprise or organization from becoming difficult-to-monitor objects due to their understanding of the internal system and access rights, and to formulate security policies more carefully and deeply.

[0004] However, during the use of the above patent, internal threats are not only related to the behavior patterns of employees but also involve multiple aspects such as the social network relationships and work performance of employees. This system only focuses on the behavior data of employees and ignores these other important factors, resulting in misidentifying the potential threats of employees. Therefore, a big data artificial intelligence monitoring system is proposed. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem existing in the prior art that the prior art only focuses on the behavior data of employees and ignores these other important factors, resulting in misidentifying the potential threats of employees, and to propose a big data artificial intelligence monitoring system.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0007] A big data artificial intelligence monitoring system, comprising:

[0008] A data collection module: collecting enterprise data in real time;

[0009] A data processing module: preprocessing the enterprise data collected by the data collection module;

[0010] A data storage module: storing the preprocessed data and encrypting and protecting the stored data through a data encryption module;

[0011] A data encryption module: encrypting the stored and transmitted data through key generation and data encryption;

[0012] A real-time monitoring module: considering employees' access rights, data sensitivity levels, employee backgrounds, turnover tendencies, accident histories, social network relationships, and work performance, constructing a GAN-SVM model to automatically identify abnormal behavior patterns, judging employees' abnormal behaviors according to the GAN-SVM model, and when detecting abnormal behaviors, sending warning messages to security administrators and executing automated response rules;

[0013] A user interaction module: responsible for providing an interaction interface and establishing a user feedback mechanism.

[0014] The above technical solution further includes:

[0015] Further, the data processing module preprocesses the enterprise data collected by the data collection module, and the preprocessing includes data cleaning, format conversion, and data standardization. Data cleaning is the first step of data preprocessing, aiming to identify and correct errors, anomalies, or duplicate data in the dataset. Format conversion is to convert the data into a format suitable for the GAN-SVM model. Data standardization is the process of converting the data into the same scale or range.

[0016] Further, the data cleaning includes missing value processing, outlier processing, and duplicate value processing. Missing value processing checks for missing values in the dataset and selects filling (such as mean filling, median filling, mode filling, or filling based on context inference) or deletion according to the actual situation. If a column has a large number of missing values and the data in that column is normally distributed, mean filling can be selected. Outlier processing identifies outliers through the Z-score and selects retention, correction, or deletion according to the actual situation. The Z-score calculation formula is Z = (X - μ) / σ, where X is the data value, μ is the mean, and σ is the standard deviation. If |Z| is greater than a certain threshold (such as 3), it may be regarded as an outlier. Duplicate value processing checks for duplicate values in the dataset and selects retaining unique values or deleting duplicate values according to the actual situation. If a row of data is completely duplicate, the duplicate row can be selected for deletion.

[0017] Further, the data encryption module includes a key generation unit and a data encryption unit. The key generation unit generates keys for data encryption and decryption, which are crucial for ensuring data security. The data encryption unit is responsible for encrypting the stored and transmitted data using the keys generated by the key generation unit. During the encryption process, the data encryption unit reads the data to be encrypted and encrypts the data according to the selected encryption algorithm and key. The encrypted data will be securely stored or transmitted to ensure that it cannot be accessed or tampered with by unauthorized personnel.

[0018] Further, the real-time monitoring module includes a feature extraction unit, a GAN-SVM model unit, a warning information sending unit, and an automated response rule execution unit. The feature extraction unit extracts data that can reflect the employee's behavior characteristics based on the employee's access rights, data sensitivity, employee background, tendency to leave the job, accident history, social network relationships, and work performance. The feature extraction unit receives the data from the data storage module, extracts the features, and then passes the data to the GAN-SVM model. The GAN-SVM model unit uses the combination of a generative adversarial network (GAN) and a support vector machine (SVM) to construct a GAN-SVM model to automatically identify abnormal behavior patterns. The GAN-SVM model unit receives the data from the feature extraction unit, runs the model, and outputs a judgment result (i.e., whether the employee's behavior is abnormal). When the GAN-SVM model determines that the employee's behavior is abnormal, the warning information sending unit sends a warning message to the security administrator. The warning information sending unit receives the judgment result of the GAN-SVM model unit and sends a warning message according to the result. The automated response rule execution unit executes corresponding security operations according to the preset automated response rules (such as automatically isolating abnormal devices, blocking malicious IPs, etc.). The automated response rule execution unit receives the judgment result of the GAN-SVM model unit or the trigger signal of the warning information sending unit and executes the corresponding automated response rule.

[0019] Further, the specific steps for the GAN-SVM model unit to construct a GAN-SVM model using the combination of a generative adversarial network (GAN) and a support vector machine (SVM) are as follows:

[0020] Data generation and enhancement: Using the generator of the GAN, realistic simulated data is generated according to the data of normal behavior patterns. By adjusting the parameters of the GAN generator, normal behavior pattern data in different scenarios is generated, thus enriching the training sample set;

[0021] Feature Extraction and Selection: Extract features from the generated simulated data and real data, where the features cover employees' access rights, data sensitivity levels, employee backgrounds, propensity to leave, and accident history, and use Principal Component Analysis (PCA) to screen out the features that are most crucial for distinguishing normal behavior from abnormal behavior;

[0022] SVM Classifier Training: Input the extracted features into the SVM classifier for training, and optimize the performance of the classifier by adjusting the kernel function of SVM (such as linear kernel, Gaussian kernel, etc.) and regularization parameters;

[0023] Model Evaluation and Optimization: Use cross-validation to evaluate the performance of the model, ensure that the model can also maintain good recognition ability on unseen data, and adjust the parameters of GAN and SVM according to the evaluation results to further optimize the performance of the model;

[0024] Comprehensive Evaluation of Internal Threat Value: After the model identifies abnormal behavior, comprehensively consider the employee's social network relationships and work performance to calculate the internal threat value, and judge whether the employee has potential threats by setting a threshold.

[0025] Furthermore, in the feature extraction and selection, the specific steps of using Principal Component Analysis (PCA) to screen out the features that are most crucial for distinguishing normal behavior from abnormal behavior are as follows:

[0026] Set the data after digitization as M samples {X 1 , X 2 ,..., X M}, and each sample has N-dimensional features Each feature X j has its own eigenvalue;

[0027] First, centralize all features, that is, remove the mean value, calculate the mean value of each feature, and then for all samples, each feature subtracts its own mean value, where the respective mean values are After centralization, calculate the covariance matrix where the diagonal elements are the variances of features X 1 and X 2 respectively, and the non-diagonal elements are the covariances. The calculation formula for cov(X 1 , X 1 ) is Thus, obtain the covariance matrix C of the M samples under these N-dimensional features;

[0028] After obtaining the covariance matrix, its eigenvalues and their corresponding eigenvectors are calculated according to the characteristic equation \(C\mu=\lambda\mu\), where \(\lambda\) is the eigenvalue and \(\mu\) is its corresponding eigenvector. The largest first \(k\) eigenvalues and their corresponding eigenvectors are selected for projection, and the projection is the process of dimensionality reduction, which reduces the original features from high dimensions to low dimensions while retaining the main information of the data.

[0029] Further, in the comprehensive evaluation of the internal threat value, the social network relationship and work performance of employees are comprehensively considered to calculate the internal threat value. The specific steps are as follows:

[0030] Consider the social network relationship of employees:

[0031] Data collection: Collect relevant information of employees in the enterprise internal social network, such as which colleagues they communicate with frequently, communication content, interaction frequency, etc. This information can be obtained through the enterprise internal social network platform, email system, instant messaging tool, etc.;

[0032] Social network analysis: Use node centrality to measure the importance and influence of employees in the social network, and identify the position of employees in the social network, such as whether they are at key nodes, whether they have close connections with other high-risk employees, etc.;

[0033] Quantitative evaluation: According to the analysis results, set a quantitative index for the threat value of employees in terms of social network relationship. For example, a scoring system based on node centrality can be set, where the higher the centrality of an employee, the higher the score, indicating that its potential threat value is also greater;

[0034] Consider the work performance of employees:

[0035] Data collection: Collect work performance data of employees, such as performance scores, work completion status, violation records, etc. This data can be obtained from the enterprise human resource management system, performance appraisal system, project management system, etc.;

[0036] Performance evaluation: Quantitatively evaluate the work performance of employees. For example, measure their work quality and efficiency through performance scores, and identify employees with poor work performance, such as those with low performance scores, frequent work mistakes or violations;

[0037] Risk association: Analyze the association between the work performance of employees and potential threats. For example, employees with poor work performance may be more likely to have dissatisfaction or bad motives, thus increasing the risk of internal threats;

[0038] Calculate the internal threat value:

[0039] Comprehensive assessment: The threat values of employees in terms of social network relationships and work performance are comprehensively evaluated, and the weighted average method is used to calculate the internal threat value. Let the social network relationship threat value be SNS_Threat, the work performance threat value be Perf_Threat, and the internal threat value be Internal_Threat. Weights w1 and w2 are set, where w1 + w2 = 1, to balance the contributions of the two factors in the calculation of the internal threat value. The calculation formula for the internal threat value is expressed as Internal_Threat = w1 * SNS_Threat + w2 * Perf_Threat.

[0040] The present invention has the following beneficial effects:

[0041] In the present invention, considering the access rights of employees, the sensitivity of data, the background of employees, the tendency to leave the job, the accident history, the social network relationship, and the work performance, a GAN-SVM model is constructed to automatically identify abnormal behavior patterns. The system can more accurately identify abnormal behavior patterns. This combination makes full use of the advantages of GAN in data generation and feature extraction, as well as the efficiency of SVM in classification tasks, thereby improving the accuracy of internal threat detection. According to the GAN-SVM model to judge the abnormal behavior of employees, the system can quickly determine whether the evaluation score of employees' access to data is abnormal. Once abnormal behavior is detected, the system will immediately send a warning message to the security administrator so that timely measures can be taken, thereby enhancing the timeliness of the system response. Brief Description of the Drawings

[0042] Figure 1 It is a system block diagram of a big data artificial intelligence monitoring system proposed by the present invention. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0044] Please refer to Figure 1 As shown, the present invention is a big data artificial intelligence monitoring system, including:

[0045] Data acquisition module: Real-time acquisition of enterprise data;

[0046] Data processing module: Preprocess the enterprise data collected by the data acquisition module;

[0047] Data storage module: Stores the preprocessed data and encrypts the stored data through the data encryption module for protection;

[0048] Data encryption module: Encrypts the stored and transmitted data through key generation and data encryption;

[0049] Real-time monitoring module: Considering the access rights of employees, data sensitivity, employee background, tendency to leave the company, accident history, social network relationships, and work performance, constructs a GAN-SVM model to automatically identify abnormal behavior patterns, determines the abnormal behavior of employees based on the GAN-SVM model, and when detecting abnormal behavior, sends a warning message to the security administrator and executes automated response rules;

[0050] User interaction module: Responsible for providing an interactive interface and establishing a user feedback mechanism.

[0051] In one embodiment, for the above data processing module, the data processing module preprocesses the enterprise data collected by the data collection module. The preprocessing includes data cleaning, format conversion, and data standardization. Data cleaning is the first step of data preprocessing, aiming to identify and correct errors, anomalies, or duplicate data in the dataset. Format conversion is to convert the data into a format suitable for the GAN-SVM model. Data standardization is the process of converting the data into the same scale or range.

[0052] In one embodiment, for the above data cleaning, data cleaning includes missing value processing, outlier processing, and duplicate value processing. Missing value processing checks for missing values in the dataset and selects filling (such as mean filling, median filling, mode filling, or filling based on context speculation) or deletion according to the actual situation. If a column has a large number of missing values and the data in that column is normally distributed, mean filling can be selected. Outlier processing identifies outliers through the Z-score and selects to retain, correct, or delete according to the actual situation. The Z-score calculation formula is Z = (X - μ) / σ, where X is the data value, μ is the mean, and σ is the standard deviation. If |Z| is greater than a certain threshold (such as 3), it may be regarded as an outlier. Duplicate value processing checks for duplicate values in the dataset and selects to retain the unique value or delete the duplicate values according to the actual situation. If a row of data is completely duplicate, the duplicate row can be selected for deletion.

[0053] In one embodiment, for the above data encryption module, the data encryption module includes a key generation unit and a data encryption unit. The key generation unit generates keys for data encryption and decryption, which are crucial for ensuring data security. The data encryption unit is responsible for encrypting the stored and transmitted data using the keys generated by the key generation unit. During the encryption process, the data encryption unit reads the data to be encrypted and encrypts the data according to the selected encryption algorithm and key. The encrypted data will be securely stored or transmitted to ensure that it cannot be accessed or tampered with by unauthorized personnel.

[0054] In one embodiment, for the above real-time monitoring module, the real-time monitoring module includes a feature extraction unit, a GAN-SVM model unit, a warning information sending unit, and an automated response rule execution unit. The feature extraction unit extracts data that can reflect the employee's behavior characteristics based on the employee's access rights, data sensitivity, employee background, turnover tendency, accident history, social network relationships, and work performance. The feature extraction unit receives data from the data storage module, extracts features, and then passes the data to the GAN-SVM model. The GAN-SVM model unit uses the combination of a generative adversarial network (GAN) and a support vector machine (SVM) to construct a GAN-SVM model to automatically identify abnormal behavior patterns. The GAN-SVM model unit receives the data from the feature extraction unit, runs the model, and outputs a judgment result (i.e., whether the employee's behavior is abnormal). When the GAN-SVM model determines that the employee's behavior is abnormal, the warning information sending unit sends a warning message to the security administrator. The warning information sending unit receives the judgment result of the GAN-SVM model unit and sends a warning message according to the result. The automated response rule execution unit executes corresponding security operations according to the preset automated response rules (such as automatically isolating abnormal devices, blocking malicious IPs, etc.). The automated response rule execution unit receives the judgment result of the GAN-SVM model unit or the trigger signal of the warning information sending unit and executes the corresponding automated response rules.

[0055] In one embodiment, for the above GAN-SVM model unit, the specific steps for the GAN-SVM model unit to construct a GAN-SVM model using the combination of a generative adversarial network (GAN) and a support vector machine (SVM) are as follows:

[0056] Data generation and enhancement: Using the generator of the GAN, realistic simulated data is generated according to the data of normal behavior patterns. By adjusting the parameters of the GAN generator, normal behavior pattern data in different scenarios is generated, thereby enriching the training sample set;

[0057] Feature Extraction and Selection: Extract features from the generated simulated data and real data. The features cover employees' access rights, data sensitivity levels, employee backgrounds, turnover tendencies, and accident histories. Use Principal Component Analysis (PCA) to screen out the features that are most crucial for distinguishing normal behavior from abnormal behavior;

[0058] SVM Classifier Training: Input the extracted features into the SVM classifier for training. Optimize the performance of the classifier by adjusting the kernel function (such as linear kernel, Gaussian kernel, etc.) and regularization parameters of the SVM;

[0059] Model Evaluation and Optimization: Use cross-validation to evaluate the performance of the model, ensuring that the model can also maintain good recognition ability on unseen data. Adjust the parameters of the GAN and SVM according to the evaluation results to further optimize the performance of the model;

[0060] Comprehensive Evaluation of the Internal Threat Value: After the model identifies abnormal behavior, comprehensively consider the employee's social network relationships and work performance to calculate the internal threat value. By setting a threshold, determine whether the employee poses a potential threat.

[0061] In one embodiment, for the above feature extraction and selection, the specific steps of using Principal Component Analysis (PCA) to screen out the features that are most crucial for distinguishing normal behavior from abnormal behavior in feature extraction and selection are as follows:

[0062] Set the digitized data as M samples {X 1 , X 2 ,..., X M}, and each sample has N-dimensional features Each feature X j has its own eigenvalue;

[0063] First, centralize all features, that is, remove the mean. Calculate the mean of each feature, and then for all samples, subtract the mean of each feature from itself. The respective means are After centralization, calculate the covariance matrix where the diagonal elements are the variances of features X 1 and X 2 , and the non-diagonal elements are the covariances. The calculation formula for cov(X 1 , X 1 ) is Thus, obtain the covariance matrix C of the M samples under these N-dimensional features;

[0064] After obtaining the covariance matrix, its eigenvalues and the corresponding eigenvectors are calculated according to the characteristic equation \(C\mu=\lambda\mu\), where \(\lambda\) is the eigenvalue and \(\mu\) is the corresponding eigenvector. The largest \(k\) eigenvalues and the corresponding eigenvectors are selected for projection. The projection is the process of dimensionality reduction, which reduces the original features from a high dimension to a low dimension while retaining the main information of the data;

[0065] Suppose there is a two-dimensional data set containing 5 samples, and each sample has two features. The data is as follows:

[0066]

[0067]

[0068] Data preprocessing: Calculate the means of Feature 1 and Feature 2, and subtract the corresponding feature values of each sample respectively to obtain the centralized data.

[0069] Calculate the covariance matrix: According to the covariance formula, calculate the covariance between Feature 1 and Feature 2 to obtain the covariance matrix.

[0070] Perform eigenvalue decomposition on the covariance matrix: Solve the eigenvalues and eigenvectors of the covariance matrix.

[0071] Select the principal components: According to the magnitudes of the eigenvalues, select the eigenvectors corresponding to the larger eigenvalues as the principal components. In this example, assume that the first principal component (the one with the largest eigenvalue) is selected.

[0072] Data transformation: Project the original data onto the eigenvector corresponding to the selected principal component to obtain the data after dimensionality reduction.

[0073] Through the above steps, principal component analysis is used to perform dimensionality reduction and feature extraction on the data, thereby screening out the features that are most critical for distinguishing normal behavior from abnormal behavior. It should be noted that in practical applications, the number of principal components selected can be determined according to the magnitudes of the eigenvalues and the cumulative variance contribution rate.

[0074] In one embodiment, for the above comprehensive evaluation of the internal threat value, in the comprehensive evaluation of the internal threat value, the social network relationships and work performance of employees are comprehensively considered to calculate the internal threat value. The specific steps are as follows:

[0075] Consider the social network relationships of employees:

[0076] Data collection: Collect relevant information of employees in the enterprise's internal social network, such as which colleagues they communicate with frequently, the content of communication, the interaction frequency, etc. This information can be obtained through the enterprise's internal social network platform, email system, instant messaging tool, etc.;

[0077] Social network analysis: Use node centrality to measure the importance and influence of employees in the social network, and identify the positions of employees in the social network, such as whether they are at key nodes, whether they have close connections with other high-risk employees, etc.;

[0078] Quantitative assessment: Based on the analysis results, set a quantitative index for the threat value of employees in terms of social network relationships. For example, a scoring system based on node centrality can be set, where employees with higher centrality get higher scores, indicating greater potential threat values;

[0079] Consider employees' job performance:

[0080] Data collection: Collect data on employees' job performance, such as performance ratings, work completion, violation records, etc. These data can be obtained from the enterprise's human resource management system, performance appraisal system, project management system, etc.;

[0081] Performance evaluation: Quantitatively evaluate employees' job performance. For example, measure their work quality and efficiency through performance ratings, and identify employees with poor job performance, such as those with low performance ratings, frequent work mistakes or violations;

[0082] Risk association: Analyze the association between employees' job performance and potential threats. For example, employees with poor job performance may be more likely to have dissatisfaction or bad motives, thus increasing the risk of internal threats;

[0083] Calculate the internal threat value:

[0084] Comprehensive evaluation: Comprehensively evaluate the threat values of employees in terms of social network relationships and job performance, and use the weighted average method to calculate the internal threat value. Let the threat value of social network relationships be SNS_Threat, the threat value of job performance be Perf_Threat, and the internal threat value be Internal_Threat. Set weights w1 and w2, where w1 + w2 = 1, to balance the contributions of the two factors in the calculation of the internal threat value. The calculation formula for the internal threat value is expressed as Internal_Threat = w1 * SNS_Threat + w2 * Perf_Threat.

[0085] Suppose the threat value SNS_Threat of an employee in terms of social network relationships is 0.6 (score based on node centrality), and the threat value Perf_Threat in terms of job performance is 0.4 (quantitative evaluation based on performance ratings).

[0086] Set weights w1 = 0.6 and w2 = 0.4, indicating that social network relationships and job performance account for 60% and 40% of the weights respectively in the calculation of the internal threat value.

[0087] Calculate the internal threat value according to the formula: Internal_Threat = 0.6 * 0.6 + 0.4 * 0.4 = 0.36 + 0.16 = 0.52.

[0088] Therefore, the internal threat value of this employee is 0.52, indicating that there is a certain potential threat.

[0089] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A big data artificial intelligence monitoring system, characterized in that: include: Data collection module: real-time collection of enterprise data; Data processing module: pre-process the enterprise data collected by the data collection module; Data storage module: stores pre-processed data and encrypts and protects the stored data through the data encryption module; Data encryption module: encrypts the stored and transmitted data through key generation and data encryption; Real-time monitoring module: Considering employees' access rights, data sensitivity, employee background, turnover tendency, accident history, social network relationships, and work performance, a GAN-SVM model is constructed to automatically identify abnormal behavior patterns. Employees' abnormal behavior is judged based on the GAN-SVM model. When abnormal behavior is detected, an early warning message is sent to the security administrator, and automated response rules are executed; User interaction module: responsible for providing interactive interface and establishing user feedback mechanism.

2. A big data artificial intelligence monitoring system according to claim 1, characterized in that: The data processing module preprocesses the enterprise data collected by the data collection module. The preprocessing includes data cleaning, format conversion and data standardization. The data cleaning is the first step of data preprocessing, which aims to identify and correct errors, anomalies or duplicate data in the data set. The format conversion is to convert the data into a format suitable for the GAN-SVM model. The data standardization is the process of converting the data to the same scale or range.

3. A big data artificial intelligence monitoring system according to claim 2, characterized in that: The data cleaning includes missing value processing, outlier processing and duplicate value processing. The missing value processing checks the missing values ​​in the data set and chooses to fill or delete according to the actual situation. The outlier processing identifies outliers through Z-score and chooses to retain, correct or delete according to the actual situation. The Z-score calculation formula is Z=(X-μ) / σ, where X is the data value, μ is the mean, and σ is the standard deviation. If |Z| is greater than a certain threshold, it may be regarded as an outlier. The duplicate value processing checks the duplicate values ​​in the data set and chooses to retain unique values ​​or delete duplicate values ​​according to the actual situation.

4. A big data artificial intelligence monitoring system according to claim 1, characterized in that: The data encryption module includes a key generation unit and a data encryption unit. The key generation unit generates keys for data encryption and decryption. The data encryption unit is responsible for encrypting the stored and transmitted data using the keys generated by the key generation unit. During the encryption process, the data encryption unit reads the data to be encrypted and encrypts the data.

5. The big data artificial intelligence monitoring system according to claim 1 is characterized in that: The real-time monitoring module includes a feature extraction unit, a GAN-SVM model unit, an early warning information sending unit and an automatic response rule execution unit. The feature extraction unit extracts data that can reflect the employee's behavior characteristics according to the employee's access rights, data sensitivity, employee background, resignation tendency, accident history, social network relationship and work performance. The feature extraction unit receives data from the data storage module, and passes the data to the GAN-SVM model after extracting the features. The GAN-SVM model unit uses the combination of a generative adversarial network and a support vector machine to construct a GAN-SVM model to automatically identify abnormal behavior patterns. The GAN-SVM model unit receives the data from the feature extraction unit, runs the model and outputs a judgment result. When the GAN-SVM model judges that the employee's behavior is abnormal, the early warning information sending unit sends the early warning information to the security administrator. The early warning information sending unit receives the judgment result of the GAN-SVM model unit and sends the early warning information according to the result. The automatic response rule execution unit performs corresponding security operations according to the preset automatic response rules. The automatic response rule execution unit receives the judgment result of the GAN-SVM model unit or the trigger signal of the early warning information sending unit and executes the corresponding automatic response rules.

6. A big data artificial intelligence monitoring system according to claim 5, characterized in that: The GAN-SVM model unit uses the combination of generative adversarial network and support vector machine to construct the GAN-SVM model in the following specific steps: Data generation and enhancement: Generate realistic simulation data based on normal behavior pattern data using the GAN generator. Generate normal behavior pattern data in different situations by adjusting the GAN generator parameters. Feature extraction and selection: Extract features from the generated simulated data and real data. The features include employee access rights, data sensitivity, employee background, turnover tendency, and accident history. Use principal component analysis (PCA) to select the most critical features for distinguishing normal and abnormal behaviors. SVM classifier training: The extracted features are input into the SVM classifier for training, and the performance of the classifier is optimized by adjusting the kernel function and regularization parameters of the SVM; Model evaluation and optimization: Use cross-validation to evaluate the performance of the model and adjust the parameters of GAN and SVM based on the evaluation results; Comprehensive assessment of internal threat value: After the model identifies abnormal behavior, it calculates the internal threat value by comprehensively considering the employee’s social network relationships and work performance, and determines whether the employee is a potential threat by setting a threshold.

7. A big data artificial intelligence monitoring system according to claim 6, characterized in that: In the feature extraction and selection, the specific steps of using principal component analysis PCA to screen out the most critical features for distinguishing normal behavior from abnormal behavior are as follows: The digitized data is set to M samples {X 1 ,X 2 ,...,X M ,},Each sample has N-dimensional features Each feature X j All have their own eigenvalues; First, all features are decentralized, that is, the mean is removed, and the average value of each feature is calculated. Then, for all samples, each feature is subtracted from its own mean, where the respective means are Find the covariance matrix after decentralization The diagonal lines are the variances of features X1 and X2, and the off-diagonal lines are the covariances. The calculation formula for cov(X1,X1) is This results in the covariance matrix C of the M samples under these N-dimensional features; After obtaining the covariance matrix, its eigenvalues ​​and their corresponding eigenvectors are calculated according to the characteristic equation Cμ=λμ, where λ is the eigenvalue and μ is its corresponding eigenvector. The largest first k eigenvalues ​​and corresponding eigenvectors are selected for projection. Projection is the process of dimensionality reduction, which reduces the original features from high dimensions to low dimensions and retains the main information of the data.

8. A big data artificial intelligence monitoring system according to claim 6, characterized in that: In the comprehensive evaluation of the internal threat value, the internal threat value is calculated by comprehensively considering the employee's social network relationship and work performance. The specific steps are: Consider employees’ social network connections: Data collection: Collect relevant information about employees in the company's internal social network; Social network analysis: Use node centrality to measure the importance and influence of employees in social networks and identify the position of employees in social networks; Quantitative evaluation: Based on the analysis results, a quantitative indicator is set for the threat value of employees in terms of social network relationships; Consider employee performance: Data collection: Collect employee performance data; Performance evaluation: Quantitatively evaluate employees’ work performance and identify employees with poor performance; Risk correlation: Analyze the correlation between employee performance and potential threats; Calculate the insider threat value: Comprehensive assessment: conduct a comprehensive assessment of the threat values ​​of employees in terms of social network relationships and work performance, and use the weighted average method to calculate the internal threat value. Suppose the social network relationship threat value is SNS_Threat, the work performance threat value is Perf_Threat, and the internal threat value is Internal_Threat. Set weights w1 and w2, where w1+w2=1, to balance the contribution of the two factors in the calculation of the internal threat value. The calculation formula for the internal threat value is expressed as Internal_Threat=w1*SNS_Threat+w2*Perf_Threat.

Images