A system anomaly detection method based on human-computer trust bidirectional detection

Abstract

The application discloses a system anomaly detection method based on human-computer trust bidirectional detection. The method measures actual operation precision by a machine. The method calculates operator theoretical operation precision through historical data. The actual operation precision obtained in the S1 step is subjected to K-S test to calculate the probability that the actual operation precision conforms to normal distribution, and the probability that conforms to normal distribution is compared with a specified test level to judge whether the system exists operator malicious operation / attack. The probability that the actual operation qualified rate is lower or higher than the theoretical operation qualified rate is compared with the specified test level to identify the system anomaly type. The method has high calculation efficiency and can effectively identify fault types.

Description

Technical Field

[0001] This invention relates to the field of human-machine trust, and in particular to a system anomaly detection method based on bidirectional human-machine trust detection. Background Technology

[0002] In recent years, industrial system safety incidents have occurred frequently, with functional safety incidents happening from time to time. Functional safety in critical manufacturing industries requires continuous attention. Critical infrastructure in the industrial manufacturing sector is of paramount importance for cybersecurity; control commands are easily maliciously altered, and operational states can be randomly disrupted. Once a problem occurs, it can have immense destructive and lethal power, impacting production safety in industrial systems for a long time. Data injection and image replay attacks via the internet into critical links of industrial systems are common attack methods.

[0003] Common trust assessment methods based on system capabilities emphasize a high level of objective trustworthiness within the system, suggesting it's not easily attacked. Therefore, existing industrial systems often focus on the potential for malicious attacks by operators, neglecting the actual possibility of system attacks. Operators are prone to over-trusting and misusing trust; excessive trust can lead to difficulty in timely detection and reporting of attacks, while insufficient trust can result in incomplete use of system functions. Conventional methods for determining whether a system has been maliciously attacked rely heavily on subjective judgment, making them unsuitable for typical real-world scenarios.

[0004] Therefore, how to use a two-way detection method to simultaneously determine whether the system has been attacked and whether the operator has engaged in malicious operation can effectively address the security risks in industrial systems and the challenges of safe production for operators, which has great practical significance. Summary of the Invention

[0005] Therefore, the technical problem this invention aims to solve is estimating the type of system anomaly when there is a discrepancy between the operator's subjective trust level and the system's capability. This addresses the limitations of methods based solely on system capability and subjective trust levels, and thus proposes a system anomaly identification method based on bidirectional human-machine trust monitoring. This method determines the fault type based on the relationship between the probability of a specific event and the magnitude of a predetermined test level, thus mitigating potential dangers in industrial system safety.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A system anomaly detection method based on human-machine trust bidirectional detection, comprising the following steps:

[0007] S1: Machine evaluation of actual operating accuracy: Based on the machine measurement results, the operating accuracy of the actual product is measured, and the actual operating accuracy and average operating pass probability are calculated;

[0008] S2: Theoretical probability of operator qualification: Calculate the theoretical probability of operator qualification based on historical data of operators or the average ability of all personnel.

[0009] S3: Calculate the probability that the actual operation accuracy obtained in step S1 conforms to a normal distribution using the KS test, and compare the probability of conforming to a normal distribution with the specified test level to determine whether the system has been maliciously operated by the operator or attacked. If not, the detection of this operation ends; if it does, proceed to the next step.

[0010] S4: Calculate the probability that the actual operation pass rate is lower or higher than the theoretical operation pass rate by using the average operation pass rate, the number of operations, and the operator's theoretical operation pass rate; compare the probability that the actual operation pass rate is lower or higher than the theoretical operation pass rate with the specified test level. When the actual operation pass rate is lower than the specified test level, there is a risk of malicious operation by the operator. When the actual operation pass rate is higher than the specified test level, the system is vulnerable to attack. Report the detection results and end the detection of this operation. Repeat the S1-S4 process for each operation until all operations have been detected.

[0011] Furthermore, in step S1, the machine's evaluation of the actual operational accuracy includes the following steps:

[0012] S11: For average actual operating accuracy Based on the machine measurement results, the operational accuracy of the actual product is measured, and the average operational accuracy is calculated; N represents the number of historical operational data points for the operator, and the machine evaluates the results of N operations, where C i Let be the actual operational precision of the i-th operation; the smaller the value, the more precise the operation. Then, the average actual operational precision... satisfy:

[0013]

[0014] S12: For the actual operating standard deviation σ, the following condition is satisfied:

[0015]

[0016] S13: For the average operational success rate Let η be the acceptable value for operational accuracy. The machine evaluates the results of N operations, where C i For actual operating accuracy, when the operating accuracy C i If the accuracy is ≤η, then the operation is qualified; if the accuracy is C i If the probability is greater than η, then the current operation is unsuccessful; therefore, the average probability of operation success is... satisfy:

[0017]

[0018] in# i ≤η} represents the set {C} i} conforms to C i The number of elements in the condition ≤η.

[0019] Furthermore, in step S2, evaluating the operator's theoretical operational accuracy includes the following steps:

[0020] If the operator has historical operation data, the evaluation of the operator's theoretical operation qualification probability P0 is transformed into the following numerical calculation problem;

[0021]

[0022] Where x i This represents the historical data indicating whether the operator performed normally or abnormally during the i-th operation. i =0 indicates that the i-th operation is successful, x i =1 indicates that the i-th operation is unqualified, which is obtained by recording the operation results where no system anomalies occur; the qualified value of operation accuracy is η, N is the number of historical operation data of the operator; P0 represents the theoretical probability of the operator's operation being qualified;

[0023] If the operator has no historical operation data, the theoretical operation qualification probability of the operator will be directly evaluated as the average of the historical operation qualification probabilities of all operators.

[0024] Furthermore, in step S3, determining whether there is malicious operator operation / system attack through KS verification includes the following steps:

[0025] S31: Average actual operational accuracy obtained in step S1 The actual standard deviation σ is used to calculate the theoretical normal distribution. And the theoretical discrete frequency distribution g(x) corresponding to the theoretical normal distribution;

[0026] S32: The magnitude of the actual operational accuracy C obtained in step S1 i Calculate the actual discrete frequency distribution f(x) obtained from the actual operation;

[0027] S33: Calculate the maximum distance D between the theoretical discrete frequency distribution g(x) and the actual discrete frequency distribution f(x):

[0028] D = maX|f(x) - g(x)|

[0029] S34: According to the theoretical normal distribution Calculate the upper bound of the distance that conforms to the normal distribution at the specified test level α = 0.01, which is D(N,α); if D ≤ D(N,α), then the actual operation conforms to the normal distribution; if D > D(N,α), then the actual operation does not conform to the normal distribution.

[0030] S35: When the actual operation conforms to a normal distribution, there is no risk of malicious operation by the operator or attack on the system; when the actual operation does not conform to a normal distribution, there is a risk of malicious operation by the operator or attack on the system.

[0031] Furthermore, the specific operations in step S4 are as follows:

[0032] S41: Average operational success probability Given the number of operations N, the summation index k, the theoretical probability of passing the operation P0, and the probability P1 that the actual pass rate is lower than the theoretical pass rate:

[0033]

[0034] S42: Average operational pass probability Given the number of operations N, the summation index k, the theoretical probability of passing the operation P0, and the probability P2 that the actual pass rate is higher than the theoretical pass rate:

[0035]

[0036] S43: For a specified test level α = 0.01, if P1 ≤ α, the probability that the actual operation pass rate is lower than the theoretical operation pass rate is less than the specified test level; if P2 ≤ α, the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level.

[0037] S44: Malicious operation is defined as an operator's abnormal operation with an excessively low pass rate. When the pass rate obtained in step S3 does not conform to a normal distribution, and the probability that the actual pass rate is lower than the theoretical pass rate is less than the specified test level, it is determined that a malicious operation has occurred, and it is recorded and reported in the system for further verification.

[0038] S45: The definition of a system failure is the existence of a systematic deviation. When the operation pass rate obtained in step S3 does not conform to the normal distribution, and the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level, it is determined that a system failure has occurred, and it is recorded and reported in the system for further verification. Each operation is judged until all operations are tested.

[0039] The beneficial effects of this invention are as follows: This invention uses the relationship between the probability conforming to a normal distribution and the actual operation pass rate being lower / higher than the theoretical operation pass rate and the specified test level to evaluate system anomalies, avoiding the neglect of attacks such as malicious tampering of system information due to excessive trust, while effectively identifying malicious operations by operators, thus providing bidirectional protection for the normal operation of the overall system; the Bayesian estimation method is used to calculate the anomaly probability, which is simple, direct, and highly efficient, and can quickly detect abnormal system results and identify the type of system anomaly. Attached Figure Description

[0040] Figure 1 This is a flowchart of a system anomaly detection method based on bidirectional human-machine trust monitoring according to an embodiment of the present invention;

[0041] Figure 2 This is a structural diagram of the field layer and operator layer of a system anomaly detection method based on human-machine trust bidirectional monitoring, according to an embodiment of the present invention. Detailed Implementation

[0042] like Figure 1 As shown, the present invention proposes a system anomaly detection method based on bidirectional human-machine trust monitoring, which includes the following steps:

[0043] S1: Machine Assessment of Actual Operating Accuracy: Based on the machine measurement results, the operating accuracy of the actual product is measured and the actual operating accuracy is calculated; this step specifically includes the following sub-steps:

[0044] S11: For average actual operating accuracy Based on the machine's measurement results, the operational accuracy of the actual product is measured, and the average operational accuracy is calculated. The machine evaluates the results of N operations, where C... i Let represent the actual operational precision of the i-th operation; a smaller value indicates greater precision. Then, the average actual operational precision... satisfy:

[0045]

[0046] S12: For the actual operating standard deviation σ, the following condition is satisfied:

[0047]

[0048] S13: For the average operational success rate Let η be the acceptable value for operational accuracy. The machine evaluates the results of N operations, where C i For actual operating accuracy, when the operating accuracy C i If the accuracy is ≤η, then the operation is qualified; if the accuracy is C i If the probability is greater than η, then the operation is considered unsuccessful. The average probability of a successful operation is then calculated. satisfy:

[0049]

[0050] in# i ≤η} represents the set {C} i} conforms to C i The number of elements in the condition ≤η.

[0051] The system parameters are selected as follows:

[0052] S2: Assess the operator's theoretical operational accuracy. This step includes the following sub-steps:

[0053] If the operator has historical operation data, the assessment of the operator's theoretical probability of passing the operation, P0, is transformed into the following numerical calculation problem.

[0054]

[0055] Where x i x represents the historical data of whether the operator performed normally or abnormally during the i-th operation. i =0 indicates that the i-th operation is successful, x i =1 indicates that the i-th operation is unsuccessful, which is obtained by recording the operation results that did not cause system anomalies; N is the number of historical operation data of this operator; P0 represents the theoretical probability of the operation being successful for this operator;

[0056] If the operator has no historical operation data, the theoretical operation qualification probability of the operator will be evaluated as the average of the historical operation qualification probabilities of all operators.

[0057] The system parameters are selected as follows: M = 100. Assume that the system has the following three cases: P0 = 0.05, 0.1, 0.2.

[0058] S3: A system anomaly detection method based on human-machine trust bidirectional detection, wherein step S3 determines whether there is malicious operation by the operator / system attack through KS verification. This step specifically includes the following sub-steps:

[0059] S31: Average actual operational accuracy obtained in step S1 The actual standard deviation σ is used to calculate the theoretical normal distribution. And the theoretical discrete frequency distribution g(x) corresponding to the theoretical normal distribution;

[0060] S32: The magnitude of the actual operational accuracy C obtained in step S1 i Calculate the actual discrete frequency distribution f(x) obtained from the actual operation;

[0061] S33: Calculate the maximum distance D between the theoretical discrete frequency distribution g(x) and the actual discrete frequency distribution f(x):

[0062] D = maX|f(x) - g(x)|

[0063] S34: According to the theoretical normal distribution Calculate the upper bound of the distance that conforms to the normal distribution at the specified test level α = 0.01, which is D(N,α). If D ≤ D(N,α), the actual operation conforms to the normal distribution; if D > D(N,α), the actual operation does not conform to the normal distribution.

[0064] S35: When the actual operation conforms to a normal distribution, there is no risk of malicious operation by the operator or attack on the system; when the actual operation does not conform to a normal distribution, there is a risk of malicious operation by the operator or attack on the system.

[0065] At the specified test level α = 0.01, when P0 = 0.1, the actual operation accuracy conforms to the normal distribution law, and there is no malicious operation by the operator / the system is attacked. When P0 = 0.05 and 0.2, the actual operation accuracy does not conform to the normal distribution law, and there is a risk of malicious operation by the operator / the system is attacked.

[0066] S4: As Figure 2 As shown, after controlling industrial operating equipment at the field level via the Internet, ransomware attacks are a common cause of industrial system attacks. Data injection and image replay attacks via the Internet overwrite original data with malicious information, writing large amounts of meaningless information into motors, industrial cameras, and industrial robotic arm systems, affecting the normal operation of industrial equipment. From the operator's perspective, malicious operations occur because of operator cognitive problems, which affect decision-making and execution steps, ultimately leading to malicious operations and the inability to use industrial equipment normally. In the overall system, the system and operators originally had a high degree of mutual trust, but due to this excessive trust, operators easily overlook malicious attacks on industrial operating equipment, impacting production safety. Industrial equipment also tends to operate solely according to operator instructions, failing to effectively identify malicious operations. To detect malicious operations or system attacks, the detection scheme in step S4 of this embodiment is as follows:

[0067] S41: The average operational success rate obtained in step S1 Number of operations N, summation index k, theoretical success rate P0 obtained from step S2, and probability P1 that the actual success rate is lower than the theoretical success rate:

[0068]

[0069] S42: Average operational success rate obtained in step S1 Number of operations N, summation index k, theoretical success rate P0 obtained from step S2, and probability P2 that the actual success rate is higher than the theoretical success rate:

[0070]

[0071] S43: At a specified test level α = 0.01, if P1 ≤ α, the probability that the actual operation pass rate is lower than the theoretical operation pass rate is less than the specified test level; if P2 ≤ α, the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level. The system calculates that when P0 = 0.05, P1 = 0.03% < 1%, meaning the probability that the actual operation pass rate is lower than the theoretical operation pass rate is less than the specified test level; and when P0 = 0.2, P2 = 0.08% < 1%, meaning the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level.

[0072] S44: Malicious operation is defined as abnormal operation by the operator with an excessively low pass rate. When the pass rate obtained in step S3 does not conform to a normal distribution, and the probability that the actual pass rate is lower than the theoretical pass rate is less than the specified test level, it is determined that a malicious operation has occurred, and it is recorded and reported in the system for further verification. In the system, P0 = 0.05 indicates malicious operation by the operator.

[0073] S45: A system fault is defined as the existence of a systematic deviation. When the operation pass rate obtained in step S3 does not conform to a normal distribution, and the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level, a system fault is determined to have occurred. This is recorded and reported in the system, and further verification is required. In the system, P0 = 0.2 indicates a system fault. Each operation is judged until all operations are completed.

[0074] In the above technical solution, the detection method based on human-machine two-way trust performs system detection based on operator data and actual operation data. Compared with the judgment method based on subjective trust level and system capability, it can effectively deal with on-site attacks such as malicious tampering and operator problems such as malicious operation.

[0075] The system anomaly detection method based on human-machine trust bidirectional monitoring described in this embodiment does not require a complex system. It estimates the anomaly probability through Bayesian calculation and determines the error type by comparing it with the tolerable error, adapting to the complex situation in industrial sites. This method is simple and direct, has high computational efficiency, and can quickly obtain system anomaly results.

[0076] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A system anomaly detection method based on bidirectional human-machine trust detection, characterized in that, Includes the following steps: S1: Machine evaluation of actual operating accuracy: Based on the machine measurement results, the operating accuracy of the actual product is measured, and the actual operating accuracy and average operating pass probability are calculated; S2: Theoretical probability of operator qualification: Calculate the theoretical probability of operator qualification based on historical data of operators or the average ability of all personnel. S3: Calculate the probability that the actual operational accuracy obtained in step S1 conforms to a normal distribution using the KS test, and compare the probability of conforming to a normal distribution with a specified test level to determine whether the system has been maliciously operated by the operator or attacked. If not, the detection of this operation ends; if it does, proceed to the next step. Step S3, determining whether there is malicious operation by the operator / attack using the KS test, includes the following steps: S31: Average actual operational accuracy obtained in step S1 Actual operating standard deviation The theoretical normal distribution is calculated. and the theoretical discrete frequency distribution corresponding to the theoretical normal distribution. ; S32: The magnitude of the actual operational precision obtained in step S1. The actual discrete frequency distribution obtained from the actual operation is calculated. ; S33: Calculate the theoretical discrete frequency distribution With respect to actual discrete frequency distribution maximum distance S34: According to the theoretical normal distribution Calculate at the specified test level Below, the upper bound of the distance that conforms to a normal distribution is: ;like If the actual operation conforms to a normal distribution; if If so, the actual operation does not conform to a normal distribution; S35: When the actual operation follows a normal distribution, there is no risk of malicious operation by the operator or attack on the system; when the actual operation does not follow a normal distribution, there is a risk of malicious operation by the operator or attack on the system. S4: Calculate the probability that the actual operation pass rate is lower or higher than the theoretical operation pass rate by using the average operation pass rate, the number of operations, and the operator's theoretical operation pass rate; compare the probability that the actual operation pass rate is lower or higher than the theoretical operation pass rate with the specified test level. When the actual operation pass rate is lower than the specified test level, there is a risk of malicious operation by the operator. When the actual operation pass rate is higher than the specified test level, the system is vulnerable to attack. Report the detection results and end the detection of this operation. Repeat the S1-S4 process for each operation until all operations have been detected. In step S4: S41: Average operational success probability Number of operations Summation index Theoretical operational success rate Calculate the probability that the actual operation pass rate is lower than the theoretical operation pass rate. : S42: Average operational pass probability Number of operations Summation index Theoretical operational success rate Calculate the probability that the actual operation pass rate is higher than the theoretical operation pass rate. : S43: For the specified test level ,like The probability that the actual operation pass rate is lower than the theoretical operation pass rate is less than the specified test level; if The probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level. S44: Malicious operation is defined as an operator's abnormal operation with an excessively low pass rate. When the pass rate obtained in step S3 does not conform to a normal distribution, and the probability that the actual pass rate is lower than the theoretical pass rate is less than the specified test level, it is determined that a malicious operation has occurred, and it is recorded and reported in the system for further verification. S45: The definition of a system failure is the existence of a systematic deviation. When the operation pass rate obtained in step S3 does not conform to a normal distribution, and the probability that the actual operation pass rate is higher than the theoretical operation pass rate is less than the specified test level, it is determined that a system failure has occurred, and it is recorded and reported in the system for further verification. Each operation is judged until all operations are completed.

2. The system anomaly detection method based on human-machine trust bidirectional detection according to claim 1, characterized in that, In step S1, the machine's assessment of actual operational accuracy includes the following steps: S11: For average actual operating accuracy Based on the machine measurement results, the operational accuracy of the actual product is measured, and the average operational accuracy is calculated. The machine evaluated the total amount of historical operation data for this operator. The result of this operation, among which For the first The smaller the value of the actual operational precision of each operation, the more precise the operation; therefore, the average actual operational precision... satisfy: S12: For the actual operating standard deviation satisfy: S13: For the average operation qualification probability , define the qualified value of the operation accuracy as . The machine evaluates the results of operations in total, where is the actual operation accuracy. When the operation accuracy , the operation of that time is qualified; when the operation accuracy , the operation of that time is unqualified. Then the average operation qualification probability satisfies: in Represents a set China conforms The number of elements in the condition.

3. The system anomaly detection method based on human-machine trust bidirectional detection according to claim 1, characterized in that, In step S2, assessing the operator's theoretical operational accuracy includes the following steps: If the operator has historical operational data, the operator's theoretical operational qualification probability will be assessed. This can be transformed into the following numerical calculation problem; in For the operator in the Normal or abnormal historical data during this operation. Indicates the first The operation was successful. Indicates the first The operation failed, and the result was obtained by recording the operation results where no system anomalies occurred; the acceptable value for operation accuracy was... , This represents the number of historical operation data points for this operator. This indicates the theoretical probability that the operator will be qualified in performing the operation. If the operator has no historical operation data, the theoretical operation qualification probability of the operator will be directly evaluated as the average of the historical operation qualification probabilities of all operators.

Images