A comprehensive security threat assessment method for power IoT terminal systems

Abstract

The present invention relates to an overall security threat assessment method for an electric power Internet of Things terminal system. For several attack surfaces of the electric power Internet of Things terminal system, vulnerability test agents for each attack surface are generated, and a multi-agent array is formed, and then a value network shared by the multi-agent array is generated; the multi-agent array is trained by fuzzy test scenarios established for several attack surfaces, and the trained multi-agent array is used to perform fuzzy testing on each attack surface to discover vulnerabilities, and the security threat index of each attack surface is calculated according to the vulnerabilities, and the overall security threat index of the electric power Internet of Things terminal system is calculated. The method of the present invention can not only simulate the impact of a single attack surface when launching an attack on the electric power Internet of Things terminal, but also simulate the situation of launching an attack on the electric power Internet of Things terminal from multiple attack surfaces, so that the simulation test is more in line with the various complex forms that may appear in actual attacks, thereby making the overall security assessment more comprehensive and accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system security, and in particular relates to an overall security threat assessment method for a power Internet of Things terminal system. Background Art

[0002] The power Internet of Things (IoT) terminal is an indispensable component of the smart grid, and its security directly affects the stability and reliability of the power system. The power IoT terminal undertakes key tasks such as data collection, monitoring and control. Therefore, ensuring its security can effectively prevent threats such as network attacks, data leakage and system failures, and ensure the overall security and data integrity of the power system. Under the current complex power grid composition and interaction structure, the power IoT terminal is connected to the power network through a variety of devices, so it is also vulnerable to different types of security threats from multiple attack surfaces.

[0003] Fuzz testing is a vulnerability testing method used to locate potential vulnerabilities. Traditional fuzz testing methods rely on random or semi-random inputs, and have low test efficiency and coverage. Since randomly generated input data is often invalid or redundant, it leads to a waste of computing resources and time. In addition, traditional methods are difficult to deeply detect boundary conditions and complex logic vulnerabilities in the code because random inputs cannot cover all possible test paths. In contrast, machine learning-based fuzz testing generates more accurate test cases by analyzing historical data and software behavior patterns, thereby improving test efficiency and coverage. This intelligent generation method can more effectively detect complex vulnerabilities and improve the comprehensiveness and accuracy of testing. By learning system behavior and dynamically adjusting test strategies, machine learning methods significantly improve the efficiency and effectiveness of testing.

[0004] However, the existing fuzz testing methods only test one network communication protocol or a single software, and have poor adaptability to the complex actual application environment of the power IoT terminal. A single test cannot fully test the multiple attack surfaces and different types of security threats to the power IoT terminal. At the same time, a single fuzz test cannot handle heterogeneous data from multiple attack surfaces. This limitation also makes it difficult to find potential combined attack vulnerabilities, which can only be triggered when multiple test cases are simultaneously input into the power IoT terminal system from multiple attack surfaces. As a result, it is impossible to accurately evaluate the overall security threats faced by the complex overall environment of the power IoT terminal. Summary of the invention

[0005] Based on the above-mentioned shortcomings and deficiencies in the prior art, one of the objects of the present invention is to at least solve one or more of the above-mentioned problems in the prior art, or to provide an overall security threat assessment method for a power Internet of Things terminal system that meets one or more of the above-mentioned needs.

[0006] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for overall security threat assessment of a power IoT terminal system, comprising the following steps:

[0008] For several attack surfaces of the power IoT terminal system, vulnerability testing agents are generated for each attack surface, and all agents are combined into a multi-agent array;

[0009] Establish fuzz testing scenarios based on several attack surfaces;

[0010] Generate a policy network for each agent and a value network shared by the multi-agent array;

[0011] Use a multi-agent reinforcement learning algorithm to train a multi-agent array, use the input of the fuzzy test scenario as the state of reinforcement learning, the mutation strategy as the action of reinforcement learning, and the generated anomalies as the reward of reinforcement learning, iterate the policy network of each agent, and use the parameters of the iterated policy network to update the value network;

[0012] Use the trained multi-agent array to perform fuzz testing on several attack surfaces simultaneously to discover vulnerabilities of various threat levels, and calculate the security threat index of each vulnerability based on the threat level of the vulnerability and the attack surface involved;

[0013] The overall security threat index of the power Internet of Things terminal system is calculated based on the security threat index of all vulnerabilities.

[0014] As a preferred implementation, several attack surfaces of the power Internet of Things terminal system include network attack surface, software attack surface, and external service attack surface.

[0015] As a further preferred implementation, for the vulnerability testing agent of the network attack surface, the input of the fuzzy testing scenario is network protocol data; for the vulnerability testing agent of the software attack surface, the input of the fuzzy testing scenario is binary data; for the vulnerability testing agent of the external service attack surface, the input of the fuzzy testing scenario is text data.

[0016] As a preferred implementation, the fuzzy test scenario is established based on the Markov process, the state matrix of the Markov process is set according to the sample length and state sequence length of the fuzzy test scenario, the action matrix is ​​set according to the number of mutation strategies and action sequence length of the fuzzy test scenario, and the reward matrix is ​​set according to the number of threat levels and reward sequence length of the fuzzy test scenario.

[0017] As a further preferred implementation, the threat level number of the fuzzy test scenario is 5, and the various threat levels include:

[0018] No exception trigger, general exception trigger, minor exception trigger, major exception trigger, critical exception trigger.

[0019] As a preferred implementation, the security threat index of each vulnerability is calculated according to the threat level of the vulnerability and the involved attack surface, including:

[0020] Set normalized indicators for different threat levels;

[0021] Calculate the security threat index of each vulnerability to the relevant attack surface based on the relevant attack surface, threat level and normalized index of each vulnerability;

[0022] The overall security threat index is calculated based on the accumulated security threat index of each attack surface.

[0023] As a preferred implementation, the overall security threat index is calculated based on the security threat index accumulated by each attack surface, including:

[0024] Set the correlation strength coefficient between attack surfaces;

[0025] According to the correlation strength coefficient between the attack surfaces, the security threat index of one attack surface is added to the security threat index of another attack surface, and then the overall security threat index is calculated.

[0026] As a preferred implementation, updating the value network includes:

[0027] Perform soft update on the value network, retain most of the parameters of the value network, and use some parameters of the iterated strategy network to replace some parameters of the value network.

[0028] On the other hand, the present invention also provides an electronic device, comprising a processor, a storage medium and a computer program, wherein the computer program is stored in the storage medium, and is characterized in that when the computer program is executed by the processor, it implements any of the above-mentioned methods for assessing the overall security threat of the power Internet of Things terminal system.

[0029] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the overall security threat assessment method for the power Internet of Things terminal system described in any one of the above items is implemented.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention provides a method for overall assessment of security threats of power IoT terminals based on multi-agent reinforcement learning fuzzy testing. Agents are constructed for various attack surfaces of the power IoT terminal respectively, and all agents are organized into a multi-agent array of a shared value network. Fuzzy test scenarios are modeled for attacks that the power IoT terminal may be subjected to, and multi-agent reinforcement learning is performed based on actual input and mutation strategies in the fuzzy test scenarios and anomalies generated during the test process, and the multi-agent array is iteratively trained. The trained multi-agent array is used to perform fuzzy testing on the entire power IoT terminal, so that the trained multi-agent array can not only simulate the impact of a single attack surface when attacking the power IoT terminal, but also simulate the situation of jointly attacking the power IoT terminal from multiple attack surfaces, so that the simulation test is more in line with the various complex forms that may occur in actual attacks, thereby making the overall security assessment more comprehensive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of an overall security threat assessment method for a power Internet of Things terminal system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] Below, the specific embodiments of the present invention are described in detail with reference to the accompanying drawings. The following description with reference to the accompanying drawings is provided to assist in the understanding of the exemplary embodiments of the present invention as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but they are to be regarded as exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Moreover, in order to make the description more clear and concise, detailed descriptions of functions and structures well known in the art will be omitted.

[0035] The present invention provides a method for evaluating the overall security threat of a power IoT terminal system. The flowchart of the embodiment is shown in Figure 1 .

[0036] The present invention first generates multiple intelligent agents through step S1, and constructs a separate policy network for each intelligent agent. Each intelligent agent is respectively set for an attack surface of a power Internet of Things terminal system, and conducts in-depth mining specifically for its given attack surface to ensure that the detection and analysis on the attack surface are more refined and comprehensive. After generating multiple intelligent agents, all intelligent agents are organized into a multi-agent array, and a shared value network is constructed for the multi-agent array composed of all intelligent agents, so that the intelligent agents can be jointly trained. Joint training enables the multi-agent array to reason across multiple related attack surfaces, thereby improving the ability to recognize complex program behaviors, especially improving the ability to recognize joint attacks on multiple attack surfaces, and then more accurately assessing the overall security threats of the power Internet of Things terminal system.

[0037] In this embodiment, there are three types of attack surfaces, namely network, software, and external service attack surfaces.

[0038] The embodiment of the present invention also provides the specific structure of the policy network and the value network. The policy network and the value network both use the same structure of the deep neural network, and use the three-layer perceptron framework. The input of the policy network and the value network are both state matrices with the shape of batch size × state dimension. The weight matrix of the hidden layer 1 is in the shape of state dimension × hidden layer 1 dimension. After the input matrix passes through the hidden layer 1, it is transformed into a matrix with the shape of batch size × hidden layer 1 dimension, and activated using the ReLU activation function; the weight matrix of the hidden layer 2 is in the shape of hidden layer 1 dimension × hidden layer 2 dimension. After the input matrix passes through the hidden layer 2, it is transformed into a matrix with the shape of batch size × hidden layer 2 dimension, and activated using the ReLU activation function; the weight matrix of the output layer is in the shape of hidden layer 2 dimension × action dimension, and the input matrix is ​​transformed into a matrix with the shape of batch size × action dimension after passing through the output layer. Each row in the output matrix is ​​a vector with a dimension equal to the action dimension, and the value of each dimension of the vector is the reward that can be obtained after the input state executes the corresponding action.

[0039] More specifically, this embodiment also provides the construction process of the above-mentioned strategy network and value network:

[0040] Define the shape of the policy network input matrix. Suppose the input of the network is Represents a feature matrix consisting of a batch of state vectors, where represents the batch size, Represents the state dimension.

[0041]

[0042] Define the parameters of hidden layer 1 of the policy network. The weight matrix of hidden layer 1 is , the bias matrix is ,in Represents the dimension of hidden layer 1.

[0043]

[0044] Define the parameters of hidden layer 2 of the policy network. The weight matrix of hidden layer 2 is , the bias matrix is ,in Represents the dimension of hidden layer 2.

[0045]

[0046] Define the policy network output layer parameters. The weight matrix of the output layer is , the bias matrix is , where k represents the action dimension.

[0047]

[0048] Define the value network parameters. The structure of the value network is consistent with the strategy network structure, but it is updated using a separate set of parameters.

[0049]

[0050]

[0051] After establishing a multi-agent array, the present invention establishes a scenario for fuzzy testing the power Internet of Things terminal system through step S2. By modeling the state space, action space, reward space, state transfer function, and reset function of the environment, the fuzzy test scenario is modeled into a multi-agent reinforcement learning environment.

[0052] In the process of establishing the fuzz test scenario, in order to enhance the effect of fuzz testing, this embodiment incorporates the Markov chain into the fuzz test, and identifies the mutation process of the test sample as a Markov process. The Markov chain helps the fuzz test to more intelligently select the next path or input to be tested by analyzing the transition probability between different states of program execution. When each agent mines a specific attack surface, the Markov chain can predict which states (or code paths) are more likely to expose vulnerabilities based on historical test data, thereby guiding the fuzz test to generate more potential inputs. This ability to predict state transitions makes fuzz testing more efficient and reduces unnecessary randomness.

[0053] Specifically, the fuzz testing scenario is established according to the following method:

[0054] Assume the length of the input fuzzy test sample is , the length of the state sequence is , then the state matrix of the environment is OK Matrix of columns ,in Indicates The test samples of time steps, Indicates The first time point of the input sample bit data; let total Variation strategies, action sequence length , then the action matrix of the environment is OK Matrix of columns ,in Indicates The action vector of time steps, Indicates Execute the first The probability of an action, when the Execute the first Action time ,otherwise ; Assume a total of security threat level, the length of the return sequence is , then the reward matrix of the environment is OK Matrix of columns ,in Indicates The reward vector for each time step, Indicates The security threat level of the vulnerability triggered at the moment The probability that The security threat level triggered at this moment is hour, ,otherwise .

[0055] Set up fuzz testing environment The state transfer function is ,set up is the current moment, the state transfer function is based on the current state and the next action to be performed As input, the output is a four-tuple ,in is the input state and action, Indicates the next state, Indicates the reward after executing the action. Suppose the fuzzy test environment The reset function is , the input of the reset function is the current state of the environment , returns the initial state of the environment .

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] The present invention also provides a specific embodiment of the above method. In this embodiment, the maximum length of the fuzzy test sample is 1000, and a total of 6 mutation operators are set, namely, byte / bit level erasure, replacement, insertion, disorder, flip, and copy. Five levels of returns are set, namely, no abnormal trigger, general abnormal trigger, minor abnormal trigger, important abnormal trigger, and critical abnormal trigger. The return values ​​of the five returns are set to 1, 2, 3, 4, and 5 respectively. At the same time, the maximum length of the state sequence is set to 100, and the state matrix of the environment , the action matrix of the environment , environmental return matrix The sizes are 100×1000, 100×6, and 100×5 respectively.

[0063] The format of the test sample of the fuzzy test scenario is set specifically according to the attack surface. In the present embodiment, since the attack surface includes the network attack surface, the software attack surface, and the external service attack surface, the test sample respectively selects the network protocol data corresponding to the network attack surface, the binary data corresponding to the software attack surface, and the text data format corresponding to the external service attack surface.

[0064] Specifically, for network protocol data, each message consists of multiple protocol fields. These fields may include source IP address, destination IP address, port number, protocol type, etc. In order to introduce network protocol data into the reinforcement learning framework, the entire message is modeled as a state vector , which contains the values ​​of all protocol fields. Network protocol messages are usually structured as a set of ordered fields, each of which has a clear meaning (such as source IP, destination IP, etc.), and these fields together define the state of the protocol message. By extracting the value of each field and combining them in order into a state vector, the state of the current message can be fully represented. This state vector It is a natural representation of the state in the reinforcement learning model. The model can evaluate and make decisions on the state of the message based on this vector. Perform fuzz testing on the network protocol. The input test sample is a message of a specific network protocol. , assuming that the message contains protocol fields, the message can be modeled as a state in the reinforcement learning framework, which can be represented by a state vector S( ) is expressed as shown in formula (5):

[0065]

[0066] in, Indicates the first i field values.

[0067] For binary data, each sample is represented as a fixed-length bit string , the bit string consists of multiple bits. The state vector of binary data It is a vector composed of the value of each bit. Binary data is essentially a sequence of bits. The value of each bit (0 or 1) represents the state of the bit position. In the reinforcement learning framework, binary data can be directly modeled as a state vector, where the value of each bit is part of the state vector. This representation is simple and effective, and can accurately capture the low-level features of binary data, which helps the model to perform fine-grained operations and analysis on the data. When performing fuzzy testing on binary data, the input test sample is a fixed-length bit string , assuming that the binary data contains indivual bit , then the binary data can be represented by a state vector S( ) It is expressed as shown in formula (6):

[0068]

[0069] in, Represents binary data i The value of the bit.

[0070] For text data, since text data usually consists of varying numbers of characters, symbols, and phrases, in order to convert text data into a state vector in the reinforcement learning framework, it is necessary to use natural language processing technology to convert the text into a word embedding vector. The word vector representation converted from the text sequence. Text data has high semantic complexity, and the order and combination of characters or phrases will affect its meaning. The word embedding technology in natural language processing captures the semantic relationship between words by mapping text into a continuous vector space. These word embedding vectors can retain the semantic information of the text and convert it into a vector representation of a fixed length. By combining the word embedding vectors of the text sequence into a state vector, the reinforcement learning model can use these vectors to understand and process the semantic structure of the text data. Perform fuzzy testing on text data. Since the text is composed of varying numbers of numbers, symbols, and Chinese and English phrases, it is necessary to perform natural language processing on the text data and convert the text data into word vectors using word embedding. Suppose the input test sample is a text sequence ,, the state vector S( ) directly uses the word vector representation converted from the text sequence, as shown in formula (7):

[0071]

[0072] in, Represents word vector i The value on the dimension, Indicates the length of the word vector.

[0073] Step S3: Use a multi-agent reinforcement learning algorithm to train the agents and multi-agent array generated in step S1. The data used for training is generated through the fuzzy test scenario established in step S2.

[0074] Use the input, mutation action and reward of the above fuzzy test scenario to conduct reinforcement learning for multiple agents. Specifically, suppose the test sample of the fuzzy test input consists of the character set The input state at a certain moment is s i It can be expressed as in s ij For character set A character in .

[0075]

[0076] Model the actions of multi-agent reinforcement learning networks. Assume that there are a total of mutation operators in the fuzzy testing process k , use number 1~ k Indicated by, the action to be performed by the test sample in the current state can be expressed as Indicates that r ij = 1 means in the current state, the next action to be performed is action j ,the remaining .

[0077]

[0078] in, .

[0079] According to the possible harm caused by triggering anomalies during fuzz testing, the types of anomalies are divided into no anomalies, general anomalies, minor anomalies, major anomalies and critical anomalies. After the initial test sample undergoes several mutations, the return value after executing the next mutation in the current state is Indicates that when an exception of the corresponding type is triggered, the dimension corresponding to the exception r ij The value is 1, otherwise it is 0; r none , r normal , r minor , r major , r critical Indicates the return when an exception of the corresponding type is triggered.

[0080]

[0081] Through the above reinforcement learning training settings, the policy network of each agent is updated during the training process. Specifically, the policy network is updated using the following method:

[0082] set up t The state at the moment is s t ,Will s t Input the policy network to get a k Dimension value vector V ( s t ), V ( s t ) i Values ​​on the dimension v i Indicates that the current state is executing i The reward for an action is recorded as the value function Q ( s t , a t ), a t Indicates i The action vector when an action is performed.

[0083]

[0084] Calculate the maximum state action function value for the next step. The next state action function uses the value function to calculate the value vector V ( s t +1 ).

[0085]

[0086]

[0087] Calculate mean square error MSELoss , the mean square error MSELoss As the loss function, after each round of training, the policy network is updated by calculating the loss function and using the policy gradient method.

[0088]

[0089] After the policy network is updated, this embodiment updates the value network according to the updated parameters of the policy network. The value network shared by the multi-agent array enables the sample of the test to train the entire multi-agent array when the fuzzy test triggers a vulnerability through multiple attack surfaces, thereby improving the multi-agent array's ability to identify vulnerabilities triggered by multiple attack surfaces.

[0090] In order to prevent the sudden propagation of errors from causing large fluctuations in the model during training, trapping in a local optimal solution or causing training failure, the value network of this embodiment is iterated using a soft update method. By introducing a smaller update coefficient, the parameters of the value network are gradually updated so that the value network slowly follows the changes in the policy network, thereby significantly improving the stability of the overall training. The gradual adjustment of soft updates helps to reduce the error propagation effect, smoothly guides the model to converge, and avoids over-optimization of the strategy. In addition, soft updates can better adapt to gradual changes in a dynamic environment, allowing the policy network to be optimized on a relatively stable objective function, thereby improving the performance and convergence speed of the reinforcement learning algorithm. The soft update method of the value network is shown below, where Indicates the update rate.

[0091]

[0092] in, are the parameters of the policy network, Parameters of the value network.

[0093] After the multi-agent array training is completed, the present invention performs step S4, using the trained multi-agent array to simultaneously perform fuzzy testing on several attack surfaces to discover vulnerabilities of various threat levels.

[0094] S5. Quantify the threat that each vulnerability poses to the system as a whole, and calculate the security threat index of each vulnerability based on the threat level of the vulnerability and the attack surface involved.

[0095] Specifically, step S5 includes the following steps:

[0096] S501. Define the threat value corresponding to the threat level. Suppose the discovered vulnerabilities can be divided into k Threat level, d i Indicates the threat level is i The normalized threat index of the vulnerability with the highest level.

[0097]

[0098] S502. Calculate the proportion of vulnerabilities of different threat levels discovered by the multi-agent array. These vulnerabilities involve various attack surfaces. For each attack surface, this step will weight the vulnerabilities involved in the attack surface by the normalized threat index of S501, and then accumulate them together to obtain the security threat index of the attack surface.

[0099] If a vulnerability is caused by the combined input of multiple attack surfaces, each attack surface involved in the vulnerability will contribute to the calculation of the security threat index.

[0100] The security threat index of any attack surface is calculated as follows: Indicates that it involves j The threat level of the attack surface is i The number of vulnerabilities, Indicates that it involves j The total number of vulnerabilities on the attack surface is calculated comprehensively to get the j Security Threat Index of Attack Surfaces S j .

[0101]

[0102] in, .

[0103] Since there may be dependencies between some attack surfaces, such as attacks from the network attack surface will affect the hardware or software attack surface at the same time, the method of this embodiment also sets the correlation strength coefficient between the attack surfaces, and adds the security threat index of one attack surface to the security threat index of another attack surface according to the correlation strength coefficient. In this way, the correlation between the attack surfaces is incorporated into the security threat index of the attack surface to improve the accuracy of the overall security threat assessment of the power IoT terminal.

[0104] Specifically, this embodiment sets a correlation matrix in step S503:R ,in R ab Represents the attack surface a and attack surface b The strength of the association between R ab Adjust the security threat index of each attack surface to reflect the potential impact of the associated attack surface.

[0105]

[0106] in R is the number of attack surfaces.

[0107] Finally, in order to obtain the overall threat index of the power Internet of Things system, this embodiment performs step S6 to calculate the overall security threat index of the power Internet of Things terminal system according to the security threat index of each attack surface.

[0108] Specifically, step S6 calculates the overall security threat index based on the security threat indexes of all attack surfaces by weighted summation. S total .

[0109]

[0110] The overall assessment method of security threats to power IoT terminals in the above-mentioned embodiment constructs agents for various attack surfaces of the power IoT terminals respectively, and organizes all agents into a multi-agent array of a shared value network, and performs fuzzy test scenario modeling on attacks that the power IoT terminals may be subjected to, performs multi-agent reinforcement learning based on actual input and mutation strategies in the fuzzy test scenarios and anomalies generated during the test, and iteratively trains the multi-agent array. The trained multi-agent array is used to perform fuzzy testing on the entire power IoT terminal, so that the trained multi-agent array can not only simulate the impact of a single attack surface when attacking the power IoT terminal, but also simulate the situation of jointly attacking the power IoT terminal from multiple attack surfaces, so that the simulation test is more in line with the various complex forms that may occur in actual attacks, thereby making the overall security assessment more comprehensive and accurate.

[0111] The embodiment of the present specification also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes the above Figure 1 The steps in the illustrated embodiment. If the components of the electronic device described above are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0112] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. The present invention is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for overall security threat assessment of a power Internet of Things terminal system, characterized in that: The steps include: For several attack surfaces of the power IoT terminal system, vulnerability testing agents for each attack surface are generated respectively, and all the agents are combined into a multi-agent array; a strategy network for each of the agents is generated, and a value network shared by the multi-agent array is generated; Establishing fuzz testing scenarios according to the several attack surfaces; Using a multi-agent reinforcement learning algorithm to train the multi-agent array, using the input of the fuzzy test scenario as the state of reinforcement learning, the mutation strategy of the fuzzy test scenario as the action of reinforcement learning, and the anomaly generated by the fuzzy test scenario as the reward of reinforcement learning, iterating the policy network of each of the agents, and using the parameters of the iterated policy network to update the value network; Using the trained multi-agent array to simultaneously perform fuzz testing on the several attack surfaces to discover vulnerabilities of various threat levels; Calculate the security threat index of each attack surface based on the threat level of the vulnerability and the attack surface involved; The overall security threat index of the power Internet of Things terminal system is calculated based on the security threat index of each attack surface.

2. The overall security threat assessment method of a power Internet of Things terminal system according to claim 1, characterized in that: The several attack surfaces of the power Internet of Things terminal system include network attack surface, software attack surface, and external service attack surface.

3. The overall security threat assessment method of a power Internet of Things terminal system according to claim 2, characterized in that: For the vulnerability testing agent of the network attack surface, the input of the fuzzy testing scenario is network protocol data; for the vulnerability testing agent of the software attack surface, the input of the fuzzy testing scenario is binary data; for the vulnerability testing agent of the external service attack surface, the input of the fuzzy testing scenario is text data.

4. The overall security threat assessment method of a power Internet of Things terminal system according to claim 1, characterized in that: The fuzzy test scenario is established based on a Markov process, the state matrix of the Markov process is set according to the sample length and state sequence length of the fuzzy test scenario, the action matrix is ​​set according to the number of mutation strategies and action sequence length of the fuzzy test scenario, and the reward matrix is ​​set according to the number of threat levels and reward sequence length of the fuzzy test scenario.

5. The overall security threat assessment method of a power Internet of Things terminal system according to claim 4, characterized in that: The threat level number of the fuzzy test scenario is 5, and the various threat levels include: No exception trigger, general exception trigger, minor exception trigger, major exception trigger, critical exception trigger.

6. The overall security threat assessment method of a power Internet of Things terminal system according to claim 1, characterized in that: The security threat index of each attack surface is calculated according to the threat level of the vulnerability and the involved attack surface, including: Set normalized indicators for different threat levels; Calculate the security threat index of each vulnerability to the relevant attack surface according to the relevant attack surface, threat level and the normalized index of each vulnerability; Calculate the security threat index of each attack surface.

7. The overall security threat assessment method of a power Internet of Things terminal system according to claim 1, characterized in that: The overall security threat index of the power IoT terminal system is calculated according to the security threat index of each attack surface, including: Set the correlation strength coefficient between attack surfaces; According to the correlation strength coefficient between the attack surfaces, the security threat index of one attack surface is added to the security threat index of another attack surface, and then the overall security threat index is calculated.

8. The overall security threat assessment method of a power Internet of Things terminal system according to claim 1, characterized in that: The strategy network and the value network adopt the same structure, including a three-layer perceptron framework.

9. The overall security threat assessment method of a power Internet of Things terminal system according to claim 8, characterized in that: The updating of the value network comprises: The value network is soft-updated, most of the parameters of the value network are retained, and some parameters of the value network are replaced by some parameters of the strategy network after iteration.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the overall security threat assessment method for the power Internet of Things terminal system described in any one of claims 1 to 9 is implemented.

Images