An Internet of Things generator data transmission security management system, method and identification method

Abstract

The present invention discloses a data transmission security management system, method and identification method for an Internet of Things generator, including a security management system body, wherein the security management system body includes a data sending platform, a security management platform and a data receiving platform, wherein the data generating platform establishes a two-way communication connection with the security management platform, the data generating platform collects and organizes the data of the Internet of Things generator and then sends it, the security management platform performs a security level check on the sent data and rejects high-risk data, and the data receiving platform receives the Internet of Things generator data that has passed the security review of the security management platform. The present invention sets up a security management platform, and the data is compared with a security comparison library and then its credit value is determined before transmission, thereby avoiding the transmission of the transmitted power signal from being stolen and interfered with during the transmission process, and even the phenomenon of being maliciously attacked during the data transmission process, thereby ensuring the normal operation of the Internet of Things generator and improving its power generation efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things generators, and in particular to a data transmission security management system, method and identification method for Internet of Things generators. Background Art

[0002] The generator is based on Faraday's law of electromagnetic induction, that is, when a piece of metal, such as a copper wire, moves in a magnetic field, an electric field will be generated inside the metal, and the charge inside the metal will flow along the electric field. If the piece of metal is not a ring, the charge will accumulate at both ends and exist in the form of voltage. When the piece of metal is connected to a loop, and the other parts of the loop except the metal are not in the magnetic field, an electric current is generated, and the generator generates electricity. Therefore, the basic principle of the generator is to fix the permanent magnet or metal and keep the metal or permanent magnet moving continuously. Motion is relative, so the piece of metal that keeps moving in the magnetic field will generate electricity continuously.

[0003] The IoT generator is an electrical device that can automatically monitor electricity demand, adjust power generation and perform optimal power generation. In order to ensure that the power generation of the IoT generator is optimal, one side of the IoT generator is connected to a data transmission device to transmit power demand information and power supply information.

[0004] However, traditional data transmission has the following disadvantages:

[0005] The data transmission of IoT generators on the market is mostly wireless transmission, and the wireless transmission environment is generally in public places, which makes it easy for the transmitted power signals to be stolen and interfered with, or even maliciously attacked during the data transmission process, which causes the IoT generator to stop operating and affects the power generation efficiency of the IoT generator. Summary of the invention

[0006] The purpose of the present invention is to provide an Internet of Things generator data transmission security management system, method and identification method to solve the problem raised in the above background technology that the data transmission of the Internet of Things generators on the market is mostly wireless transmission, and the wireless transmission environment is generally in public places, which makes it easy for the transmitted power signal to be stolen and interfered, and even to be maliciously attacked during the data transmission process, which causes the Internet of Things generator to stop operating and affects the power generation efficiency of the Internet of Things generator.

[0007] To achieve the above object, the present invention provides the following technical solutions: an Internet of Things generator data transmission safety management system, comprising a safety management system body, the safety management system body comprising a data sending platform, a safety management platform and a data receiving platform, the data generation platform establishes a two-way communication connection with the safety management platform, and the safety management platform is connected with the data receiving platform;

[0008] The data generation platform collects and organizes the data of the IoT generator and then sends it. The security management platform verifies the security level of the sent data and rejects high-risk data. The data receiving platform receives the IoT generator data after security review by the security management platform.

[0009] As a preferred technical solution of the present invention, the data sending platform includes a mode setting module, a data acquisition module, a feature extraction module, a data access module and a data service module. The data acquisition module collects warning data, fault data, operation data, power generation data and shutdown data of the Internet of Things generator. The feature extraction module extracts the collected data features of the Internet of Things generator. The data access module connects the data generation platform with the Internet of Things generator. The data service module stores the collected data on the data sending platform for backup. The mode setting module sets the operation of the Internet of Things generator to manual mode, automatic mode and mute mode respectively.

[0010] As a preferred technical solution of the present invention, the security management platform includes a data comparison module, a data analysis module, a data enhancement module, a security comparison library, a data modification module, a data storage module and a data transmission module. The data comparison module compares the data extracted from the warning features sent by the data sending platform with the data transmitted previously. The data analysis module analyzes the credit value of the data to be generated, and returns it to the data sending platform if the credit value is lower than the credit value in the security comparison library. The data enhancement module enhances the safety factor of the data through data analysis. The security comparison library compares and analyzes the data through a credit table. The data modification module compares the data of the data sending platform with the real-time data of the Internet of Things generator and corrects it in time if there is any error. The data storage module stores each data of the data sending platform to facilitate subsequent data comparison and to understand the subtle changes of the Internet of Things generator. The data transmission module transmits the data in the data acquisition platform.

[0011] The invention discloses an identification method for an IoT generator data transmission security management system, which is based on the combination of deep reinforcement learning and ensemble learning for intelligent decision making, security event detection and response abnormal behavior identification. The specific process of the adaptive security strategy is as follows:

[0012] Step 1: Identify the problem and context

[0013] Problem definition: The task of the security management platform is to monitor and process the data sent by the data collection platform to ensure the security and reliability of the data; the specific tasks include data monitoring and data processing;

[0014] Data monitoring: real-time monitoring of data to detect abnormalities or security vulnerabilities;

[0015] Data processing: Based on the monitoring results, abnormal data is processed, such as identifying and marking abnormal data, encrypting or filtering data to ensure system security;

[0016] Environment modeling: The security management platform environment can be modeled as a multi-component environment, each with specific functions and interactions;

[0017] Data comparison module: detects differences between incoming data and historical data;

[0018] Data analysis module: Analyzes the credit value of the data. If it is lower than the credit value in the security comparison library, an exception is returned;

[0019] Data enhancement module: process abnormal data, encrypt or filter data;

[0020] Security comparison library: stores the credit value of historical data for comparison and analysis with current data;

[0021] Data modification module: to modify the data to ensure the accuracy and consistency of the data;

[0022] Data storage module: stores each data for subsequent comparison and analysis;

[0023] Data transmission module: responsible for data transmission, ensuring timely transmission and integrity of data;

[0024] In environmental modeling, the relevant parameter variables can be specified as follows:

[0025] D: Dataset, which represents the data set received from the data collection platform;

[0026] Dt: The data set at time t, which means the data received from the data collection platform at time t;

[0027] A: Action space, which represents the set of actions that the security management platform can perform and the set of data processing operations;

[0028] S: state space, which represents the set of states that the security management platform may be in, including data monitoring state and data processing state;

[0029] R: Reward function, used to evaluate the quality of each action to guide the learning and decision-making of the agent;

[0030] Q(s, a): state-action value function, which represents the long-term reward obtained by performing action a in state s;

[0031] ∈: exploration rate in the greedy strategy, indicating the probability that the agent chooses a random action when making decisions;

[0032] α: learning rate, used to adjust the weight when updating the state-action value function;

[0033] Step 2: Design the intelligent agent: monitor and process data in real time, accurately judge the environmental status, formulate effective decision-making strategies, learn and optimize behavioral strategies, ensure the security and reliability of the system, and ensure that the security management platform can effectively monitor and process data from the data collection platform to ensure the security and reliability of the data;

[0034] 2.1 State space: Ensure that the agent can obtain key environmental information, data comparison results, the credit value of the data analysis module and the safety factor of the data enhancement module, so as to accurately judge the current environmental status; the state space of the agent includes the output status of each component of the security management platform, which is crucial to the decision-making of the agent; the specific state variables are as follows:

[0035] Sc: The output of the data comparison module, indicating the data comparison result, whether there is abnormal data or data tampering;

[0036] Sa: The credit value of the data analysis module, which indicates the credit level of the data and is used to evaluate the credibility of the data;

[0037] Se: the safety factor of the data enhancement module, indicating the safety level of the data after enhancement;

[0038] The state s of the agent can be expressed as s = (Sc, Sa, Se);

[0039] 2.2 Action Space: Ensure that the agent can take appropriate actions to process data, perform data modification operations or data enhancement operations to maintain the security and stability of the system; the action space that the agent can perform covers the operable actions in the security management platform, including data modification and data enhancement; the specific action variables are as follows:

[0040] Am: data modification operation, correcting erroneous data or deleting abnormal data;

[0041] Ae: data enhancement operations, data encryption, data cleaning or data reconstruction;

[0042] The action space A of the agent can be expressed as A = {Am, Ae};

[0043] 2.3 Reward function: Through the design of reward function, the performance of the agent in handling abnormal situations is clearly evaluated, so as to encourage the agent to learn the correct behavior strategy;

[0044] Importance and scope of impact of abnormal data: Different types of abnormal data have different degrees of impact on the security and reliability of the system; different degrees of positive or negative rewards are given according to the importance and scope of impact of abnormal data;

[0045] Time efficiency and resource consumption for handling abnormal data: Handling abnormal data requires different time and computing resources; some abnormal situations may be easier to solve, while others require more time and resources; therefore, time efficiency and resource consumption are considered in the reward function;

[0046] R(s,a)=rpositive·importance weight-rnegative·complexity weight-rtime·time consumption-rresource·resource consumption

[0047] in:

[0048] R(s, a): the reward obtained by the agent for performing action a in state s;

[0049] rpositive: positive reward value when processing abnormal data, which is a large positive number;

[0050] r negative: negative reward value when an abnormal situation is not handled correctly, which is a large negative number;

[0051] Importance weight: indicates the importance and impact range of abnormal data, and is defined and calculated based on the abnormal type and data impact range;

[0052] Complexity weight: indicates the complexity of processing abnormal data, which is defined and calculated based on the computing resources and time required during the processing;

[0053] Time consumption: indicates the time required to process abnormal data, which is defined and calculated based on actual conditions;

[0054] Resource consumption: represents the computing resources required to process abnormal data, which is defined and calculated based on actual conditions;

[0055] Step 3: Build a deep reinforcement learning model:

[0056] 3.1 Building a deep Q network model:

[0057] Neural network structure: In the security management platform, a deep Q network model is designed with the following structure:

[0058] Input layer: the dimension size of the state space, including the output of the data comparison module, the credit value of the data analysis module, and the safety factor of the data enhancement module;

[0059] Hidden layer: can include multiple fully connected layers, each hidden layer can have a different number of neurons;

[0060] Output layer: Each action corresponds to an output node, which represents the Q-value of the corresponding action;

[0061] Loss function:

[0062] The mean square error is used to measure the difference between the predicted Q-value and the target Q-value. It is used to measure the average quadratic difference between the predicted value and the true value.

[0063]

[0064] The first term in the formula is the square of the difference between the predicted value and the target value for each sample, summed over all samples, and then divided by the number of samples N to get the average loss value.

[0065] in,

[0066] N is the number of samples, which means the number of samples in the training set;

[0067] Q(si,ai) is the predicted Q-value of the deep Q network for state si and action ai;

[0068] yi is the target Q-value, usually ri+γmaxa′Q(si+1,a′),

[0069] Where ri is the reward after executing action ai,

[0070] si+1 is the next state after executing action ai;

[0071] γ is a discount factor that balances the importance of current rewards and future rewards;

[0072] maxa′Q(si+1, a′) means selecting the action that maximizes the Q-value in the next state si+1.

[0073] Training Algorithm:

[0074] The deep Q network algorithm is used for training. The purpose is to optimize the network parameters so that the predicted Q-value is as close as possible to the target Q-value, thereby maximizing the cumulative reward. The Q-learning update rule is used during training:

[0075] Q(s,a)←Q(s,a)+α[r+γmaxa′Q(s′,a′)-Q(s,a)]

[0076] That is, each time the Q-value is updated, the estimated value of the Q-value is adjusted according to the current state and the action performed and the optimal action for the next state.

[0077] in,

[0078] Q(s, a) is the Q-value of taking action a in state s;

[0079] α is the learning rate, which is used to control the step size of parameter update;

[0080] r is the reward obtained after performing action a;

[0081] γ is a discount factor that balances the importance of current rewards and future rewards.

[0082] s′ is the next state after executing action a;

[0083] a′ is the optimal action in the next state s′, that is, the action with the largest Q-value.

[0084] 3.2 Introducing Experience Replay:

[0085] Experience replay buffer: During the training process, an experience replay buffer is maintained to store the historical experience of the agent; each experience contains a state s, an action a taken, a reward r obtained, the next state s′ after the action is executed, and a flag indicating whether it is finished or not; the size of the experience replay buffer can be set according to the actual situation, and a large enough size should be selected to cover the experience of multiple training cycles;

[0086] Experience replay process: During each training, a batch of experience is randomly sampled from the experience replay buffer. The experience replay process is as follows: A batch of experience is randomly sampled from the experience replay buffer, denoted as B, where N is the batch size, which means we randomly select a batch of experiences from the experience replay buffer to use for training the deep Q-network.

[0087] N: the number of samples in the batch, i.e. the empirical number of sampling;

[0088] (si,ai,ri,si+1,donei): the specific content of each experience, where:

[0089] si: current status;

[0090] ai: the action performed by the agent in the current state;

[0091] ri: reward obtained after executing action ai;

[0092] si+1: the next state after executing action ai;

[0093] donei: A flag indicating whether the terminal state has been reached. If the terminal state has been reached, it is True, otherwise it is False;

[0094] For each experience {(si, ai, ri, si′, donei)}, calculate the target Q-value yi. The calculation method of the target Q-value depends on whether the end state of the episode is reached = donei;

[0095]

[0096] Among them, ri is the reward after performing action ai, si′ is the next state after performing action ai, and γ is a discount factor used to balance the importance of current rewards and future rewards.

[0097] Use the loss function to calculate the loss:

[0098] Loss = (yi-Q(si,ai)) 2

[0099] The parameters of the deep Q network are updated using the gradient descent method to minimize the loss of all samples in the batch; the update rule is as follows

[0100]

[0101] in,

[0102] θ is the parameter of the deep Q network;

[0103] α is the learning rate, which is used to control the magnitude of each update;

[0104] In order to deal with the problem of sampling bias in the experience replay process, the importance sampling weight is introduced to make more effective use of experience; for each experience (si, ai, ri, si′, donei), its importance sampling weight wi is calculated:

[0105]

[0106] Where P(ai|si) is the probability that the deep Q network selects action ai in state si, and β is the exponent of the importance sampling weight;

[0107] When updating the network parameters, the loss is multiplied by the importance sampling weights:

[0108] Loss = wi*(yi-Q(si,ai)) 2

[0109] This can more effectively handle sampling bias problems and improve training efficiency and stability;

[0110] 3.3 Combining reinforcement learning and security management platform:

[0111] Parameters of interaction between intelligent agent and security management platform:

[0112] Interaction frequency: The frequency of information interaction between the intelligent agent and the security management platform;

[0113] Information transmission method: the information transmission method between the intelligent agent and the security management platform, such as synchronous transmission or asynchronous transmission;

[0114] Expansion of the agent state space: In addition to the output states of each component of the security management platform, the agent state space also includes other state information related to system security, such as network traffic and user behavior. This enables the agent to perceive the security status of the system more comprehensively and make more accurate decisions.

[0115] Combined with the functions of the security management platform: During the learning process of the intelligent agent, combined with the data monitoring, processing and reinforcement of the security management platform, the decision-making of the intelligent agent is guided and optimized; when abnormal data is detected, the intelligent agent can automatically adjust the strategy and respond in time, thereby improving the security and reliability of the system;

[0116] Step 4: Build a reinforcement learning model: Select Bagging as the ensemble learning method; Bagging constructs multiple data sets by sampling the data set with replacement and integrates their outputs to improve the generalization ability of the model;

[0117] 4.1 Dataset Division:

[0118] In the security management platform, an original data set containing historical monitoring data and security event records is required. In order to construct multiple training subsets, the original data set is sampled with replacement. The original data set contains 1,000 records, and a part of the data is randomly selected as a training subset. Then the data is replaced and sampled again to form another training subset. This is repeated multiple times to construct multiple training subsets.

[0119] Original data set: contains system monitoring data, historical records, and security events;

[0120] Training subset: Multiple subsets are extracted from the original dataset through sampling with replacement, and each subset is used to train a deep Q network model;

[0121] 4.2 Q network training:

[0122] For each training subset, a deep Q network model is trained separately; during the training process, different initialization methods are used or different data subsets are trained to increase the diversity of the model; for the first training subset, the default parameter settings are used for training; for the second training subset, the parameters are randomly initialized for training; for the third training subset, different learning rates or optimization algorithms are used for training;

[0123] Deep Q-Network Model: Each model is a deep Q-network consisting of multiple layers of neural networks, used to learn the mapping relationship between environment states and actions;

[0124] Initialization method: You can use different initialization parameters, different weight initialization or different bias initialization;

[0125] Data subset training: Each deep Q network model is trained on a training subset to learn the data features and patterns in that subset;

[0126] Learning rate: the rate at which the deep Q network model adjusts its parameters during training;

[0127] Optimization algorithm: Algorithms used to optimize the parameters of deep Q-network models, stochastic gradient descent, Adam;

[0128] 4.3 Integrated output:

[0129] After training on each training subset, multiple deep Q network models are obtained. Next, the outputs of these models are integrated. A simple method is to use voting, that is, to select the action corresponding to the maximum Q value of each Q network output as the final integrated output. There are 3 training subsets, each of which corresponds to a deep Q network model. The three models select actions A, B, and C, and the corresponding Q values ​​are 10, 15, and 12, respectively. Action B is selected as the final integrated output.

[0130] Integrated output: Vote or average the output of each deep Q network model, and select the action with the most votes as the final integrated output;

[0131] Voting method: For each state, the Q values ​​of the outputs of multiple deep Q network models are compared, and the action with the largest Q value is selected as the final action.

[0132] Compared with the prior art, the present invention combines deep reinforcement learning and enhanced learning to achieve the following beneficial effects:

[0133] Improve security and reliability: Through deep reinforcement learning, the security management platform can dynamically adjust decisions based on the current state of the system and environmental changes, so as to respond to various security threats and attacks in a timely manner. Combined with the integrated method of reinforcement learning, it can effectively integrate the outputs of multiple deep reinforcement learning models, improving the security and reliability of the system.

[0134] Improve decision-making efficiency: Deep reinforcement learning models can make decisions based on real-time feedback from the environment, thereby optimizing the operating efficiency of the system. Through ensemble learning methods, the opinions of multiple models can be combined to further improve the accuracy and efficiency of decision-making.

[0135] Strong adaptability: The deep reinforcement learning model has strong adaptive capabilities and can dynamically adjust strategies according to changes in the environment. Combined with the integrated method of reinforcement learning, it can flexibly adapt to the security management needs in different environments, improving the adaptability and flexibility of the system.

[0136] Improved diversity and robustness: By introducing diversity when constructing base learners, such as different parameter settings, initialization methods, or data subsets, the diversity and robustness of the integrated model are enhanced, making the system more resilient to various threats and attacks.

[0137] Improve the overall performance of the system: Combining the advantages of deep reinforcement learning and reinforcement learning can effectively improve the overall performance of the security management platform, including security, reliability, efficiency and adaptability, thereby better protecting the security and stable operation of the system.

[0138] As a preferred technical solution of the present invention, the security comparison library includes an unauthorized element, an unauthorized element, a high-risk element, a sensitive element, an operation and maintenance desensitizing element, a risk warning element and a risk display element. The unauthorized element compares whether the data transmitted by the data collection platform has an unauthorized risk, the unauthorized element compares whether the data transmitted by the data collection platform has unauthorized behavior, the high-risk element compares whether the data transmitted by the data collection platform has high temperature, the sensitive element compares whether the data transmitted by the data collection platform has sensitive parts, the operation and maintenance desensitizing element compares whether the data transmitted by the data collection platform is related to operation and maintenance, the risk warning element warns whether the data transmitted by the data collection platform has risks, and the risk display element intuitively displays the risks of the data transmitted by the data collection platform.

[0139] As a preferred technical solution of the present invention, the data transmission module includes a WIFI transmission element, a 2G transmission element, a 3G transmission element, a 4G transmission element and a 5G transmission element. The WIFI transmission element transmits the data after security management to the data receiving platform via WIFI, the 2G transmission element transmits the data after security management to the data receiving platform via 2G, the 3G transmission element transmits the data after security management to the data receiving platform via 3G, the 4G transmission element transmits the data after security management to the data receiving platform via 4G, and the 5G transmission element transmits the data after security management to the data receiving platform via 5G.

[0140] As a preferred technical solution of the present invention, the data receiving platform includes a data receiving module, a generator algorithm model and an edge computing controller. The data receiving module receives the data transmitted by the security management platform. The generator algorithm model automatically matches the optimal power value corresponding to different electrical quantities, and then automatically adjusts the power to optimize the power generation efficiency of the Internet of Things. The edge computing controller is installed on the Internet of Things generator to facilitate the corresponding transmission of the information generated by the data receiving platform to the Internet of Things generator.

[0141] The present invention provides a method for a data transmission safety management system for an Internet of Things generator, comprising the following steps:

[0142] Step 1: Data collection: The data transmission platform collects information about IoT generators, extracts features, determines the main data to be transmitted, and sends the data to the security management platform;

[0143] Step 2: Data comparison: The security management platform compares the attribute feature data of the data sending platform with the stored historical behavior feature data of the data sending platform, calculates its credit value, and determines whether the credit value exceeds the credit value in the security comparison library. If so, the data sending platform is prohibited from continuing to transmit data. Otherwise, the data sending platform is allowed to continue to transmit data to the data receiving platform.

[0144] Step 3: Data analysis: After segmenting the prohibited data, compare it with the security comparison database to identify dangerous data nodes and remove them;

[0145] Step 4: Data reinforcement: The data to be transmitted is segmented and nodes are set up, and encryption measures are set up at each node;

[0146] Step 5: Data transmission: Transmit the encrypted data to the data receiving platform;

[0147] Step 6. Data processing: The data receiving platform automatically calculates the most matching power of the IoT motor based on the transmitted data.

[0148] Compared with the prior art, the present invention has the following beneficial effects:

[0149] 1. By setting up a data receiving platform, the edge computing controller of the Internet of Things integrates an algorithm to automatically detect the power demand and automatically calculate the most matching power, so as to perform intelligent adjustment without human intervention, solving the problem of low efficiency of human adjustment. The power generation of the IoT generator is optimized: through the optimal algorithm model of the edge computing controller, it automatically matches the best power value that best suits the current power demand, and automatically and accurately adjusts the power generation of the unit to achieve the optimal, safe and guaranteed. On the one hand, the edge computing controller analyzes the safety parameter baseline of the generator set based on historical big data to keep the active power value below the safety level; on the other hand, it monitors the changes in the operating parameters of the unit in real time. When the parameters change that affect the safe operation of the equipment, the active power value is automatically adjusted to ensure the safe and stable operation of the equipment.

[0150] 2. By setting up a security management platform, the data is compared with the security comparison library to determine its credit value before transmission, which avoids the theft and interference of the transmitted power signal during the transmission process, and even the phenomenon of malicious attacks during the data transmission process, thus ensuring the normal operation of the IoT generator and improving its power generation efficiency.

[0151] 3. The present invention is based on the combination of deep reinforcement learning and enhanced learning for intelligent decision-making, security incident detection and response to abnormal behavior identification; combined with reinforcement learning and security management platform: the decision of the intelligent agent is applied to the security management platform: the strategies learned by the intelligent agent are applied to the various components of the security management platform to dynamically adjust data monitoring, processing and reinforcement according to actual conditions; the intelligent agent is enabled to make the best decision in real time according to the current environmental status and security requirements, thereby improving the security and reliability of the system; interaction between the intelligent agent and the security management platform: an effective interaction mechanism between the intelligent agent and the security management platform is established, so that the intelligent agent can obtain the status information of the security management platform in a timely manner, and apply its own decision results to the actual operation of the security management platform; a close integration of the intelligent agent and the security management platform is achieved, thereby better protecting the security and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0152] Figure 1 This is a schematic diagram of the architecture of the security management system of the present invention;

[0153] Figure 2 This is a schematic diagram of the architecture of the data sending platform of the present invention;

[0154] Figure 3 This is a schematic diagram of the architecture of the security management platform of the present invention;

[0155] Figure 4 It is a schematic diagram of the architecture of the security comparison library of the present invention;

[0156] Figure 5 It is a schematic diagram of the architecture of the data transmission module of the present invention;

[0157] Figure 6 It is a schematic diagram of the architecture of the data receiving platform of the present invention;

[0158] Figure 7 is a flow chart of the present invention;

[0159] Figure 8 It is a flow chart of the data transmission module of the present invention. DETAILED DESCRIPTION

[0160] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0161] See also Figure 1-8 , the present invention provides an Internet of Things generator data transmission safety management system, including a safety management system body, the safety management system body includes a data sending platform, a safety management platform and a data receiving platform, the data generation platform establishes a two-way communication connection with the safety management platform, and the safety management platform is connected with the data receiving platform;

[0162] The data generation platform collects and organizes the IoT generator data and then sends it. The security management platform verifies the security level of the sent data and rejects high-risk data. The data receiving platform receives the IoT generator data after security review by the security management platform.

[0163] The data sending platform includes a mode setting module, a data acquisition module, a feature extraction module, a data access module and a data service module. The data acquisition module collects the warning data, fault data, operation data, power generation data and shutdown data of the Internet of Things generator. The feature extraction module extracts the collected data features of the Internet of Things generator. The data access module connects the data generation platform with the Internet of Things generator. The data service module stores the collected data on the data sending platform for backup. The mode setting module sets the operation of the Internet of Things generator to manual mode, automatic mode and mute mode respectively.

[0164] The security management platform includes a data comparison module, a data analysis module, a data enhancement module, a security comparison library, a data modification module, a data storage module and a data transmission module. The data comparison module compares the data extracted from the warning features sent by the data sending platform with the data transmitted previously. The data analysis module analyzes the credit value of the data to be generated. If the credit value is lower than the credit value in the security comparison library, it is returned to the data sending platform. The data enhancement module enhances the security factor of the data through data analysis. The security comparison library compares and analyzes the data through the credit table. The data modification module compares the data of the data sending platform with the real-time data of the Internet of Things generator. If there is any error, it will be corrected in time. The data storage module stores each data of the data sending platform to facilitate subsequent data comparison and to understand the subtle changes of the Internet of Things generator. The data transmission module transmits the data in the data acquisition platform.

[0165] The security comparison library includes unauthorized elements, unauthorized elements, high-risk elements, sensitive elements, operation and maintenance desensitized elements, risk warning elements and risk display elements. The unauthorized elements compare whether the data transmitted by the data collection platform has unauthorized risks. The unauthorized elements compare whether the data transmitted by the data collection platform has unauthorized behaviors. The high-risk elements compare whether the data transmitted by the data collection platform has high temperatures. The sensitive elements compare whether the data transmitted by the data collection platform has sensitive areas. The operation and maintenance desensitized elements compare whether the data transmitted by the data collection platform is related to operation and maintenance. The risk warning elements warn whether the data transmitted by the data collection platform has risks. The risk display elements intuitively display the risks of the data transmitted by the data collection platform.

[0166] The data transmission module includes WIFI transmission element, 2G transmission element, 3G transmission element, 4G transmission element and 5G transmission element. The WIFI transmission element transmits the data after security management to the data receiving platform via WIFI, the 2G transmission element transmits the data after security management to the data receiving platform via 2G, the 3G transmission element transmits the data after security management to the data receiving platform via 3G, the 4G transmission element transmits the data after security management to the data receiving platform via 4G, and the 5G transmission element transmits the data after security management to the data receiving platform via 5G.

[0167] The data receiving platform includes a data receiving module, a generator algorithm model and an edge computing controller. The data receiving module receives the data transmitted by the security management platform. The generator algorithm model automatically matches the optimal power value corresponding to different electrical quantities, and then automatically adjusts the power to optimize the power generation efficiency of the Internet of Things. The edge computing controller is installed on the Internet of Things generator to facilitate the corresponding transmission of the information generated by the data receiving platform to the Internet of Things generator.

[0168] The present invention provides a method for a data transmission safety management system for an Internet of Things generator, comprising the following steps:

[0169] Step 1: Data collection: The data transmission platform collects information about IoT generators, extracts features, determines the main data to be transmitted, and sends the data to the security management platform;

[0170] Step 2: Data comparison: The security management platform compares the attribute feature data of the data sending platform with the stored historical behavior feature data of the data sending platform, calculates its credit value, and determines whether the credit value exceeds the credit value in the security comparison library. If so, the data sending platform is prohibited from continuing to transmit data. Otherwise, the data sending platform is allowed to continue to transmit data to the data receiving platform.

[0171] Step 3: Data analysis: After segmenting the prohibited data, compare it with the security comparison database to identify dangerous data nodes and remove them;

[0172] Step 4: Data reinforcement: The data to be transmitted is segmented and nodes are set up, and encryption measures are set up at each node;

[0173] Step 5: Data transmission: Transmit the encrypted data to the data receiving platform;

[0174] Step 6. Data processing: The data receiving platform automatically calculates the most matching power of the IoT motor based on the transmitted data.

[0175] The invention discloses an identification method for an IoT generator data transmission security management system, which is based on deep reinforcement learning and enhanced learning combined for intelligent decision-making, security event detection and response abnormal behavior identification. The specific process is as follows:

[0176] Step 1: Identify the problem and context

[0177] Problem definition: The task of the security management platform is to monitor and process the data sent by the data collection platform to ensure the security and reliability of the data; the specific tasks include data monitoring and data processing;

[0178] Data monitoring: real-time monitoring of data to detect abnormalities or security vulnerabilities;

[0179] Data processing: Based on the monitoring results, abnormal data is processed, identified and marked, and data is encrypted or filtered to ensure system security;

[0180] Environment modeling: The security management platform environment can be modeled as a multi-component environment, each with specific functions and interactions;

[0181] Data comparison module: detects differences between incoming data and historical data;

[0182] Data analysis module: Analyzes the credit value of the data. If it is lower than the credit value in the security comparison library, an exception is returned;

[0183] Data enhancement module: process abnormal data, encrypt or filter data;

[0184] Security comparison library: stores the credit value of historical data for comparison and analysis with current data;

[0185] Data modification module: modify the data to ensure the accuracy and consistency of the data;

[0186] Data storage module: stores each data for subsequent comparison and analysis;

[0187] Data transmission module: responsible for data transmission, ensuring timely transmission and integrity of data;

[0188] In environmental modeling, the relevant parameter variables can be specified as follows:

[0189] D: Dataset, which represents the data set received from the data collection platform;

[0190] Dt: The data set at time t, which means the data received from the data collection platform at time t;

[0191] A: Action space, which represents the set of actions that the security management platform can perform and the set of data processing operations;

[0192] S: state space, which represents the set of states that the security management platform may be in, including data monitoring state and data processing state;

[0193] R: Reward function, used to evaluate the quality of each action to guide the learning and decision-making of the agent;

[0194] Q(s, a): state-action value function, which represents the long-term reward obtained by performing action a in state s;

[0195] ∈: exploration rate in the greedy strategy, indicating the probability that the agent chooses a random action when making decisions;

[0196] α: learning rate, used to adjust the weight when updating the state-action value function;

[0197] Step 2: Design the intelligent agent: monitor and process data in real time, accurately judge the environmental status, formulate effective decision-making strategies, learn and optimize behavioral strategies, ensure the security and reliability of the system, and ensure that the security management platform can effectively monitor and process data from the data collection platform to ensure the security and reliability of the data;

[0198] 2.1 State space: Ensure that the agent can obtain key environmental information, data comparison results, the credit value of the data analysis module and the safety factor of the data enhancement module, so as to accurately judge the current environmental status; the state space of the agent includes the output status of each component of the security management platform, which is crucial to the decision-making of the agent; the specific state variables are as follows:

[0199] Sc: The output of the data comparison module, indicating the data comparison result, whether there is abnormal data or data tampering;

[0200] Sa: The credit value of the data analysis module, which indicates the credit level of the data and is used to evaluate the credibility of the data;

[0201] Se: the safety factor of the data enhancement module, indicating the safety level of the data after enhancement;

[0202] The state s of the agent can be expressed as s = (Sc, Sa, Se);

[0203] 2.2 Action Space: Ensure that the agent can take appropriate actions to process data, perform data modification operations or data enhancement operations to maintain the security and stability of the system; the action space that the agent can perform covers the operable actions in the security management platform, including data modification and data enhancement; the specific action variables are as follows:

[0204] Am: data modification operation, correcting erroneous data or deleting abnormal data;

[0205] Ae: data enhancement operations, data encryption, data cleaning or data reconstruction;

[0206] The action space A of the agent can be expressed as A = {Am, Ae};

[0207] 2.3 Reward function: Through the design of reward function, the performance of the agent in handling abnormal situations is clearly evaluated, so as to encourage the agent to learn the correct behavior strategy;

[0208] Importance and scope of impact of abnormal data: Different types of abnormal data have different degrees of impact on the security and reliability of the system; different degrees of positive or negative rewards are given according to the importance and scope of impact of abnormal data;

[0209] Time efficiency and resource consumption for handling abnormal data: Handling abnormal data requires different time and computing resources; some abnormal situations may be easier to solve, while others require more time and resources; therefore, time efficiency and resource consumption are considered in the reward function;

[0210] R(s,a)=rpositive·importance weight-rnegative·complexity weight-rtime·time consumption-rresource·resource consumption

[0211] in:

[0212] R(s, a): the reward obtained by the agent for performing action a in state s;

[0213] rpositive: positive reward value when processing abnormal data, which is a large positive number;

[0214] r negative: negative reward value when an abnormal situation is not handled correctly, which is a large negative number;

[0215] rTime: Negative reward value corresponding to time consumption: This part takes into account the time required to process abnormal data; the more time consumed, the larger the corresponding negative reward value; this value can be set based on the time required to process abnormal data to encourage the agent to handle abnormal situations as quickly as possible;

[0216] rResources: Negative reward value corresponding to resource consumption: This part takes into account the computing resource consumption required to process abnormal data; similar to time consumption, the more resources consumed, the larger the corresponding negative reward value; this value can be set based on the computing resources required to process abnormal data to encourage the agent to use resources as efficiently as possible;

[0217] Importance weight: indicates the importance and impact range of abnormal data, and is defined and calculated based on the abnormal type and data impact range;

[0218] Complexity weight: indicates the complexity of processing abnormal data, which is defined and calculated based on the computing resources and time required during the processing;

[0219] Time consumption: indicates the time required to process abnormal data, which is defined and calculated based on actual conditions;

[0220] Resource consumption: represents the computing resources required to process abnormal data, which is defined and calculated based on actual conditions;

[0221] Step 3: Build a deep reinforcement learning model:

[0222] 3.1 Building a deep Q network model:

[0223] Neural network structure: In the security management platform, a deep Q network model is designed with the following structure:

[0224] Input layer: the dimension size of the state space, including the output of the data comparison module, the credit value of the data analysis module, and the safety factor of the data enhancement module;

[0225] Hidden layer: can include multiple fully connected layers, each hidden layer can have a different number of neurons;

[0226] Output layer: Each action corresponds to an output node, which represents the Q-value of the corresponding action;

[0227] Loss function:

[0228] The mean square error is used to measure the difference between the predicted Q-value and the target Q-value. It is used to measure the average quadratic difference between the predicted value and the true value.

[0229]

[0230] The first term in the formula is the square of the difference between the predicted value and the target value for each sample, summed over all samples, and then divided by the number of samples N to get the average loss value.

[0231] in,

[0232] N is the number of samples, which means the number of samples in the training set;

[0233] Q(si,ai) is the predicted Q-value of the deep Q network for state si and action ai;

[0234] yi is the target Q-value, usually ri+γmaxa′Q(si+1,a′),

[0235] Where ri is the reward after executing action ai,

[0236] si+1 is the next state after executing action ai;

[0237] γ is a discount factor that balances the importance of current rewards and future rewards;

[0238] maxa′Q(si+1, a′) means selecting the action that maximizes the Q-value in the next state si+1.

[0239] Training Algorithm:

[0240] The deep Q network algorithm is used for training. The purpose is to optimize the network parameters so that the predicted Q-value is as close as possible to the target Q-value, thereby maximizing the cumulative reward. The Q-learning update rule is used during training:

[0241] Q(s,a)←Q(s,a)+α[r+γmaxa′Q(s′,a′)-Q(s,a)]

[0242] That is, each time the Q-value is updated, the estimated value of the Q-value is adjusted according to the current state and the action performed and the optimal action for the next state.

[0243] in,

[0244] Q(s, a) is the Q-value of taking action a in state s;

[0245] α is the learning rate, which is used to control the step size of parameter update;

[0246] r is the reward obtained after performing action a;

[0247] γ is a discount factor that balances the importance of current rewards and future rewards.

[0248] s′ is the next state after executing action a;

[0249] a′ is the optimal action in the next state s′, that is, the action with the largest Q-value.

[0250] 3.2 Introducing Experience Replay:

[0251] Experience replay buffer: During the training process, an experience replay buffer is maintained to store the historical experience of the agent; each experience contains a state s, an action a taken, a reward r obtained, the next state s′ after the action is executed, and a flag indicating whether it is finished or not; the size of the experience replay buffer can be set according to the actual situation, and a large enough size should be selected to cover the experience of multiple training cycles;

[0252] Experience replay process: During each training, a batch of experience is randomly sampled from the experience replay buffer. The experience replay process is as follows: A batch of experience is randomly sampled from the experience replay buffer, denoted as B, where N is the batch size, which means we randomly select a batch of experiences from the experience replay buffer to use for training the deep Q-network.

[0253] For each experience {(si, ai, ri, si′, donei)}, calculate the target Q-value yi. The calculation method of the target Q-value depends on whether the end state of the episode is reached = donei;

[0254]

[0255] Among them, ri is the reward after performing action ai, si′ is the next state after performing action ai, and γ is a discount factor used to balance the importance of current rewards and future rewards.

[0256] Use the loss function to calculate the loss:

[0257] Loss = (yi-Q(si,ai)) 2

[0258] The parameters of the deep Q network are updated using the gradient descent method to minimize the loss of all samples in the batch; the update rule is as follows

[0259]

[0260] in,

[0261] θ is the parameter of the deep Q network;

[0262] α is the learning rate, which is used to control the magnitude of each update;

[0263] In order to deal with the problem of sampling bias in the experience replay process, the importance sampling weight is introduced to make more effective use of experience; for each experience (si, ai, ri, si′, donei), its importance sampling weight wi is calculated:

[0264]

[0265] Where P(ai|si) is the probability that the deep Q network selects action ai in state si, and β is the exponent of the importance sampling weight;

[0266] When updating the network parameters, the loss is multiplied by the importance sampling weights:

[0267] Loss = wi*(yi-Q(si,ai)) 2

[0268] This can more effectively handle sampling bias problems and improve training efficiency and stability;

[0269] 3.3 Combining reinforcement learning and security management platform:

[0270] The decision-making of the intelligent agent is applied to the security management platform: the strategies learned by the intelligent agent are applied to various components of the security management platform to dynamically adjust data monitoring, processing and reinforcement according to actual conditions; the intelligent agent makes the best decision in real time according to the current environmental status and security requirements, thereby improving the security and reliability of the system;

[0271] Interaction between the intelligent agent and the security management platform: Establish an effective interaction mechanism between the intelligent agent and the security management platform so that the intelligent agent can obtain the status information of the security management platform in a timely manner and apply its own decision results to the actual operation of the security management platform; realize the close integration of the intelligent agent and the security management platform, so as to better protect the security and reliability of the system;

[0272] Parameters of interaction between intelligent agent and security management platform:

[0273] Interaction frequency: The frequency of information interaction between the intelligent agent and the security management platform;

[0274] Information transmission method: the information transmission method between the intelligent agent and the security management platform, such as synchronous transmission or asynchronous transmission;

[0275] Expansion of the agent state space: In addition to the output states of each component of the security management platform, the agent state space also includes other state information related to system security, such as network traffic and user behavior. This enables the agent to perceive the security status of the system more comprehensively and make more accurate decisions.

[0276] Combined with the functions of the security management platform: During the learning process of the intelligent agent, combined with the data monitoring, processing and reinforcement of the security management platform, the decision-making of the intelligent agent is guided and optimized; when abnormal data is detected, the intelligent agent can automatically adjust the strategy and respond in time, thereby improving the security and reliability of the system;

[0277] Step 4: Build a reinforcement learning model: Select Bagging as the ensemble learning method; Bagging constructs multiple data sets by sampling the data set with replacement and integrates their outputs to improve the generalization ability of the model;

[0278] 4.1 Dataset Division:

[0279] In the security management platform, an original data set containing historical monitoring data and security event records is required. In order to construct multiple training subsets, the original data set is sampled with replacement. The original data set contains 1,000 records, and a part of the data is randomly selected as a training subset. Then the data is replaced and sampled again to form another training subset. This is repeated multiple times to construct multiple training subsets.

[0280] Original data set: contains system monitoring data, historical records, and security events;

[0281] Training subset: Multiple subsets are extracted from the original dataset through sampling with replacement, and each subset is used to train a deep Q network model;

[0282] 4.2Q network training:

[0283] For each training subset, a deep Q network model is trained separately; during the training process, different initialization methods are used or different data subsets are trained to increase the diversity of the model; for the first training subset, the default parameter settings are used for training; for the second training subset, the parameters are randomly initialized for training; for the third training subset, different learning rates or optimization algorithms are used for training;

[0284] Deep Q-Network Model: Each model is a deep Q-network consisting of multiple layers of neural networks, used to learn the mapping relationship between environment states and actions;

[0285] Initialization method: You can use different initialization parameters, different weight initialization or different bias initialization;

[0286] Data subset training: Each deep Q network model is trained on a training subset to learn the data features and patterns in that subset;

[0287] Learning rate: the rate at which the deep Q network model adjusts its parameters during training;

[0288] Optimization algorithm: Algorithms used to optimize the parameters of deep Q-network models, stochastic gradient descent, Adam;

[0289] 4.3 Integrated output:

[0290] After training on each training subset, multiple deep Q network models are obtained. Next, the outputs of these models are integrated. A simple method is to use voting, that is, to select the action corresponding to the maximum Q value of each Q network output as the final integrated output. There are 3 training subsets, each of which corresponds to a deep Q network model. The three models select actions A, B, and C, and the corresponding Q values ​​are 10, 15, and 12, respectively. Action B is selected as the final integrated output.

[0291] Integrated output: Vote or average the output of each deep Q network model, and select the action with the most votes as the final integrated output;

[0292] Voting method: For each state, the Q values ​​of the outputs of multiple deep Q network models are compared, and the action with the largest Q value is selected as the final action.

[0293] In the present invention, the data sending platform collects information about the Internet of Things generator and then extracts features to determine the main data to be transmitted, and sends the data to the security management platform. The security management platform compares the attribute feature data of the data sending platform with the stored historical behavior feature data of the data sending platform, calculates its credit value, and determines whether the credit value exceeds the credit value in the security comparison library. If so, the data sending platform is prohibited from continuing to transmit the data. Otherwise, the data sending platform is allowed to continue to pass the data to the data receiving platform. After the prohibited data is segmented, it is compared with the security comparison library for the second time to determine the dangerous data nodes and eliminate them. The data to be transmitted is segmented and nodes are established, and encryption measures are established at each node. The encrypted data is transmitted to the data receiving platform, and the data receiving platform automatically calculates the most matching power of the Internet of Things motor according to the transmitted data.

[0294] In the present invention, the data transmission platform function is embedded into the front-end Internet of Things generator device in an embedded manner, and the Internet of Things generator device is automatically connected to the VPN in a power-on self-starting manner to achieve the use effect. In order to better manage the front-end Internet of Things front-end devices, the IP address of each Internet of Things front-end device is bound to the data transmission platform. Therefore, no matter what the reason for the restart of the Internet of Things power generation device, after the Internet of Things power generation device automatically connects to the VPN, the original IP address is still used to transmit the collected data to the server of the data transmission platform, which is convenient for the collection server and the administrator on the data transmission platform to accurately locate the data collection location. The data transmission platform collects the information of the Internet of Things generator and then extracts features. Determine the main data to be transmitted and send the data to the security management platform. In order to prevent the IoT generator equipment from being maliciously replaced and avoid attacks on the IoT generator data collection business, the security management platform adopts the hardware feature code authentication method to perform hardware platform-based identity authentication for each front-end IoT power generation equipment. Only IoT power generation equipment that has been reviewed and approved by the administrator can be connected normally, and by binding the hardware feature code with the IP address, the use effect of accurate positioning of the collected data can be achieved. The data to be transmitted is segmented and nodes are set up, and encryption measures are set up at each node. The encrypted data is transmitted to the data receiving platform, and the data receiving platform automatically calculates the most matching power of the IoT motor based on the transmitted data.

[0295] In the present invention, the data acquisition module collects the warning data, fault data, operation data, power generation data and shutdown data of the Internet of Things generator, the feature extraction module extracts the features of the collected Internet of Things generator data, the data access module connects the data generation platform with the Internet of Things generator, the data service module stores the collected data on the data sending platform for backup, the mode setting module sets the operation of the Internet of Things generator to manual mode, automatic mode and mute mode respectively, the data comparison module compares the data extracted from the warning features sent by the data sending platform with the data transmitted previously, the data analysis module analyzes the credit value of the data to be generated, and returns it to the data sending platform if the credit value is lower than the credit value in the security comparison library, and the data enhancement module performs data analysis on the data sent through the data analysis module. The security factor of the analyzed data is enhanced, the security comparison library compares and analyzes the data through the credit table, the data modification module compares the data of the data sending platform with the real-time data of the IoT generator, and if there is any error, it will be corrected in time. The data storage module stores each data of the data sending platform for subsequent data comparison, so as to understand the subtle changes of the IoT generator. The data transmission module transmits the data in the data acquisition platform, and the data receiving module receives the data transmitted by the security management platform. The generator algorithm model automatically matches the optimal power value corresponding to different electrical quantities, and then automatically adjusts the power to optimize the power generation efficiency of the IoT. The edge computing controller is installed on the IoT generator to facilitate the corresponding transmission of the information generated by the data receiving platform to the IoT generator.

[0296] The above modules can be constructed by mathematical models or implemented through code editing (mode setting module, feature extraction module, data access module and data service module, data comparison module, data analysis module, data enhancement module, security comparison library, data modification module, data transmission module), or they can be physical sensing and storage modules (data storage module, data acquisition module).

[0297] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An identification method for an Internet of Things generator data transmission safety management system, including a safety management system body, characterized in that: The security management system body includes a data sending platform, a security management platform and a data receiving platform. The data sending platform establishes a two-way communication connection with the security management platform, and the security management platform is connected to the data receiving platform; The data sending platform collects and organizes the data of the IoT generator and then sends it. The security management platform performs a security level check on the sent data and rejects high-risk data. The data receiving platform receives the IoT generator data after security review by the security management platform. The data transmission platform includes a mode setting module, a data acquisition module, a feature extraction module, a data access module and a data service module. The data acquisition module collects warning data, fault data, operation data, power generation data and shutdown data of the Internet of Things generator. The feature extraction module extracts the features of the collected Internet of Things generator data. The data access module connects the data transmission platform with the Internet of Things generator. The data service module stores the collected data on the data transmission platform for backup. The mode setting module sets the operation of the Internet of Things generator to manual mode, automatic mode and mute mode respectively. The security management platform includes a data comparison module, a data analysis module, a data enhancement module, a security comparison library, a data modification module, a data storage module and a data transmission module. The data comparison module compares the data extracted from the warning features sent by the data sending platform with the previously transmitted data. The data analysis module analyzes the credit value of the data to be sent, and returns it to the data sending platform if the credit value is lower than the credit value in the security comparison library. The data enhancement module enhances the security factor of the data analyzed. The specific process of the identification method is as follows: Step 1: Identify the problem and context Problem definition: The task of the security management platform is to monitor and process the data sent by the data collection platform to ensure the security and reliability of the data; the specific tasks include data monitoring and data processing; Data monitoring: real-time monitoring of data to detect abnormalities or security vulnerabilities; Data processing: Based on the monitoring results, abnormal data is processed, identified and marked, and data is encrypted or filtered to ensure system security; Environment modeling: The security management platform environment is modeled as a multi-component environment, each component has functions and can interact with each other; Data comparison module: detects differences between incoming data and historical data; Data analysis module: analyzes the credit value of the data. If it is lower than the credit value in the security comparison library, an exception is returned; Data enhancement module: process abnormal data, encrypt or filter data; Security comparison library: stores the credit value of historical data for comparison and analysis with current data; Data modification module: modify the data to ensure the accuracy and consistency of the data; Data storage module: stores each data for subsequent comparison and analysis; Data transmission module: responsible for data transmission, ensuring timely transmission and integrity of data; In environmental modeling, the relevant parameter variables are described as follows: D: Dataset, which represents the data set received from the data collection platform; Dt: The data set at time t, which means the data received from the data collection platform at time t; A: Action space, which represents the set of actions performed by the security management platform and the set of data processing operations; S: state space, which represents the set of states that the security management platform may be in, including data monitoring state and data processing state; R: Reward function, used to evaluate the quality of each action to guide the learning and decision-making of the agent; Q(s,a): state-action value function, which represents the long-term reward obtained by performing action a in state s; ∈: exploration rate in the greedy strategy, indicating the probability that the agent chooses a random action when making decisions; α: learning rate, used to adjust the weight when updating the state-action value function; Step 2: Design the intelligent agent: monitor and process data in real time, accurately judge the environmental status, formulate effective decision-making strategies, learn and optimize behavioral strategies, ensure the security and reliability of the system, and ensure that the security management platform can effectively monitor and process data from the data collection platform to ensure the security and reliability of the data; 2.1 State space: Ensure that the agent can obtain key environmental information, data comparison results, the credit value of the data analysis module and the safety factor of the data enhancement module, so as to accurately judge the current environmental status; the state space of the agent includes the output status of each component of the security management platform, which is crucial to the decision-making of the agent; the specific state variables are as follows: Sc: The output of the data comparison module, indicating the data comparison result, whether there is abnormal data or data tampering; Sa: The credit value of the data analysis module, which indicates the credit level of the data and is used to evaluate the credibility of the data; Se: the safety factor of the data enhancement module, indicating the safety level of the data after enhancement; The state s of the agent is represented by s = (Sc, Sa, Se); 2.2 Action Space: Ensure that the agent can take appropriate actions to process data, perform data modification operations or data enhancement operations to maintain the security and stability of the system; the action space performed by the agent covers the operable actions in the security management platform, including data modification and data enhancement; the specific action variables are as follows: Am: data modification operation, correcting erroneous data or deleting abnormal data; Ae: data enhancement operations, data encryption, data cleaning or data reconstruction; The action space A of the agent is represented by A = {Am, Ae}; 2.3 Reward function: Through the design of reward function, the performance of the agent in handling abnormal situations is clearly evaluated, so as to encourage the agent to learn the correct behavior strategy; Importance and scope of impact of abnormal data: Different types of abnormal data have different degrees of impact on the security and reliability of the system; different degrees of positive or negative rewards are given according to the importance and scope of impact of abnormal data; Time efficiency and resource consumption for handling abnormal data: Handling abnormal data requires different time and computing resources; some abnormal situations may be easier to solve, while others require more time and resources; therefore, time efficiency and resource consumption are considered in the reward function; R(s,a)=rpositive·importance weight-rnegative·complexity weight-rtime·time consumption-rresources·resource consumption in: R(s,a): the reward obtained by the agent for performing action a in state s; rpositive: positive reward value when processing abnormal data, a positive number; r negative: negative reward value when an abnormal situation is not handled correctly, which is a negative number; rTime: Negative reward value corresponding to time consumption: This part takes into account the time required to process abnormal data; the more time consumed, the larger the corresponding negative reward value; this value is set based on the time required to process abnormal data to encourage the agent to handle abnormal situations as quickly as possible; rResources: Negative reward value corresponding to resource consumption: This part takes into account the computing resource consumption required to process abnormal data; similar to time consumption, the more resources consumed, the larger the corresponding negative reward value; this value is set based on the computing resources required to process abnormal data to encourage the agent to use resources as efficiently as possible; Importance weight: indicates the importance and impact of abnormal data; Complexity weight: indicates the complexity of processing abnormal data; Time consumption: indicates the time required to process abnormal data; Resource consumption: represents the computing resources required to process abnormal data; Step 3: Build a deep reinforcement learning model: 3.1 Building a deep Q network model: Neural network structure: In the security management platform, a deep Q network model is designed with the following structure: Input layer: the dimension size of the state space, including the output of the data comparison module, the credit value of the data analysis module, and the safety factor of the data enhancement module; Hidden layer: includes multiple fully connected layers, each hidden layer has a different number of neurons; Output layer: Each action corresponds to an output node, which represents the Q-value of the corresponding action; Loss function: The mean square error is used to measure the difference between the predicted Q-value and the target Q-value, which is used to measure the average quadratic difference between the predicted value and the true value; The first term in the formula is to square the difference between the predicted value and the target value of each sample, sum it up for all samples, and then divide it by the number of samples N to get the average loss value; in, N is the number of samples, which means the number of samples in the training set; Q(si,ai) is the predicted Q-value of the deep Q network for state si and action ai; yi is the target Q-value, usually ri+γmaxa′Q(si+1,a′), Where ri is the reward after executing action ai, si+1 is the next state after executing action ai; γ is a discount factor that balances the importance of current rewards and future rewards; maxa′Q(si+1,a′) means choosing the action that maximizes the Q-value in the next state si+1; Training Algorithm: The deep Q network algorithm is used for training. The purpose is to optimize the network parameters so that the predicted Q-value is as close as possible to the target Q-value, thereby maximizing the cumulative reward. The Q-learning update rule is used during the training process: Q(s,a)←Q(s,a)+α[r+γmaxa′Q(s′,a′)-Q(s,a)] That is, each time the Q-value is updated, the estimated value of the Q-value is adjusted according to the current state and the action performed and the optimal action for the next state; in, Q(s,a) is the Q-value of taking action a in state s; α is the learning rate, which is used to control the step size of parameter update; r is the reward obtained after performing action a; γ is a discount factor that balances the importance of current rewards and future rewards; s′ is the next state after executing action a; a′ is the optimal action in the next state s′, that is, the action with the largest Q-value; 3.2 Introducing Experience Replay: Experience replay buffer: During the training process, an experience replay buffer is maintained to store the historical experience of the agent; each experience contains a state s, an action a taken, a reward r obtained, the next state s′ after the action is executed, and a flag indicating whether it is finished or not; the size of the experience replay buffer is set according to the actual situation, and is large enough to cover the experience of multiple training cycles; Experience replay process: During each training, a batch of experience is randomly sampled from the experience replay buffer. The experience replay process is as follows: A batch of experience is randomly sampled from the experience replay buffer, denoted as B, where N is the batch size, which means we randomly select a batch of experiences from the experience replay buffer to be used for training the deep Q-network; N: the number of samples in the batch, i.e. the empirical number of sampling; (si,ai,ri,si+1,donei): the specific content of each experience, where: si: current status; ai: the action performed by the agent in the current state; ri: reward obtained after executing action ai; si+1: the next state after executing action ai; donei: A flag indicating whether the terminal state has been reached. If the terminal state has been reached, it is True, otherwise it is False; For each experience {(si,ai,ri,si′,donei)}, calculate the target Q-value yi. The calculation method of the target Q-value depends on whether the end state of the episode is reached =donei; Where ri is the reward after executing action ai, si′ is the next state after executing action ai, and γ is a discount factor used to balance the importance of current rewards and future rewards; Use the loss function to calculate the loss: Loss = (yi-Q(si,ai)) 2 The parameters of the deep Q network are updated using the gradient descent method to minimize the loss of all samples in the batch; the update rule is as follows in, θ is the parameter of the deep Q network; represents the gradient of the parameter θ; α is the learning rate, which is used to control the magnitude of each update; In order to deal with the problem of sampling bias in the experience replay process, the importance sampling weight is introduced to make more effective use of experience; for each experience (si, ai, ri, si′, donei), its importance sampling weight wi is calculated: Among them, P(ai|si) is the probability that the deep Q network selects action ai in state si, and β is the exponent of the importance sampling weight; When updating the network parameters, the loss is multiplied by the importance sampling weights: Loss = wi*(yi-Q(si,ai)) 2 This can more effectively handle the sampling bias problem and improve the efficiency and stability of training; 3.3 Combining reinforcement learning and security management platform: Parameters of interaction between intelligent agent and security management platform: Interaction frequency: The frequency of information interaction between the intelligent agent and the security management platform; Information transmission method: the information transmission method between the intelligent agent and the security management platform, such as synchronous transmission or asynchronous transmission; Expansion of the agent state space: In addition to the output states of each component of the security management platform, the agent state space also includes other state information related to system security, such as network traffic and user behavior. This enables the agent to more comprehensively perceive the security status of the system and make more accurate decisions. Combined with the functions of the security management platform: During the learning process of the intelligent agent, combined with the data monitoring, processing and reinforcement of the security management platform, the decision-making of the intelligent agent is guided and optimized; when abnormal data is detected, the intelligent agent automatically adjusts the strategy and responds in time; Step 4: Build a reinforcement learning model: Select Bagging as the ensemble learning method; Bagging constructs multiple data sets by sampling the data set with replacement and integrates their outputs; 4.1 Dataset Division: In the security management platform, an original data set containing historical monitoring data and security event records is required. In order to construct multiple training subsets, the original data set is sampled with replacement. The original data set contains 1,000 records, and a part of the data is randomly selected as a training subset. Then the data is replaced and sampled again to form another training subset. This is repeated multiple times to construct multiple training subsets. Original data set: contains system monitoring data, historical records, and security events; Training subset: Multiple subsets are extracted from the original dataset through sampling with replacement, and each subset is used to train a deep Q network model; 4.2Q network training: For each training subset, a deep Q network model is trained separately; during the training process, different initialization methods are used or different data subsets are trained to increase the diversity of the model; for the first training subset, the default parameter settings are used for training; for the second training subset, the parameters are randomly initialized for training; for the third training subset, different learning rates or optimization algorithms are used for training; 4.3 Integrated output: After training on each training subset, multiple deep Q network models were obtained. Next, the outputs of these models were integrated. The action corresponding to the maximum Q value of each Q network output was selected as the final integrated output by voting. There were three training subsets, each corresponding to a deep Q network model. The three models selected actions A, B, and C, and the corresponding Q values ​​were 10, 15, and 12, respectively. Action B was selected as the final integrated output. Integrated output: Vote or average the output of each deep Q network model, and select the action with the most votes as the final integrated output; Voting method: For each state, the Q values ​​of the outputs of multiple deep Q network models are compared, and the action with the largest Q value is selected as the final action.

2. The method according to claim 1, characterized in that: The security comparison library compares and analyzes the data through the credit table. The data modification module compares the data of the data sending platform with the real-time data of the Internet of Things generator and corrects any errors in a timely manner. The data storage module stores each data of the data sending platform to facilitate subsequent data comparison and understanding of subtle changes in the Internet of Things generator. The data transmission module transmits data in the data acquisition platform.

3. The method according to claim 1, characterized in that: The security comparison library includes unauthorized elements, unauthorized elements, high-risk elements, sensitive elements, operation and maintenance desensitized elements, risk warning elements and risk display elements. The unauthorized elements compare whether the data transmitted by the data collection platform has unauthorized risks, the unauthorized elements compare whether the data transmitted by the data collection platform has unauthorized behaviors, the high-risk elements compare whether the data transmitted by the data collection platform has high temperatures, the sensitive elements compare whether the data transmitted by the data collection platform has sensitive areas, the operation and maintenance desensitized elements compare whether the data transmitted by the data collection platform is related to operation and maintenance, the risk warning elements warn whether the data transmitted by the data collection platform has risks, and the risk display elements intuitively display the risks of the data transmitted by the data collection platform.

4. The method according to claim 1, characterized in that: The data transmission module includes a WIFI transmission element, a 2G transmission element, a 3G transmission element, a 4G transmission element and a 5G transmission element.

5. The method according to claim 4, characterized in that: The WIFI transmission element transmits the data after security management to the data receiving platform via WIFI, the 2G transmission element transmits the data after security management to the data receiving platform via 2G, the 3G transmission element transmits the data after security management to the data receiving platform via 3G, the 4G transmission element transmits the data after security management to the data receiving platform via 4G, and the 5G transmission element transmits the data after security management to the data receiving platform via 5G.

6. The method according to claim 1, characterized in that: The data receiving platform includes a data receiving module, a generator algorithm model and an edge computing controller. The data receiving module receives the data transmitted by the security management platform. The generator algorithm model automatically matches the optimal power value corresponding to different electrical quantities, and then automatically adjusts the power to optimize the power generation efficiency of the Internet of Things. The edge computing controller is installed on the Internet of Things generator to facilitate the corresponding transmission of the information generated by the data receiving platform to the Internet of Things generator.

7. A method for managing the data transmission security of an Internet of Things generator according to any one of claims 1 to 6, characterized in that: The following steps are involved: Step 1: Data collection: The data transmission platform collects information about IoT generators, extracts features, determines the main data to be transmitted, and sends the data to the security management platform; Step 2: Data comparison: The security management platform compares the attribute feature data of the data sending platform with the stored historical behavior feature data of the data sending platform, calculates its credit value, and determines whether the credit value exceeds the credit value in the security comparison library. If so, the data sending platform is prohibited from continuing to transmit data. Otherwise, the data sending platform is allowed to continue to transmit data to the data receiving platform. Step 3: Data analysis: After segmenting the prohibited data, compare it with the security comparison database to identify dangerous data nodes and remove them; Step 4: Data reinforcement: The data to be transmitted is segmented and nodes are set up, and encryption measures are set up at each node; Step 5: Data transmission: Transmit the encrypted data to the data receiving platform; Step 6. Data processing: The data receiving platform automatically calculates the most matching power of the IoT motor based on the transmitted data.

Images