Sensing network malicious node detection method based on redeemable deep reinforcement learning

Abstract

The invention belongs to the technical field of information security detection, and relates to a sensor network malicious node detection method based on redeemable deep reinforcement learning. The method specifically comprises the following steps: firstly, collecting network node transmission data, and providing multi-dimensional data support for subsequent node trust evaluation; a dynamic trust model is constructed based on deep reinforcement learning, real-time evaluation of node trust values is realized, and trusted nodes and suspicious nodes are preliminarily distinguished according to the trust values obtained through evaluation; finally, a trust decision-making mechanism supporting'node redemption 'is built based on a type-2 fuzzy logic system, and differentiated processing of redeeming misjudged nodes and isolating malicious nodes is achieved by quantitatively analyzing the damage degree of suspicious node behaviors. Compared with an existing detection method, the method has the advantages that the problem of misjudgment of normal nodes is effectively solved while the high detection rate of the malicious nodes is ensured, and key technical support is provided for long-term reliable deployment of the sensor network.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of trust management mechanism of network information security, and particularly relates to a malicious node detection method for a sensor network based on a redeemable deep reinforcement learning. BACKGROUND

[0002] As a core technology of the perception layer of the Internet of Things, a sensor network (WSN) is widely used in fields such as environmental monitoring, intelligent transportation and industrial control. However, the WSN is often deployed in an open and complex environment, and the nodes have limited resources and communication links that are easily disturbed. The WSN faces security risks such as malicious intrusion, data tampering and transmission anomalies. Therefore, it is a key measure to protect the reliable operation of the WSN to build a security protection mechanism that is suitable for the characteristics of the WSN.

[0003] A trust management mechanism is a core technology for security protection of a sensor network. The mechanism can distinguish between trusted and malicious nodes by quantifying the trust value of a node, thereby making up for the deficiency of traditional encryption technology in judging internal malicious nodes. Meanwhile, the mechanism can capture the behavior characteristics of a node, isolate malicious nodes and optimize network resources, and has become an important means to improve the anti-interference capability of a sensor network.

[0004] However, the existing trust management mechanism still has obvious limitations. First, the environment of a sensor network is dynamic and variable, and normal nodes are prone to abnormal behavior due to environmental interference or temporary failure. Traditional mechanisms rely on static weights and are difficult to distinguish between non-malicious anomalies (such as temporary node failure) and malicious attacks, which may misjudge normal nodes as malicious nodes. Second, with the development of artificial intelligence, attack patterns are evolving towards dynamicity, and malicious nodes can dynamically adjust strategies to evade detection. However, the existing mechanism is designed for static attacks and lacks adaptive ability, which cannot effectively cope with dynamic attacks and cannot meet the security needs of complex scenarios. SUMMARY

[0005] In view of the deficiencies of the prior art, the application provides a malicious node detection method for a sensor network based on a redeemable deep reinforcement learning. The method comprises the following steps:

[0006] S1: Obtain transmission data of a node of a sensor network, wherein the transmission data comprises transmission data in an initial running stage of the network and transmission data in a normal running stage of the network;

[0007] S2: Extract features from the transmission data to obtain trust evidence representing different features, and use the fact that there are fewer malicious nodes in the initial running stage of the network to construct a model training data set based on the transmission data in the initial running stage, as a benchmark data set for pre-training of a trust model;

[0008] S3: A dynamic trust model is constructed based on a deep reinforcement learning algorithm. A deep deterministic policy gradient (DDPG) algorithm is used as the training framework, and the state space, action space, and reward function are defined. An Actor-Critic dual network structure is constructed based on the above parameters, and the network is pre-trained using the constructed training data set to obtain a pre-trained trust model. At this time, the model has the ability to identify the normal node's basic behavior pattern. During normal operation, the latest transmission data of the node is input into the pre-trained model, and the real-time weight of each trust evidence is output. When abnormal behavior of the node is detected, the weight distribution of each trust evidence is immediately adjusted and the model parameters are updated, so that the trust evaluation ability is continuously optimized with the change of the network state, and the evolution of the model from the "initial baseline model" to the "dynamic adaptive model" is promoted.

[0009] S4: Based on the real-time weight output by the trust model, a trust decision mechanism based on a two-type fuzzy logic system is constructed. The execution process of this mechanism is as follows: First, the trust evidence and the corresponding weight output by the reinforcement learning model are weighted and summed to obtain the node trust value, and the trust value is used to distinguish between trusted nodes and suspicious nodes. Second, an interval two-type fuzzy logic system is introduced, with relative energy waste (REW) and relative invalid forwarding (RIF) as inputs. Through the processes of fuzzification, fuzzy reasoning, type reduction, and defuzzification, the damage degree of the suspicious node behavior is quantified. Finally, according to the quantification result of the damage degree, a judgment is made. If the damage degree is at a low level, it is judged as an accidental fault node, and a redemption operation is performed. If the damage degree is at a high level, it is judged as a malicious node, and an isolation operation is performed.

[0010] The transmission data process of the sensor network specifically includes:

[0011] S5: The process of obtaining the transmission data of the sensor network includes: using a clustering algorithm to divide the nodes in the sensor network into multiple clusters, and selecting the cluster head node of each cluster through performance evaluation. The member nodes within a cluster are close to the cluster head node, so the member nodes within the cluster use single-hop communication to send their transmission data in network operation to the cluster head node. The cluster head nodes use a multi-hop communication protocol to send the collected transmission data of the cluster nodes to the background server, realizing centralized storage and processing of the transmission data.

[0012] The trust evidence specifically includes:

[0013] S6: The trust evidence specifically includes communication evidence F representing the communication success rate of the node, real-time evidence R measuring the real-time performance of the communication, energy evidence E reflecting the abnormal degree of energy consumption of the node, and length evidence L identifying the abnormality of the data packet length transmission mode.

[0014] The construction of the dynamic trust model includes:

[0015] S7: A dynamic trust model based on deep reinforcement learning algorithm is constructed, which is realized through a Markov decision process (s, a, r, p, γ) framework. In this model, s is the input state of the model, a is the output action of the model, r is the reward function, p is the state transition probability function, and γ is the discount factor.

[0016] S8: Combine the trust evidence and set the input state of the model as follows:

[0017] s = (F, R, E, L)

[0018] Where F represents communication evidence, R represents real-time evidence, E represents energy evidence, and L represents hair length evidence.

[0019] S9: Define the action space of the Markov decision process, i.e., the output action a of the model is the weight combination of the trust evidence, which is specifically as follows:

[0020] a = (k1, k2, k3, k4)

[0021] Where (k1, k2, k3, k4) correspond to the weights of (F, R, E, L), respectively.

[0022] S10: The reward function r is defined as the feedback value of the combination of "weight distribution uniformity" and "weight constraint satisfaction degree", and the calculation formula is:

[0023] r = H(a) · r conditions

[0024] Where H(a) is the entropy value of the action, which measures the uniformity of the action distribution; r conditions is the reward for meeting the constraints, which aims to ensure the rationality and pertinence of the action weight distribution. Its value is determined by the compliance check of multi-dimensional constraint items, including the consistency evaluation of the order of evidence and weight, the weight threshold evaluation for different numerical level evidence, and the independent score of each check. The cumulative score is obtained through this design to ensure that the weight distribution not only fits the actual information value of the evidence, but also avoids the occurrence of extreme weight.

[0025] S11: Define the discount factor γ, which is used to calculate the cumulative reward value in the whole process. The larger the value of γ, the more the model focuses on long-term returns.

[0026] S12: Combine the defined state space, action space, reward function, and discount factor to build the Actor-Critic double network structure of DDPG algorithm. The Actor network outputs the weight of each evidence, and the Critic network evaluates the action value corresponding to the weight.

[0027] The construction process of the trust decision mechanism is as follows:

[0028] S13: The construction process of the trust decision-making mechanism based on the type-two fuzzy logic system is as follows:

[0029] S14: First, the trust evidence is weighted and summed with the corresponding weights output by the reinforcement learning model to obtain the node trust value. A preset trust value threshold is set, and nodes below the threshold are judged as suspicious nodes.

[0030] S15: Next, the relative energy waste (REW) and relative ineffective forwarding (RIF) of the suspicious nodes are input into the interval type II fuzzy logic system. Through fuzzification, rule reasoning, type reduction and defuzzification operations, the numerical value of the node damage degree is obtained.

[0031] S16: Fuzzification: Map REW and RIF to the three fuzzy sets “Low”, “Medium” and “High” respectively, and determine the membership degree of each input parameter to the fuzzy set;

[0032] S17: Fuzzy reasoning: The Takagi-sugeno-Kang (TSK) rule base is adopted, which includes 9 rules such as "If REW is Low and RIF is Low, then the degree of destruction is 0". The type II confidence of each rule is calculated based on the membership degree of the input parameters, and the global type II fuzzy set of the degree of destruction is obtained by fusion.

[0033] S18: Type Reduction: A type of fuzzy set that calculates the degree of destruction based on rule confidence;

[0034] S19: Defuzzification: Converts a type of fuzzy set into a specific numerical value indicating the degree of destruction;

[0035] S20: Finally, if the value is lower than the set threshold, the suspicious node will be identified as a faulty node and a redemption operation will be performed; otherwise, it will be identified as a malicious node and an isolation operation will be performed.

[0036] The application has the beneficial effects that: a comprehensive solution is provided for the core pain points of wireless sensor network trust management: on the one hand, through the combination of multi-dimensional trust evidence and interval type-2 fuzzy logic redemption mechanism, the "non-malicious anomaly caused by environmental interference / node failure" and the "malicious anomaly caused by attack" can be accurately distinguished, the normal node misjudgment rate is effectively reduced, the network resource waste and the sensing coverage vulnerability are reduced; on the other hand, the trust modeling core is constructed by combining the deep reinforcement learning based on DDPG algorithm and the Markov decision process, the dynamic attack strategy of the malicious node can be captured in real time, the evidence weight is dynamically optimized through reward feedback, the limitation of the traditional model "only capable of coping with known static attacks" is broken through, the adaptive ability to dynamic attacks is enhanced; at the same time, the model balances the short-term evaluation accuracy and the long-term stability through the discount factor, and combines the redemption mechanism to retain the recoverable nodes, which not only guarantees the real-time and robustness of trust evaluation, but also optimizes the network resource utilization, indirectly prolongs the life cycle of the sensor network, and fully adapts to the complex and changeable sensor network deployment scene requirements. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of the malicious node detection method according to the application is shown in the figure.

[0038] Figure 2 A DDPG algorithm training framework diagram according to the application is shown in the figure.

[0039] Figure 3 A type-2 fuzzy logic system block diagram according to the application is shown in the figure.

[0040] Figure 4 A membership function diagram according to the application is shown in the figure. DETAILED DESCRIPTION

[0041] The preferred embodiments of the application will be described in detail below with reference to the accompanying drawings; it should be understood that the preferred embodiments are only for illustrating the application, and are not intended to limit the protection scope of the application.

[0042] As shown in the figure, a sensor network malicious node detection method based on redeemable deep reinforcement learning includes: Figure 1

[0043] S1: Obtain the transmission data of the sensor network node, wherein the transmission data includes the transmission data when the network is initially running and the transmission data when the network is normally running;

[0044] ​The specific process is: the nodes in the sensor network are divided into multiple clusters by using a clustering algorithm, and the cluster head nodes of each cluster are screened through performance evaluation; the member nodes in the cluster are close to the cluster head nodes, so the member nodes in the cluster send their transmission data in network operation to the cluster head nodes by using single-hop communication mode; the cluster head nodes adopt multi-hop communication protocol to send the collected transmission data of the cluster nodes to the background server after aggregation, so as to realize centralized storage and processing of the transmission data.

[0045] S2: feature extraction is performed on the transmission data to obtain trust evidence representing different features, and based on the characteristics that there are fewer malicious nodes in the initial running stage of the network, a model training data set is constructed based on the transmission data in the initial running stage, which is used as the benchmark data set for pre-training of the trust model.

[0046] The server calculates the trust evidence corresponding to each node based on the collected transmission data;

[0047] The trust evidence mainly includes communication evidence F representing the success rate of node communication, real-time evidence R measuring the real-time performance of communication, energy evidence E reflecting the abnormal degree of node energy consumption, and length evidence L identifying the abnormal transmission mode of data packet length.

[0048] The communication evidence F represents the success rate of node communication, which is mainly represented by the communication success rate of transmission behavior. In a period of time, the number of communication successes is s, and the number of communication failures is f, then the communication evidence F is calculated as follows:

[0049]

[0050] Wherein,

[0051] The real-time evidence R is defined as the real-time forwarding rate for the demand of time-sensitive scene, which quantifies the communication delay caused by malicious delay or environmental interference.

[0052]

[0053] Wherein, n delay represents the number of delayed communications, n success represents the number of successful communications.

[0054] The malicious node will cause its energy consumption to be different from that of the normal node when it implements malicious behavior, so the energy evidence focuses on the change trend of the energy consumption rate, and the calculation formula is as follows:

[0055]

[0056] Wherein, σ(t) is the standard deviation of the energy consumption rate of the node in a period of time, and Δ(t) is the range of the energy consumption rate of the node in a period of time.

[0057] In WSN, nodes will transmit data according to some communication protocol standards, so the length of data packets sent by normal nodes in a period of time has certain regularity. Malicious nodes will discard data packets or conduct DoS attacks, so the length of data packets sent by malicious nodes will be significantly different from that of normal nodes. Therefore, the length of the evidence L is calculated based on the ratio of the length of data packets transmitted by the node to the number of communications, and the calculation formula is as follows:

[0058]

[0059] Where y(t) is the ratio of the total length of data packets transmitted by the node to the number of communications in a period of time. is the average value of all nodes y(t).

[0060] Generally, at the initial stage of network deployment, the number of malicious nodes is small, and the transmission data at this time is used to build a training data set.

[0061] S3: Based on the deep reinforcement learning algorithm, a dynamic trust model is constructed, the deep deterministic policy gradient (DDPG) algorithm is used as the training framework, and the state space, action space and reward function are defined; Based on the above parameters, an Actor-Critic double network structure is constructed, and the training data set is used to pre-train the network to obtain a pre-trained trust model, at this time the model has the ability to distinguish the basic behavior pattern of normal nodes; In normal operation, input the latest transmission data of the node into the pre-trained model, and output the real-time weight of each trust evidence; When the node's bad behavior is monitored, immediately adjust the weight distribution of each trust evidence and update the model parameters, so that the trust evaluation ability is continuously optimized with the change of network state, and the evolution of the model from "initial baseline model" to "dynamic adaptive model" is promoted.

[0062] The dynamic trust model constructed based on the deep reinforcement learning algorithm realizes modeling through the Markov decision process (s, a, r, p, γ) framework. In this model, s is the input state of the model, a is the output action of the model, r is the reward function, p is the state transition probability function, and γ is the discount factor.

[0063] The input state of the model is set as:

[0064] s = (F, R, E, L)

[0065] Where F represents the communication evidence, R represents the real-time evidence, E represents the energy evidence, and L represents the length of the evidence;

[0066] The action space of the Markov model, i.e. the output action a of the model, is defined as the combination of the weights of the trust evidence:

[0067] a = (k1, k2, k3, k4)

[0068] where (k1, k2, k3, k4) correspond to the weights of (F, R, E, L) respectively.

[0069] The reward function r is defined as the feedback value of the comprehensive "weight distribution uniformity" and "weight constraint satisfaction", and the calculation formula is:

[0070] r = H(a) r conditions

[0071] where H(a) is the entropy value of the action, which measures the uniformity of the action distribution; r conditions is the reward of the constraint condition. The calculation formula is as follows:

[0072]

[0073] r conditions = r condition1 + r condition2 + r condition3 + r condition4

[0074] where ε is a small constant to avoid taking the logarithm of zero. r condition is the reward of checking whether the evidence meets different conditions respectively, 1 if it meets, and 0 if it does not meet. condition1 checks whether the order of the evidence is consistent with the order of the weights. condition2, condition3, condition4 check whether different weights meet the lower limit restriction: the weight of the smallest evidence is greater than 0.5, the weight of the second smallest evidence is greater than 0.25, and the weight of the second largest evidence is greater than 0.125. Through this design, it is ensured that the weight distribution is consistent with the actual information value of the evidence, and the emergence of extreme weights is avoided.

[0075] As Figure 2 shown, the Critic policy network and the Actor policy network in the DDPG algorithm are constructed.

[0076] The Critic policy network includes Critic online network and Critic target network. The Critic online network is responsible for the iterative update of network parameters μ, and is responsible for calculating the target Q value y i = r i + YQ'(s i+1 , π'(s i+1 | θ') | μ'); the Critic policy network parameter μ' is copied from μ regularly; the parameter update of the Critic network is realized by calculating the loss of the Critic network, and then using gradient descent:

[0077]

[0078] The actor policy network includes an actor online network and an actor target network. The actor online network is responsible for iterative updating of policy network parameters θ, and is responsible for selecting a current action a according to a current state s, for generating s', r with the environment, and the actor target network is responsible for selecting an optimal next action a' according to a sampled next state s' in the experience replay pool. The actor target network θ' is periodically copied from θ. The parameter updating of the actor network is achieved by using gradient ascent through the following steps: calculating the actor network target based on the value evaluation of the action by the critic network, and then obtaining the policy gradient:

[0079]

[0080] The updating mode of the target network parameters θ', μ' is as follows:

[0081] θ'←τθ+(1-τ)θ'

[0082] μ'←τμ+(1-τ)μ'

[0083] S5: Based on the real-time weight output by the trust model, a trust decision mechanism based on a two-type fuzzy logic system is constructed. The execution process of the mechanism is as follows: first, the trust evidence and the corresponding weight output by the reinforcement learning model are weighted and summed to obtain a node trust value, and the trust value is used to distinguish between trust nodes and suspicious nodes; second, an interval two-type fuzzy logic system is introduced, with relative energy waste (REW) and relative invalid forwarding (RIF) as inputs, and the damage degree of the suspicious node behavior is quantified through the processes of fuzzification, fuzzy reasoning, type reduction and defuzzification; finally, according to the quantification result of the damage degree, it is determined whether the damage degree is at a low level or not. If the damage degree is at a low level, it is determined that the node is an accidental failure node, and a redemption operation is performed; if the damage degree is at a high level, it is determined that the node is a malicious node, and an isolation operation is performed.

[0084] As shown in Figure 3 , the interval two-type fuzzy logic system includes four core modules: a fuzzy device, a fuzzy reasoning device, a type reduction device and a defuzzification device. The functions and implementation processes of the modules are as follows:

[0085] The fuzzy device module maps the input parameters REW and RIF into corresponding two-type fuzzy sets (i.e. output membership interval) according to the fuzzy set definition in the knowledge base, providing a basis for subsequent fuzzy reasoning.

[0086] REW represents the degree of invalid consumption of other nodes on the transmission path caused by the transmission failure of the suspicious node, and the calculation formula is as follows:

[0087]

[0088] wherein, the actual energy waste amount of the suspicious node, the average energy waste amount of the trusted nodes in the same time network. RIF represents the probability of non-delayed communication failure of the suspicious node, and the calculation formula is:

[0089]

[0090] wherein, the actual number of invalid forwarding of the suspicious node, the average number of invalid forwarding of the trusted nodes in the same time network.

[0091] As shown in Figure 4 three fuzzy sets of "Low", "Medium" and "High" are defined for REW and RIF, the fuzzy set "Medium" adopts a triangular membership function, and the fuzzy sets "Low" and "High" adopt irregular trapezoidal membership functions. After inputting a group of specific REW and RIF values into the fuzzy ware, the membership intervals of the six groups of bivalent fuzzy sets are output through the fuzzy membership function calculation, reflecting the uncertainty range of the parameters belonging to the corresponding fuzzy set.

[0092] The fuzzy inference ware module inputs the six groups of membership intervals output by the fuzzy ware, combines with the preset fuzzy rule base, maps the system input into the bivalent fuzzy set of the damage degree, so as to realize logical reasoning. The fuzzy rule base adopts Takagi-Sugeno-Kang (TSK) type IF-THEN rule to construct the rule base, and the specific definition is shown in Table 1: Table 1 Rule base

[0093] Rule Number REW RIF Degree of Damage 1 Low Low 0 2 Medium Low 0.1 3 High Low 0.3 4 Low Medium 0.3 5 Medium Medium 0.5 6 High Medium 0.7 7 Low High 0.7 8 Medium High 0.9 9 High High 1

[0094] The specific process of logical reasoning includes: matching the input membership intervals one by one with the premise conditions of the 9 rules, and calculating the confidence interval of each rule; for each successfully matched rule, the confidence interval is combined with the "damage degree" to generate the bivalent fuzzy set of the damage degree: taking the damage degree as the core value, and the confidence interval as the interval set of the uncertainty range.

[0095] The defuzzifier module compresses the bivalent fuzzy set of the damage degree into a single deterministic interval, and specifically adopts the "interval barycenter defuzzification method": first, the minimum value of the left endpoint and the maximum value of the right endpoint of all confidence intervals are extracted to form an initial merged interval; second, the abnormal sub-interval which has no overlap with the majority of fuzzy intervals in the initial interval is removed; finally, the intersection interval of the remaining effective intervals is calculated as the deterministic interval after defuzzification.

[0096] The defuzzifier module takes the "midpoint value" of the deterministic interval [L, U] after defuzzification as the accurate value of the damage degree by the "interval midpoint method", and the calculation formula is:

[0097]

[0098] Finally, according to the damage degree output by the deobfuscator, a redemption threshold is set, and the specific value needs to consider network security and node utilization. If the damage degree of the suspicious node is less than the redemption threshold, it is determined that the node anomaly is caused by environmental interference, the network access permission of the node is retained, the suspicious label is cleared, the node is allowed to continue to participate in transmission, and this time of anomaly is recorded as a failure type anomaly, and the subsequent system preferentially improves the node state through network maintenance; Otherwise, it is determined that the node anomaly is caused by malicious attack, the communication link between the node and the network is immediately cut off, it is marked as a malicious node, and it is added to the network blacklist to prevent it from further consuming network resources or damaging data transmission.

[0099] The application is suitable for sensor network trust management, especially for scenes with complex environmental interference and dynamic malicious node attacks, and typical applications include industrial Internet of Things monitoring, environmental perception, intelligent traffic node communication, etc.

[0100] Traditional sensor network trust management relies on static rules, which is easy to mis-isolate normal nodes and miss malicious nodes, and there are resource waste and security risks. The application can adapt to network changes, filter interference and accurately identify malicious nodes through "reinforcement learning dynamic evaluation + two-type fuzzy logic trust decision", reduce the mis-isolation rate of normal nodes, reduce data packet loss and energy waste, and provide technical support for scenes with high requirements for data reliability and network continuity.

[0101] The above only describes the preferred embodiments of the application and is not used to limit the application. Obviously, those skilled in the art can make various modifications and changes to the application without departing from the spirit and scope of the application. Therefore, if these modifications and changes of the application belong to the scope of the claims of the application and equivalent technologies, the application also intends to include these modifications and changes.

Claims

1. A method for detecting malicious nodes in sensor networks based on redeemable deep reinforcement learning, characterized in that, include: S1: Obtain the transmission data of the sensor network node, including the transmission data during the initial operation of the network and the transmission data during normal operation of the network. S2: Extract features from the transmitted data to obtain trust evidence representing different features. Taking advantage of the fact that there are fewer malicious nodes in the initial stage of network operation, build a model training dataset based on the transmitted data in the initial stage of operation as a benchmark dataset for pre-training the trust model. S3: A dynamic trust model is constructed based on deep reinforcement learning algorithms. The Deep Deterministic Policy Gradient (DDPG) algorithm is used as the training framework, and the state space, action space, and reward function are clearly defined. Based on the above parameters, an Actor-Critic dual network structure is constructed. The network is pre-trained using the constructed training dataset to obtain a pre-trained trust model. At this point, the model has the ability to identify the basic behavioral patterns of normal nodes. During normal operation, the latest transmitted data of the nodes is input into the pre-trained model, and the real-time weights of each trust evidence are output. When a node's misbehavior is detected, the weight allocation of each trust evidence is immediately adjusted and the model parameters are updated, so that the trust evaluation capability is continuously optimized as the network state changes, driving the model to evolve from an "initial baseline model" to a "dynamically adapted model". S4: Based on the real-time weights output by the trust model, a trust decision-making mechanism based on a type-II fuzzy logic system is constructed. The execution process of this mechanism is as follows: First, the trust evidence and the corresponding weights output by the reinforcement learning model are weighted and summed to obtain the node trust value, and this trust value is used to distinguish between trustworthy nodes and suspicious nodes. Second, an interval type-II fuzzy logic system is introduced, with relative energy waste (REW) and relative invalid forwarding (RIF) as inputs. Through fuzzification, fuzzy inference, type reduction, and defuzzification processes, the degree of damage to the behavior of suspicious nodes is quantified. Finally, a judgment is made based on the quantification result of the degree of damage. If the degree of damage is at a low level, it is judged as an occasional fault node, and a redemption operation is performed. If the level of damage is high, it is identified as a malicious node, and isolation operations are performed.

2. The method for detecting malicious nodes in a sensor network based on redeemable deep reinforcement learning according to claim 1, characterized in that: S5: The process of acquiring the transmission data of the sensor network includes: using a clustering algorithm to divide the nodes in the sensor network into multiple clusters, and selecting the cluster head node of each cluster through performance evaluation; since the member nodes within a cluster are close to the cluster head node, the member nodes within the cluster use a single-hop communication method to send their own transmission data during network operation to the cluster head node; the cluster head nodes use a multi-hop communication protocol to summarize the collected transmission data of the nodes within the cluster and send it to the backend server to realize centralized storage and processing of the transmission data.

3. The method for detecting malicious nodes in a sensor network based on redeemable deep reinforcement learning according to claim 1, characterized in that: S6: Trust evidence specifically includes communication evidence F, which characterizes the success rate of node communication; real-time evidence R, which measures the real-time performance of communication; energy evidence E, which reflects the degree of abnormality in node energy consumption; and transmission length evidence L, which identifies abnormalities in packet length transmission patterns.

4. The method for detecting malicious nodes in a sensor network based on redeemable deep reinforcement learning according to claim 1, characterized in that: S7: A dynamic trust model built on deep reinforcement learning algorithm, which is modeled through Markov decision process (s,a,r,p,γ) framework. In this model, s is the input state of the model, a is the output action of the model, r is the reward function, p is the state transition probability function, and γ is the discount coefficient. S8: Based on trust evidence, set the model input state as follows: s = (F, R, E, L) Where F represents communication evidence, R represents real-time evidence, E represents energy evidence, and L represents transmission length evidence; S9: Define the action space of the Markov decision process, that is, the output action 'a' of the model is a weighted combination of trust evidence, in the following form: a = (k1, k2, k3, k4) Where (k1,k2,k3,k4) correspond to the weights of (F,R,E,L), respectively. S10: The reward function r is defined as a feedback value that combines "weight distribution uniformity" and "weight constraint satisfaction," and the calculation formula is as follows: r=H(a)·r conditions Where H(a) is the entropy value of the action, which measures the uniformity of the action distribution; r conditions To ensure that the reward meets the constraints, the allocation of action weights is designed to be reasonable and targeted. The values ​​are determined through compliance checks of multi-dimensional constraints, including the consistency assessment of the ranking of evidence and weights, and the assessment of the weight thresholds for evidence of different numerical levels. This design ensures that the weight allocation is both in line with the actual information value of the evidence and avoids the occurrence of extreme weights. S11: Define the discount factor γ, which is used to calculate the cumulative return over the entire process; the larger the value of γ, the more the model focuses on long-term returns. S12: Combining the defined state space, behavior space, reward function, and discount factor, construct the Actor-Critic dual network structure of the DDPG algorithm. The Actor network outputs the weights of each piece of evidence, and the Critic network evaluates the action value corresponding to the weights.

5. The method for detecting malicious nodes in a sensor network based on redeemable deep reinforcement learning according to claim 1, characterized in that: S13: The specific process of the trust decision-making mechanism based on the type-two fuzzy logic system is as follows: S14: First, the trust evidence is weighted and summed with the corresponding weights output by the reinforcement learning model to obtain the node trust value. A preset trust value threshold is set, and nodes below the threshold are judged as suspicious nodes. S15: Next, the relative energy waste (REW) and relative ineffective forwarding (RIF) of the suspicious nodes are input into the interval type II fuzzy logic system. Through fuzzification, rule reasoning, type reduction and defuzzification operations, the numerical value of the node damage degree is obtained. S16: Fuzzification: Map REW and RIF to the three fuzzy sets "Low", "Medium" and "High" respectively, and determine the membership degree of each input parameter to the fuzzy set; S17: Fuzzy reasoning: The Takagi-Sugeno-Kang (TSK) rule base is adopted, which includes 9 rules such as "If REW is Low and RIF is Low, then the degree of destruction is 0". The type II confidence of each rule is calculated based on the membership degree of the input parameters, and the global type II fuzzy set of the degree of destruction is obtained by fusion. S18: Type Reduction: A type of fuzzy set that calculates the degree of destruction based on rule confidence; S19: Defuzzification: Converts a type of fuzzy set into a specific numerical value indicating the degree of destruction; S20: Finally, if the value is lower than the set threshold, the suspicious node will be identified as a faulty node and a redemption operation will be performed; otherwise, it will be identified as a malicious node and an isolation operation will be performed.

Images