A method for detecting and defending against cache pollution attacks based on deep reinforcement learning

CN122513780APending Publication Date: 2026-08-04TIANJIN UNIV OF TECH & EDUCATION (TEACHER DEV CENT OF CHINA VOCATIONAL TRAINING & GUIDANCE)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610984347.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-03
Publication Date
2026-08-04

AI Technical Summary

Technical Problem

[0004]然而,边缘计算环境具有开放性,实体来源多样,物联网设备可轻易地发起任务请求,使得边缘计算环境面临缓存污染攻击的威胁

Benefits of technology

[0057] On the one hand, the dynamic nature of edge computing environments, such as the movement of IoT devices and changes in task request frequency, makes conventional methods difficult to adapt. The method of this invention, through interaction with the environment and continuous training updates, can adapt well to the dynamic changes in edge computing environments. On the other hand, task requests and actions cause fluctuations in edge caching. The method of this invention improves convergence by virtually reapplying action data to the reward data of the original edge environment state. Furthermore, conventional cache pollution attack defense methods do not consider the impact of falsely rejecting normal task requests and forwarding suspected malicious requests. Therefore, this invention, based on the statistical characteristics of IoT devices, jointly restricts service content caching and task request forwarding, balancing false rejections and forwarding burden while improving performance indicators such as detection rate and edge cache hit rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122513780A_ABST
    Figure CN122513780A_ABST
Patent Text Reader

Abstract

A cache pollution attack detection and defense method based on deep reinforcement learning is presented. It includes steps such as constructing state data, constructing an online neural network and a target neural network, constructing and training the online neural network, and performing cache pollution attack detection and defense. The advantages of this invention are: by interacting with the environment and continuously updating the training, it can adapt well to the dynamic changes of the edge computing environment. Task requests and actions can cause fluctuations in edge caching. This invention improves the convergence of the method by virtually reapplying action data to the reward data of the original edge environment state. Furthermore, conventional cache pollution attack defense methods do not consider the impact of falsely rejecting normal task requests and forwarding suspected malicious requests. Therefore, this invention, based on the statistical characteristics of IoT devices, jointly restricts service content caching and restricts task request forwarding, improving performance indicators such as detection rate and edge cache hit rate while balancing false rejections and forwarding burden.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of edge computing security technology, and in particular relates to a method for detecting and defending against cache pollution attacks based on deep reinforcement learning. Background Technology

[0002] Because IoT devices have limited resources, they struggle to handle computationally intensive and latency-sensitive tasks. Compared to cloud computing, edge computing provides IoT devices with storage and computing resources locally, effectively reducing the number of task requests IoT devices make to the cloud, thereby alleviating network congestion and reducing response latency. Although edge computing has more abundant resources than IoT devices, its distributed nature means that its resources are still less than those of centralized cloud computing, making it difficult to provide IoT devices with all the services they need to make task requests.

[0003] Against this backdrop, edge caching, as a key technology in edge computing, plays a crucial role. By caching some popular service content, it improves cache hit rate, fully utilizing edge computing storage resources. This allows most tasks to be processed by edge computing, with only a small portion needing to be forwarded to cloud computing, thus enhancing service quality and user experience.

[0004] However, the open nature of edge computing environments, coupled with the diverse origins of entities and the ease with which IoT devices can initiate task requests, exposes them to cache poisoning attacks. Attackers can reduce the caching chance of popular services by continuously requesting different non-popular service content or repeatedly requesting the same set of non-popular service content, while caching non-popular service content, thus lowering the cache hit rate and severely impacting system performance. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a method for detecting and defending against cache pollution attacks based on deep reinforcement learning.

[0006] To achieve the above objectives, the cache pollution attack detection and defense method based on deep reinforcement learning provided by this invention includes the following steps performed in sequence:

[0007] S1) Based on the task request information initiated by the IoT device, construct multiple environmental state features of the edge node, and obtain state data from the above environmental state features;

[0008] S2) Construct the online neural network and the target neural network, and combine the above... The state data for a given time period is input into an online neural network to obtain action data for edge nodes. This action data is then virtually applied back to the original state data to obtain reward data, and the next state data is collected. The state data, action data, reward data, and next state data of a time period constitute a training data item, and all training data items constitute the training data experience pool.

[0009] S3) Disorder the training data items in the training data experience pool, then randomly select multiple training data items to form a training data set, and use this training data set as input to train the online neural network. Construct an objective function based on the training data set and the target neural network, and then construct a loss function to update the parameters of the online neural network based on the training data set and its corresponding objective function. This updates the parameters of the online neural network and obtains the trained online neural network.

[0010] S4) Based on the statistical characteristics of IoT devices, the trained online neural network is used to detect and defend against cache pollution attacks by jointly restricting service content caching and restricting task request forwarding.

[0011] In step S1), the method for constructing multiple environmental state features of the edge node based on the task request information initiated by the IoT device, and obtaining state data from the above environmental state features, is as follows:

[0012] S101) The edge nodes collect task request information initiated by IoT devices during the current time period, and construct the following edge environment features based on this information, as shown in the following formula:

[0013] Edge cache hit rate ,for The percentage of task requests that are successfully completed within a given time period is expressed as follows:

[0014]

[0015] in, for The total number of task requests within the time period. for The number of times a task request was successfully completed within a given time period;

[0016] Task request ratio ,for Task request service content within the time period The ratio is expressed as:

[0017]

[0018] in, Service content for task requests The number of times;

[0019] Task Request Types , used to reflect The types of task requests within a given time period are represented as follows:

[0020]

[0021] in, for A vector set of task request ratios for each type of service content within a given time period. for The L0 vector norm;

[0022] Average task request interval ,for The average time interval between requests for the same service within a given period. Service content for task requests The average request time interval is expressed as:

[0023]

[0024] in, for Task request service content within the time period The Sub-task request and the first The time interval for the next task request The maximum time interval;

[0025] S102) Based on the above-mentioned edge environment characteristics, obtain Status data for a given time period .

[0026] In step S2), the construction of the online neural network and the target neural network involves the above-mentioned... The state data for a given time period is input into an online neural network to obtain action data for edge nodes. This action data is then virtually applied back to the original state data to obtain reward data, and the next state data is collected. The state data, action data, reward data, and next state data for a given time period constitute a training data item. The method for constructing a training data experience pool from all training data items is as follows:

[0027] (S201) Construct an online neural network whose input is state data, namely the edge cache hit rate, task request ratio, task request type, and average task request interval constructed from task request information. Its output dimension is the same as the number of defensive action categories that the edge node can adopt, and the output result is the action value of each action. This online neural network uses a multilayer perceptron, and the ReLU activation function is applied between each layer. The activation function is not used when the last layer outputs. Construct a target neural network. The input, output, and network structure of this neural network are the same as those of the online neural network, only the neural network parameters are different. Therefore, under the same input data, the action value evaluations of each action output by the two neural networks are different. The output of the target neural network serves as an important basis for constructing the target label of the online neural network output, helping the online neural network to update.

[0028] S202) When After the task request for the specified time period arrives, the edge node uses Status data for a given time period The online neural network is trained by using random numbers as input. During training, to improve exploration and increase the diversity of training data, random numbers are used. With probability Randomly select action data Based on the evaluation results of each action value by the online neural network, if the task request is considered a normal task request, then from the action space... Select action data Edge nodes cache the necessary service content; if a task request is considered to be abnormal, it is processed from the action space. Select action data Edge nodes do not cache the services they require to avoid consuming limited storage space; among them, the action space... Strategies to be taken after detecting cache pollution attacks on task requests; probability The decay rate decreases with each training session, and is multiplied by a decay factor after each training session. until probability Change to minimum value So far, therefore, probability The decay process is represented as:

[0029]

[0030] The action data is represented as follows:

[0031]

[0032] in, To be based on state data With online neural network parameters Obtained motion data The value of the action;

[0033] S203) Virtually reapply the above motion data to... The edge caching environment during a specific time period is used to obtain the virtual edge cache hit rate. This is then combined with the edge cache hit rate obtained in step S1). This yields reward data used to guide the training direction. , is represented as:

[0034]

[0035] S204) will move data After interacting with the edge environment, collect the next state data. ;

[0036] S205) Collect status data for each time period. Motion data Reward data and next state data , constitute training data items ;

[0037] S206) consists of a training data experience pool composed of all training data items. .

[0038] In step S3), the order of training data items in the training data experience pool is shuffled, and multiple training data items are randomly selected to form a training dataset, which is then used as input to train the online neural network. An objective function is constructed based on the training data set and the target neural network. Then, a loss function is constructed based on the training data set and its corresponding objective function to update the parameters of the online neural network, thereby updating the parameters of the online neural network. The method for obtaining the trained online neural network is as follows:

[0039] S301) The training data experience pool The order of the training data items is shuffled, and multiple training data items are randomly selected to form a training data set. Used for online neural network training;

[0040] S302) Based on the above training data set Construct the objective function using the target neural network; when using the training dataset Data items in When, the objective function Represented as:

[0041]

[0042] in, This is a discount factor, representing the degree of importance placed on the value of future actions; For the target neural network parameters;

[0043] S303) Based on the training data set And its corresponding objective function, constructing a loss function for updating the parameters of the online neural network; training data items The loss function is expressed as:

[0044]

[0045] in, Represents the training data set The expected mean of all training data items;

[0046] S304) Minimize the above loss function using gradient descent. This makes the online neural network parameters This allows for updates, resulting in a trained online neural network.

[0047] In step S4), the method for detecting and defending against cache pollution attacks by jointly restricting service content caching and task request forwarding based on the statistical characteristics of IoT devices and utilizing the trained online neural network is as follows:

[0048] S401) The IoT device, acting as the task requester, initiates a task request to the edge node in order to borrow the resources and related cached services of the edge node to complete the calculation and processing of the task.

[0049] S402) Edge nodes use the online neural network trained above to identify task requests;

[0050] S403) When a task request is identified as a normal request, the relevant cached service content remains in a state that can be cached by edge nodes, and the corresponding task processing operation is performed.

[0051] S404) When a task request is identified as a suspected cache poisoning attack, restrict the caching of the relevant cache service content for that task request;

[0052] S405) Statistically analyze the request frequency of relevant cached service content restricted by the task requests of each IoT device, and identify IoT devices whose request frequency exceeds the threshold as suspected malicious IoT devices;

[0053] S406) To avoid mistakenly rejecting normal task requests, task requests suspected of being cache poisoning attacks are re-evaluated based on whether the task requester is a suspected malicious IoT device.

[0054] (S407) If a task request suspected of being a cache poisoning attack comes from a suspected malicious IoT device, it will not be forwarded to the cloud computing node when the cache misses, in order to avoid wasting network resources.

[0055] (S408) If a task request suspected of being a cache poisoning attack comes from a normal IoT device, it will continue to be forwarded to the cloud computing node to process the task request when the cache misses, thus avoiding the mistaken rejection of normal task requests.

[0056] The cache pollution attack detection and defense method based on deep reinforcement learning provided by this invention has the following beneficial effects:

[0057] On the one hand, the dynamic nature of edge computing environments, such as the movement of IoT devices and changes in task request frequency, makes conventional methods difficult to adapt. The method of this invention, through interaction with the environment and continuous training updates, can adapt well to the dynamic changes in edge computing environments. On the other hand, task requests and actions cause fluctuations in edge caching. The method of this invention improves convergence by virtually reapplying action data to the reward data of the original edge environment state. Furthermore, conventional cache pollution attack defense methods do not consider the impact of falsely rejecting normal task requests and forwarding suspected malicious requests. Therefore, this invention, based on the statistical characteristics of IoT devices, jointly restricts service content caching and task request forwarding, balancing false rejections and forwarding burden while improving performance indicators such as detection rate and edge cache hit rate. Attached Figure Description

[0058] Figure 1 The flowchart illustrates the cache pollution attack detection and defense method based on deep reinforcement learning provided by this invention.

[0059] Figure 2 This is a flowchart of the process for defending against cache contamination attacks. Detailed Implementation

[0060] The present invention will now be described in detail with reference to the accompanying drawings.

[0061] like Figure 1 As shown, the cache pollution attack detection and defense method based on deep reinforcement learning provided by this invention includes the following steps performed in sequence:

[0062] S1) Based on the task request information initiated by the IoT device, construct multiple environmental state features of the edge node, and obtain state data from the above environmental state features;

[0063] The method is as follows:

[0064] S101) The edge nodes collect task request information initiated by IoT devices during the current time period, and construct the following edge environment features based on this information, as shown in the following formula:

[0065] Edge cache hit rate ,for The percentage of task requests that are successfully completed within a given time period is expressed as follows:

[0066]

[0067] in, for The total number of task requests within the time period. for The number of times a task request is hit within a time period, i.e., the number of times the edge node provides the relevant services required for the task request and provides task processing services; this edge environment feature can reflect the caching status and provide a basis for obtaining reward data.

[0068] Task request ratio ,for Task request service content within the time period The ratio is expressed as:

[0069]

[0070] in, Service content for task requests The number of times;

[0071] Task Request Types , used to reflect The types of task requests within a given time period are represented as follows:

[0072]

[0073] in, , for A vector set of task request ratios for each type of service content within a given time period. for The L0 vector norm;

[0074] Average task request interval ,for The average time interval between requests for the same service within a given period. Service content for task requests The average request time interval is expressed as:

[0075]

[0076] in, for Task request service content within the time period The Sub-task request and the first The time interval for the next task request The maximum time interval;

[0077] S102) Based on the above-mentioned edge environment characteristics, obtain Status data for a given time period .

[0078] S2) Construct the online neural network and the target neural network, and combine the above... The state data for a given time period is input into an online neural network to obtain action data for edge nodes. This action data is then virtually applied back to the original state data to obtain reward data, and the next state data is collected. The state data, action data, reward data, and next state data of a time period constitute a training data item, and all training data items constitute the training data experience pool.

[0079] The method is as follows:

[0080] (S201) Construct an online neural network for cache pollution attack detection and parameter updates. Its input consists of state data, including edge cache hit rate, task request ratio, task request type, and average task request interval, all constructed from task request information. Its output dimension is the same as the number of defensive action categories that edge nodes can employ, and the output is the action value of each action, representing the quality of executing that action under the current input state. This online neural network uses a multilayer perceptron, applying the ReLU activation function between layers, with no activation function used in the output of the last layer. A target neural network is then constructed. This target neural network has the same input, output, and network structure as the online neural network, differing only in its parameters. Therefore, under the same input data, the two neural networks output different action value evaluations for each action. The output of the target neural network serves as a crucial basis for constructing the target label for the online neural network, aiding in its updates.

[0081] S202) When After the task request for the specified time period arrives, the edge node uses Status data for a given time period The online neural network is trained by using random numbers as input. During training, to improve exploration and increase the diversity of training data, random numbers are used. ( ) with probability Randomly select action data Based on the evaluation results of each action value by the online neural network, if the task request is considered a normal task request, then from the action space... Select action data Edge nodes cache the necessary service content; if a task request is considered to be abnormal, it is processed from the action space. Select action data Edge nodes do not cache the services they require to avoid consuming limited storage space; among them, the action space... This refers to the countermeasures that can be taken after detecting cache pollution attacks on task requests, which determines whether to cache the relevant service content required by the task request; probability. The decay rate decreases with each training session, and is multiplied by a decay factor after each training session. ( ), until the probability Change to minimum value So far, therefore, probability The decay process is represented as:

[0082]

[0083] The action data is represented as follows:

[0084]

[0085] in, To be based on state data With online neural network parameters Obtained motion data The value of the action;

[0086] (S203) To avoid the volatility of the edge caching environment, the above action data is virtually reapplied to... The edge caching environment during a specific time period is used to obtain the virtual edge cache hit rate. This is then combined with the edge cache hit rate obtained in step 1). This yields reward data used to guide the training direction. , is represented as:

[0087]

[0088] S204) When edge nodes take different action data At that time, with the next period The arrival of new task requests will cause different changes in the characteristics of the edge environment; therefore, the motion data... After interacting with the edge environment, collect the next state data. This allows for adaptation to dynamic changes in the edge environment, while also enabling better determination of the accuracy of action values ​​output by the online neural network during training.

[0089] S205) Collect status data for each time period. Motion data Reward data and next state data , constitute training data items ;

[0090] S206) consists of a training data experience pool composed of all training data items. .

[0091] S3) Disorder the training data items in the training data experience pool, then randomly select multiple training data items to form a training data set, and use this training data set as input to train the online neural network. Construct an objective function based on the training data set and the target neural network, and then construct a loss function to update the parameters of the online neural network based on the training data set and its corresponding objective function. This updates the parameters of the online neural network and obtains the trained online neural network.

[0092] The method is as follows:

[0093] S301) The training data experience pool The order of the training data items is shuffled, and multiple training data items are randomly selected to form a training data set. Used for online neural network training;

[0094] S302) Based on the above training data set An objective function is constructed with the target neural network. This objective function is used to evaluate the accuracy of the output action value, and then update the parameters of the online neural network. This allows for the output of more accurate action values, thereby precisely selecting the optimal action data; when using training datasets... Data items in When, the objective function Represented as:

[0095]

[0096] in, This is a discount factor, representing the degree of importance placed on the value of future actions; These are the target neural network parameters, used to predict the value of actions in the next time period, thereby ensuring the accuracy of the online neural network parameters. The training stability; specifically, the predicted action value of the next state in the objective function is based on the next state data. As input, based on the target neural network parameters Perform linear and nonlinear transformations to obtain the target neural network parameters. The update involves copying the current parameters of the online neural network after a period of training. Received;

[0097] S303) Based on the training data set And its corresponding objective function, constructing a loss function for updating the parameters of the online neural network; the loss function represents the difference between the action value output by the online neural network and its objective function; training data items The loss function is expressed as:

[0098]

[0099] in, Represents the training data set The expected mean of all training data items;

[0100] S304) Minimize the above loss function using gradient descent. This makes the online neural network parameters This allows for updates, resulting in a trained online neural network.

[0101] S4) as Figure 2 As shown, based on the statistical characteristics of IoT devices, the trained online neural network is used to detect and defend against cache pollution attacks by jointly restricting service content caching and task request forwarding.

[0102] The method is as follows:

[0103] S401) The IoT device, acting as the task requester, initiates a task request to the edge node in order to borrow the resources and related cached services of the edge node to complete the calculation and processing of the task.

[0104] S402) Edge nodes use the online neural network trained above to identify task requests;

[0105] S403) When a task request is identified as a normal request, the relevant cached service content remains in a state that can be cached by edge nodes, and the corresponding task processing operation is performed.

[0106] S404) When a task request is identified as a suspected cache poisoning attack, restrict the caching of the relevant cache service content for that task request;

[0107] S405) Statistically analyze the request frequency of relevant cached service content restricted by the task requests of each IoT device, and identify IoT devices whose request frequency exceeds the threshold as suspected malicious IoT devices;

[0108] S406) To avoid mistakenly rejecting normal task requests, task requests suspected of being cache poisoning attacks are re-evaluated based on whether the task requester is a suspected malicious IoT device.

[0109] (S407) If a task request suspected of being a cache poisoning attack comes from a suspected malicious IoT device, it will not be forwarded to the cloud computing node when the cache misses, in order to avoid wasting network resources.

[0110] (S408) If a task request suspected of being a cache poisoning attack comes from a normal IoT device, it will continue to be forwarded to the cloud computing node to process the task request when the cache misses, thus avoiding the mistaken rejection of normal task requests.

Claims

1. A method for detecting and defending against cache pollution attacks based on deep reinforcement learning, characterized in that: The method for detecting and defending against cache pollution attacks based on deep reinforcement learning includes the following steps performed in sequence: S1) Based on the task request information initiated by the IoT device, construct multiple environmental state features of the edge node, and obtain state data from the above environmental state features; S2) Construct the online neural network and the target neural network, and combine the above... The state data for a given time period is input into an online neural network to obtain action data for edge nodes. This action data is then virtually applied back to the original state data to obtain reward data, and the next state data is collected. The state data, action data, reward data, and next state data of a time period constitute a training data item, and all training data items constitute the training data experience pool. S3) Disorder the training data items in the training data experience pool, then randomly select multiple training data items to form a training data set, and use this training data set as input to train the online neural network. Construct an objective function based on the training data set and the target neural network, and then construct a loss function to update the parameters of the online neural network based on the training data set and its corresponding objective function. This updates the parameters of the online neural network and obtains the trained online neural network. S4) Based on the statistical characteristics of IoT devices, the trained online neural network is used to detect and defend against cache pollution attacks by jointly restricting service content caching and restricting task request forwarding.

2. The cache pollution attack detection and defense method based on deep reinforcement learning according to claim 1, characterized in that: In step S1), the method for constructing multiple environmental state features of the edge node based on the task request information initiated by the IoT device, and obtaining state data from the above environmental state features, is as follows: S101) The edge nodes collect task request information initiated by IoT devices during the current time period, and construct the following edge environment features based on this information, as shown in the following formula: Edge cache hit rate ,for The percentage of task requests that are successfully completed within a given time period is expressed as follows: ; in, for The total number of task requests within the time period. for The number of times a task request was successfully completed within a given time period; Task request ratio ,for Task request service content within the time period The ratio is expressed as: ; in, Service content for task requests The number of times; Task Request Types , used to reflect The types of task requests within a given time period are represented as follows: ; in, for A vector set of task request ratios for each type of service content within a given time period. for The L0 vector norm; Average task request interval ,for The average time interval between requests for the same service within a given period. Service content for task requests The average request time interval is expressed as: ; in, for Task request service content within the time period The Sub-task request and the first The time interval for the next task request The maximum time interval; S102) Based on the above-mentioned edge environment characteristics, obtain Status data for a given time period .

3. The cache pollution attack detection and defense method based on deep reinforcement learning according to claim 1, characterized in that: In step S2), the construction of the online neural network and the target neural network involves the above-mentioned... The state data for a given time period is input into an online neural network to obtain action data for edge nodes. This action data is then virtually applied back to the original state data to obtain reward data, and the next state data is collected. The state data, action data, reward data, and next state data for a given time period constitute a training data item. The method for constructing a training data experience pool from all training data items is as follows: (S201) Construct an online neural network whose input is state data, namely the edge cache hit rate, task request ratio, task request type, and average task request interval constructed from task request information. Its output dimension is the same as the number of defensive action categories that the edge node can adopt, and the output result is the action value of each action. This online neural network uses a multilayer perceptron, and the ReLU activation function is applied between each layer. The activation function is not used when the last layer outputs. Construct a target neural network. The input, output, and network structure of this neural network are the same as those of the online neural network, only the neural network parameters are different. Therefore, under the same input data, the action value evaluations of each action output by the two neural networks are different. The output of the target neural network serves as an important basis for constructing the target label of the online neural network output, helping the online neural network to update. S202) When After the task request for the specified time period arrives, the edge node uses Status data for a given time period The online neural network is trained by using random numbers as input. During training, to improve exploration and increase the diversity of training data, random numbers are used. With probability Randomly select action data Based on the evaluation results of each action value by the online neural network, if the task request is considered a normal task request, then from the action space... Select action data Edge nodes cache the necessary service content; if a task request is considered to be abnormal, it is processed from the action space. Select action data Edge nodes do not cache the services they require to avoid consuming limited storage space; among them, the action space... Strategies to be taken after detecting cache pollution attacks on task requests; probability The decay rate decreases with each training session, and is multiplied by a decay factor after each training session. until probability Change to minimum value So far, therefore, probability The decay process is represented as: ; The action data is represented as follows: ; in, To be based on state data With online neural network parameters Obtained motion data The value of the action; S203) Virtually reapply the above motion data to... The edge caching environment during a specific time period is used to obtain the virtual edge cache hit rate. This is then combined with the edge cache hit rate obtained in step S1). This yields reward data used to guide the training direction. , is represented as: ; S204) will move data After interacting with the edge environment, collect the next state data. ; S205) Collect status data for each time period. Motion data Reward data and next state data , constitute training data items ; S206) consists of a training data experience pool composed of all training data items. .

4. The cache pollution attack detection and defense method based on deep reinforcement learning according to claim 1, characterized in that: In step S3), the order of training data items in the training data experience pool is shuffled, and multiple training data items are randomly selected to form a training dataset, which is then used as input to train the online neural network. An objective function is constructed based on the training data set and the target neural network. Then, a loss function is constructed based on the training data set and its corresponding objective function to update the parameters of the online neural network, thereby updating the parameters of the online neural network. The method for obtaining the trained online neural network is as follows: S301) The training data experience pool The order of the training data items is shuffled, and multiple training data items are randomly selected to form a training data set. Used for online neural network training; S302) Based on the above training data set Construct the objective function using the target neural network; when using the training dataset Data items in When, the objective function Represented as: ; in, This is a discount factor, representing the degree of importance placed on the value of future actions; For the target neural network parameters; S303) Based on the training data set And its corresponding objective function, constructing a loss function for updating the parameters of the online neural network; training data items The loss function is expressed as: ; in, Represents the training data set The expected mean of all training data items; S304) Minimize the above loss function using gradient descent. This makes the online neural network parameters This allows for updates, resulting in a trained online neural network.

5. The cache pollution attack detection and defense method based on deep reinforcement learning according to claim 1, characterized in that: In step S4), the method for detecting and defending against cache pollution attacks by jointly restricting service content caching and task request forwarding based on the statistical characteristics of IoT devices and utilizing the trained online neural network is as follows: S401) The IoT device, acting as the task requester, initiates a task request to the edge node in order to borrow the resources and related cached services of the edge node to complete the calculation and processing of the task. S402) Edge nodes use the online neural network trained above to identify task requests; S403) When a task request is identified as a normal request, the relevant cached service content remains in a state that can be cached by edge nodes, and the corresponding task processing operation is performed. S404) When a task request is identified as a suspected cache poisoning attack, restrict the caching of the relevant cache service content for that task request; S405) Statistically analyze the request frequency of relevant cached service content restricted by the task requests of each IoT device, and identify IoT devices whose request frequency exceeds the threshold as suspected malicious IoT devices; S406) To avoid mistakenly rejecting normal task requests, task requests suspected of being cache poisoning attacks are re-evaluated based on whether the task requester is a suspected malicious IoT device. (S407) If a task request suspected of being a cache poisoning attack comes from a suspected malicious IoT device, it will not be forwarded to the cloud computing node when the cache misses, in order to avoid wasting network resources. (S408) If a task request suspected of being a cache poisoning attack comes from a normal IoT device, it will continue to be forwarded to the cloud computing node to process the task request when the cache misses, thus avoiding the mistaken rejection of normal task requests.