Intelligent purchase supply chain network intrusion detection method and system

Through the selection of high-quality nodes and optimized bandwidth allocation in data middle-end layer, the inefficiency problem caused by the differences in data distribution and model performance in traditional federated learning is solved, and the performance and training efficiency of smart procurement supply chain network intrusion detection models are improved.

CN118921184APending Publication Date: 2024-11-08DATA TECHNOLOGY BRANCH OF NATIONAL ENERGY GROUP MATERIALS CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410849621.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional federated learning methods in smart procurement supply chain networks have low performance and training efficiency of intrusion detection models due to the differences in data distribution and model performance between enterprise nodes. There is a lack of effective standards to select high-quality nodes to participate in federated learning, which affects the performance of global models and Training time.

Method used

The state information of enterprise nodes is recovered through the middle-end data layer, candidate nodes are selected and migration performance indicators are calculated, and node selection and bandwidth allocation are optimized based on Markov decision-making process and deep deterministic strategy gradient algorithm to build the problem of maximizing migration efficiency and ensure nodes are guaranteed. The quality and efficiency of participation in federal learning.

Benefits of technology

The performance of the federated learning smart procurement supply chain network intrusion detection model has been improved, the training time has been shortened, the network security has been ensured, and the training efficiency has been improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921184A_ABST
    Figure CN118921184A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an intelligent purchase supply chain network intrusion detection method and system, and belongs to the technical field of network security. The method comprises the following steps: recovering state information of each enterprise node transmitted by an enterprise node layer, and selecting candidate enterprise nodes participating in a federation algorithm based on the state information; obtaining a migration performance index value of each candidate enterprise node participating in the federation algorithm; constructing a migration efficiency function, and constructing a migration efficiency maximization problem based on the migration efficiency function; and solving the migration efficiency maximization problem based on Markov decision process construction and a depth deterministic strategy gradient algorithm, updating an intrusion detection local model of each enterprise node based on a solving result, and executing corresponding network intrusion detection based on the intrusion detection local model of each enterprise node. According to the scheme, the problems of poor safety performance and low training efficiency of the existing network intrusion detection scheme are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a smart procurement supply chain network intrusion detection method and a smart procurement supply chain network intrusion detection system. Background Art

[0002] With the acceleration of the digitalization process of material enterprises and large-scale network access, the smart procurement supply chain network, as a key link connecting upstream and downstream enterprise information, can use intelligent logistics management to improve product production and material management efficiency and reduce resource consumption. However, the smart procurement supply chain network involves a large amount of sensitive data and transaction information, including supplier information, purchase orders, inventory information, etc. Once it is attacked by hackers or data is leaked, it will cause serious losses to the company's economic interests and reputation, and seriously affect the data security governance capabilities of the material company.

[0003] As one of the key technologies to enhance the network defense capabilities of the supply chain, the intrusion detection system (IDS) provides a strong guarantee for network security by monitoring suspected attack behaviors in network traffic.

[0004] As the core of artificial intelligence data analysis applications, machine learning algorithms can learn and extract complex features from large amounts of data, and are therefore widely used in intrusion detection systems. Given that sensitive data in smart procurement supply chain networks is crucial to the decision-making of material companies, its security cannot be underestimated. Once leaked or tampered with, it will bring huge economic losses.

[0005] Federated learning (FL) is particularly suitable for the actual deployment of smart procurement supply chain networks due to its distributed characteristics and data privacy protection mechanism. However, traditional federated learning methods face challenges in this network. Due to differences in data distribution and model performance among enterprise nodes, poor node participation will affect global model performance and training efficiency. There is currently a lack of effective standards to select high-quality nodes to participate in federated learning, which limits the performance and construction efficiency of intrusion detection models. In addition, differences in node geographical location, communication conditions, and computing power make training time unpredictable, which becomes a key factor restricting the efficiency of federated learning. In view of the poor security performance and low training efficiency of existing network intrusion detection schemes, a new network intrusion detection scheme needs to be proposed. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a smart procurement supply chain network intrusion detection method and system to at least solve the problems of poor security performance and low training efficiency in existing network intrusion detection solutions.

[0007] In order to achieve the above-mentioned objectives, the first aspect of the present invention provides a smart procurement supply chain network intrusion detection method, the method is executed by a data middle platform layer, the data middle platform layer is connected to an enterprise node layer composed of multiple enterprise nodes, and the method includes: recovering the status information of each enterprise node transmitted by the enterprise node layer, and selecting candidate enterprise nodes participating in the federal algorithm based on the status information; respectively calculating the migration performance index of each candidate enterprise node participating in the federal algorithm, and obtaining the migration performance index value of each candidate enterprise node participating in the federal algorithm; constructing a migration efficiency function based on the migration performance index value of each candidate enterprise node participating in the federal algorithm and a pre-built intrusion detection system model, and constructing a migration efficiency maximization problem based on the migration efficiency function; solving the migration efficiency maximization problem based on the Markov decision process construction and the deep deterministic policy gradient algorithm, updating the intrusion detection local model of each enterprise node based on the solution result, and performing corresponding network intrusion detection based on the intrusion detection local model of each enterprise node.

[0008] Optionally, the selecting of candidate enterprise nodes for participating in the federation algorithm based on the state information includes: calculating the distribution difference between the state parameters of each enterprise node and preset target data based on KL divergence, and taking enterprise nodes whose distribution difference is less than a preset distribution difference threshold as candidate enterprise nodes for participating in the federation algorithm; wherein the calculation rule for calculating the distribution difference between the state parameters of each enterprise node and the preset target data based on KL divergence is:

[0009]

[0010] in, is the distribution difference between the state parameter of enterprise node n and the preset target data; is the state parameter of enterprise node n; P T The preset target data.

[0011] Optionally, the migration performance indicators of each candidate enterprise node participating in the federated algorithm are calculated separately, including: calculating the migration gain of enterprise node n based on a first accuracy rate obtained by enterprise node n based on verification data in the kth round of federated learning, and a second accuracy rate obtained by a model constructed using only preset target data based on verification data; calculating the cumulative migration gain score of each enterprise node based on the migration gain of each enterprise node and the global model migration gain stored in the data middle platform layer; and calculating the migration performance indicator value of each enterprise node based on the cumulative migration gain score of each enterprise node.

[0012] Optionally, the calculation rule of the migration gain of the enterprise node n is:

[0013]

[0014] in, is the migration gain of enterprise node n; Acc is the first accuracy rate obtained by enterprise node n based on the verification data in the kth round of federated learning; T The second accuracy rate obtained based on the validation data for the model built using only the preset target data.

[0015] Optionally, the calculation rule of the cumulative migration gain score of each enterprise node is:

[0016]

[0017] in, is the cumulative migration gain score of enterprise node n in the kth round of federated learning; is the global model migration gain in the kth round of federated learning; a1, a2 and a3 are preset constants whose sum is 1; Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected. is the cumulative migration gain score of enterprise node n in the k-1th round of federated learning.

[0018] Optionally, the calculation rule of the migration performance index value of each enterprise node is:

[0019]

[0020] in, is the migration performance index value of enterprise node n in the kth round of federated learning; is the distribution difference between the state parameters of enterprise node n and the preset target data.

[0021] Optionally, the migration efficiency function is constructed based on the migration performance index value of each candidate enterprise node participating in the federation algorithm and the pre-built intrusion detection system model, including: constructing a migration efficiency function TLE based on the balance between maximizing the migration performance of the intrusion detection model and minimizing the training time, and the migration efficiency function TLE in the kth round of federated learning k It is expressed as:

[0022]

[0023] in, is the normalized migration performance index value; is the migration performance index value of enterprise node n in the kth round of federated learning; is the minimum value of the migration performance index in the kth round of federated learning; is the maximum value of the migration performance index in the kth round of federated learning; T ave To evenly distribute server bandwidth B to all nodes S The training time required; is the total training time of the kth round of federated learning.

[0024] Optionally, the calculation rule for the total training time of the kth round of federated learning is: the training time of each enterprise node in the corresponding round of federated learning is calculated based on the enterprise node calculation model; the communication time of each enterprise node in the corresponding round of federated learning is calculated based on the enterprise node communication model; the total training time of the corresponding round of federated learning is calculated based on the longest training time of the training time of each enterprise node in the corresponding round of federated learning and the communication time of each enterprise node in the corresponding round of federated learning.

[0025] Optionally, the enterprise node calculation model is:

[0026]

[0027] Among them, D n is the local data size of enterprise node n; C n is the number of CPU cycles required to train one sample on node n; is the current CPU frequency of node n in the kth round of federated learning; is the training time of node n; the enterprise node communication model is:

[0028]

[0029] in, is the effective transmission rate of node n in the kth round of federated learning; is the bandwidth size allocated to enterprise node n in the kth round of federated learning; is the channel gain of the enterprise node in the kth round of federated learning; p n is the transmission power of enterprise node n; n0 is the background noise; is the communication time of node n.

[0030] Optionally, the total training time of the corresponding round of federated learning is calculated based on the longest training time of the training time of each enterprise node in the corresponding round of federated learning and the communication time of each enterprise node in the corresponding round of federated learning. The corresponding calculation rule is:

[0031]

[0032] Optionally, constructing a migration efficiency maximization problem based on the migration efficiency function includes:

[0033] Based on the migration efficiency function, the optimization goal is converted to obtain the maximum migration efficiency in each round of federated learning, and the migration efficiency maximization problem is obtained, which is expressed as:

[0034] max TLE k

[0035]

[0036] Among them, TLE k is the migration efficiency of the kth round of federated learning; Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected. B is the bandwidth allocated to enterprise node n in the kth round of federated learning; s is the total allocable bandwidth; C n is the set of all enterprise nodes; C k is the set of all target nodes.

[0037] Optionally, the method of solving the migration efficiency maximization problem based on the Markov decision process construction and the deep deterministic policy gradient algorithm includes: converting the migration efficiency maximization problem into a corresponding reward function based on the Markov decision process construction; solving the reward function based on the deep deterministic policy gradient algorithm to obtain a solution result.

[0038] Optionally, the Markov decision process-based construction converts the migration efficiency maximization problem into a corresponding reward function, including: based on a penalty function method, converting the constrained migration efficiency maximization problem into an unconstrained migration efficiency maximization problem; wherein the introduced penalty function is:

[0039]

[0040] in, is the allocated bandwidth of node n; the corresponding reward function is set based on the penalty function, which is expressed as:

[0041] r(s (t) ,a (t) )=TLE (t) +υF

[0042] in, Indicates the system status of the current data platform layer; is the CPU cycle frequency of enterprise node n; is the channel gain of enterprise node n; is the migration performance evaluation index of enterprise node n; It represents the t-th action of the system at the middle platform layer, including the enterprise node selection decision and bandwidth allocation decision; is the selection status of enterprise node n.

[0043] Optionally, solving the reward function based on the deep deterministic policy gradient algorithm to obtain a solution result includes: based on the deep deterministic policy gradient algorithm, using an action value function to approximate the expected value of the cumulative reward; the action value function is expressed as:

[0044]

[0045] Where E is the expectation; γ∈[0,1] represents the discount factor; π is the optimal node selection and bandwidth allocation strategy function, expressed as:

[0046]

[0047] Based on the expected value approximation of the cumulative reward, the optimal behavior strategy function is constructed, which is expressed as:

[0048]

[0049] Iterative training is performed based on the optimal behavior strategy function to obtain a training result as a solution result.

[0050] Optionally, the intrusion detection local model of each enterprise node is updated based on the solution results, and corresponding network intrusion detection is performed based on the intrusion detection local model of each enterprise node, including: performing intrusion detection local model update based on the enterprise nodes selected from the candidate enterprise nodes participating in the federation algorithm and the allocated bandwidth of each enterprise node, and performing local network intrusion detection of each enterprise node based on the updated intrusion detection local model.

[0051] The second aspect of the present invention provides a data middle platform layer, which is connected to an enterprise node layer composed of multiple enterprise nodes; the data middle platform layer includes: a collection unit, which is used to recover the status information of each enterprise node transmitted by the enterprise node layer, and select candidate enterprise nodes participating in the federal algorithm based on the status information; a processing unit, which is used to respectively calculate the migration performance index of each candidate enterprise node participating in the federal algorithm, and obtain the migration performance index value of each candidate enterprise node participating in the federal algorithm; a training unit, which is used to construct a migration efficiency function based on the migration performance index value of each candidate enterprise node participating in the federal algorithm and a pre-built intrusion detection system model, and construct a migration efficiency maximization problem based on the migration efficiency function; a detection unit, which is used to solve the migration efficiency maximization problem based on Markov decision process construction and deep deterministic policy gradient algorithm, update the intrusion detection local model of each enterprise node based on the solution result, and perform corresponding network intrusion detection based on the intrusion detection local model of each enterprise node.

[0052] The third aspect of the present invention provides a smart procurement supply chain network intrusion detection system, which includes the above-mentioned data middle platform layer; the system also includes an enterprise node layer composed of multiple enterprise nodes; the enterprise node layer is connected to the data middle platform layer.

[0053] On the other hand, the present invention provides a computer-readable storage medium, which stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned smart procurement supply chain network intrusion detection method.

[0054] Through the above technical solution, an indicator to measure the performance of the node local model is designed through migration gain, so as to comprehensively consider the data distribution and model performance of the enterprise node, and can guide the nodes with small data distribution differences and high model migration gain to participate in federated learning, ensuring the performance of the federated learning smart procurement supply chain network intrusion detection model; finally, based on the proposed migration performance evaluation index, from the perspective of balancing high migration performance and low training delay, a migration efficiency function is designed to transform the node selection and bandwidth allocation optimization strategy into an optimization problem that maximizes migration efficiency. It is used to screen out high-quality nodes to participate in federated learning, so as to quickly and accurately complete the training and update of the detection model, ensuring network security while improving training efficiency.

[0055] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0057] Figure 1 It is a flowchart of the steps of a smart procurement supply chain network intrusion detection method provided by an embodiment of the present invention;

[0058] Figure 2 This is a training time comparison diagram for the same accuracy of the present invention provided by one embodiment of the present invention;

[0059] Figure 3 It is a comparison chart of the training time when the method of the present invention provided by one embodiment of the present invention and the algorithm based on the traditional FL selection of all enterprise nodes to participate in learning achieve the same false alarm rate;

[0060] Figure 4 It is a comparison chart of the detection rates of the method of the present invention provided in one embodiment of the present invention and the algorithm based on the traditional FL selecting all enterprise nodes to participate in learning;

[0061] Figure 5 It is a performance comparison diagram of an intrusion detection model of the method of the present invention provided in one embodiment of the present invention and an intrusion detection solution CNN-auto based on CNN automatic selection;

[0062] Figure 6 It is a system structure diagram of a smart procurement supply chain network intrusion detection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0064] Figure 1 This is a method flow chart of a smart procurement supply chain network intrusion detection method provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a smart procurement supply chain network intrusion detection method, the method comprising:

[0065] Step S10: Recover the status information of each enterprise node transmitted by the enterprise node layer, and select candidate enterprise nodes participating in the federation algorithm based on the status information.

[0066] Specifically, the distribution difference between the state parameters of each enterprise node and the preset target data is calculated based on the KL divergence, and the enterprise nodes whose distribution difference is less than the preset distribution difference threshold are taken as candidate enterprise nodes participating in the federation algorithm; wherein, the calculation rule for calculating the distribution difference between the state parameters of each enterprise node and the preset target data based on the KL divergence is:

[0067]

[0068] in, is the distribution difference between the state parameter of enterprise node n and the preset target data; is the state parameter of enterprise node n; P T The preset target data.

[0069] In the embodiment of the present invention, the greater the distribution difference between the preset target data, the more likely the corresponding enterprise node will cause negative migration when participating in federated learning. In order to avoid the negative migration phenomenon, enterprise nodes with distribution differences less than a preset distribution difference threshold are selected as candidate enterprise nodes for participating in the federated algorithm, and enterprise nodes with distribution differences exceeding the preset distribution difference threshold are filtered out.

[0070] Step S20: Calculate the migration performance index of each candidate enterprise node participating in the federation algorithm respectively, and obtain the migration performance index value of each candidate enterprise node participating in the federation algorithm.

[0071] Specifically, the migration gain of enterprise node n is calculated based on the first accuracy rate obtained by enterprise node n based on the verification data in the kth round of federated learning and the second accuracy rate obtained by the model built using only the preset target data based on the verification data; the cumulative migration gain score of each enterprise node is calculated based on the migration gain of each enterprise node and the global model migration gain stored in the data middle platform layer; the migration performance index value of each enterprise node is calculated based on the cumulative migration gain score of each enterprise node.

[0072] In one possible implementation, by comparing the accuracy of the model using node n on the target Compared with the model accuracy Acc built using only target data T , calculate the migration gain It is expressed as:

[0073] Furthermore, considering the difference between the migration gain of node n and the migration gain of the global model, as well as the difference between the migration gain of node n and other nodes, the cumulative migration gain score is calculated according to the following formula:

[0074]

[0075] The higher the score, the higher the model quality of node n.

[0076] Preferably, the data distribution quality and local model performance of node n are comprehensively considered to calculate the migration performance evaluation index It is expressed as:

[0077]

[0078] Where ε1,ε2∈[0,1], to prevent the numerator and denominator from being 0. Migration performance score Higher enterprise

[0079] The business node has a greater probability of being selected. When the migration performance of the selected node is poor, the probability of selecting the node again will be greatly reduced, and the nodes with negative migration gain will no longer participate in the aggregation, thereby reducing the impact of low-quality nodes on the global intrusion detection model.

[0080] Step S30: construct a migration efficiency function based on the migration performance index value of each candidate enterprise node participating in the federation algorithm and a pre-built intrusion detection system model, and construct a migration efficiency maximization problem based on the migration efficiency function.

[0081] Specifically, the migration efficiency function is constructed based on the migration performance index value of each candidate enterprise node participating in the federation algorithm and the pre-built intrusion detection system model, including: constructing a migration efficiency function TLE based on the balance between maximizing the migration performance of the intrusion detection model and minimizing the training time, and the migration efficiency function TLE in the kth round of federated learning k It is expressed as:

[0082]

[0083] in, is the normalized migration performance index value; is the migration performance index value of enterprise node n in the kth round of federated learning; is the minimum value of the migration performance index in the kth round of federated learning; is the maximum value of the migration performance index in the kth round of federated learning; T ave To evenly distribute server bandwidth B to all nodes S The training time required; is the total training time of the kth round of federated learning.

[0084] Preferably, the calculation rule for the total training time of the kth round of federated learning is: the training time of each enterprise node in the corresponding round of federated learning is calculated based on the enterprise node calculation model; the communication time of each enterprise node in the corresponding round of federated learning is calculated based on the enterprise node communication model; the total training time of the corresponding round of federated learning is calculated based on the longest training time of the training time of each enterprise node in the corresponding round of federated learning and the communication time of each enterprise node in the corresponding round of federated learning.

[0085] In one possible implementation, in the kth round of federated learning, the time required for enterprise node n to complete training in the process of calculating the local local CNN model is:

[0086]

[0087] Where D n is the local data size of enterprise node n, C n is the number of CPU cycles required to train one example on node n. The current CPU frequency of node n in the kth round of federated learning reflects the computing power of the node. The computing power of nodes of different enterprises is different.

[0088] Preferably, after the local model training is completed, the selected enterprise node needs to upload the learned local model parameters (weights and biases of the CNN model) to the server for aggregation. In the kth round of federated learning, the effective transmission rate of enterprise node n is Expressed by Shannon's formula, it is expressed as:

[0089]

[0090] in is the bandwidth size allocated to enterprise node n in the kth round of federated learning. is the channel gain of the enterprise node in the kth round of federated learning. n is the transmission power of enterprise node n, and n0 is the background noise.

[0091] The communication time of enterprise node n is calculated by the node effective transmission rate and the node local model parameter size, and its formula is expressed as:

[0092]

[0093] Since enterprise nodes upload models in parallel during the federated learning process, the training time of the intrusion detection model is determined by the slowest participating node. The total training time of the kth round of federated learning is expressed as:

[0094]

[0095] in Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected.

[0096] Preferably, constructing the migration efficiency maximization problem based on the migration efficiency function includes: based on the migration efficiency function, converting the optimization target into obtaining the maximum migration efficiency in each round of federated learning, and obtaining the migration efficiency maximization problem, which is expressed as:

[0097]

[0098] Among them, TLE k is the migration efficiency of the kth round of federated learning; Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected. B is the bandwidth allocated to enterprise node n in the kth round of federated learning; s is the total allocable bandwidth; C n is the set of all enterprise nodes; C k is the set of all target nodes.

[0099] Step S40: Solve the migration efficiency maximization problem based on the Markov decision process construction and the deep deterministic policy gradient algorithm, update the intrusion detection local model of each enterprise node based on the solution result, and perform corresponding network intrusion detection based on the intrusion detection local model of each enterprise node.

[0100] Specifically, when reinforcement learning explores the environment, the agent interacts continuously with the environment, which reflects the Markov process. The federated learning intrusion detection model designed by the present invention is suitable for dynamically changing smart procurement supply chain networks. The data center performs node selection and bandwidth allocation based on information such as enterprise node performance, communication conditions, and computing power to obtain the reward value of environmental feedback and evaluate the pros and cons of decisions. In order to maximize the cumulative rewards, the data center selects the optimal nodes and bandwidth through the DDPG (Deep Deterministic Policy Gradient Algorithm) algorithm to complete the model building task, which is also a Markov decision process. The optimization goal is to maximize migration efficiency.

[0101] In one possible implementation, the key elements of the reinforcement learning problem are as follows:

[0102] 1) State S: For each step t of the DDPG network training process, the system state is determined by the CPU cycle frequency of enterprise node n and channel gain And migration performance evaluation indicators Composition. Used to evaluate model calculation time; For evaluating communication time; Represents the migration performance of enterprise node n. State s (t) ∈S is defined as

[0103] 2) Action A: In the intelligent procurement supply chain network scenario constructed by the present invention, the action of each step t is the enterprise node selection decision and bandwidth allocation decision. (t) ∈A is defined as:

[0104]

[0105] 3) Reward function r: The goal of reinforcement learning is to obtain the maximum cumulative reward, that is, to maximize the reward function, which coincides with the goal of maximizing the migration efficiency proposed by the present invention. Therefore, the optimization goal of the present invention is associated with the reward function of reinforcement learning, and the penalty function method is used to transform the constrained optimization problem into an unconstrained optimization problem for solution. S Will After normalization, the penalty function F is introduced and expressed as:

[0106]

[0107] The reward function is then set as:

[0108] r(s (t) ,a (t) )=TLE (t) +υF

[0109] Where υ is the penalty factor, which is used to adjust the value of the penalty function. The reward function is the sum of the migration efficiency function and the penalty function, so as to maximize the migration efficiency function under the bandwidth constraint to achieve a balance between maximizing the migration gain and minimizing the training time. In the reinforcement learning process, the data middle station, as an intelligent agent, aims to maximize the cumulative reward and continuously explores and learns the node selection and bandwidth allocation decisions that maximize the reward function. Therefore, in any state, the reward value corresponding to the optimal decision action that maximizes the migration efficiency is also the highest, thereby completing the transformation of the problem.

[0110] Further, the performing of solving the reward function based on the deep deterministic policy gradient algorithm to obtain a solution result includes: based on the deep deterministic policy gradient algorithm, using an action value function to approximate the expected value of the cumulative reward; the action value function is expressed as:

[0111]

[0112] Where E is the expectation; γ∈[0,1] represents the discount factor; π is the optimal node selection and bandwidth allocation strategy function, expressed as:

[0113]

[0114] Based on the expected value approximation of the cumulative reward, the optimal behavior strategy function is constructed, which is expressed as:

[0115]

[0116] Iterative training is performed based on the optimal behavior strategy function to obtain a training result as a solution result.

[0117] In one possible implementation, the algorithm comprehensively considers the migration performance, communication conditions, and computing conditions of enterprise nodes, and makes a trade-off between training time and migration performance, aiming to achieve efficient node selection and bandwidth allocation decisions. The ultimate goal of the DDPG algorithm is to quickly generate the optimal node selection and bandwidth allocation strategy when the data center receives the status information of the enterprise node. The strategy function is expressed as:

[0118]

[0119] The DDPG algorithm uses the action value function Q to approximate the expected value of the cumulative reward. The Q function is defined as:

[0120]

[0121] Where E represents the expectation and γ∈[0,1] represents the discount factor. When γ approaches 1, it means that long-term rewards are more important than short-term rewards. Conversely, when it approaches 0, short-term rewards are more important. The goal of DDPG learning is to maximize the expected cumulative reward and determine the optimal value of the Q function Q * , so as to find the optimal behavior strategy (optimal node selection and bandwidth allocation strategy). The expression of the optimal behavior strategy is:

[0122]

[0123] During the training of the DDPG algorithm, the agent explores and learns the optimal policy function and value function by interacting with the network environment. Assume that during each training, the agent randomly extracts J samples from the experience replay pool, and the samples are represented as (s j ,a j ,r j ,s j+1 ). The definition of the target Q value is expressed as:

[0124] y j = r(s j ,aj )+γQ′(s j+1 ,π′(s j+1 |θ π′ )|θ Q′ )

[0125] Then, by minimizing the loss function, the Critic current network is updated, and the policy gradient is calculated to update the Actor current network. The gradient formula definition is expressed as:

[0126]

[0127] Then, the corresponding target network parameters are iteratively updated every τ steps to stabilize the learning process. The soft update formula is expressed as:

[0128]

[0129] Preferably, the intrusion detection local model of each enterprise node is updated based on the solution results, and the corresponding network intrusion detection is performed based on the intrusion detection local model of each enterprise node, including: performing intrusion detection local model update based on the enterprise nodes selected from the candidate enterprise nodes participating in the federation algorithm and the allocated bandwidth of each enterprise node, and performing local network intrusion detection of each enterprise node based on the updated intrusion detection local model.

[0130] In the embodiment of the present invention, the scheme of the present invention takes into account that the data quality differences of various enterprises in the smart procurement supply chain network lead to different migration performance of enterprise network nodes, and designs a migration performance evaluation index. First, the pros and cons of the nodes are evaluated from the data perspective, and the KL divergence is used to measure the similarity between the target data and the enterprise node data; secondly, the pros and cons of the nodes are evaluated from the model perspective, and three standards are designed through migration gain to measure the performance of the node local model, thereby comprehensively considering the data distribution and model performance of the enterprise nodes, and being able to guide nodes with small data distribution differences and high model migration gain to participate in federated learning, ensuring the performance of the federated learning smart procurement supply chain network intrusion detection model; finally, based on the proposed migration performance evaluation index, from the perspective of balancing high migration performance and low training delay, a migration efficiency function is designed to transform the node selection and bandwidth allocation optimization strategy into an optimization problem that maximizes migration efficiency.

[0131] Example:

[0132] like Figure 2, which is a training time comparison chart of the present invention with the same accuracy. It can be seen that when the same accuracy is achieved, the training time used by the algorithm of the present invention is shorter. This is due to the use of the DDPG algorithm. Before each round of training, based on the similarity between the data of each enterprise node and the target data and the migration gain in the previous round, as well as the current computing power and channel conditions of each enterprise node, the method of the present invention selects enterprise nodes with high migration performance evaluation indicators and both high computing power and high channel gain, ensuring the performance of the global model, and allocating appropriate bandwidth to individual enterprise nodes with high migration performance evaluation indicators but poor computing power or channel conditions, shortening the training time.

[0133] like Figure 3 , which is a comparison of the training time when the method of the present invention and the algorithm based on traditional FL selects all enterprise nodes to participate in learning to achieve the same false alarm rate. It can be seen from the figure that as the false alarm rate decreases, the training time of the two methods gradually increases. When the same false alarm rate is achieved, the training time required for the method of the present invention is lower. This is because before each round of training starts, the algorithm of the present invention combines the similarity between the data of each enterprise node and the target data, the migration gain of the previous round, and the computing power and channel conditions of each enterprise node. The method of the present invention selects enterprise nodes with high migration performance evaluation indicators, high computing power and high channel gain to ensure the overall model performance. At the same time, the present invention allocates appropriate bandwidth to enterprise nodes with higher migration performance evaluation indicators but poor computing power or channel conditions, thereby shortening the training time.

[0134] like Figure 4 , which is a comparison of the detection rates of the method of the present invention and the algorithm based on traditional FL that selects all enterprise nodes to participate in learning. As the detection rate increases, the training time also gradually increases. When the same detection rate is achieved, the training time required for the method of the present invention is lower. This is because the present invention adopts the DDPG algorithm. Before each round of training starts, by comprehensively considering the similarity between the data of each enterprise node and the target data, the migration gain of the previous round, and the computing power and channel conditions of each enterprise node, the method of the present invention selects enterprise nodes with high migration performance evaluation indicators, high computing power and high channel gain to ensure the performance of the global model. At the same time, for some enterprise nodes with higher migration performance evaluation indicators but poor computing power or channel conditions, the present invention allocates appropriate bandwidth to shorten the training time.

[0135] like Figure 5, which is a performance comparison of the intrusion detection model of the method of the present invention and the intrusion detection scheme CNN-auto based on CNN automatic selection. The purple part is the CNN method without using transfer learning, which is used as the calculation basis of the migration gain. It can be seen that the accuracy of the method of the present invention is 3.78% higher than that of the CNN-auto method, the detection rate is 0.96% higher, and the false alarm rate is 7.3% lower. This proves that in the case of enterprise data islands, the performance of the intrusion detection model can be improved by combining multiple enterprise node training models. Compared with the CNN method without using transfer learning, the intrusion detection model constructed by the method of the present invention has an accuracy increase of 10.16%, a detection rate increase of 8.36%, and a false alarm rate reduction of 11.89% after transfer learning. It can be seen that the method of the present invention can efficiently construct an intrusion detection model with good performance.

[0136] Figure 6 This is a diagram of the data middle platform system structure provided by an embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides a data middle platform layer, and the data middle platform layer includes: a collection unit, which is used to recover the status information of each enterprise node transmitted by the enterprise node layer, and select candidate enterprise nodes participating in the federal algorithm based on the status information; a processing unit, which is used to respectively calculate the migration performance index of each candidate enterprise node participating in the federal algorithm, and obtain the migration performance index value of each candidate enterprise node participating in the federal algorithm; a training unit, which is used to construct a migration efficiency function based on the migration performance index value of each candidate enterprise node participating in the federal algorithm and a pre-built intrusion detection system model, and construct a migration efficiency maximization problem based on the migration efficiency function; a detection unit, which is used to solve the migration efficiency maximization problem based on the Markov decision process construction and the deep deterministic policy gradient algorithm, update the intrusion detection local model of each enterprise node based on the solution result, and perform corresponding network intrusion detection based on the intrusion detection local model of each enterprise node.

[0137] The solution of the present invention also provides a smart procurement supply chain network intrusion detection system, which includes the above-mentioned data middle platform layer; the system also includes an enterprise node layer composed of multiple enterprise nodes; the enterprise node layer is connected to the data middle platform layer.

[0138] An embodiment of the present invention also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed on a computer, the computer executes the above-mentioned smart procurement supply chain network intrusion detection method.

[0139] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions for making a single-chip microcomputer, a chip or a processor (processor) perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0140] The optional embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, the technical scheme of the embodiments of the present invention can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0141] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A smart procurement supply chain network intrusion detection method, characterized in that: The method is executed by a data middle platform layer, and the data middle platform layer is connected to an enterprise node layer composed of multiple enterprise nodes. The method includes: Recovering the status information of each enterprise node transmitted by the enterprise node layer, and selecting candidate enterprise nodes participating in the federation algorithm based on the status information; Calculate the migration performance index of each candidate enterprise node participating in the federation algorithm respectively, and obtain the migration performance index value of each candidate enterprise node participating in the federation algorithm; A migration efficiency function is constructed based on the migration performance index values ​​of the candidate enterprise nodes participating in the federation algorithm and the pre-constructed intrusion detection system model, and a migration efficiency maximization problem is constructed based on the migration efficiency function; The migration efficiency maximization problem is solved based on the Markov decision process construction and the deep deterministic policy gradient algorithm, the intrusion detection local model of each enterprise node is updated based on the solution result, and the corresponding network intrusion detection is performed based on the intrusion detection local model of each enterprise node.

2. The method according to claim 1, characterized in that The selecting of candidate enterprise nodes participating in the federation algorithm based on the state information includes: Based on the KL divergence, the distribution difference between the state parameters of each enterprise node and the preset target data is calculated, and the enterprise nodes whose distribution difference is less than the preset distribution difference threshold are regarded as candidate enterprise nodes participating in the federation algorithm; wherein, The calculation rule for calculating the distribution difference between the state parameters of each enterprise node and the preset target data based on KL divergence is: in is the distribution difference between the state parameter of enterprise node n and the preset target data; is the state parameter of enterprise node n; P T The preset target data.

3. The method according to claim 1, characterized in that The migration performance indicators of candidate enterprise nodes participating in the federation algorithm are calculated separately, including: The migration gain of enterprise node n is calculated based on a first accuracy rate obtained by enterprise node n based on verification data in the kth round of federated learning and a second accuracy rate obtained by a model built using only preset target data based on verification data; Based on the migration gain of each enterprise node and the global model migration gain stored in the data middle platform layer, the cumulative migration gain score of each enterprise node is calculated respectively; The migration performance index value of each enterprise node is calculated based on the cumulative migration gain score of each enterprise node.

4. The method according to claim 3, characterized in that The calculation rule of the migration gain of the enterprise node n is: in, is the migration gain of enterprise node n; is the first accuracy rate obtained by enterprise node n based on the verification data in the kth round of federated learning; Acc T The second accuracy rate obtained based on the validation data for the model built using only the preset target data.

5. The method according to claim 4, characterized in that The calculation rule of the cumulative migration gain score of each enterprise node is: in, is the cumulative migration gain score of enterprise node n in the kth round of federated learning; is the global model transfer gain in the kth round of federated learning; a1, a2 and a3 are preset constants whose sum is 1; Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected. is the cumulative migration gain score of enterprise node n in the k-1th round of federated learning.

6. The method according to claim 5, characterized in that The calculation rules for the migration performance index value of each enterprise node are as follows: in, is the migration performance index value of enterprise node n in the kth round of federated learning; ε1,ε2∈[0,1], used to prevent the numerator and denominator from being 0; is the distribution difference between the state parameters of enterprise node n and the preset target data.

7. The method according to claim 1, characterized in that The migration efficiency function is constructed based on the migration performance index values ​​of the candidate enterprise nodes participating in the federation algorithm and the pre-built intrusion detection system model, including: Based on the balance between maximizing the migration performance of the intrusion detection model and minimizing the training time, the migration efficiency function TLE is constructed. The migration efficiency function TLE in the kth round of federated learning k It is expressed as: in, is the normalized migration performance index value; is the migration performance index value of enterprise node n in the kth round of federated learning; is the minimum value of the migration performance index in the kth round of federated learning; is the maximum value of the migration performance index in the kth round of federated learning; T ave To evenly distribute server bandwidth B to all nodes S The training time required; is the total training time of the kth round of federated learning.

8. The method according to claim 7, characterized in that The calculation rule for the total training time of the kth round of federated learning is: The training time of each enterprise node in the corresponding round of federated learning is calculated based on the enterprise node calculation model; Based on the enterprise node communication model, the communication time of each enterprise node in the corresponding round of federated learning is calculated; The total training time of the corresponding round of federated learning is obtained by calculating the longest training time of the training time of each enterprise node in the corresponding round of federated learning and the communication time of each enterprise node in the corresponding round of federated learning.

9. The method according to claim 8, characterized in that The enterprise node calculation model is: Among them, D n is the local data size of enterprise node n; C n is the number of CPU cycles required to train one sample on node n; is the current CPU frequency of node n in the kth round of federated learning; is the training time of node n; The enterprise node communication model is: in, is the effective transmission rate of node n in the kth round of federated learning; is the bandwidth size allocated to enterprise node n in the kth round of federated learning; is the channel gain of the enterprise node in the kth round of federated learning; p n is the transmission power of enterprise node n; n0 is the background noise; is the communication time of node n.

10. The method according to claim 8, characterized in that The total training time of the corresponding round of federated learning is calculated based on the longest training time of the training time of each enterprise node in the corresponding round of federated learning, and the communication time of each enterprise node in the corresponding round of federated learning. The corresponding calculation rule is:

11. The method according to claim 1, characterized in that: The constructing of the migration efficiency maximization problem based on the migration efficiency function includes: Based on the migration efficiency function, the optimization goal is converted to obtain the maximum migration efficiency in each round of federated learning, and the migration efficiency maximization problem is obtained, which is expressed as: max TLE k Among them, TLE k is the migration efficiency of the kth round of federated learning; Indicates the selection status of enterprise node n in the kth round of federated learning. A value of 1 indicates that enterprise node n is selected to participate in this round of federated learning, and a value of 0 indicates that it is not selected. is the bandwidth size allocated to enterprise node n in the kth round of federated learning; B is the total allocatable bandwidth; Bs C n A collection of all enterprise nodes; C k is the set of all target nodes.

12. The method according to claim 1, characterized in that The Markov decision process-based construction and deep deterministic policy gradient algorithm are used to solve the migration efficiency maximization problem, including: The migration efficiency maximization problem is converted into a corresponding reward function based on a Markov decision process; The reward function is solved based on the deep deterministic policy gradient algorithm to obtain a solution result.

13. The method according to claim 12, characterized in that The Markov decision process-based construction converts the migration efficiency maximization problem into a corresponding reward function, including: Based on the penalty function method, the constrained migration efficiency maximization problem is transformed into an unconstrained migration efficiency maximization problem; The penalty function introduced is: in, The allocated bandwidth for node n; The corresponding reward function is set based on the penalty function, which is expressed as: r(s (t) ,a (t) )=TLE (t) +υF in, Indicates the system status of the current data platform layer; is the CPU cycle frequency of enterprise node n; is the channel gain of enterprise node n; is the migration performance evaluation index of enterprise node n; It represents the t-th action of the system at the middle platform layer, including the enterprise node selection decision and bandwidth allocation decision; is the selection status of enterprise node n.

14. The method according to claim 13, characterized in that The step of solving the reward function based on the deep deterministic policy gradient algorithm to obtain a solution result includes: Based on the deep deterministic policy gradient algorithm, the expected value of the cumulative reward is approximated by an action-value function; the action-value function is expressed as: Among them, E is the expectation; γ∈[0,1] represents the discount factor; π is the optimal node selection and bandwidth allocation strategy function, expressed as: Based on the expected value approximation of the cumulative reward, the optimal behavior strategy function is constructed, which is expressed as: Among them, Q * is the optimal value of the Q function; π * (s) is the optimal behavioral strategy; Iterative training is performed based on the optimal behavior strategy function to obtain a training result as a solution result.

15. The method according to claim 1, characterized in that The updating of the intrusion detection local model of each enterprise node based on the solution result and the execution of the corresponding network intrusion detection based on the intrusion detection local model of each enterprise node include: The intrusion detection local model is updated based on the enterprise nodes selected from the candidate enterprise nodes participating in the federation algorithm and the allocated bandwidth corresponding to each enterprise node, and the local network intrusion detection of each enterprise node is performed based on the updated intrusion detection local model.

16. A data middle platform layer, characterized in that: The data middle platform layer is connected to the enterprise node layer composed of multiple enterprise nodes; The data middle platform layer includes: A collection unit, used to recover the status information of each enterprise node transmitted by the enterprise node layer, and select candidate enterprise nodes participating in the federation algorithm based on the status information; A processing unit, used to respectively calculate the migration performance index of each candidate enterprise node participating in the federation algorithm, and obtain the migration performance index value of each candidate enterprise node participating in the federation algorithm; A training unit, used to construct a migration efficiency function based on the migration performance index value of each candidate enterprise node participating in the federation algorithm and a pre-built intrusion detection system model, and to construct a migration efficiency maximization problem based on the migration efficiency function; The detection unit is used to solve the migration efficiency maximization problem based on the Markov decision process construction and the deep deterministic policy gradient algorithm, update the intrusion detection local model of each enterprise node based on the solution result, and perform corresponding network intrusion detection based on the intrusion detection local model of each enterprise node.

17. A smart procurement supply chain network intrusion detection system, characterized in that: The system includes the data middle platform layer as claimed in claim 16; The system also includes an enterprise node layer consisting of a plurality of enterprise nodes; The enterprise node layer is connected to the data middle platform layer.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the smart procurement supply chain network intrusion detection method described in any one of claims 1-15.

Citation Information

Cited By

  • General-calculation integrated multi-mode federated learning task resource scheduling method and medium

    CN120416051A