A trusted multi-objective neuroevolution intrusion detection framework

Abstract

The application belongs to the technical field of internet of things intrusion detection, and proposes a trusted multi-objective neural evolution intrusion detection framework, which can guarantee the intrusion detection alarm data as much as possible not to be tampered with. For the information security protection of intrusion detection alarm data, through node credit rating, a reward and punishment mechanism is designed to strengthen the uploading of trusted alarm data by nodes; in terms of intrusion detection model, for the automatic design problem of deep learning based intrusion detection model, a multi-objective evolutionary algorithm is used to evolve some convolutional neural network structures with small parameter quantity and as high accuracy as possible; and a new convolution block structure is designed for the search space to further enhance the model feature extraction capability; after neural network architecture search, multiple Pareto solutions are generated, and finally a self-adaptive weight decision method is proposed to decide the final solution.

Description

Technical Field

[0001] This invention belongs to the field of IoT intrusion detection technology, specifically a reliable multi-target neural evolution intrusion detection framework. Background Technology

[0002] The Internet of Things (IoT), also known as the Internet of Smart Objects, is a network connecting various terminal devices using internet addressing and communication technologies. Composed of numerous interconnected devices, it plays a crucial role in the wave of information technology development. Today, IoT devices are becoming increasingly intelligent in fields such as industrial production, intelligent transportation, and smart homes. Approximately 127 new IoT devices connect to the internet every second, including those in factories, healthcare, and automotive products. Low-cost IoT devices are more vulnerable to malware attacks than host machines. Hackers can leverage IoT botnets composed of numerous devices to launch DDoS attacks, overwhelming nodes or servers with massive traffic, rendering services unusable or disrupting the network.

[0003] The Internet of Things (IoT) is a complex system. A network intrusion can impact not only the network itself but also its surrounding ecosystem. The cybersecurity company Imperva suffered a massive attack when a botnet coordinated a DDoS attack from 402,000 different IPs, resulting in failed business transactions, data loss, damage to brand trust, and significant financial losses. Therefore, the key to protecting IoT security lies in network traffic detection and early detection of potential attacks.

[0004] In his paper "An anomaly-based intrusion detection system for the smart grid based on cart decision tree," Panagiotis proposed a decision tree-based intrusion detection method to protect smart grids. While decision tree-based methods offer good accuracy, they struggle to detect intrusion activities with the increasing volume of network traffic and high-bandwidth samples.

[0005] In "An improved convolutional neural network model for intrusion detection in networks," Khan proposed a convolutional neural network intrusion detection model. By increasing the number of convolutional kernels, the original features are mapped to a high-dimensional space, increasing the ability to learn features. However, this method relies on manually designed neural network structures, requiring frequent trial and error. Andresini, in "Nearest cluster-based intrusion detection through convolutional neural networks," proposed a novel deep learning method that represents network flow as a 2D image. Its unique feature is the use of an autoencoder to extract input features, followed by nearest neighbor search and K-Means clustering to output a 2D image. However, this method is based on existing neural network models, which are designed for large image datasets and lack universality.

[0006] Modern IoT applications typically employ a distributed architecture, distributing computing and storage resources across different nodes to provide users with a superior experience. However, these server nodes are vulnerable to botnet attacks, leading to service outages. Furthermore, modern botnet attacks are often characterized by high speed and large scale, posing challenges to intrusion detection. Ensuring the reliability of alert data and logs from each server node is also a problem.

[0007] While some works utilize deep learning for intrusion detection, existing deep learning-based intrusion detection methods rely on manually designed neural network structures, requiring frequent trial and error. Furthermore, their scalability is poor, failing to meet ever-increasing computational demands and more complex task requirements. Moreover, these methods cannot guarantee the immutability and privacy of alert data and logs across nodes in a distributed environment.

[0008] This invention mainly addresses the following four issues: (1) Traditional intrusion detection systems are deployed on servers, and their logs are easily attacked and the alarm information is tampered with by hackers. This causes a problem of trust in the system; (2) Most deep learning-based intrusion detection methods require manual design of network structures, which requires frequent trial and error and consumes a lot of time; (3) Considering the number of parameters and accuracy of neural network structures, a neural network structure is searched; (4) Multiple Pareto solutions are generated after searching for neural network architectures, and a suitable final solution needs to be selected for deployment.

[0009] Most deep learning-based intrusion detection algorithms rely on manually designed neural network structures, which is time-consuming and requires specialized knowledge. To address these issues, this invention proposes a reliable multi-target neural evolutionary intrusion detection framework. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention aims to improve system reliability and ensure that system alarm data and logs are not altered. Simultaneously, it achieves lightweight model design while maintaining model accuracy. To this end, this invention designs a reliable intrusion detection framework.

[0011] The technical solution of the present invention:

[0012] (1) A trustworthy multi-target neural evolution intrusion detection framework, based on the immutability and traceability of blockchain, to improve the trustworthiness of the system;

[0013] A trust assessment module is defined. The trust of alarm data or nodes is assessed through smart contracts, taking into account data access control: the trust score of a node changes dynamically, as shown in equation (1), which is the trust assessment module; when a node first joins the network, the smart contract initializes it with a minimum trust value S. trust (t0); The trust assessment module calculates the current score based on the node's historical trust score and current behavior; After a node joins the network, the smart contract will update the node's trust value over time;

[0014]

[0015] Where f1, f2, ... f j These are the feature scores that influence the node's trust value; they can be negative or positive, w1, w2, ... w j The weighting factor is determined by the system administrator based on the actual situation. It's a decay function; the influence of past actions on the current trust value decreases over time. Malicious behavior on a node can reduce the current trust value, even causing it to become negative. And for that node t... n Trust ratings over time will also be affected.

[0016] When a node writes a hash value to the blockchain, it must first use access control to define read and write rights according to the access control list rules; each node must maintain a minimum trust level S. trust Only those with a trust value below S have the right to participate in data reading and writing; trust The node at (t0) will have its access to alarm data revoked and will be unable to write hash values ​​to the blockchain;

[0017] When the node reaches the minimum trust level Strust (t0) When preparing to write data to the blockchain, the data needs to be encrypted. The specific steps include:

[0018] Step 1: Encrypt alarm data or log data using AES key and generate AES ciphertext;

[0019] Step 2: Asymmetric encryption of the AES key, using RSA asymmetric encryption to generate RSA ciphertext;

[0020] Step 3: The AES ciphertext and RSA ciphertext are uploaded to IPFS respectively, and IPFS will return two different hash values;

[0021] Step 4: Store the hash values ​​generated in IPFS using AES and RSA ciphertexts in the blockchain via a remote procedure call blockchain contract;

[0022] (2) Implement a two-layer optimized neural evolution algorithm

[0023] Based on a multi-objective neural architecture search process, this algorithm searches for the structure of convolutional neural networks using a multi-objective evolutionary algorithm. The algorithm divides the search for neural network structure into two parts to search for lightweight neural networks. The first part optimizes the block-level connections of the upper feature extraction layer, and the second part optimizes the internal connections of neurons in the lower classification layer, ultimately obtaining the Pareto solution set of the intrusion detection model. Specifically:

[0024] Step 1: Collect network traffic data and preprocess it to build a dataset;

[0025] Step 2: Construct a two-layer optimized multi-objective neural evolution algorithm framework;

[0026] Step 2.1: Construct multiple different block-level structures as components of a two-layer optimized multi-objective neural evolution algorithm framework;

[0027] Step 2.2: Construct new convolutional blocks to add to the search space; the feature map is used as input, and the feature map is first convolved in the spatial dimension through depthwise convolution, where one channel is handled by only one convolutional kernel; after the depthwise convolution, a 1×1 channel-wise pointwise convolution is performed; then layer normalization is performed to ensure the stability of the data features; followed by the activation layer, which uses Gaussian error linear units to map the feature map;

[0028] Step 2.3: The lower classification layer constructs an adaptive sparse connection layer using the SET algorithm to optimize the connection weights of the fully connected neurons;

[0029] Step 3: The feature data is classified using the softmax function;

[0030] Step 4: Initialize the population;

[0031] Step 5: Use the NSGA-III multi-objective optimization algorithm to search the block-level connections of the upper feature extraction layer, and use the SET algorithm to adaptively sparsely evolve the neuron connection weights of the lower classification layer.

[0032] Step 6: After the search is completed, a Pareto solution set is generated; the solutions in the Pareto solution set are retrained one by one using the training set in the dataset constructed in Step 1 to obtain the corresponding performance indicators, namely accuracy and the number of model parameters.

[0033] (3) Adaptive decision-making process.

[0034] After a two-layer optimized neural evolution algorithm search, a Pareto solution set and corresponding performance metrics are obtained. After retraining the searched architecture, suitable final solutions are selected and deployed across distributed nodes. For the individual differences of each node, the weights of each node should be adaptively adjusted. Nodes with good computational performance are relatively insensitive to the number of model parameters, and their weights are correspondingly smaller. Nodes with poor computational performance are sensitive to the number of model parameters, and some accuracy can be sacrificed to ensure latency. When using TOPSIS to calculate the weighted decision matrix, the weights are calculated based on the computational performance of each node. Specific steps include:

[0035] Step 1: The weight vector W is obtained by normalizing the performance metrics of each node;

[0036] Step 2: The searched solutions are used as the decision set. The decision set u is standardized to reduce the impact of data at different scales.

[0037] Step 3: After standardizing the decision set, we obtain r. Multiply the weight vector with r to construct the standardized decision matrix wr.

[0038] Step 4: After constructing the weighted decision matrix, it is necessary to calculate the Euclidean distance between the individual and the positive and negative ideal solutions. The ideal solution in this paper It is the largest value in the first column of the weighted matrix wr, representing the negative ideal solution. It is the minimum value of the second column of the weighted matrix wr, z i1 The first column contains the indicators for each individual, z. i2 The second column contains indicators for each individual.

[0039]

[0040]

[0041] Step 5: Using Euclidean distance Calculate the final satisfaction score S for each solution. i After calculating the satisfaction level, normalize it; finally, the satisfaction score S is calculated. i The highest-valued solution is used as the final decision solution.

[0042]

[0043] The constructed standardized weighted decision matrix wr is shown in equation (5);

[0044]

[0045] Where F represents the computational performance index, a ij p represents the standardized individual accuracy. ij This represents the standardized individual parameter values.

[0046] The beneficial effects of this invention are as follows: This invention proposes a reliable intrusion detection framework to ensure that intrusion detection alert data is not tampered with as much as possible. It strengthens the uploading of reliable alert data by designing reward and punishment mechanisms through node credit rating. Regarding the intrusion detection model, a multi-objective evolutionary algorithm is used to evolve convolutional neural network structures with small parameter counts and maximizing accuracy. Finally, an adaptive weight decision method is used to determine the final solution. Attached Figure Description

[0047] Figure 1 This is a structural diagram of the present invention.

[0048] Figure 2 A diagram of the new convolutional block structure.

[0049] Figure 3 A diagram of a two-layer optimized multi-objective neural evolution architecture. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0051] (1) As Figure 1 As shown, humans interact with distributed hosts through IoT devices, and hackers can launch botnet attacks on these distributed hosts by attacking these IoT devices. Illegal and legitimate network traffic from the IoT will be collected as training data. The raw traffic cannot be used directly and needs to be preprocessed. A suitable architecture is searched using a neural evolution algorithm; the algorithm details will be described in detail below. The searched model is deployed on distributed devices to monitor traffic. The intrusion detection system deployed on the distributed hosts will continue to collect alert data and store it encrypted on a blockchain. These alerts, after data processing, will continue to be used as training data to evolve the structure of the neural network.

[0052] This invention uses transparent and secure smart contracts for node credit rating. Reward and penalty mechanisms are designed to encourage nodes to upload reliable alert data or logs. The smart contracts are deployed by the administrator and cannot be tampered with once deployed.

[0053] A node's trust score changes dynamically, as shown in the formula. When a node first joins the network, the contract initializes it with a minimum trust value S. trust (t0). The trust assessment module calculates the current score based on the node's historical trust score and current behavior. After a node joins the network, the trust assessment smart contract updates the node's trust value over time.

[0054]

[0055] Where f1, f2, ... f j These are the feature scores that influence the node's trust value; they can be negative or positive, w1, w2, ... w j The weighting factor is determined by the system administrator based on the actual situation. It's a decay function; the influence of past actions on the current trust value decreases over time. Malicious behavior on a node can reduce the current trust value, even causing it to become negative. And for that node t... n Trust ratings over time will also be affected.

[0056] The characteristic scores of trust values ​​are usually determined based on the specific application scenario. The characteristic scores of node trust values ​​can be mapped using a mapping table. For example, if the number of successful transaction commits is greater than 1 and less than 3, then the score is mapped to 1 point.

[0057] When a node writes a hash value to the blockchain, it must first use access control to define read and write rights according to the access control list rules. Each node must maintain a minimum trust level S. trust Only nodes with a trust rating of (t0) have the right to participate in reading and writing data. Nodes with low trust ratings will have their alert data access rights revoked and will be unable to write hash values ​​to the blockchain.

[0058] When the node reaches the minimum trust level S trust (t0) When preparing to write data to the blockchain, the data needs to be encrypted. The specific steps include:

[0059] Step 1: Use the AES key to perform AES symmetric encryption on the alarm data or log data and generate AES ciphertext.

[0060] Step 2: Asymmetric encryption of the AES key. The AES key is asymmetrically encrypted using RSA to generate RSA ciphertext.

[0061] Step 3: The AES ciphertext and RSA ciphertext are uploaded to IPFS respectively, and IPFS will return two different hash values.

[0062] Step 4: Store the hash values ​​generated in IPFS using AES and RSA ciphertexts in the blockchain via remote procedure call to the blockchain contract.

[0063] (2) For the part on the bi-layer optimized multi-objective neural evolution algorithm, please refer to [link / reference]. Figure 2 , Figure 3 This algorithm is based on a multi-objective neural architecture search process, using a multi-objective evolutionary algorithm to search for the structure of a neural network. The algorithm divides the search for the neural network structure into two parts to search for lightweight neural networks. The first part optimizes the block-level connections of the upper feature extraction layer, and the second part optimizes the internal connections of neurons in the lower classification layer, ultimately obtaining the Pareto solution set of the intrusion detection model. Specifically:

[0064] Step 1: Collect network traffic data and preprocess it to build a dataset;

[0065] Step 2: Construct a two-layer optimized multi-objective neural evolution algorithm; specifically described as follows:

[0066] Step 2.1: Construct multiple different block-level structures as components of a two-layer optimized multi-objective neural evolution algorithm framework;

[0067] Step 2.2: Construct new convolutional blocks to add to the search space to enhance the feature processing capabilities of individual neural networks. For example... Figure 2 As shown, this invention designs a novel convolutional block structure for space search. The feature map is used as input, and it is first convolved spatially using depthwise convolution, where each channel is handled by only one convolutional kernel. After depthwise convolution, a 1×1 channel-wise pointwise convolution is performed. This combination of operations results in low parameter count and computational cost. After pointwise convolution, layer normalization is performed to ensure the stability of data features. This is followed by an activation layer that uses Gaussian error linear units to map the feature map.

[0068] Step 2.3: Construct an adaptive sparse connection layer in the lower classification layer to optimize the connection weights of neurons in the fully connected layer;

[0069] Step 3: The feature data is classified using the softmax function;

[0070] Step 4: Initialize the population;

[0071] Step 5: Use the NSGA-III multi-objective optimization algorithm to search the block-level connections of the upper feature extraction layer, and use the SET algorithm to adaptively sparsely evolve the neuron connection weights in the lower classification layer.

[0072] Step 6: After the search is complete, the final architecture is retrained using the training set in the dataset constructed in Step 1.

[0073] After retraining the searched architecture, a suitable final solution needs to be selected and deployed across distributed nodes. Due to differences in computing power among distributed nodes, the computational performance of different nodes must be considered during decision-making. TOPSIS is used to make decisions regarding different nodes. The original TOPSIS determines the weights of each objective using entropy weighting or decision-maker subjectivity. This method lacks flexibility and may not meet the actual needs of decision-makers. For the individual differences of each node, the weights of each node should be adaptively adjusted. Nodes with good computational performance are relatively insensitive to the number of model parameters, and their weights for parameters are correspondingly smaller. Nodes with poor computational performance are sensitive to the number of model parameters, and some accuracy can be sacrificed to ensure latency.

[0074] The specific steps of adaptive decision-making include:

[0075] Step 1: The weight vector W is obtained by normalizing the performance metrics of each node;

[0076] Step 2: The searched solutions are used as the decision set. The decision set u is standardized to reduce the impact of data at different scales.

[0077] Step 3: After standardizing the decision set, we obtain r. Multiply the weight vector with r to construct the standardized decision matrix wr.

[0078] Step 4: After constructing the weighted decision matrix, it is necessary to calculate the Euclidean distance between the individual and the positive and negative ideal solutions. The ideal solution in this paper It is the largest value in the first column of the weighted matrix wr, representing the negative ideal solution. It is the minimum value of the second column of the weighted matrix wr, z i1 The first column contains the indicators for each individual, z. i2 The second column contains indicators for each individual.

[0079]

[0080]

[0081] Step 5: Using Euclidean distance Calculate the final satisfaction score S for each solution. iAfter calculating the satisfaction level, normalize it; finally, the satisfaction score S is calculated. i The highest-valued solution is used as the final decision solution.

[0082]

[0083] The constructed standardized weighted decision matrix wr is shown in equation (5);

[0084]

[0085] Where F represents the computational performance index, a ij p represents the standardized individual accuracy. ij This represents the standardized individual parameter values.

[0086] Example:

[0087] The decision-making process of the model was simulated. Before deploying the model, a decision needs to be made on the searched solution set to select suitable solutions for deployment. The TOPSIS method is used to make decisions on the solution set. Since edge inference mostly relies on the CPU, we mainly use the CPU clock frequency as an evaluation index of computing performance to simulate the computing performance of distributed nodes. In practical application environments, the calculation of performance indicators can be determined according to the actual situation. As shown in Table 1, the CPU clock frequencies of five nodes are simulated as indicators for calculating weights. First, the clock frequencies of these five nodes are normalized to obtain the weight vector W. i1 W i2 The weight vector representing node i.

[0088]

[0089] Taking the four solutions found in the UNSW-NB15 dataset as an example, A1-expand, A2-expand, A3-expand, and A4-expand constitute the decision set u. Then, the decision set is standardized to eliminate the influence of different units, resulting in the standardized decision set r.

[0090]

[0091] The standardized weighted decision matrix wr is constructed as shown below, where each row of the weight vector W represents the vector corresponding to each node.

[0092]

[0093] After calculating the weighted decision matrix, it is necessary to calculate the Euclidean distance between the individual and the positive and negative ideal solutions. Here, the positive ideal solution is the largest value in the first column of the weighted matrix wr, and the negative ideal solution is the smallest value in the second column of the weighted matrix wr. The calculated positive and negative ideal solutions are shown in Table 2. The final satisfaction score is then calculated using Euclidean distance. After calculating the satisfaction score, it is normalized. The solution with the highest final satisfaction score is taken as the final decision solution. The satisfaction scores for each node are shown below.

[0094] Satosfaction1=(0,0.01612298,0.01206036,1) T

[0095] Satosfaction2=(0,0.75240592,0.69360826,1) T

[0096] Satosfaction3=(1,0.32004034,0,0.64855572) T

[0097] Satosfaction4=(0.53307902,0,1,0.45960502) T

[0098] Satosfaction5=(0.98177149,0,1,0.97568223) T

[0099] The final decisions, calculated across the five nodes, are shown in Table 3. The table reveals that, through the adaptive decision-making process, nodes with high computational performance, such as N1 and N2, tend to produce solutions with higher accuracy. They are not sensitive to the number of model parameters. Nodes with lower computational performance, such as N4 and N5, produce decision solutions that favor models with fewer parameters.

[0100] Table 1 CPU clock frequency of each node

[0101]

[0102] Table 2 Positive and Negative Ideal Solutions for Each Node

[0103]

[0104] Table 3 Decision solutions for each node

[0105]

Claims

1. A reliable multi-target neural evolutionary intrusion detection framework, characterized in that, The steps are as follows: (1) Define the trust assessment module The creditworthiness of alert data or nodes is assessed through smart contracts, and data access control is considered: The trust score of a node changes dynamically, as shown in equation (1), which is the trust evaluation module; when a node first joins the network, the smart contract initializes it with a minimum trust value S. trust (t0); The trust assessment module calculates the current score based on the node's historical trust score and current behavior; After a node joins the network, the smart contract will update the node's trust value over time; Where f1, f2, ... f j These are the feature scores that influence the node's trust value; they can be negative or positive; w1, w2, ... w j The weighting factor is determined by the system administrator based on the actual situation; It is a decreasing function; the older the historical trust value, the smaller its impact on the current trust value. Malicious behavior by a node will cause the current trust value to decrease, even to a negative value, and will also affect the trust value of that node t. n Trust ratings over time will also be affected; When a node writes a hash value to the blockchain, it must first use access control to define read and write rights according to the access control list rules; each node must maintain a minimum trust level S. trust Only those with a trust value below S have the right to participate in data reading and writing; trust The node at (t0) will have its access to alarm data revoked and will be unable to write hash values ​​to the blockchain; When the node reaches the minimum trust level S trust (t0) When preparing to write data to the blockchain, the data needs to be encrypted. The specific steps include: Step 1: Encrypt alarm data or log data using AES key and generate AES ciphertext; Step 2: Perform asymmetric encryption on the AES key, using RSA asymmetric encryption to encrypt the AES key and generate RSA ciphertext; Step 3: Upload the AES and RSA ciphertexts to IPFS. IPFS will return two hash values. Step 4: Store the hash values ​​generated in IPFS using AES and RSA ciphertexts in the blockchain via a remote procedure call blockchain contract; (2) Implement a two-layer optimized neural evolution algorithm Based on a multi-objective neural architecture search process, this algorithm searches for the structure of convolutional neural networks using a multi-objective evolutionary algorithm. To find lightweight neural networks, the search is divided into two parts: the first part optimizes the block-level connections of the upper feature extraction layer, and the second part optimizes the internal connections of neurons in the lower classification layer, ultimately obtaining the Pareto solution set of the intrusion detection model. Specifically: Step 1: Collect and preprocess network intrusion data to build a dataset; Step 2: Construct a two-layer optimized multi-objective neural evolution algorithm framework; Step 2.1: Construct multiple different block-level structures as components of a two-layer optimized multi-objective neural evolution algorithm framework; Step 2.2: Construct new convolutional blocks to add to the search space; the feature map is used as input, and first it is convolved in the spatial dimension by depthwise convolution, with one convolutional kernel responsible for only one channel; after depthwise convolution, 1×1 pointwise convolution is performed in the channel dimension; after pointwise convolution, layer normalization is performed; then the activation layer is activated, and Gaussian error linear units are used to map the feature map; Step 2.3: The lower classification layer constructs an adaptive sparse connection layer using the SET algorithm to optimize the connection weights of the fully connected neurons; Step 3: Use a softmax classifier to classify the feature data; Step 4: Initialize the population; Step 5: Optimize the block-level connections of the upper feature extraction layer using the NSGA-III multi-objective optimization algorithm, and perform adaptive sparse evolution of neuron connection weights in the lower classification layer using the SET algorithm; Step 6: After the search is completed, a Pareto solution set is generated; the solutions in the Pareto solution set are retrained one by one using the training set in the dataset constructed in Step 1 to obtain the corresponding performance indicators, namely accuracy and the number of model parameters. (3) Adaptive decision-making process After searching using a two-layer optimized neural evolution algorithm, a Pareto solution set and corresponding performance metrics are obtained. Suitable final solutions need to be selected and deployed across distributed nodes. For the individual differences of each node, the weights of each node should be adaptively adjusted. When calculating the weighted decision matrix using TOPSIS, the weights are calculated based on the computational performance of each node. Specific steps include: Step 1: Normalize the weight vector index to obtain the weight vector W; Step 2: The searched solutions are used as the decision set. The decision set u is standardized to eliminate the influence of different dimensions. Step 3: After decision-making and standardization, obtain r, and construct the standardized decision matrix wr; Step 4: After constructing the weighted decision matrix, it is necessary to calculate the Euclidean distance between the individual and the positive and negative ideal solutions. Positive Ideal Solution It is the largest value in the first column of the weighted matrix wr, representing the negative ideal solution. It is the minimum value of the second column of the weighted matrix wr, z i1 The first column contains the indicators for each individual, z. i2 The second column contains indicators for each individual. Step 5: Using Euclidean distance Calculate the final satisfaction score S for each solution. i After calculating the satisfaction level, normalize it; the solution with the highest satisfaction score is taken as the final decision solution. The constructed standardized weighted decision matrix wr is shown in equation (5); Where F represents the computational performance index, a ij p represents the standardized individual accuracy. ij This represents the standardized individual parameter values.

Images