DNNs security protection method for activation function

Abstract

A DNNs security protection method oriented to an activation function comprises the following steps: constructing a mountain-like activation function on the basis of an existing activation function, and performing spatial reconstruction on neurons; according to a spatial constraint relationship of neurons, adaptively and dynamically adjusting a peak value; a TOP-1 peak value enhancement method based on static statistical data driving is adopted to obtain a static peak value so as to enhance the peak value characteristic of a mountain-like activation function. According to the invention, secondary fine tuning of the model is not needed, and effective discrimination and activation can be carried out on neuron input by using a dynamic and static dual-state data driving mode at the same time; meanwhile, the method significantly improves the accuracy and reliability of the model, and has the characteristics of low overhead and high robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep neural network model security protection, and in particular to a method for improving model reliability using an activation function. Background Art

[0002] In deep neural networks (DNNs), activation functions enable the network to learn complex patterns and relationships by introducing nonlinear characteristics, which is crucial to the performance of deep learning models. At present, the research on activation functions in deep neural networks mainly focuses on factors such as saturation, nonlinearity, continuity, differentiability, and computational efficiency. However, as a key component of neural networks, activation functions should also have the ability to discriminate input data in order to limit the propagation of irrelevant and erroneous neuron input perturbations. This requirement is particularly important for maintaining the reliability of the system, especially in artificial intelligence-driven safety-critical systems (AISCS), which are often vulnerable to various security challenges. For example, individuals with physical access can use laser injection attacks to interfere with the relevant hardware deployed in deep neural networks. Therefore, it is of great theoretical and practical significance to deeply study activation functions to enable DNNs to resist these security threats.

[0003] In recent years, there has been an increasing number of methods targeting the output range of neural network activation values ​​to enhance model reliability, such as ClipAct, Ranger, and BReLU. ClipAct is an error mitigation technique that compresses high-intensity error activation values ​​by replacing unbounded activation functions with clipped versions. Ranger is an automatic conversion method that selectively limits the range of activation values ​​in deep neural networks (DNNs), thereby reducing large deviations caused by critical faults. However, both ClipAct and Ranger rely on static statistics derived from training data to set thresholds for each neuron, which makes them somewhat deficient in dynamic input perception. BReLU uses a gradient ascent algorithm to determine the upper limit of ReLU activations on all layers, aiming to suppress fault propagation by limiting the maximum ReLU activation value of each layer. This method mainly focuses on the ReLU activation function and ignores the processing of other activation functions. In addition, these methods may sacrifice a certain degree of model accuracy while improving model reliability. At the same time, they impose additional storage requirements on the thresholds associated with each neuron. Therefore, how to achieve a dual improvement in model accuracy and reliability using less storage resources remains a research problem that needs to be solved. Summary of the invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention proposes a deep neural network (DNNs) security protection method for the activation function. This method does not require secondary fine-tuning of the model, and can simultaneously use the dynamic and static dual-state data driving method to effectively identify and activate neuron inputs; at the same time, this method significantly improves the accuracy and reliability of the model, and has the characteristics of low overhead and high robustness.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A DNNs security protection method for activation functions, the method comprising the following steps:

[0007] Step 1: Based on the target neural network N, spatially reconstruct the input neurons of each layer of activation function to obtain the spatial constraint relationship R of each neuron;

[0008] Step 2: Combined with the neuron spatial constraint relationship R, introduce learnable parameters, all of which are initially set to 1, and are subsequently trained and updated together with the model weights; adaptively and dynamically adjust the neuron peak to obtain the final dynamic peak;

[0009] Step 3: Combine dynamic peak p d Based on the original activation function, a mountain-like activation function F_M is constructed according to formula (6). When the neuron input value z is less than or equal to the peak value p of the mountain-like activation function d When the input value z is greater than the peak value p d When , z may be disturbed, so different degrees of suppression are required;

[0010]

[0011] Step 4: Replace the activation function in the target neural network N with the proposed mountain-like activation function F_M;

[0012] Step 5: Use the training data set to train the modified target neural network N to obtain the trained model M;

[0013] Step 6: Use the TOP-1 peak enhancement strategy to obtain the enhanced peak value Peak of neuron n n ;

[0014] Step 7: According to formula (9), the mountain-like activation function F_M in the model M is modified again, without the need for secondary training, to obtain the final target model that can be used directly;

[0015]

[0016] Further, the process of step 1 is as follows:

[0017] 1.1. Determine the dimension of the activation function input feature map: If the activation function input feature map is a 3-D feature map, the size is expressed as H*W*C, then directly proceed to step 1.4; if the activation function input feature map is a 1-D or 2-D feature map, then proceed to step 1.2; where H, W and C represent the height, width and number of channels of the feature map respectively;

[0018] 1.2. The 1-D or 2-D feature map of each activation function input in N is expressed as: L*C or L, and is filled with a zero value using formula (1) so that its size becomes L'*C or L' respectively; where L and L' represent the length of the feature map before and after filling respectively;

[0019]

[0020] 1.3. Combine formula (2) to transform the spatial dimensions of the feature maps of size L'*C and L' in 1.1, so that they become 2-D and 3-D feature maps of size H*W*C and H*W respectively;

[0021] H=W=sqrt(L') (2)

[0022] 1.4. Use Cartesian 2-D or 3-D coordinate system to map feature space. Put the 2-D feature map into Cartesian 2-D coordinate system, that is, the mapping size on the x-axis is 1*W; the mapping size on the y-axis is 1*H; put the 3-D feature map into Cartesian 3-D coordinate system, that is, the mapping size on the y-z plane is H*W; the mapping size on the x-z plane is H*C; the mapping size on the x-y plane is W*C;

[0023] 1.5. In the Cartesian 2-D coordinate system, along<x,y> Perform maximum pooling in different directions of the axis to obtain peak (maximum) matrices of size <1*W, 1*H> <P x ,P y >, P x ,P y Represents the peak matrices in the x and y dimensions respectively;

[0024] In a Cartesian 3-D coordinate system, along<x,y,z> The maximum value pooling is performed in different directions of the axis, and the sizes are<H*W,H*C,W*C> The peak (maximum) matrix <P x ,P y ,P z >, P x ,P y ,P z Represent the peak matrices in three dimensions: x, y, and z;

[0025] 1.6. Establish the neuron space constraint relationship R, according to the maximum value constraint relationship of neuron n in two-dimensional space: the value V of neuron n n are respectively less than or equal to the maximum value (p) of the two dimensions (x, y) to which they are associated x ,p y ), p x ∈P x ,p y ∈P y , as in formula (3-1), and then formula (4-1);

[0026]

[0027] According to the maximum value constraint relationship of neuron n in three-dimensional space: the value of neuron n V n are respectively less than or equal to the maximum value (p) of the three dimensions (x, y, z) to which they are associated x ,p y ,p z ), p x ∈P x ,p y ∈P y ,p z ∈P z , as in formula (3), and then formula (4);

[0028]

[0029] Furthermore, in step 2, combined with the neuron space constraint relationship R, according to the neuron n maximum value constraint relationship in the two-dimensional space, they are p x ,p y Introducing learnable parameters w x ,w y , the initial value is set to 1, and then updated with the model weight training; adaptively and dynamically adjust the neuron peak value to obtain the final dynamic peak value p d , see formula (5-1);

[0030] p d =w x ·α·p x +w y β p y (5-1)

[0031] According to the neuron n-value constraint relationship in three-dimensional space, they are p x ,p y ,p z Introducing learnable parameters w x ,w y ,w z, the initial value is set to 1, and then it is trained and updated together with the model weights; the neuron peak value is adaptively and dynamically adjusted to obtain the final dynamic peak value p d , see formula (5);

[0032] p d =w x ·α·p x +w y β p y +w z ·γ·p z (5).

[0033] Furthermore, the process of step 6 is as follows:

[0034] 6.1. Based on the training data, derive the dynamic peak matrix P of each image on the dimensions x,y or x,y,z after all the mountain-like activation functions x and P y or P x , P y and P z ;

[0035] 6.2. Batch Processing to Export Dynamic Peak Matrix P x and P y or P x , P y and P z , and sort them at the element level, selecting the maximum peak of each neuron to form a statistically based static peak matrix P' x and P' y or P' x , P' y and P' z ;

[0036] 6.3. The static peak matrix P' x and P' y or P' x , P' y and P' z Upload back to the trained model M;

[0037] 6.4. For the static peak matrix P' x and P' y or P' x , P' y and P' z Expand the dimension and get P x + and P y + or P x + , P y+ and P z + , P x + and P y + or P + x , P + y and P + z The dimensions are the same as the spatial dimensions of the input feature map;

[0038] 6.5. According to formula (7), P x + and P y + or P x + , P y + and P z + Perform reverse spatial reconstruction to obtain the final static peak matrix P s ;

[0039]

[0040] 6.6. According to formula (8), the peak value of neuron n is revised and enhanced to obtain the enhanced neuron peak value Peak n ;

[0041]

[0042] The technical concept of the present invention is: in view of the complexity of the actual operating environment, the deep learning model is susceptible to external interference. As an important component of the neural network, the activation function should not only enhance the representation ability of the model, but also have the ability to distinguish the neuron input. To this end, we propose a DNNs security protection method for activation functions. First, this method constructs a mountain-like activation function based on the existing activation function and spatially reconstructs the neurons. Subsequently, the peak value is adaptively and dynamically adjusted according to the spatial constraint relationship of the neurons. Finally, a TOP-1 peak enhancement method driven by static statistical data is used to obtain the static peak value to enhance the peak characteristics of the mountain-like activation function.

[0043] A DNN security protection method is proposed for activation functions. Based on the maximum limit of different neuron activation input values, this method constructs a mountain-like activation function with neuron disturbance suppression function. In addition, according to the dimension of neurons in space, a dynamic neuron peak constraint relationship is established to achieve adaptive adjustment of neuron peaks. In order to further improve the ability of activation functions to distinguish abnormal inputs, this method introduces a TOP-1 peak enhancement strategy based on static data statistics. Based on the above method, the activation function can not only improve the expressive power of the model, but also effectively enhance the reliability of the model. At the same time, this method does not require secondary training of the model, and is applicable to a variety of currently popular activation functions, and has broad application prospects.

[0044] In the present invention, first, according to the target neural network N, the input neurons of each layer of the activation function are spatially reconstructed to obtain the spatial constraint relationship R of each neuron. Next, a learnable parameter is introduced to dynamically and adaptively adjust the peak value of the neuron, thereby constructing a mountain-type activation function and replacing the original activation function in the neural network N. Subsequently, the neural network is trained using the training data to obtain the target model M. Each neuron peak value derived from M is sorted and processed to select the final static peak value. Next, the processed static peak value is imported into the model M, and reversely reconstructed in M. Finally, the dynamic and static peak values ​​are combined to effectively distinguish the activation input of the neuron. At this point, the modification of the model M is completed, and the model M is updated and stored.

[0045] The beneficial effects of the present invention are mainly manifested in:

[0046] 1. According to the characteristics of the currently popular activation functions, a mountain-like activation function is constructed to effectively limit the abnormal activation input of neurons and show strong generalization ability;

[0047] 2. Through the spatial reconstruction of neurons, the mountain-like activation function can dynamically and adaptively adjust the activation value to adapt to different input data. This not only introduces spatial information, but also enhances the fitting ability of the activation function and the ability to distinguish the input;

[0048] 3. A static peak enhancement strategy based on statistical data was introduced to consolidate and strengthen the mountain-like activation function;

[0049] 4. Due to the reverse reconstruction method, only the compressed peak matrix needs to be stored, and there is no need to store the peak value of each neuron. Therefore, only a very small storage overhead is required to ensure the reliability of the model in the face of large disturbances;

[0050] 5. The activation function enhancement strategy based on dynamic and static dual states can not only improve the accuracy of the model, but also effectively improve the reliability of the model and achieve a significant improvement in the overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of the DNNs security protection method for activation functions.

[0052] Figure 2 DualD and DualD are used on ReLU, H-Swish and Mish respectively. - The performance change after the upgrade and the growth rate of Params and FLOPs.

[0053] Figure 3 This is a comparison chart of the ADR rate of model accuracy after injecting random bit flip faults. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings.

[0055] Reference Figure 1 , a DNNs security protection method for activation functions, the method comprising the following steps:

[0056] Step 1: Based on the target neural network N, spatially reconstruct the input neurons of each layer of activation function to obtain the spatial constraint relationship R of each neuron. The process is as follows:

[0057] 1.1. Determine the dimension of the activation function input feature map: If the activation function input feature map is a 3-D feature map, the size is expressed as H*W*C, then directly proceed to step 1.4; if the activation function input feature map is a 1-D or 2-D feature map, then proceed to step 1.2; where H, W and C represent the height, width and number of channels of the feature map respectively;

[0058] 1.2. The 1-D or 2-D feature map of each activation function input in N is expressed as: L*C or L, and is filled with a zero value using formula (1) so that its size becomes L'*C or L' respectively; where L and L' represent the length of the feature map before and after filling respectively;

[0059]

[0060] 1.3. Combine formula (2) to transform the spatial dimensions of the feature maps of size L'*C and L' in 1.1, so that they become 2-D and 3-D feature maps of size H*W*C and H*W respectively;

[0061] H=W=sqrt(L') (2)

[0062] 1.4. Use Cartesian 2-D or 3-D coordinate system to map feature space. Put the 2-D feature map into Cartesian 2-D coordinate system, that is, the mapping size on the x-axis is 1*W; the mapping size on the y-axis is 1*H; put the 3-D feature map into Cartesian 3-D coordinate system, that is, the mapping size on the y-z plane is H*W; the mapping size on the x-z plane is H*C; the mapping size on the x-y plane is W*C;

[0063] 1.5. In the Cartesian 2-D coordinate system, along<x,y> Perform maximum pooling in different directions of the axis to obtain peak (maximum) matrices of size <1*W, 1*H> <P x ,P y >, P x ,P y Represents the peak matrices in the x and y dimensions respectively;

[0064] In a Cartesian 3-D coordinate system, along<x,y,z> The maximum value pooling is performed in different directions of the axis, and the sizes are<H*W,H*C,W*C> The peak (maximum) matrix <P x ,P y ,P z >, P x ,P y ,P z Represent the peak matrices in three dimensions: x, y, and z;

[0065] 1.6. Establish the neuron space constraint relationship R, according to the maximum value constraint relationship of neuron n in two-dimensional space: the value V of neuron n n are respectively less than or equal to the maximum value (p) of the two dimensions (x, y) to which they are associated x ,p y ), p x ∈P x ,p y ∈P y , as in formula (3-1), and then formula (4-1);

[0066]

[0067] According to the maximum value constraint relationship of neuron n in three-dimensional space: the value of neuron n V n are respectively less than or equal to the maximum value (p) of the three dimensions (x, y, z) to which they are associated x ,p y ,p z ), p x ∈P x ,p y ∈P y ,p z ∈P z, as in formula (3), and then formula (4);

[0068]

[0069] (Since the subsequent operations of 2-D and 3-D feature maps are similar, the 2-D feature map and the 3-D feature map are described in parallel using an “or” relationship.)

[0070] Step 2: Combine the neuron space constraint relationship R, according to the neuron n maximum value constraint relationship in two-dimensional space, they are p x ,p y Introducing learnable parameters w x ,w y , the initial value is set to 1, and then updated with the model weight training; adaptively and dynamically adjust the neuron peak value to obtain the final dynamic peak value p d , see formula (5-1);

[0071] p d =w x ·α·p x +w y β p y (5-1)

[0072] According to the neuron n-value constraint relationship in three-dimensional space, they are p x ,p y ,p z Introducing learnable parameters w x ,w y ,w z , the initial value is set to 1, and then it is trained and updated together with the model weights; the neuron peak value is adaptively and dynamically adjusted to obtain the final dynamic peak value p d , see formula (5);

[0073] p d =w x ·α·p x +w y β p y +w z ·γ·p z (5)

[0074] Step 3: Combine dynamic peak p d Based on the original activation function (such as the common ReLU, H-Swish, Mish and GELU), a mountain-like activation function F_M is constructed according to formula (6). When the neuron input value z is less than or equal to the peak value p of the mountain-like activation function d When the input value z is greater than the peak value p dWhen , z may be disturbed, so different degrees of suppression are required;

[0075]

[0076] Step 4: Replace the activation function in the target neural network N with the proposed mountain-like activation function F_M;

[0077] Step 5: Use the training data set to train the modified target neural network N to obtain the trained model M;

[0078] Step 6: Use the TOP-1 peak enhancement strategy to obtain the enhanced peak value Peak of neuron n n , the process is as follows:

[0079] 6.1. Based on the training data, derive the dynamic peak matrix P of each image on the dimensions x,y or x,y,z after all the mountain-like activation functions x and P y or P x , P y and P z ;

[0080] 6.2. Batch Processing to Export Dynamic Peak Matrix P x and P y or P x , P y and P z , and sort them at the element level, selecting the maximum peak of each neuron to form a statistically based static peak matrix P' x and P' y or P' x , P' y and P' z ;

[0081] 6.3. The static peak matrix P' x and P' y or P' x , P' y and P' z Upload back to the trained model M;

[0082] 6.4. For the static peak matrix P' x and P' y or P' x , P' y and P' z Expand the dimension and get P x + and P y + or P x + , P y+ and P z + , P x + and P y + or P + x , P + y and P + z The dimensions are the same as the spatial dimensions of the input feature map;

[0083] 6.5. According to formula (7), P x + and P y + or P x + , P y + and P z + Perform reverse spatial reconstruction to obtain the final static peak matrix P s ;

[0084]

[0085] 6.6. According to formula (8), the peak value of neuron n is revised and enhanced to obtain the enhanced neuron peak value Peak n ;

[0086]

[0087] Step 7: Modify the mountain-like activation function F_M in the model M according to formula (9) without secondary training to obtain the final target model that can be used directly;

[0088]

[0089] This example uses the Caltech-101 dataset using the VGG16 model (using ReLU, H-Swish, and Mish respectively) as an example to demonstrate the results of our proposed method (see Figure 2 and Figure 3 ). For the sake of convenience, the proposed method will be referred to as DualD in the following text, and the version without TOP-1 peak enhancement strategy will be referred to as DualD - .

[0090] In this embodiment, Figure 2The figure shows the change Δdif in the four performance indicators of accuracy, precision, recall and F1-score, as well as the growth rate of parameters and floating-point operations per second (FLOPs) after applying the method in this paper, compared with when the method is not used. It can be clearly seen from the figure that the proposed method performs well in the above four performance indicators. Specifically, the present invention only needs to increase the number of parameters by 0.001% and the number of FLOPs by 0.186% in the VGG16 model to achieve at least 0.5% performance improvement. In addition, in order to evaluate the effectiveness of the method in this paper in improving the reliability of the model, Figure 3 We show the performance of the VGG16 model on the Caltech-101 dataset after random bit flipping faults (failure rates of 1e-6, 1e-5, and 1e-4, respectively) using the original activation functions (ACT: ReLU, H-Swish, and Mish) and applying our method (+DualD - The results show that when the failure rates are 1e-6, 1e-5, and 1e-4, our method (+DualD - The average ADRs of the proposed method and the method with DualD are 1.369%, 11.210% and 62.872% respectively, which are reduced by about 16 times, 7 times and 0.6 times respectively compared with the ADR without our method. This shows that the present invention has a significant improvement effect on improving the reliability of the model under fault conditions.

[0091] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A DNNs security protection method for activation functions, characterized in that: The method comprises the following steps: Step 1: Based on the target neural network N, spatially reconstruct the input neurons of each layer of activation function to obtain the spatial constraint relationship R of each neuron; Step 2: Combined with the neuron spatial constraint relationship R, introduce learnable parameters, all of which are initially set to 1, and are subsequently trained and updated together with the model weights; adaptively and dynamically adjust the neuron peak to obtain the final dynamic peak; Step 3: Combine dynamic peak p d Based on the original activation function, a mountain-like activation function F_M is constructed according to formula (6). When the neuron input value z is less than or equal to the peak value p of the mountain-like activation function d When the input value z is greater than the peak value p d When , z may be disturbed, so different degrees of suppression are required; Step 4: Replace the activation function in the target neural network N with the proposed mountain-like activation function F_M; Step 5: Use the training data set to train the modified target neural network N to obtain the trained model M; Step 6: Use the TOP-1 peak enhancement strategy to obtain the enhanced peak value Peak of neuron n n ; Step 7: Modify the mountain-like activation function F_M in the model M according to formula (9) without secondary training to obtain the final target model that can be used directly; 2. A DNNs security protection method for activation functions according to claim 1, characterized in that: The process of step 1 is as follows: 1.

1. Determine the dimension of the activation function input feature map: If the activation function input feature map is a 3-D feature map, the size is expressed as H*W*C, then directly proceed to step 1.4; if the activation function input feature map is a 1-D or 2-D feature map, then proceed to step 1.2; where H, W and C represent the height, width and number of channels of the feature map respectively; 1.

2. The 1-D or 2-D feature map of each activation function input in N is expressed as: L*C or L, and is filled with a zero value using formula (1) so that its size becomes L'*C or L' respectively; where L and L' represent the length of the feature map before and after filling respectively; s = ceil(sqrt(L)) L'=s*s (1) a=L'-L 1.

3. Combine formula (2) to transform the spatial dimensions of the feature maps of size L'*C and L' in 1.1, so that they become 2-D and 3-D feature maps of size H*W*C and H*W respectively; H=W=sqrt(L') (2) 1.

4. Use Cartesian 2-D or 3-D coordinate system to map feature space. Put the 2-D feature map into Cartesian 2-D coordinate system, that is, the mapping size on the x-axis is 1*W; the mapping size on the y-axis is 1*H; put the 3-D feature map into Cartesian 3-D coordinate system, that is, the mapping size on the y-z plane is H*W; the mapping size on the x-z plane is H*C; the mapping size on the x-y plane is W*C; 1.

5. In the Cartesian 2-D coordinate system, along<x,y> Perform maximum pooling in different directions of the axis to obtain peak (maximum) matrices of size <1*W, 1*H> <P x ,P y >, P x ,P y Represents the peak matrices in the x and y dimensions respectively; In a Cartesian 3-D coordinate system, along<x,y,z> The maximum value pooling is performed in different directions of the axis, and the sizes are<H*W,H*C,W*C> The peak (maximum) matrix <P x ,P y ,P z >, P x ,P y ,P z Represent the peak matrices in three dimensions: x, y, and z; 1.

6. Establish the neuron space constraint relationship R, according to the maximum value constraint relationship of neuron n in two-dimensional space: the value V of neuron n n are respectively less than or equal to the maximum value (p) of the two dimensions (x, y) to which they are associated x ,p y ), p x ∈P x ,p y ∈P y , as in formula (3-1), and then formula (4-1); R:V n st≤.α·pα x ++ββ=·p1 y (4-1) According to the maximum value constraint relationship of neuron n in three-dimensional space: the value of neuron n V n are respectively less than or equal to the maximum value (p) of the three dimensions (x, y, z) to which they are associated x ,p y ,p z ), p x ∈P x ,p y ∈P y ,p z ∈P z , as in formula (3), and then formula (4); R:V n ≤s.tα.·p x a++bbb·p+ y γ+=γ1·p z (4)。 3. A DNNs security protection method for activation functions as described in claim 2, characterized in that: In step 2, combined with the neuron space constraint relationship R, according to the neuron n maximum value constraint relationship in the two-dimensional space, they are p x ,p y Introducing learnable parameters w x ,w y ,, the initial values ​​are all set to 1, and are subsequently trained and updated together with the model weights; Adaptively and dynamically adjust the neuron peak value to obtain the final dynamic peak value p d , see formula (5-1); p d =w x ·a·p x +w y ·b·p y (5-1) According to the neuron n-value constraint relationship in three-dimensional space, they are p x ,p y ,p z Introducing learnable parameters w x ,w y ,w z , the initial value is set to 1, and is subsequently trained and updated along with the model weights; Adaptively and dynamically adjust the neuron peak value to obtain the final dynamic peak value p d , see formula (5); p d =w x ·a·p x +w y ·b·p y +w z ·g·p z (5)。 4. A DNNs security protection method for activation functions as described in any one of claims 1 to 3, characterized in that: The process of step 6 is as follows: 6.

1. Based on the training data, derive the dynamic peak matrix P of each image on the dimensions x,y or x,y,z after all the mountain-like activation functions x and P y or P x , P y and P z ; 6.

2. Batch processing to export the dynamic peak matrix P x and P y or P x , P y and P z , and sort them at the element level, selecting the maximum peak of each neuron to form a statistically based static peak matrix P' x and P' y or P' x , P' y and P' z ; 6.

3. The static peak matrix P' x and P' y or P' x , P' y and P' z Upload back to the trained model M; 6.

4. For the static peak matrix P' x and P' y or P' x , P' y and P' z Expand the dimension and get P x + and P y + or P x + , P y + and P z + , P x + and P y + or P + x , P + y and P + z The dimensions are the same as the spatial dimensions of the input feature map; 6.

5. According to formula (7), P x + and P y + or P x + , P y + and P z + Perform reverse spatial reconstruction to obtain the final static peak matrix P s ; 6.

6. According to formula (8), the peak value of neuron n is revised and enhanced to obtain the enhanced neuron peak value Peak n ;

Images