A neural architecture search-based anti-single event upset network structure optimization method and system

Abstract

The application discloses a single event upset resistant network structure optimization method and system based on neural architecture search, for simulating single event upset interference effect, first, different layers of the neural network model are distinguished according to different data contained in the layers, then the neural network model parameters, the neural network model input and output data are sampled in a given proportion respectively, then the extracted data is converted into binary, then the converted binary bits are sampled and bit flipping operation is performed, the data after single event upset interference simulation is obtained, the flipped binary data is converted into decimal data again, and finally the single event upset is accurately simulated. For the single event upset resistant model structure optimization algorithm, the single event upset interference simulation algorithm is introduced, the neural architecture search method is combined, the packaged single event upset simulation function is combined with the neural architecture search algorithm, and the structure optimization of the single event upset resistant neural network model is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to an anti-single event upset network structure optimization method and system based on neural architecture search. BACKGROUND

[0002] When an artificial intelligence chip is carried by a satellite as a satellite chip and applied to a space environment, the chip will be affected by space radiation, especially the radiation and impact phenomenon caused by high-energy particles in space, which will cause the chip to have a single event upset effect. The interference of the single event upset to the storage unit of the intelligent chip will cause errors in the parameters stored in the memory unit, and the errors mapped into the neural network will cause the output result of the neural network to deviate. To solve the above problems, researchers improve the anti-single event upset performance of the chip from two aspects of hardware and software: 1) from the aspect of hardware, the early anti-single event upset method mainly focuses on improving the manufacturing process of the chip, for example, using SOI devices. The buried oxide layer of the SOI device structure completely isolates the devices, which can fundamentally eliminate the single event latch-up effect, but the structure cannot prevent the penetration of protons, and the protons penetrating from the bottom will still cause the single event upset effect; 2) from the aspect of software, researchers try to solve the influence of the single event upset interference on the neural network model from the aspect of network structure. Lee et al. analyzed the fault tolerance of the feedforward neural network model and proposed to randomly disconnect the weights during model training to improve the error recovery capability. Assoum et al. analyzed the robustness of artificial neural networks against single event upset in space environment. Arechiga and Michaels et al. analyzed the influence of weight errors caused by single event upset interference on the model when the network model has different network structures such as VGG16, ResNet50 and InceptionV3 during the inference stage, which leads to the performance reduction of the neural network model when it suffers from single event upset. SUMMARY

[0003] The application aims to solve the problem of the single event upset effect caused by high-energy particles in space interfering with the performance of the neural network model in the prior art, and provides an anti-single event upset network structure optimization method and system based on neural architecture search.

[0004] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0005] The anti-single event upset network structure optimization method based on neural architecture search provided by the application comprises the following steps:

[0006] obtaining a simulation function according to a single event upset interference model;

[0007] constructing an anti-single event model architecture search space according to the simulation function;

[0008] According to the anti-single particle model architecture search space, a super network is constructed;

[0009] According to the constructed super network, a search strategy is determined, and the anti-single particle flip network structure optimization is realized.

[0010] Preferably, the method for obtaining the single particle flip interference model is as follows:

[0011] The parameters that need to be simulated by the single particle flip are converted from decimal to binary;

[0012] Randomly sample the binary bits of the data and determine the binary bits that need to be flipped, and then perform a bit flip operation on the selected binary bits;

[0013] The flipped binary data is converted back to decimal data, and the single particle flip interference model is finally obtained.

[0014] Preferably, the data in the single particle flip interference model is equally disturbed.

[0015] Preferably, the specific steps for constructing the search space of the anti-single particle flip optimization model are as follows:

[0016] Step 1: Define the search space

[0017] The defined search space satisfies the following conditions: each basic unit is composed of six nodes, the first two nodes are input layer fixed, and the last four nodes are intermediate nodes that need to be determined by searching; the input of each basic unit is the output of the previous two basic units, and the output of each basic unit is the connection of the intermediate node calculation results on the depth channel;

[0018] Step 2: Construct the search space

[0019] After defining the search space conditions, select avg_pool, max_pool, skip_connect, sep_conv, dil_conv and conv operations as optional basic operations of the search space operations, and introduce a single particle flip simulation function in each node. First, introduce the Seu function of the single particle flip interference simulation module seu_float_simple in the operations optional operation module, then traverse each layer defined in self.op in the operations optional operation module forward function and use the Seu function to perform single particle flip simulation operation, and finally define a global variable self.count at the model initialization to calculate the number of modified values.

[0020] Preferably, if self.classifier encapsulates a linear layer and an activation function, each layer is traversed and single event flips are performed.

[0021] Preferably, the method for constructing the super network is as follows:

[0022] The first step, for the flow of data between different nodes, the optional basic operation in the search space is used to form a basic unit with the nodes; on this basis, the calculation results of 1 / 4 of the intermediate nodes of the basic unit are randomly selected and connected into a search block in the channel dimension, and the remaining 3 / 4 features are combined with the output of the search block to form a new feature description by concat;

[0023] The second step, the feature map obtained by concat is subjected to a Softmax operation, so that it forms a continuous differentiable operation, and this part of the operation can realize network edge normalization to obtain a super network introducing a single event flip simulation function.

[0024] Preferably, a search strategy using gradient descent is used, the architecture search parameters are regarded as parameters to be optimized, after completing the network parameter training of each epoch, the obtained network model parameters are fixed, and the architecture search parameters are updated according to the evaluation results.

[0025] The anti-single event flip network structure optimization system based on neural architecture search provided by the application comprises:

[0026] A simulation function acquisition module is configured to acquire a simulation function according to a single event flip interference model;

[0027] A search space construction module is configured to construct an anti-single event model architecture search space according to the simulation function;

[0028] A super network construction module is configured to construct a super network according to the anti-single event model architecture search space;

[0029] A search strategy determination module is configured to determine a search strategy according to the constructed super network, and realize anti-single event flip network structure optimization.

[0030] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the anti-single event flip network structure optimization method based on neural architecture search when executing the computer program.

[0031] A computer readable storage medium stores a computer program, and the computer program realizes the steps of the anti-single event flip network structure optimization method based on neural architecture search when executed by a processor.

[0032] Compared with the prior art, the present application has the following beneficial effects:

[0033] The anti-single event upset network structure optimization method based on neural architecture search provided by the present application is used for effectively simulating the influence of the single event upset phenomenon on the performance of the neural network deployed on an artificial intelligence chip, and a single event upset interference effect simulation method is provided to accurately simulate the influence of the single event upset effect on the performance of the neural network model.

[0034] Further, limiting the search space to meet certain conditions can ensure that the search space is a space composed of the data flow mode between the four nodes.

[0035] The anti-single event upset network structure optimization system based on neural architecture search provided by the present application realizes the optimization of the anti-single event upset network structure by dividing the system into an analog function acquisition module, a search space construction module, a super network construction module and a search strategy determination module. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those of ordinary skill in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 The flowchart of the anti-single event upset network structure optimization method based on neural architecture search of the present application.

[0038] Figure 2 The flowchart of the single event upset interference simulation module of the present application.

[0039] Figure 3 The flowchart of the single event upset simulation function of the present application.

[0040] Figure 4A flow chart of the anti-single event upset model structure optimization algorithm of the present application.

[0041] Figure 5 A diagram of the anti-single event upset network structure optimization system of the present application based on neural architecture search. DETAILED DESCRIPTION

[0042] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0043] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0044] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0045] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is usually placed, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, therefore, it cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0046] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0047] In the description of the embodiments of the present application, it also needs to be explained that, unless otherwise explicitly specified and limited, if the terms "arrange", "install", "connect", "connect" appear, they should be understood in a broad sense, for example, they can be fixedly connected, or can be detachably connected, or integrally connected; can be mechanically connected, or can be electrically connected; can be directly connected, or can be indirectly connected through an intermediate medium, or can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0048] The present application will be further described in detail below in combination with the drawings:

[0049] The present application proposes an anti-single event upset network structure optimization method based on neural architecture search, as shown in Figure 1 The method comprises the following steps:

[0050] S1, obtaining a simulation function according to a single event upset interference model;

[0051] The method for obtaining the single event upset interference model is as follows:

[0052] The parameters that need to be simulated by single event upset are converted from decimal to binary;

[0053] Randomly sampling the binary bits of the data and determining the binary bits that need to be flipped, and then performing a bit flip operation on the selected binary bits;

[0054] The flipped binary data is converted back to decimal data, and the single event upset interference model is finally obtained.

[0055] The data in the single event upset interference model is equally disturbed.

[0056] S2, constructing an anti-single event model architecture search space according to the simulation function;

[0057] The specific steps of constructing the search space of the anti-single event upset optimization model are as follows:

[0058] First step: limit the search space

[0059] The limited search space meets the following conditions: each basic unit is composed of six nodes, of which the first two nodes are input layer fixed and unchanged, and the last four nodes are intermediate nodes that need to be determined by searching; The input of each basic unit is the output of the previous two basic units, and the output of each basic unit is the connection of the intermediate node calculation result on the depth channel;

[0060] Second step: construct the search space

[0061] After defining the search space condition, select avg_pool, max_pool, skip_connect, sep_conv, dil_conv and conv operations as optional basic operations of the search space operations, and introduce single particle flip simulation function in each node. First, introduce the Seu function of the single particle flip interference simulation module seu_float_simple in the operations optional operation module, then traverse each layer defined in self.op in the operations optional operation module forward function and use the Seu function to perform single particle flip simulation operation, and finally define a global variable self.count in the model initialization place to calculate the number of modification values.

[0062] If self.classifier encapsulates a linear layer and an activation function, traverse each layer and perform single particle flip.

[0063] S3, constructing a super network according to the anti-single particle model architecture search space;

[0064] The method for constructing a super network is as follows:

[0065] First, for the flow of data between different nodes, the optional basic operations in the search space and the nodes constitute basic units; on this basis, randomly select 1 / 4 of the intermediate node calculation results of the basic units and connect them in the channel dimension to form a search block, and the remaining 3 / 4 features are concatenated with the output of the search block to form new feature descriptions.

[0066] Second, perform a Softmax operation on the feature maps obtained by concat to make them form a continuous differentiable operation, which can realize network edge normalization and obtain a super network with single particle flip simulation function.

[0067] S4, determine the search strategy according to the constructed super network to realize the optimization of the anti-single particle flip network structure.

[0068] Using the gradient descent search strategy, the architecture search parameters are regarded as the parameters to be optimized. After completing the network parameter training of each epoch, the obtained network model parameters are fixed, and the architecture search parameters are updated according to the evaluation results.

[0069] As Figure 2As shown in the flowchart of the single event upset interference simulation module. Since the probability of single event upset occurring in space environment can be considered to be equal, the interference probability of the network model data (including model parameters and model intermediate data) is equal, so the model parameters are selected by random sampling, and the parameters are converted to binary, and the decimal parameters after single event upset are converted to decimal parameters. As shown in Figure 2 As shown, r is an analog control parameter, which is used to control whether single event upset simulation is required for the parameter, when r = 0, it represents that single event upset simulation is not required for the data, when r = 1, it represents that single event upset simulation is required for the data. For the model parameters that need to be simulated by single event upset, first convert the parameter from decimal to binary, then randomly sample the binary bits of the data to determine the binary bits that need to be inverted, then perform a bit flip operation on the selected binary bits, and finally convert the flipped binary parameters to decimal parameters, and finally obtain the model interfered by single event upset.

[0070] As shown in Figure 3 The single event upset simulation function flowchart of the present application is shown. The input of the single event upset simulation function Seu() includes three variables: input data (converted to tensor type), flip parameter proportion control parameter (prop and self.count); The output of Seu() includes two variables: flipped output data (converted to tensor type) and self.count. The functions needed to realize the single event upset simulation function are: Modify(), when r = 1, the single event upset simulation is performed on the obtained data, and the toBinary(), Bit_change() and toBase() functions are called. The toBinary() function converts decimal data to binary. Bit_change() randomly flips one bit of the obtained binary number. The toBase() function converts binary data to decimal. The Seu() function randomly obtains the parameters that need to be flipped in the input neural network model data (model output data, etc.) according to the proportion, and calls the Modify() function, which performs single event upset simulation on the obtained data when r = 1. The Modify() function first calls the toBinary() function to convert the selected parameters to binary form, then calls the Bit_change() function to randomly flip each binary parameter obtained, and finally calls the toBase() function to convert the flipped binary parameters to decimal parameters, and finally obtains the simulated data and modifies the neural network model data to single event upset parameters.

[0071] As shown in Figure 4The flowchart of the anti-single event upset model structure optimization algorithm proposed in the application is shown. Due to the introduction of the Dropout mechanism in the training process of the neural network model, the effective allocation of model parameters can be realized, and the anti-interference ability of the model itself is improved. Through the above mechanism, it can be found that the information distribution of the network model is adjusted, so that the information of the network model is stored in the parameters of the network model in a certain required distribution manner, thereby ensuring that the information of the network model is not concentrated in certain specific parameters, and the optimization of the information distribution of the network model can be effectively realized, and the anti-interference ability of the network model can be improved. Therefore, single event upset interference can be introduced in the training process of the network model to further improve the robustness of the network model when subjected to single event upset interference. On the basis of the above optimized model, in order to further excavate the model structure more robust to single event upset phenomenon, the model structure is searched by further combining the neural network structure search method, so as to realize the optimization of the anti-single event upset network model structure.

[0072] The specific steps are as follows:

[0073] Step 1, constructing an anti-single event upset model architecture search space

[0074] In the neural structure search algorithm, the search space defines whether the network architecture is searched in a discrete or continuous space, and in how large a space the architecture search is performed. Although theoretically the larger the search space, the more likely it is to find the optimal network structure, in the implementation process, the constraints of time and equipment need to be considered, and the search space is generally a limited subset space under certain premise conditions. The specific steps of defining the search space of the anti-single event upset optimization model are as follows:

[0075] Step 1.1 limit the search space

[0076] The present application intends to limit the search space to meet the following conditions: each basic unit is composed of six nodes (network layers), of which the first two nodes are input layer fixed and unchanged, and four nodes are intermediate nodes that need to be determined by searching; the input of each basic unit is the output of the previous two basic units, and the output of each basic unit is the connection (concat) of the calculation results of the intermediate nodes in the depth channel. Through the above limiting conditions, it can be ensured that the search space is composed of the flow of data (operations) among the four nodes.

[0077] Step 1.2 construct the search space

[0078] After defining the search space condition, operations such as avg_pool, max_pool, skip_connect, sep_conv, dil_conv and conv are selected as optional basic operations of the search space operations, and a single particle flip simulation function is introduced in each node (network layer). First, the Seu function of the single particle flip interference simulation module seu_float_simple is introduced in the operations optional operation module, then the Seu function is used to perform single particle flip simulation operation in each layer defined in self.op in the operations optional operation module forward function, and finally a global variable self.count is defined in the model initialization place to calculate the number of modification values.

[0079] Step 2, constructing a super network

[0080] On the basis of constructing the search space, the data flow between different nodes (network layers) is formed by the combination of various optional basic operations in the search space, and each operation corresponds to a coefficient a representing the weight of different operations in the data flow (mixed operation). The above mixed operation and node constitute the basic unit of model architecture search at the same time, and the network composed of such basic units is a super network. In the model architecture search space constructed in step 2.1, firstly, for the data flow between different nodes (network layers), the optional basic operations in the search space are used to construct the basic unit with the nodes; on this basis, 1 / 4 of the intermediate node calculation results of the basic unit are randomly selected and connected in the channel dimension to form a search block, and the remaining 3 / 4 of the features are combined with the output of the search block to form new feature description; secondly, the feature map obtained by concat is subjected to a Softmax operation, so that it constitutes a continuous differentiable operation, which can realize network edge normalization; finally, the super network with the single particle flip simulation function is obtained.

[0081] Step 3, determine the search strategy

[0082] On the basis of the constructed super network model, the invention uses a gradient descent search strategy, and regards the architecture search parameters as parameters to be optimized (similar to the network parameters in the general machine learning method). After completing the network parameter training of one epoch, the obtained network model parameters are fixed, and the architecture search parameters are updated according to the evaluation results. The updating method is similar to the updating of network parameters.

[0083] Step 4, performance evaluation of architecture search

[0084] The hardware platform for implementing the architecture search method is: NVIDIA GeForce RTX 2080, PyTorch 1.11.0 + CUDA 10.2 + TorchVision 0.12.0 running environment. In the super network constructed on the basis of the single event upset simulation function proposed in step 1, first, the super network is pre-trained for 15 epochs; then, it is trained for 1 epoch, and according to the gradient descent search strategy, the coefficients a corresponding to different operations in the architecture search hybrid operation and the parameters b for realizing network normalization are updated, and then the basic unit finally constituting the network structure is updated, and the updated network structure is used to train the data set for 1 epoch (until the training iteration number of architecture parameter update and network structure parameter update reaches 50 epochs); secondly, according to the architecture parameters after 50 iterations, the final single event upset resistant network structure is determined, and it is trained on the data set until it converges, and finally the optimized single event upset resistant network structure model is obtained and performance evaluation is carried out.

[0085] The anti-single event upset network structure optimization method based on neural architecture search needs five steps, single event upset interference effect simulation, constructing anti-single event upset model architecture search space, constructing super network, determining search strategy, architecture search performance evaluation. Take ResNet-18 model as backbone network based on PC-DARTS neural architecture search algorithm training CIFAR10 dataset on 1 GPU (NVIDIA GeForce RTX 2080) server as an example: (1) Single event upset interference effect simulation, for the parameters in the neural network model that need to be simulated by single event upset, first convert the parameter from decimal to binary, then randomly sample the binary bits of the data and determine the binary bits that need to be flipped, then perform bit flip operation on the selected binary bits, finally convert the flipped binary data back to decimal data, and finally get the simulated data and modify the neural network model parameters to the single event upset simulated data. (2) Constructing anti-single event upset model architecture search space, limiting the search space to the space composed of data flow between four nodes (operations), and selecting avg_pool, max_pool, skip_connect, sep_conv, dil_conv and conv as the basic operations of the search space operations. Introduce the single event upset simulation function in each node (network layer), first introduce the seu function of seu_float_simple single event upset interference simulation module in the operations selectable operation module, then traverse each layer defined in self.op in the operations selectable operation module forward function and use the seu function to realize the single event upset simulation operation, finally define a global variable self.count in the model initialization place to calculate the number of modified values. (3) Constructing super network, first, for the data flow between different nodes (network layers), adopt the selectable basic operations in the search space and nodes to form basic units; On this basis, randomly select 1 / 4 of the intermediate nodes of the basic units to calculate the results and connect them in the channel dimension to form search blocks, and the remaining 3 / 4 features are concatenated with the output of the search blocks to form new feature descriptions; Secondly, the feature maps obtained by concat are subjected to a Softmax operation, which makes them form a continuous differentiable operation. This part of the operation can realize network edge normalization; Finally, the super network with the single event upset simulation function is obtained. (4) Determine the search strategy, use the gradient descent search strategy, treat the architecture search parameters as the parameters to be optimized (similar to the network parameters in general machine learning methods), after completing the network parameter training of one epoch, fix the obtained network parameters, and update the architecture search parameters according to the evaluation results. The update method is also similar to the update of network parameters.(5) Architecture search performance evaluation, the specific implementation of the architecture search is carried out in the NVIDIAGeForce RTX 2080, PyTorch 1.11.0+CUDA 10.2+TorchVision 0.12.0 running environment. In the super network constructed on the basis of introducing the single event upset simulation function, first, the super network is pre-trained for 15 epochs; Then train for 1 epoch and update the coefficients a corresponding to different operations in the architecture search mixed operation and the parameters b for realizing network normalization according to the search strategy of gradient descent, and then update the basic unit finally constituting the network structure, and train for 1 epoch (until the training iteration number of architecture parameter update and network structure parameter update reaches 50 epochs) on the dataset using the updated network structure; secondly, according to the architecture parameters after 50 iterations, the final single event upset resistant network structure is determined, and it is trained on the dataset until convergence, and finally the optimized single event upset resistant network structure model is obtained and the performance is evaluated.

[0086] The anti-single event upset network structure optimization system based on neural architecture search provided by the application comprises an analog function acquisition module, a search space construction module, a super network construction module and a search strategy determination module. Figure 5

[0087] The analog function acquisition module is used for acquiring an analog function according to a single event upset interference model.

[0088] The search space construction module is used for constructing an anti-single event model architecture search space according to the analog function.

[0089] The super network construction module is used for constructing a super network according to the anti-single event model architecture search space.

[0090] The search strategy determination module is used for determining a search strategy according to the constructed super network, and realizing anti-single event upset network structure optimization.

[0091] The terminal device provided by the embodiment comprises a processor, a memory and a computer program stored in the memory and executable on the processor. The processor executes the computer program to realize the steps in each method embodiment. Alternatively, the processor executes the computer program to realize the functions of each module / unit in each device embodiment.

[0092] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the application.

[0093] ​The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device can include, but is not limited to, a processor and a memory.

[0094] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.

[0095] The memory can be used to store the computer program and / or modules, and the processor can realize various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory.

[0096] The modules / units integrated in the terminal device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0097] The application provides an anti-single event upset network structure optimization method based on neural architecture search, which is used for improving the model performance of a neural network model under the influence of single event upset. The method comprises a single event upset simulation method and an anti-single event upset model structure optimization algorithm. For the single event upset interference effect simulation function, first, the data in different layers of the neural network model are distinguished according to the difference, then the neural network model parameters, the neural network model input and output data are sampled according to a given proportion, then the sampled data are converted into binary data, then the converted binary bits are sampled and flipped, the flipped binary data are converted into decimal data again, and finally the accurate simulation of the single event upset is realized. For the anti-single event upset model structure optimization algorithm, the single event upset interference simulation algorithm is introduced, the neural structure search method is combined, the packaged single event upset simulation function is combined with the neural structure search algorithm, and finally the structure optimization of the anti-single event upset neural network model is realized. The method takes image recognition as the neural network model performance evaluation task, and uses the CIFAR10 database as the model performance evaluation data set. The experimental results show that when the image recognition task is performed on the CIFAR10 data set, the Top-1 accuracy of the image recognition task performed on the CIFAR10 data set by the method can reach 94.78%, compared with the network model without using the structure optimization method of the application, the method can effectively improve the performance of the network model under the single event upset interference, and the robustness of the neural network model is enhanced.

[0098] Has the following advantages:

[0099] 1) The application proposes an anti-single event upset network structure optimization method based on neural architecture search, which further improves the reliability and stability of the neural network model in the field of aviation and aerospace.

[0100] 2) The application proposes a single event upset interference effect simulation method to effectively simulate the influence of the single event upset phenomenon on the performance of the neural network deployed on the artificial intelligence chip, and accurately simulates the influence of the single event upset effect on the performance of the neural network model.

[0101] 3) The application further optimizes the structure of the anti-single event upset interference neural network model based on the neural structure search algorithm, and verifies the effect of the optimized model structure on the classical image recognition task.

[0102] The above merely describes the preferred embodiments of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A neural architecture search based anti-single event upset network structure optimization method, characterized in that, Comprise the following steps: According to the single event upset interference model to obtain the simulation function; the method for obtaining the single event upset interference model is as follows: Convert the parameters that need to be simulated by single event upset from decimal to binary; Randomly sample the binary bits of the converted binary data and determine the binary bits that need to be flipped, and then perform a bit flip operation on the selected binary bits; Convert the flipped binary data back to decimal data to obtain the single event upset interference model; The specific steps of constructing the anti-single event model architecture search space according to the simulation function are as follows: Step 1: Limit the search space, which meets the following conditions: each basic unit consists of six nodes, the first two nodes are input layer fixed, and the last four nodes are intermediate nodes that need to be determined by searching; the input of each basic unit is the output of the previous two basic units, and the output of each basic unit is the connection of the intermediate node calculation results in the depth channel; Step 2: Build the search space, after limiting the search space conditions, select avg_pool, max_pool, skip_connect, sep_conv, dil_conv and conv operations as the optional basic operations of the search space operations, and introduce the single event upset simulation function in each node, first introduce the Seu function of the single event upset interference simulation module seu_float_simple in the operations optional operation module, then traverse each layer defined in self.op in the operations optional operation module forward function and use the Seu function to perform single event upset simulation operation, finally define a global variable self.count at the model initialization to calculate the number of modified values; According to the anti-single event model architecture search space to construct a super network; the method for constructing the super network is as follows: Step 1, for the flow of data between different nodes, adopt the optional basic operations in the search space and the nodes to form basic units; on this basis, randomly select 1 / 4 of the intermediate node calculation results of the basic units and connect them in the channel dimension to form a search block, and the remaining 3 / 4 features are concatenated with the output of the search block to form new feature descriptions; Step 2, perform a Softmax operation on the feature maps obtained by concatenation to make them form a continuous differentiable operation, the Softmax operation can realize network edge normalization, and the super network introducing the single event upset simulation function is obtained; According to the constructed super network to determine the search strategy, realize the optimization of the anti-single event upset network structure; the optimized neural network model realizes the image recognition task.

2. The neural architecture search based anti-single event upset network structure optimization method according to claim 1, wherein, The data in the single event upset interference model is equally disturbed.

3. The neural architecture search based anti-single event upset network structure optimization method according to claim 1, wherein, If self.classifier encapsulates a linear layer and an activation function, traverse each layer and perform single event upset.

4. The neural architecture search based anti-single event upset network structure optimization method according to claim 1, wherein, The search strategy using gradient descent regards the architecture search parameters as parameters to be optimized, and after completing one epoch of network parameter training, the obtained network model parameters are fixed, and one update of the architecture search parameters is performed according to the evaluation result.

5. A neural architecture search based anti-single event upset network structure optimization system, comprising: The method of any one of claims 1-4, comprising: An analog function acquisition module, configured to acquire an analog function according to a single event upset interference model; A search space construction module, configured to construct an anti-single event model architecture search space according to the analog function; A super network construction module, configured to construct a super network according to the anti-single event model architecture search space; A search strategy determination module, configured to determine a search strategy according to the constructed super network, and realize anti-single event upset network structure optimization. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the anti-single event upset network structure optimization method based on neural architecture search according to any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor to implement the steps of the anti-single event upset network structure optimization method based on neural architecture search according to any one of claims 1-4.

Images