Accelerator, device and application method for adversarial deep learning defense reasoning

Abstract

The application discloses an accelerator, device and application method for adversarial deep learning defense reasoning, the accelerator of the application, including scheduling controller, processing core array, core private cache, global cache and memory access component, the scheduling controller has access structure for host CPU access, the scheduling controller is connected with processing core array, core private cache and memory access component respectively, the core private cache is connected through global cache and memory access component, the core private cache is also directly connected with memory access component, the processing core array is connected by multiple processing core subarrays through processing core interconnection module, and the processing core subarray is composed of multiple calculation units.The application aims at realizing the optimization design of the accelerator for adversarial deep learning defense reasoning, supporting various different adversarial machine learning defense algorithms while supporting deep learning functions, and providing a high-performance computing platform for a trusted artificial intelligence system.

Description

Technical Field

[0001] This invention relates to processor design technology, and more specifically to an accelerator, device, and application method for adversarial deep learning defensive inference. Background Technology

[0002] With the development of adversarial machine learning, adversarial example attacks have become a major threat to deep learning systems and their applications. Therefore, adversarial machine learning defense algorithms (hereinafter referred to as defense algorithms) have been proposed to avoid or mitigate the damage caused by adversarial example attacks, providing important security guarantees for trustworthy artificial intelligence systems. Executing a deep learning inference process with the ability to defend against adversarial examples is called defensive inference. Building upon conventional deep learning inference computation, the execution of defensive inference, in specific domains including but not limited to bits, real numbers, complex numbers, and spectral analysis, places additional functional and performance demands on the computing platform, requiring broad support for various different defense algorithms. Currently, mainstream accelerators cannot simultaneously meet the aforementioned functional and performance requirements for adversarial machine learning defense while executing conventional deep learning. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide an accelerator, device and application method for adversarial deep learning defense inference, which addresses the above-mentioned problems of the prior art. This invention aims to optimize the design of the accelerator for adversarial deep learning defense inference, and can widely support a variety of different adversarial machine learning defense algorithms while supporting deep learning functions, thus providing a high-performance computing platform for artificial intelligence systems.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] An accelerator for adversarial deep learning defense inference includes a scheduler controller, a processing core array, a core private cache, a global cache, and a memory access unit. The scheduler controller has an access structure for host CPU access. The scheduler controller is connected to the processing core array, the core private cache, and the memory access unit. The core private cache is connected to the global cache and the memory access unit, and is also directly connected to the memory access unit. The processing core array is composed of multiple identical or different processing core subarrays connected together through inter-processing core connection modules. Each processing core subarray consists of multiple computing units interconnected through internal processing core connection modules. The scheduler controller configures whether each processing core subarray is enabled to adjust the computing power and configures the pipeline execution order of each computing unit in the enabled processing core subarray to adapt to the execution order required by the service and defense tasks in the adversarial deep learning defense inference algorithm.

[0006] Optionally, the multiple computing units in the processing core subarray include some or all of the following: a real / complex arithmetic unit, a nonlinear unit, a bitwise operator, and a pseudo-random noise sampler; the real / complex arithmetic unit is used for real or complex number operations and internally contains multiple sets of basic arithmetic units of different types, including multipliers, adders, comparators, shifters, and data type converters; the basic arithmetic units are used to directly calculate data of basic bit width or to calculate data of combined bit width by doubling or multiplying the data; when the real / complex arithmetic unit is used for real number operations, all basic arithmetic units perform real number calculations, and when the real / complex arithmetic unit is used for real number operations, all basic arithmetic units perform real number calculations. For complex number calculations, the basic arithmetic units are configured in pairs, with one arithmetic unit in each pair calculating the real part of the complex number and the other calculating the imaginary part. The nonlinear unit contains a nonlinear function table implemented based on a lookup table (LUT). When it is necessary to calculate the value of a function at a specified point, an approximate result is obtained by reading the closest data in the nonlinear function table and interpolating based on this data. The bit arithmetic unit includes logical operation modules for AND, OR, NOT, XOR, NAND, and NOR. The pseudo-random noise sampler is a linear feedback shift register used for random sampling of noise data stored in the linear feedback shift register based on random offsets.

[0007] Furthermore, the present invention also provides a computer device including a microprocessor and an accelerator interconnected, wherein the accelerator is the aforementioned accelerator for adversarial deep learning defensive inference.

[0008] Furthermore, the present invention also provides a method for applying the aforementioned accelerator for adversarial deep learning defense inference, comprising the following steps:

[0009] S1, configure the pipeline execution order of each computing unit in the enabled processing core subarray according to the execution order required by the service tasks and defense tasks in the adversarial deep learning defense inference algorithm; determine the computational load of the service tasks in the adversarial deep learning defense inference algorithm respectively. The computational load of defense tasks And the computational speed of a single processing core subarray performing service tasks. And the calculation speed of defense tasks ;

[0010] S2, based on Constraints are used to adjust the proportion of processing core subarrays allocated to service tasks within the processing core array. The ratio of defense missions Find the relative relationships that satisfy the given conditions The optimal proportion allocated to service tasks. The ratio of defense missions , where relative relationship This indicates a relationship that is greater than, less than, or has no constraint.

[0011] S3, adjust the proportion of the processing core subarray in the processing core array of the accelerator for adversarial deep learning defense inference allocated to the defense tasks, so that the proportion allocated to the service tasks is... The proportion allocated to defense tasks is ;

[0012] S4, compile the adversarial deep learning defense inference algorithm into a binary executable file, and import the binary executable file into the accelerator used for adversarial deep learning defense inference for execution.

[0013] Optionally, step S2 includes:

[0014] S2.1, initialize the total ratio n of service tasks and defense tasks, and satisfy... and the proportion allocated to service tasks Let n be the total proportion;

[0015] S2.2, according to The proportion of those who received defense missions ;

[0016] S2.3, Calculate the execution time of the service task according to the following formula. and the execution time of defense missions :

[0017] ,

[0018] ,

[0019] S2.4, Determine the given relative relationship Whether it holds true, and the relative relationships involved. The constraint is greater than, less than, or has no constraint. If the constraint is true, record the current proportion allocated to the service task. The ratio of defense missions The proportion allocated to service tasks The new proportion allocated to service tasks is obtained by subtracting the preset change amount. Proceed to step S2.2; otherwise, use the proportion allocated to the service task obtained from the previous record. The ratio of defense missions The optimal proportion allocated to service tasks was found. The ratio of defense missions .

[0020] Optionally, step S4, compiling the adversarial deep learning defense inference algorithm into a binary executable file, includes:

[0021] S4.1, Obtain information about the service task and defense task in the input adversarial deep learning defense inference algorithm, wherein the information about the service task and defense task includes the algorithm model and parameters;

[0022] S4.2 Generate computation graphs for service tasks and defense tasks respectively. The computation graph is expressed in the form of a directed acyclic graph (DAG), with nodes representing variables and directed edges representing computations. The destination node of a directed edge is called a child node, and the source node is called a parent node. The computation graph defines how to use the parent node to compute the child node, and describes the specific operations and intermediate variables used in the process of executing the complete defense reasoning to calculate the output step by step from the input.

[0023] S4.3, convert the directed edges representing computation in the computation graph into a sequence of operators composed of combinations of operators, wherein the operators refer to a sequence of instructions for a series of specific operations performed by the accelerator on the data;

[0024] S4.4, convert the operator sequence into a computation instruction sequence, where each computation instruction in the computation instruction sequence represents a basic action of the accelerator;

[0025] S4.5 outputs the computation instruction sequence as a binary executable file.

[0026] Optionally, when generating computation graphs for the service task and the defense task in step S4.2, the computation graph is expressed in the form of a directed acyclic graph (DAG). The DAG uses nodes to represent variables and directed edges to represent computations. The destination node of a directed edge is called a child node and the source node is called a parent node. The directed edges define how the parent node is used to compute the child node. Thus, the DAG describes the specific operations and intermediate variables used in the process of executing a complete defense reasoning, from the input to the output step by step.

[0027] Optionally, when converting the computation graph into an operator sequence composed of operator combinations in step S4.3, this includes referencing operators from the defensive inference enhancement instruction set to convert the computation graph into an operator sequence composed of operator combinations. The operators in the defensive inference enhancement instruction set include some or all of the following: basic arithmetic combinations, convolution, matrix multiplication, filtering and spectral analysis transformation, activation functions, bit logic computation, set operations, data noise addition, and mask-based selective computation. The basic arithmetic combinations are implemented based on a real / complex arithmetic operator, and are used to form data processing including calculating specific statistical indicators, linear / nonlinear transformations, specified threshold filtering, and mask generation from basic operations such as real / complex addition and subtraction, multiplication, fast reciprocal square root, size comparison, precision conversion, and sorting implemented based on a real / complex arithmetic operator. The convolution and matrix multiplication are multiply-add tree structures or systolic array structures implemented based on a real / complex arithmetic operator. The system is structured as follows: the filtering is implemented using a given filter based on matrix multiplication and convolution; the spectral analysis transformation is implemented by generating a specific filter for frequency domain transformation using a nonlinear unit, and then completing the spectral analysis transformation using the filtering function, including discrete Fourier transform, discrete cosine transform, and discrete wavelet transform; the activation function is implemented by a nonlinear unit; the bit logic calculation is implemented by a bit arithmetic unit; the set operation is implemented by mapping a set within a finite range to a bit sequence, and performing bit-by-bit logical operations through the bit arithmetic unit to obtain the result of the set operation; the data noise addition is implemented by generating noise using a pseudo-random noise sampler and adding it to the data using basic arithmetic operators in the real / complex arithmetic unit; the mask-based selective calculation is implemented by processing data points at specified mask values ​​in a specified data block according to the mask, otherwise not processing the data points to achieve selective calculation.

[0028] Optionally, step S4.3 includes: first, finding multiple directed edges in the computation graph that can be locally merged into one operator through pattern matching, and then locally merging these multiple directed edges into one operator; then, performing constant folding, changing variables in the computation graph whose processes can be predetermined and whose values ​​are fixed into constants; and finally, performing scheduling optimization, using the execution order and resource allocation of each node as the search space, starting from the original input state, finding a better solution through heuristic search, and finally determining the execution order and corresponding resource allocation of each node.

[0029] Optionally, in step S1, when configuring the pipeline execution order of each computational unit in the enabled processing core subarray according to the execution order required by the service task and the defense task in the adversarial deep learning defense inference algorithm, the pipeline execution order of each computational unit in the enabled processing core subarray includes: multiplying the multiple pairs of input feature map-weight data in a multiplier, then feeding them into an accumulator to accumulate them to obtain a feature map data, then feeding the feature map data into a nonlinear unit to obtain the activated value, and then feeding the activated value into a comparator to compare it with the threshold and output the comparison result.

[0030] Compared with existing technologies, the present invention has the following main advantages: The accelerator for adversarial deep learning defense inference of the present invention includes a scheduling controller, a processing core array, a core private cache, a global cache, and a memory access unit. The present invention realizes an optimized design for an accelerator for adversarial deep learning defense inference. While supporting deep learning functions, it can widely support a variety of different adversarial machine learning defense algorithms. It can be used in the design of artificial intelligence compilers and corresponding hardware accelerator architectures that support a variety of different defense inference algorithms. It can provide a high-performance computing platform for artificial intelligence systems, especially adversarial deep learning defense inference, and has the advantages of simple structure and high efficiency. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the accelerator structure in an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the processing core array in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the compilation process of the adversarial deep learning defense inference algorithm in an embodiment of the present invention.

[0034] Legend: 1. Scheduling controller; 2. Processing core array; 21. Processing core subarray; 211. Processing core internal connection module; 212. Computing unit; 22. Processing core inter-connection module; 3. Core private cache; 4. Global cache; 5. Memory access unit. Detailed Implementation

[0035] like Figure 1As shown, this embodiment provides an accelerator for adversarial deep learning defense inference, including a scheduling controller 1, a processing core array 2, a core private cache 3, a global cache 4, and a memory access unit (DMA) 5. The scheduling controller 1 has an access structure for host CPU access. The scheduling controller 1 is connected to the processing core array 2, the core private cache 3, and the memory access unit 5. The core private cache 3 is connected to the memory access unit 5 through the global cache 4, and is also directly connected to the memory access unit 5. The scheduling controller 1 controls the entire operation of the accelerator. The core private cache 3 and the global cache 4 constitute a two-level cache hierarchy for the processing core array 2. The memory access unit (DMA) 5 is responsible for external data transfer. Data is read from external memory by the memory access unit (DMA) 5, sent to the global cache 4 according to the schedule, and the core private cache 3 retrieves data from the global cache 4 according to calculation instructions. Multiple cores in the processing core array 2 read data from their respective core private caches 3 and perform calculations. The calculation results are temporarily stored in the core private cache 3 and then sent outside the accelerator by the memory access unit (DMA) 5.

[0036] like Figure 2 As shown, the processing core array 2 in this embodiment is composed of multiple identical or different processing core sub-arrays 21 connected together by a processing core interconnection module 22. Each processing core sub-array 21 consists of multiple computing units 212 interconnected by internal processing core interconnection modules 211. The scheduling controller 1 configures whether each processing core sub-array 21 is enabled to adjust the computing power, and configures the pipeline execution order of each computing unit 212 in the enabled processing core sub-array 21 to adapt to the execution order required by the service and defense tasks in the adversarial deep learning defense inference algorithm. Figure 2 As shown, the portion within the dashed box represents a single processing core subarray 21, which contains several different functional computing units 212 (the specific number can be configured according to application requirements; four are shown as an example in the figure). These units provide different types of computational operation instructions. Data for a single operation can be processed sequentially by different computing units 212 within the processing core subarray 21. The computational order can be determined by configuring the internal connection devices of the processing core subarray 21. Taking a threshold comparison operation after convolution as an example: after multiple pairs of feature map-weight data enter the processing core, they are first multiplied in a multiplier, then accumulated in an accumulator to obtain a single feature map data. This feature map data is then fed into a nonlinear unit to obtain an activated value, which is then compared with a threshold value in a comparator.

[0037] In this embodiment, the multiple computing units 212 in the processing core subarray 21 include some or all of the following: a real / complex number arithmetic unit, a nonlinear unit, a bit arithmetic unit, and a pseudo-random noise sampler, wherein:

[0038] U1: The real / complex arithmetic unit is used for real or complex number operations. It contains multiple sets of basic arithmetic units of different types. These units include multipliers, adders, comparators, shifters, and data type converters. The basic arithmetic units are used to directly calculate data of a basic bit width or to calculate data of a combined bit width by doubling or multiplying the data. When the real / complex arithmetic unit is used for real number operations, all basic arithmetic units perform real number calculations. When the real / complex arithmetic unit is used for complex number calculations, the basic arithmetic units are configured in pairs, with one unit in each pair calculating the real part of the complex number and the other unit calculating the imaginary part. Specifically, in this embodiment, the real / complex arithmetic unit contains multiple sets of multipliers, adders, comparators, shifters, data type converters, and other arithmetic units. The data bit width is 8-bit, with a single 8-bit arithmetic unit as the basic arithmetic unit. Multiple basic arithmetic units can be combined into a higher-precision arithmetic unit (16-bit, 32-bit arithmetic units, etc.). When performing single-precision (8-bit precision) calculations, the basic arithmetic units are directly used to calculate the data. When performing double-precision or higher precision calculations (16-bit, 32-bit, etc.), multiple basic arithmetic units are combined into an arithmetic unit of the corresponding precision. When performing real number calculations, all arithmetic units perform real number calculations. When performing complex number calculations, the arithmetic units are configured in pairs, with one arithmetic unit in each pair calculating the real part of the complex number and the other calculating the imaginary part.

[0039] U2: The nonlinear unit contains a nonlinear function table implemented based on a lookup table (LUT). When it is necessary to calculate the value of a function at a specified point, an approximate result is obtained by reading the closest data in the nonlinear function table and interpolating based on this data. The principle of the nonlinear unit is to record several sampling points (input value-function value data pairs) of a specified function within a specified domain and range. When it is necessary to calculate the value of a function at a specified point, an approximate result is obtained by reading the closest data in the function storage table and interpolating based on this data.

[0040] U3: The bit arithmetic unit includes logic operation modules for AND, OR, NOT, XOR, NAND, and NOR. The bit arithmetic unit performs bit-by-bit digital logic calculations, including but not limited to bit-by-bit AND, OR, NOT, XOR, NAND, and NOR calculations. When used for set operations, it maps a set within a finite range to a bit sequence, and obtains the result of the set operation through bit-by-bit logical operations (e.g., logical OR to obtain the union, logical AND to obtain the intersection).

[0041] U4: The pseudo-random noise sampler is a linear feedback shift register used for random sampling of noise data stored in the linear feedback shift register based on random offsets.

[0042] Based on the aforementioned functional components, operator functions are implemented, including but not limited to the following operators, and more complex operators F1 to F9 formed by combining the following operators. F1: Basic arithmetic and data processing combined with basic arithmetic, implementing basic operations including but not limited to real / complex number addition and subtraction, multiplication, fast reciprocal square root, size comparison, precision conversion, and sorting based on real / complex number arithmetic units. Data processing combined with basic arithmetic includes calculating specific statistical indicators, linear / nonlinear transformations, specified threshold filtering, and mask generation. F2: Convolution (convolutional neural network layer) and matrix multiplication (fully connected neural network layer), implementing convolution and matrix multiplication based on a multiply-add tree structure and a systolic array structure formed by connecting real / complex number arithmetic units across cores. F3: Filtering, implementing data filtering using a given filter based on matrix multiplication and convolution functions. F4: Spectral analysis transformation, including discrete Fourier transform, discrete cosine transform, and discrete wavelet transform. Specific filters for frequency domain transformation are generated by nonlinear units, and then the spectral analysis transformation is completed by the filtering function. F5: Activation function, completed by nonlinear units. F6: Bit logic computation, performing bit-by-bit logical operations on bit data using basic logic operators. F7: Set operations, mapping a finite set to a bit sequence, obtaining the result of the set operation through bit-by-bit logical operations. F8: Adding noise to data, generating noise using a pseudo-random noise sampler and adding it to the data using basic arithmetic operators. F9: Selective computation based on a mask, processing data points in a specified data block where the mask value is specified, otherwise leaving the data points unprocessed, thus achieving selective computation.

[0043] In addition, this embodiment also provides a computer device including a microprocessor and an accelerator interconnected, wherein the accelerator is the aforementioned accelerator for adversarial deep learning defense inference.

[0044] like Figure 2As shown, multiple (the specific number depends on the requirements; here, six are used as an example) identical or different processing core subarrays 21 are connected together through the processing core connection module 22 to form the processing core array 2. By configuring the processing core connection module 22, computing power can be allocated and adjusted, and the processing core array 2 can be logically divided into multiple smaller processing arrays. Taking a common form of defensive inference as an example: while executing a standard neural network (DNN) classifier that provides services, an adversarial example detection algorithm is executed simultaneously. The detection algorithm determines whether the input is an adversarial example, i.e., the service task and the defense task. The computational cost of the standard neural network model is 100 TOPS, and the computational cost of the detection algorithm is 20 TOPS. In order to determine whether the input is malicious before possible erroneous results occur, the time constraint requires that the completion time of the detection algorithm is earlier than the completion time of the standard neural network model. Therefore, the processing core array is allocated into two logical subarrays, with 60% of the processing core resources allocated to the service task and 40% allocated to the defense task, executing the standard neural network model (providing services) and the adversarial example detection algorithm (providing defense), respectively. The specific allocation method in actual use is given by the optimization steps of the defense inference compiler during the compilation process. Specifically, this embodiment also provides an application method for the aforementioned accelerator for adversarial deep learning defense inference, including the following steps:

[0045] S1, configure the pipeline execution order of each computing unit 212 in the enabled processing core subarray 21 according to the execution order required by the service tasks and defense tasks in the adversarial deep learning defense inference algorithm; determine the computational load of the service tasks in the adversarial deep learning defense inference algorithm respectively. The computational load of defense tasks And the computational speed of a single processing core subarray 21 performing service tasks. And the calculation speed of defense tasks ;

[0046] S2, based on The constraints are adjusted to modify the proportion of service tasks allocated to the processing core subarray 21 in the processing core array 2. The ratio of defense missions Find the relative relationships that satisfy the given conditions The optimal proportion allocated to service tasks. The ratio of defense missions , where relative relationship This can be interpreted as greater than, less than, or no constraint relationship, which can be specified according to actual needs, given the relative relationship. When there are no constraints, then As long as it meets the requirements Both are valid;

[0047] S3, adjust the proportion of processing core subarray 21 in the processing core array 2 of the accelerator used for adversarial deep learning defense inference allocated to defense tasks, so that the proportion allocated to service tasks is... The proportion allocated to defense tasks is ;

[0048] S4 compiles the adversarial deep learning defense inference algorithm into a binary executable file, and imports the binary executable file into the accelerator used for adversarial deep learning defense inference for execution.

[0049] In this embodiment, step S2 includes:

[0050] S2.1, initialize the total ratio n of service tasks and defense tasks, and satisfy... and the proportion allocated to service tasks Let n be the total proportion;

[0051] S2.2, according to The proportion of those who received defense missions ;

[0052] S2.3, Calculate the execution time of the service task according to the following formula. and the execution time of defense missions :

[0053] ,

[0054] ,

[0055] S2.4, Determine the given relative relationship Whether it holds true, and the relative relationships involved. This can be interpreted as greater than, less than, or no constraint relationship, which can be specified according to actual needs, given the relative relationship. When there are no constraints, then If true, then record the current proportion allocated to service tasks. The ratio of defense missions The proportion allocated to service tasks The new proportion allocated to service tasks is obtained by subtracting the preset change amount. Proceed to step S2.2; otherwise, use the proportion allocated to the service task obtained from the previous record. The ratio of defense missions The optimal proportion allocated to service tasks was found. The ratio of defense missions .

[0056] like Figure 3As shown, step S4 of this embodiment, compiling the adversarial deep learning defense inference algorithm into a binary executable file, includes:

[0057] S4.1 Obtain information about the service task and defense task in the input adversarial deep learning defense inference algorithm. The information of both the service task and the defense task includes the algorithm model and parameters. The model of the service task is the main DNN architecture, which is generally described by a high-level programming language (the neural network that provides services as the application itself, such as face recognition, LLM, etc.). The defense task is the algorithm for defense, such as adversarial example detection algorithm, neural network robustness enhancement algorithm, etc. The two together constitute the defense inference task.

[0058] S4.2 generates computation graphs for service tasks and defense tasks respectively. The computation graph is expressed in the form of a directed acyclic graph (DAG), with nodes representing variables and directed edges representing computations. The destination node of a directed edge is called a child node, and the source node is called a parent node. The computation graph defines how to use the parent node to compute the child node, and describes the specific operations and intermediate variables used in the process of executing the complete defense reasoning from the input to the output step by step.

[0059] S4.3 transforms the directed edges representing computations in the computation graph into operator sequences composed of combinations of operators. An operator (compute) is a sequence of instructions for a specific operation performed by the accelerator on data; an operator (compute) represents an operation performed on a block of data and is also the smallest unit for scheduling computation tasks. Each operator represents a specific series of operations performed by the hardware on the data and corresponds to a specific sequence of instructions to be executed. The entire operator sequence represents the complete computation process. In a one-to-one correspondence, each computation node is transformed into a corresponding operator sequence (e.g., a convolution node is transformed into a load-convolution-store-load-convolution-store operator sequence; multiple loads are because the available cache space for that node is limited).

[0060] S4.4 converts the operator sequence into a computation instruction sequence. Each computation instruction in the sequence represents a basic action of the accelerator. For example, the "load" instruction means that the accelerator hardware retrieves data from a specified memory and places it into a specified cache, and the "bias" instruction means that all data in a data block is increased or decreased by a specified value. The resulting instruction sequence represents all the operations of the complete defense inference computation. The accelerator hardware completes the entire task by executing the instructions sequentially.

[0061] S4.5 outputs the computation instruction sequence as a binary executable file.

[0062] In this embodiment, when generating computation graphs for the service task and the defense task in step S4.2, the computation graph is expressed in the form of a directed acyclic graph (DAG). The DAG uses nodes to represent variables and directed edges to represent computations. The destination node of a directed edge is called a child node, and the source node is called a parent node. The directed edges define how the parent node is used to compute the child node. Thus, the DAG describes the specific operations and intermediate variables used in the process of executing a complete defense reasoning, from the input to the output step by step.

[0063] In this embodiment, when converting the computation graph into an operator sequence composed of operator combinations in step S4.3, it includes referencing operators from the defensive inference enhancement instruction set to convert the computation graph into an operator sequence composed of operator combinations. The operators in the defensive inference enhancement instruction set include some or all of the following: basic arithmetic combinations, convolution, matrix multiplication, filtering and spectral analysis transformation, activation functions, bit logic calculations, set operations, data noise addition, and mask-based selective computation. The basic arithmetic combinations are implemented based on a real / complex arithmetic operator, and are used to form data processing including calculating specific statistical indicators, linear / nonlinear transformations, specified threshold filtering, and mask generation from basic operations such as real / complex addition and subtraction, multiplication, fast reciprocal square root, size comparison, precision conversion, and sorting implemented based on the real / complex arithmetic operator. Convolution and matrix multiplication are implemented based on a multiply-add tree structure or... The system implements a pulsating array structure; filtering is used to filter data using a given filter based on matrix multiplication and convolution functions; spectral analysis transformation is used to generate specific filters for frequency domain transformation from nonlinear units, and then the filtering function completes the spectral analysis transformation, including discrete Fourier transform, discrete cosine transform, and discrete wavelet transform; activation functions are completed by nonlinear units; bit logic computation is completed by a bit arithmetic unit; set operations are used to map sets within a finite range to bit sequences, and the bit arithmetic unit performs bit-by-bit logical operations to obtain the results of set operations; data noise addition is used to generate noise from a pseudo-random noise sampler and add it to the data using basic arithmetic operators in the real / complex arithmetic unit; mask-based selective computation is used to process data points in a specified data block based on a mask, except for data points that are not processed, thus achieving selective computation.

[0064] In this embodiment, step S4.3 includes: firstly, by pattern matching, finding multiple directed edges in the computation graph that can be locally merged into one operator, and then locally merging these multiple directed edges into one operator; then, performing constant folding, changing variables in the computation graph whose processes can be predetermined and whose values ​​are fixed into constants; finally, performing scheduling optimization, using the execution order and resource allocation of each node as the search space, starting from the original input state, finding a better solution through heuristic search, and finally determining the execution order and corresponding resource allocation of each node.

[0065] In this embodiment, when configuring the pipeline execution order of each computing unit 212 in the enabled processing core subarray 21 according to the execution order required by the service task and defense task in the adversarial deep learning defense inference algorithm in step S1, it includes performing a threshold comparison operation after the service task and defense task perform convolution. The pipeline execution order of each computing unit 212 in the enabled processing core subarray 21 includes: multiplying the multiple pairs of input feature map-weight data in a multiplier, then feeding them into an accumulator to accumulate and obtain a feature map data, then feeding the feature map data into a nonlinear unit to obtain the activated value, and then feeding the activated value into a comparator to compare it with the threshold and output the comparison result.

[0066] In summary, addressing the issue that current deep learning system computing platforms cannot effectively and broadly support multiple adversarial machine learning defense algorithms (and therefore cannot effectively support defense inference), this embodiment presents an accelerator for adversarial deep learning defense inference. This accelerator design aims to support general deep learning applications while simultaneously providing general support for multiple defense inference methods. It involves a multi-functional processing core, reconfigurable connection devices within and between processing cores, an overall design of the defense inference accelerator, a defense inference enhancement instruction set, and a defense inference compiler. The multi-functional processing core, composed of the processing core array 2, can perform functions commonly used in adversarial machine learning defense algorithms, including but not limited to specific computational operations in particular domains such as bits, real numbers, complex numbers, spectral analysis, and statistics. Its structure includes computational components such as multiply-accumulate units (MAC), accumulation units, nonlinear units, and noise sampling units. Its pipeline can be reconfigured according to configuration instructions to adapt to the execution order of specific algorithms. The reconfigurable connection devices between processing cores are used to transfer single computation results and operands between computing cores. The connection method can be dynamically adjusted online according to configuration instructions to adapt to the computational data flow of specific algorithms, including but not limited to systolic arrays and multiply-accumulate trees. The overall design methodology for the defensive inference accelerator includes dynamic configuration design methods for computational and storage resources and dynamic scheduling design methods for computational tasks. It can dynamically schedule resources and tasks according to specific computational tasks to achieve optimal performance. The defensive inference enhancement instruction set contains instructions required to execute various adversarial machine learning defense algorithms, applicable to a wide range of different methods and algorithms. Its operations include, but are not limited to, computational operations in specific domains such as bits, real numbers, complex numbers, spectral analysis, and statistics. The defensive inference compiler is used to compile a given defensive inference model with defense algorithms into an executable program that can be computed on the defensive inference accelerator platform. This includes, but is not limited to, methods for defensive inference such as computational primitive transformation, intermediate representation (IR) transformation, operator fusion, computational graph optimization, instruction selection, and code generation. This embodiment of the accelerator and its application method for adversarial deep learning defensive inference can support the execution of various adversarial machine learning defense algorithms while supporting general deep learning applications, achieving high-performance, reprogrammable deep learning defensive inference acceleration. This embodiment describes an accelerator and its application methods for adversarial deep learning defense inference, including but not limited to multifunctional reconfigurable processing core arrays in specific fields such as bits, real numbers, complex numbers, and spectral analysis. These arrays can reconfigure the connection relationships between processing cores online according to software programming, thereby supporting a wide range of different adversarial machine learning defense algorithms while supporting deep learning functions, providing a high-performance computing platform for reliable artificial intelligence systems.

[0067] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An accelerator for adversarial deep learning defense inference, characterized in that, The system includes a scheduling controller (1), a processing core array (2), a core private cache (3), a global cache (4), and a memory access unit (5). The scheduling controller (1) has an access structure for host CPU access. The scheduling controller (1) is connected to the processing core array (2), the core private cache (3), and the memory access unit (5). The core private cache (3) is connected to the memory access unit (5) through the global cache (4). The core private cache (3) is also directly connected to the memory access unit (5). The processing core array (2) consists of multiple identical or different processing cores. The core subarray (21) is connected together through the processing core interconnection module (22). The processing core subarray (21) consists of multiple computing units (212) that are interconnected through the processing core interconnection module (211). The scheduling controller (1) configures whether each processing core subarray (21) is enabled to adjust the computing power and configures the pipeline execution order of each computing unit (212) in the enabled processing core subarray (21) to adapt to the execution order required by the service task and defense task in the adversarial deep learning defense inference algorithm. The processing core subarray (21) includes multiple computing units (212) comprising some or all of a real / complex arithmetic unit, a nonlinear unit, a bit arithmetic unit, and a pseudo-random noise sampler; the real / complex arithmetic unit is used for real or complex number operations and contains multiple sets of basic arithmetic units of different types, including multipliers, adders, comparators, shifters, and data type converters; the basic arithmetic units are used to directly calculate data of basic bit width or to calculate data of combined bit width by doubling or multiplying the data; when the real / complex arithmetic unit is used for real number operations, all basic arithmetic units perform real number calculations, and when the real / complex arithmetic unit is used for real number operations, all basic arithmetic units perform real number calculations. When the arithmetic unit is used for complex number calculations, the basic arithmetic units are configured in pairs, with one arithmetic unit in each pair calculating the real part of the complex number and the other calculating the imaginary part. The nonlinear unit internally contains a nonlinear function table implemented based on a lookup table (LUT). When it is necessary to calculate the value of a function at a specified point, an approximate result is obtained by reading the closest data in the nonlinear function table and interpolating based on this data. The bit arithmetic unit includes logical operation modules for AND, OR, NOT, XOR, NAND, and NOR. The pseudo-random noise sampler is a linear feedback shift register used for random sampling of noise data stored in the linear feedback shift register based on random offsets.

2. A computer device comprising a microprocessor and an accelerator interconnected, characterized in that, The accelerator is the accelerator for adversarial deep learning defense inference as described in claim 1.

3. A method for applying the accelerator for adversarial deep learning defensive inference as described in claim 1, characterized in that, Includes the following steps: S1, configure the pipeline execution order of each computing unit (212) in the enabled processing core subarray (21) according to the execution order required by the service tasks and defense tasks in the adversarial deep learning defense inference algorithm; determine the computational amount of the service tasks in the adversarial deep learning defense inference algorithm respectively. The computational load of defense tasks And the computational speed of a single processing core subarray (21) performing service tasks. And the calculation speed of defense tasks ; S2, based on Constraints are applied to adjust the proportion of the processing core subarray (21) in the processing core array (2) allocated to service tasks. The ratio of defense missions Find the relative relationships that satisfy the given conditions The optimal proportion allocated to service tasks. The ratio of defense missions , where relative relationship This indicates a relationship that is greater than, less than, or has no constraint. S3, adjust the proportion of the processing core subarray (21) in the processing core array (2) of the accelerator for adversarial deep learning defense inference allocated to the defense tasks, such that the proportion allocated to the service tasks is... The proportion allocated to defense tasks is ; S4, compile the adversarial deep learning defense inference algorithm into a binary executable file, and import the binary executable file into the accelerator used for adversarial deep learning defense inference for execution.

4. The application method of the accelerator for adversarial deep learning defensive inference according to claim 3, characterized in that, Step S2 includes: S2.1, The total ratio of initialization service tasks to defense tasks. And satisfy and the proportion allocated to service tasks For the total proportion ; S2.2, according to The proportion of those who received defense missions ; S2.3, Calculate the execution time of the service task according to the following formula. and the execution time of defense missions : , , S2.4, Determine the given relative relationship Whether it holds true, and the relative relationships involved. The constraint is greater than, less than, or has no constraint. If the constraint is true, record the current proportion allocated to the service task. The ratio of defense missions The proportion allocated to service tasks The new proportion allocated to service tasks is obtained by subtracting the preset change amount. Proceed to step S2.2; otherwise, use the proportion allocated to the service task obtained from the previous record. The ratio of defense missions The optimal proportion allocated to service tasks was found. The ratio of defense missions .

5. The application method of the accelerator for adversarial deep learning defensive inference according to claim 4, characterized in that, Step S4, compiling the adversarial deep learning defense inference algorithm into a binary executable file, includes: S4.1, Obtain information about the service task and defense task in the input adversarial deep learning defense inference algorithm, wherein the information about the service task and defense task includes the algorithm model and parameters; S4.2 Generate computation graphs for service tasks and defense tasks respectively. The computation graph is expressed in the form of a directed acyclic graph (DAG), with nodes representing variables and directed edges representing computations. The destination node of a directed edge is called a child node, and the source node is called a parent node. The computation graph defines how to use the parent node to compute the child node, and describes the specific operations and intermediate variables used in the process of executing the complete defense reasoning to calculate the output step by step from the input. S4.3, convert the directed edges representing computation in the computation graph into a sequence of operators composed of combinations of operators, wherein the operators refer to a sequence of instructions for a series of specific operations performed by the accelerator on the data; S4.4, convert the operator sequence into a computation instruction sequence, where each computation instruction in the computation instruction sequence represents a basic action of the accelerator; S4.5 outputs the computation instruction sequence as a binary executable file.

6. The application method of the accelerator for adversarial deep learning defense inference according to claim 5, characterized in that, In step S4.2, when generating computation graphs for the service task and the defense task respectively, the computation graph is expressed in the form of a directed acyclic graph (DAG). The DAG uses nodes to represent variables and directed edges to represent computations. The destination node of a directed edge is called a child node and the source node is called a parent node. The directed edges define how the parent node is used to compute the child node. Thus, the DAG describes the specific operations and intermediate variables used in the process of executing a complete defense reasoning, from the input to the output step by step.

7. The application method of the accelerator for adversarial deep learning defense inference according to claim 6, characterized in that, In step S4.3, when converting the computation graph into a sequence of operators composed of operator combinations, it includes referencing operators from the defensive inference enhancement instruction set to convert the computation graph into a sequence of operators composed of operator combinations. The operators in the defensive inference enhancement instruction set include some or all of the following: basic arithmetic combinations, convolution, matrix multiplication, filtering and spectral analysis transformations, activation functions, bit logic calculations, set operations, data noise addition, and mask-based selective computation. The basic arithmetic combinations are implemented based on a real / complex arithmetic operator, and are used to form data processing including calculating specific statistical indicators, linear / nonlinear transformations, specified threshold filtering, and mask generation from basic operations such as real / complex addition and subtraction, multiplication, fast reciprocal square root, size comparison, precision conversion, and sorting implemented based on the real / complex arithmetic operator. Matrix multiplication is implemented using a multiply-add tree structure or a pulsating array structure based on a real / complex arithmetic operator; filtering is used to filter data using a given filter based on matrix multiplication and convolution functions; the spectral analysis transformation is used to generate specific filters for frequency domain transformation by a nonlinear unit, and then the filtering function completes the spectral analysis transformation, including discrete Fourier transform, discrete cosine transform, and discrete wavelet transform; the activation function is completed by a nonlinear unit; the bit logic calculation is completed by a bit arithmetic operator; the set operation is used to map a set within a finite range to a bit sequence, and the result of the set operation is obtained by performing bit-by-bit logical operations through the bit arithmetic operator; the data noise addition is used to generate noise by a pseudo-random noise sampler and add it to the data through basic arithmetic operators in the real / complex arithmetic operator; The mask-based selective computation is used to process data points in a specified data block where the mask value is specified, and otherwise no data points are processed, in order to achieve selective computation.

8. The application method of the accelerator for adversarial deep learning defense inference according to claim 6, characterized in that, Step S4.3 includes: first, using pattern matching, finding multiple directed edges in the computation graph that can be locally merged into one operator, and then locally merging these multiple directed edges into one operator; then, performing constant folding, changing variables in the computation graph whose processes can be predetermined and whose values ​​are fixed into constants; finally, performing scheduling optimization, using the execution order and resource allocation of each node as the search space, starting from the original input state, finding a better solution through heuristic search, and finally determining the execution order and corresponding resource allocation of each node.

9. The application method of the accelerator for adversarial deep learning defensive inference according to claim 3, characterized in that, In step S1, when configuring the pipeline execution order of each computing unit (212) in the enabled processing core subarray (21) according to the execution order required by the service task and defense task in the adversarial deep learning defense inference algorithm, the service task and defense task perform a threshold comparison operation after performing convolution. The pipeline execution order of each computing unit (212) in the enabled processing core subarray (21) includes: multiplying the multiple pairs of input feature map-weight data in the multiplier, then feeding them into the accumulator to accumulate and obtain a feature map data, then feeding the feature map data into the nonlinear unit to obtain the activated value, and then feeding the activated value into the comparator to compare with the threshold and output the comparison result.

Images