Neural network model channel alignment implementation method
Patent Information
- Application Number
- CN202310926629.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-26
- Publication Date
- 2025-01-28
AI Technical Summary
The prior art has problems such as low efficiency, poor scalability, loose code, low reusability and low portability in the channel alignment process of neural network models, resulting in low model deployment efficiency.
By judging the type of operator, different alignment methods are used for channel alignment. For operators containing weights, the number of channels modification and the reciprocating alignment of weight data; for operators that do not contain weight, just modify the number of channels for alignment. Specific steps include judging operator type, conducting channel modification, and weight data alignment operations.
The reasoning speed of the neural network model on the end -side deployment has been improved, the use of memory is optimized, the duplicability of the code and the expansion of alignment, and reducing the complexity and maintenance difficulty of the model deployment.
Smart Images
Figure CN119378609A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of neural network, and in particular relates to a method for realizing channel alignment of a neural network model. Background Art
[0002] With the rapid development of science and technology and the growth of demand, artificial intelligence is subtly affecting all walks of life, and AI intelligent algorithms have brought huge impacts on people's work and life. The primary task of implementing AI technology is the research and development of AI chips. How to make the algorithm perfectly adapt to AI chips so that it can run on the chip with high precision and efficiency is the core key technology currently being developed by various chip companies.
[0003] The existing technology aims to traverse all nodes in the network graph, and perform the weight and quantization parameter alignment operation on a single node when the number of channels of a single node cannot be divided by 32. For complex structures in the graph, it is usually necessary to add a corresponding function to perform a specific alignment task, which results in a single implementation of the entire alignment module, poor scalability and low efficiency. Therefore, the existing technology has loose code, low reusability, low portability and difficult maintenance.
[0004] In addition, common terms in the prior art include:
[0005] Model deployment: Deploy the trained machine learning model to production environment.
[0006] Neural network model graph: contains the model's calculation logic and nodes with parameters, and is a directed graph structure;
[0007] Network node: It is the smallest component in the neural network model structure, also commonly referred to as each layer or each operator of the network, containing the operator's weight, bias and other related parameter information. Summary of the invention
[0008] In order to solve the above problems, the purpose of this application is: the purpose of this method is to increase the data transfer speed and better utilize the memory space, thereby speeding up the inference speed of the neural network model deployed on the terminal side.
[0009] Specifically, the present invention provides a method for realizing channel alignment of a neural network model, the method comprising the following steps:
[0010] S1. Determine the operator type to determine the alignment method, that is, divide the operator according to whether the operator node contains weight parameters. In the neural network operator, ordinary convolution, depth convolution, group convolution, etc. are operators with weights; pooling operators, splicing operators, etc. are operators without weights. The method adopted for the channel alignment task is to operate the operator according to the type of operator. If the operator contains weights, proceed to step S2; if the operator does not contain weights, proceed to step S3;
[0011] S2. For operators with weights, the alignment operation needs to consider the modification of the number of channels and the rearrangement and padding alignment of the weight data, that is, the alignment of the input and output channels, the rearrangement and padding alignment of the weight data:
[0012] (1) The formula for modifying the number of channels is:
[0013] The following formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32. For example, if the original value is 30, the calculated value is 32; if the original value is 58, the calculated value is 64, etc. Detailed explanations of each parameter are given below:
[0014] Chn_new=(Chn+0x1F)&~0x1F)
[0015] Where Chn represents the original channel value, Chn_new represents the channel value after alignment according to multiples of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn.
[0016] (2) Rearrangement and padding of weight data:
[0017] Usually, the weight data of the first layer of convolution in the network is rearranged and padded. When the convolution kernel size of the first layer of convolution in the network is 3*3, the original weight arrangement is shown in the following table:
[0018]
[0019] When the stride of the convolution is 1, the results of the rearrangement and padding alignment of the weight data are shown in the following table:
[0020] A0b A0 A0r 0 B0b B0g B0r 0 C0b C0g C0r 0 0 0 0 0 ......(16 zeros are omitted here) A1b A1g A1r 0 B1b B1g B1r 0 C1b C1g C1r 0 0 0 0 0 ......(16 zeros are omitted here) A1b A2 A2r 0 B2b B2g B2r 0 C2b C2g C2r 0 0 0 0 0 ......(16 zeros are omitted here)
[0021] When the stride of the convolution is 2, the results of the weight data rearrangement and padding alignment are shown in the following table:
[0022] A0b A0 A0r B0b A1b A1g A1r B0g A2b A2 A2r B0r B1b B1g B1r 0 C0b C0g C0r 0 C1b C1g C1r 0 C2b C2g C2r 0 B2b B2g B2r 0 ;
[0023] S3. For operators without weights, the alignment operation only needs to modify the number of channels, that is, the alignment of input and output channels:
[0024] The formula for modifying the number of channels is: The following formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32. For example, the original value is 30, and the calculated value is 32; the original value is 58, and the calculated value is 64, etc. Detailed explanations of each parameter are provided below:
[0025] Chn_new=(Chn+0x1F)&~0x1F)
[0026] Among them, Chn represents the original channel value, Chn_new represents the channel value after alignment according to the multiple of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn.
[0027] In step S1, the weighted operators include ordinary convolution and grouped convolution; the weighted operators include pooling layer and splicing layer.
[0028] In step S2 and step S3, for the alignment of node channels including input channels and output channels, it is necessary to confirm the channel alignment scheme according to the complexity of the connection relationship between nodes, and align the channels according to the connection relationship between nodes:
[0029] Since a network graph contains many attached nodes and their inter-node connections, the complex inter-node connections will also affect the alignment of operators. Therefore, the channel alignment scheme is determined based on the complexity of the inter-node connections: Assume that the node Layer0 has three convolution outputs, the output channels of Conv0 are 16, the output channels of Conv1 are 16, and the output channels of Conv2 are 32.
[0030] (1) For nodes Conv0 and Conv1, their original output channels are both 16, which are not aligned. After channel alignment based on multiples of 32, the channels are both 32, which are aligned.
[0031] (2) For node Concat0, its original output channel is the concatenation of the output channels of node Conv0 and node Conv1, that is, 32, which has been aligned. However, since the output channels of node Conv0 and node Conv1 are both aligned to 32, the output channel of node Concat0 must be aligned to 64 and realigned;
[0032] (3) For node Conv2, its original output channel is 32, which has been aligned. However, since there is an addition operation between node Conv2 and node Concat0, the output channel needs to be consistent. Therefore, since the output channel of node Concat0 is aligned to 64, the output channel of node Conv2 must be aligned to 64 and realigned.
[0033] (4) For node Add0, its original output channel is 32, which is aligned. Due to the realignment of nodes Conv2 and Concat0, the output channel of node Add0 must be aligned to 64 and realigned.
[0034] In the method, assuming that the weight data size corresponding to a convolution kernel is 6x6x3x24, that is, the product of height, width, number of input channels and number of output channels, it is necessary to align to 32 numbers by adding 8 zeros to the output channel. The weight size of the first Conv2DInt8 operator is 6x6x3x24, and its child node, that is, the weight size of the second Conv2DInt8 operator is 3x3x24x40; first, the weight of the first Conv2DInt8 operator is aligned to 6x6x3x32 by adding 8 zeros to the output channel. Since the second Conv2DInt8 operator is connected to the first Conv2DInt8 operator, the number of input channels of the second Conv2DInt8 operator should be consistent with the number of output channels of the first Conv2DInt8 operator. Therefore, the weight of the second Conv2DInt8 operator needs to be aligned to 3x3x32x64 by adding 8 zeros to the input channel and 24 zeros to the output channel.
[0035] In the method, according to the underlying hardware computing logic, if the operator data supported by the underlying layer is arranged in NDHWC32 format, the underlying data is transmitted and calculated in groups of 32 data, which helps to maximize the efficiency of the chip; when converting the Magik model, the channels of the node are aligned by a multiple of 32 to match the computing logic of the underlying data (data reading is more convenient), and the weight data is rearranged and aligned to maximize the use of the underlying 32 groups of computing logic (for example, a network with an image as input is usually calculated by passing four-channel data into the network (at this time, when the underlying calculation is performed, each group of 32 calculations is performed, and only 4 calculations are effective. If the weight data and eigenvalues are rearranged and aligned, the number of effective calculations in each group of 32 calculations can be increased, which can effectively improve the efficiency of the calculation).
[0036] The channel alignment operations of most operators are similar, and the alignment operations can be implemented through corresponding functions. For example, affine_node_channels can be used to align the vector class parameters of nodes, including input_channels (input channels), output_channels (output channels), and output_shape (output shape); affine_node_attr can be used to align the tensor class parameters of nodes, including the "weight_max" tensor in the const node containing the weight value, the "value" tensor in the const node containing the bias value, and other tensors used for quantization operations.
[0037] The method is applicable to the neural network model deployment stage.
[0038] Therefore, the advantages of this application are: this method uses a dedicated alignment function to perform alignment tasks according to the operator node type, and can reasonably perform channel alignment tasks according to the complex connection relationship between nodes, which not only improves the reusability of the code but also facilitates the alignment of new complex network structures. The weight alignment method efficiently utilizes memory (using the technology of allocating once and reusing multiple times), and at the same time, an extensible programming method (i.e., the in-class function registration method) is used to process special network structure models, and the alignment of other parameters other than weights is realized by calling the public interface in the class. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.
[0040] Figure 1(a) is a schematic diagram of the ONNX model visualization data alignment before alignment.
[0041] Figure 1(b) is a schematic diagram of the ONNX model visualization data alignment after alignment.
[0042] Figure 2 is a schematic diagram of the process of this method.
[0043] Figure 3 It is a schematic diagram of the alignment scheme for confirming channels based on the complexity of the connection relationship between nodes in this method. DETAILED DESCRIPTION
[0044] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.
[0045] Usually, the processing method of data related to the neural network model needs to be adapted according to the operating logic of the chip. In most cases, in order to better transmit and calculate data on the chip, it is necessary to perform more reasonable data rearrangement or data alignment operations on the weight data in the operator.
[0046] This method belongs to the optimization method of data arrangement alignment in the deployment stage of neural network model. It realizes a simple and easy to expand optimization method of data arrangement alignment of neural network model, and can effectively handle models with complex network structures. The method mainly consists of the following two parts: Figure 2 As shown:
[0047] S1. Determine the operator type to determine the alignment method, that is, divide the operator according to whether the operator node contains weight parameters. In the neural network operator, ordinary convolution, depth convolution, group convolution, etc. are operators with weights; pooling operators, splicing operators, etc. are operators without weights. The method adopted for the channel alignment task is to operate the operator according to the type of operator. If the operator contains weights, proceed to step S2; if the operator does not contain weights, proceed to step S3;
[0048] S2. For operators with weights, the alignment operation needs to consider the modification of the number of channels and the rearrangement and padding alignment of the weight data, that is, the alignment of the input and output channels, the rearrangement and padding alignment of the weight data:
[0049] (1) Modification of the number of channels: The formula is:
[0050] Chn_new=(Chn+0x1F)&~0x1F)
[0051] The above formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32, where Chn represents the original channel value, Chn_new represents the channel value after alignment according to a multiple of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn.
[0052] (2) Rearrangement and padding of weight data:
[0053] Usually, the weight data of the first layer of convolution in the network is rearranged and padded. When the convolution kernel size of the first layer of convolution in the network is 3*3, the original weight arrangement is shown in the following table:
[0054]
[0055] When the stride of the convolution is 1, the results of the rearrangement and padding alignment of the weight data are shown in the following table:
[0056] A0b A0 A0r 0 B0b B0g B0r 0 C0b C0g C0r 0 0 0 0 0 ......(16 zeros are omitted here) A1b A1g A1r 0 B1b B1g B1r 0 C1b C1g C1r 0 0 0 0 0 ......(16 zeros are omitted here) A1b A2 A2r 0 B2b B2g B2r 0 C2b C2g C2r 0 0 0 0 0 ......(16 zeros are omitted here)
[0057] When the stride of the convolution is 2, the results of the weight data rearrangement and padding alignment are shown in the following table:
[0058] A0b A0 A0r B0b A1b A1g A1r B0g A2b A2 A2r B0r B1b B1g B1r 0 C0b C0g C0r 0 C1b C1g C1r 0 C2b C2g C2r 0 B2b B2g B2r 0 ;
[0059] S3. For operators without weights, the alignment operation only needs to modify the number of channels, that is, the alignment of input and output channels:
[0060] The formula for modifying the number of channels is: The following formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32. For example, the original value is 30, and the calculated value is 32; the original value is 58, and the calculated value is 64, etc.:
[0061] Chn_new=(Chn+0x1F)&~0x1F)
[0062] Among them, Chn represents the original channel value, Chn_new represents the channel value after alignment according to the multiple of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn.
[0063] In step S1, the weighted operators include ordinary convolution and grouped convolution; the weighted operators include pooling layer and splicing layer.
[0064] In step S2 and step S3, for the alignment of node channels including input channels and output channels, it is necessary to confirm the channel alignment scheme according to the complexity of the connection relationship between nodes, and align the channels according to the connection relationship between nodes:
[0065] Since a network graph contains many subsidiary nodes and their inter-node connections, the complex inter-node connections will also affect the alignment of operators; therefore, the alignment scheme of the channel is determined by the complexity of the inter-node connections: Figure 3 As shown, assume that node Layer0 has three convolution outputs, assume that the output channels of Conv0 are 16, the output channels of Conv1 are 16, and the output channels of Conv2 are 32;
[0066] (1) For nodes Conv0 and Conv1, their original output channels are both 16, which are not aligned. After channel alignment based on multiples of 32, the channels are both 32, which are aligned.
[0067] (2) For node Concat0, its original output channel is the concatenation of the output channels of node Conv0 and node Conv1, that is, 32, which has been aligned. However, since the output channels of node Conv0 and node Conv1 are both aligned to 32, the output channel of node Concat0 must be aligned to 64 and realigned;
[0068] (3) For node Conv2, its original output channel is 32, which has been aligned. However, since there is an addition operation between node Conv2 and node Concat0, the output channel needs to be consistent. Therefore, since the output channel of node Concat0 is aligned to 64, the output channel of node Conv2 must be aligned to 64 and realigned.
[0069] (4) For node Add0, its original output channel is 32, which is aligned. Due to the realignment of nodes Conv2 and Concat0, the output channel of node Add0 must be aligned to 64 and realigned.
[0070] As attached Figure 1(a)-1(b) As shown in the figure, the entire network takes the NOp node as input, and the operators passed through by the model inference are convolution operators (Conv2Dint8 nodes, where the figure contains additional information 1 for the Conv2DInt8 operator: the shape and size of the weight, and additional information 2: the shape and size of the bias. For example, the weight size of the first Conv2Dint8 node connected to the NOp node in Figure 1(a) is 6x6x3x24, where the first number '6' and the second number '6' represent the width and height of the convolution kernel, respectively, and the third number '3' represents the number of input channels of the node. 3, the fourth number '24' indicates that the number of output channels of this node is 24), the addition operator (EltwiseAddInt8 node) and the concatenation operator (ConcatInt8 node), that is, assuming that the weight data size corresponding to a convolution kernel is 6x6x3x24, that is, the product of height, width, number of input channels and number of output channels, the weight size of the first Conv2DInt8 operator is 6x6x3x24, and its child node, that is, the weight size of the second Conv2DInt8 operator is 3x3x24x40 (as shown in Figure 1(a)).
[0071] First, the weight of the first Conv2DInt8 operator is aligned to 6x6x3x32 by padding the output channel with 8 zeros. Since the second Conv2DInt8 operator is connected to the first Conv2DInt8 operator, the number of input channels of the second Conv2DInt8 operator should be consistent with the number of output channels of the first Conv2DInt8 operator. Therefore, the weight of the second Conv2DInt8 operator needs to be aligned to 3x3x32x64 by padding the input channel with 8 zeros and the output channel with 24 zeros (as shown in Figure 1(b)).
[0072] This method realizes the efficient use of memory by aligning weights, that is, the technology of allocating once and reusing multiple times is adopted: according to the underlying hardware computing logic, if the operator data supported by the underlying layer is arranged in NDHWC32 format, the underlying data is transmitted and calculated in groups of 32 data, which helps to maximize the efficiency of the chip; when converting the Magik model, the channels of the nodes are aligned by multiples of 32 to match the computing logic of the underlying data (data reading is more convenient), and the rearrangement and alignment of the weight data is also to maximize the use of the underlying 32 groups of computing logic. For example, a network with images as input is usually calculated by passing four-channel data into the network. At this time, when the underlying calculation is performed, only 4 effective calculations are performed in each group of 32 calculations. If the weight data and eigenvalues are rearranged and aligned, the number of effective calculations in each group of 32 calculations can be increased, which can effectively improve the efficiency of the calculation.
[0073] This method also uses an extensible programming method (i.e., in-class function registration method) to process special network structure models, and realizes the alignment of other parameters besides weights by calling the public interface in the class": The channel alignment operations of most operators are similar, and the alignment operations can be realized through the corresponding functions. For example, affine_node_channels can be used to realize the alignment of vector class parameters of nodes, including input_channels (input channels), output_channels (output channels), and output_shape (output shape); affine_node_attr can be used to realize the alignment of tensor class parameters of nodes, including the "weight_max" tensor in the const node containing weight values, the "value" tensor in the const node containing bias values, and other tensors used for quantization operations.
[0074] The method is applicable to the neural network model deployment stage.
[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for realizing channel alignment of a neural network model, characterized in that: The method comprises the following steps: S1. Determine the operator type to determine the alignment method, that is, divide the operator according to whether the operator node contains a weight parameter. The method adopted for the channel alignment task is to operate the operator according to the type of the operator. If the operator contains a weight, proceed to step S2; if the operator does not contain a weight, proceed to step S3; S2. For operators with weights, the alignment operation needs to consider the modification of the number of channels and the rearrangement and padding alignment of the weight data, that is, the alignment of the input and output channels, the rearrangement and padding alignment of the weight data: (1) The formula for modifying the number of channels is: The following formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32. Chn_new=(Chn+0x1F)&~0x1F) Where Chn represents the original channel value, Chn_new represents the channel value after alignment according to multiples of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn. (2) Rearrangement and padding of weight data: Usually, the weight data of the first layer of convolution in the network is rearranged and padded. When the convolution kernel size of the first layer of convolution in the network is 3*3, the original weight arrangement is shown in the following table: When the stride of the convolution is 1, the results of the rearrangement and padding alignment of the weight data are shown in the following table: When the stride of the convolution is 2, the results of the weight data rearrangement and padding alignment are shown in the following table: ； S3. For operators without weights, the alignment operation only needs to modify the number of channels, that is, the alignment of input and output channels: The formula for modifying the number of channels is: The following formula is the corresponding C++ implementation code, which means that a value is calculated to obtain a minimum number that is larger than the value and a multiple of 32. Chn_new=(Chn+0x1F)&~0x1F) Among them, Chn represents the original channel value, Chn_new represents the channel value after alignment according to the multiple of 32, the decimal number corresponding to 0x1F is 31, and the decimal number corresponding to ~0x1F is -32. The symbol ~ represents the binary bitwise NOT operation, and the symbol & represents the binary bitwise AND operation. The result of the formula calculation is the multiple of 32 that is larger than the value Chn and closest to the value Chn.
2. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: In step S1, the weighted operators include ordinary convolution and grouped convolution; the weighted operators include pooling layer and concatenation layer.
3. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: In step S2 and step S3, for the alignment of node channels including input channels and output channels, it is necessary to confirm the channel alignment scheme according to the complexity of the connection relationship between nodes, and align the channels according to the connection relationship between nodes: Since a network graph contains many attached nodes and their inter-node connections, the complex inter-node connections will also affect the alignment of operators. Therefore, the channel alignment scheme is determined based on the complexity of the inter-node connections: Assume that the node Layer0 has three convolution outputs, the output channels of Conv0 are 16, the output channels of Conv1 are 16, and the output channels of Conv2 are 32. (1) For nodes Conv0 and Conv1, their original output channels are both 16, which are not aligned. After channel alignment based on multiples of 32, the channels are both 32, which are aligned. (2) For node Concat0, its original output channel is the concatenation of the output channels of node Conv0 and node Conv1, that is, 32, which has been aligned. However, since the output channels of node Conv0 and node Conv1 are both aligned to 32, the output channel of node Concat0 must be aligned to 64 and realigned; (3) For node Conv2, its original output channel is 32, which has been aligned. However, since there is an addition operation between node Conv2 and node Concat0, the output channel needs to be consistent. Therefore, since the output channel of node Concat0 is aligned to 64, the output channel of node Conv2 must be aligned to 64 and realigned. (4) For node Add0, its original output channel is 32, which is aligned. Due to the realignment of nodes Conv2 and Concat0, the output channel of node Add0 must be aligned to 64 and realigned.
4. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: In the method, assuming that the weight data size corresponding to a convolution kernel is 6x6x3x24, that is, the product of height, width, number of input channels and number of output channels, it is necessary to align to 32 numbers by adding 8 zeros to the output channel. The weight size of the first Conv2DInt8 operator is 6x6x3x24, and its child node, that is, the weight size of the second Conv2DInt8 operator is 3x3x24x40; first, the weight of the first Conv2DInt8 operator is aligned to 6x6x3x32 by adding 8 zeros to the output channel. Since the second Conv2DInt8 operator is connected to the first Conv2DInt8 operator, the number of input channels of the second Conv2DInt8 operator should be consistent with the number of output channels of the first Conv2DInt8 operator. Therefore, the weight of the second Conv2DInt8 operator needs to be aligned to 3x3x32x64 by adding 8 zeros to the input channel and 24 zeros to the output channel.
5. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: In the method, according to the underlying hardware computing logic, the operator data supported by the underlying layer is arranged in NDHWC32 format, and the underlying data is usually transmitted and calculated in groups of 32 data, which helps to maximize the efficiency of the chip; When converting the Magik model, the node channels are aligned by multiples of 32 to match the calculation logic of the underlying data. The weight data is rearranged and aligned to maximize the use of the underlying 32 sets of calculation logic. Assume that the network with images as input is calculated by passing four-channel data into the network. At this time, when calculating at the bottom layer, only 4 effective calculations are performed in each set of 32 calculations. If the weight data and eigenvalues are rearranged and aligned, the number of effective calculations in each set of 32 calculations can be increased, so as to effectively improve the efficiency of the calculation.
6. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: In the method, the channel alignment operation of the operator can be implemented through the corresponding function to implement the alignment operation, including using affine_node_channels to implement the alignment of the vector class parameters of the node, including the input channel input_channels, the output channel output_channels, and the output shape output_shape; using affine_node_attr to implement the alignment of the tensor class parameters of the node, including the weight_max tensor in the const node containing the weight value, the value tensor in the const node containing the bias value, and other tensors used for quantization operations.
7. The method for realizing channel alignment of a neural network model according to claim 1, characterized in that: The method is applicable to the neural network model deployment stage.