Universal neural network model based on hierarchical architecture

Abstract

The invention discloses a general neural network model based on a layered architecture, and relates to four planes of a general neural network, namely a region plane, an algorithm plane, a control plane and a neural plane. The design concept that control and calculation of a traditional neural network architecture are separated is proposed, nerves and a control plane are designed, the number of connecting lines between neurons is remarkably reduced, and compatibility of multiple connection modes is achieved. A traditional neural network function module is systematically decomposed, an algorithm plane is constructed according to a calculation process, a data flow direction and task implementation of the function module, and a collaborative interaction mechanism among units of the algorithm plane is established. A regional plane is constructed to solve the problem of adaptive preprocessing of cross-modal data, and receiving and preprocessing of original data are realized.

Description

Technical Field

[0001] The present invention relates to the field of neural networks, and in particular to a general neural network model based on a layered architecture. Background Art

[0002] Neural networks, a core technology in the field of artificial intelligence, have made breakthrough progress in recent years. However, their training still primarily relies on hardware such as GPUs, CPUs, TPUs, and FPGAs, lacking general-purpose hardware specifically developed for the connectivity characteristics and functional modules of neural networks. The root cause of this problem lies in the lack of a universal, standardized architecture that is compatible with multiple neural network architectures, making it difficult to develop universal hardware suitable for different neural network models.

[0003] Furthermore, the sheer number of connections between neurons in current neural network architectures significantly increases the complexity of hardware design, severely restricting the hardware implementation of neural networks. Different types of neural networks (such as fully connected networks, convolutional networks, and recurrent networks) exhibit significant differences in their neuronal connectivity, further complicating the development of a universal neural network architecture.

[0004] Furthermore, neural networks have evolved into a variety of architectural types, such as convolutional neural networks, recurrent neural networks, graph neural networks, and generative adversarial networks. These neural networks exhibit significant diversity in their functional module designs. Neural networks of different architectural types contain a variety of different functional modules, and multiple functional modules can also be integrated within the same neural network.

[0005] To address the above issues, it is necessary to propose a technical solution to exponentially reduce the number of physical connections between neurons, be compatible with multiple connection methods of neural networks, and achieve collaboration and interaction between functional modules of the new neural network architecture. Summary of the Invention

[0006] In order to address the above-mentioned deficiencies in the prior art, the present invention proposes a universal neural network model based on a layered architecture, an architecture that separates control and computation, and a universal neural network functional module collaborative interaction mechanism, which can effectively construct a standardized neural network architecture.

[0007] The present invention adopts the following technical solutions:

[0008] 1. A general neural network model based on a layered architecture, characterized by comprising a neural plane, a control plane, an algorithm plane, and a regional plane;

[0009] S1: The regional plane comprises a raw data receiving unit, a pre-processing unit, an output data receiving unit and a data post-processing unit, wherein the pre-processing unit comprises a dimension identifier and a feature extractor;

[0010] S2: The algorithm plane includes a data receiving unit, a label receiving unit, a data output unit, a model storage unit, a loss function storage unit, a model decomposition unit and a parameter calculation unit, wherein the model decomposition unit is composed of a module decomposer, a data decomposer, a step memory and a zero filling memory, a sum memory, and a nonlinear function memory;

[0011] S3: The control plane includes a data storage unit, a controller unit and an algorithm receiving unit, wherein the data storage unit is divided into a data receiving unit, a data output unit, a data decomposition unit and a controller selection unit;

[0012] S4: The neural plane includes a weight-feature multiplier, a temporary storage unit, a summation unit and a nonlinear function calculation unit.

[0013] 2. The universal neural network model according to claim 1 is characterized in that the universal neural network standardized model is a universal neural network model with scalability.

[0014] 3. A regional plane of the general neural network model according to claim S1, characterized in that it includes the following contents:

[0015] S11: The regional plane includes a raw data receiving unit (RRDRU), a preprocessing unit (RPU), an output data receiving unit (RODRU) and a data post-processing unit (RDPU), wherein the preprocessing unit includes a dimension identifier (RDI) and a feature extractor (RFE).

[0016] S12: The raw data receiving unit receives the raw input data and its corresponding label. The raw data can be represented as a matrix X′∈R M×N×D′ , which is a three-dimensional tensor where each slice matrix X of X′ d′ ′∈R M′×N′ is an input feature map. The original data can be video, audio or text signals, etc. The label can be represented as Y′∈R G×F×P′ , where each slice matrix Y of Y′ p′ ∈R G′×F′ is an input feature map, 1≤p′≤P′.

[0017] S13: After the original data is processed by the feature extractor in the pre-processing unit RPU, the data is converted into X∈R M ×N×D , where each slice matrix X d ∈R M×N is an input feature map, 1≤d≤D. The dimension identifier in the pre-processing unit RPU can automatically identify the dimension of the original input data X'.

[0018] After the label is processed by the feature extractor in the preprocessing unit RPU, the data is converted to Y∈R G×F×P , where each slice matrix Y p ∈R G×F is an input feature map, 1≤p≤P. The dimension identifier in the pre-processing unit RPU can automatically identify the dimension of the label.

[0019] The core functions of the preprocessing unit include a dimension identifier and a feature extractor. The dimension identifier automatically identifies the dimensions of the input data, while the feature extractor performs preliminary preprocessing and feature extraction on the input data. The dimension identification and feature extraction algorithms can be customized based on actual needs. This design allows the preprocessing unit to adaptively identify the dimensions of cross-modal data, preprocess it, and finally achieve dimensionality reduction using appropriate feature extraction methods.

[0020] S14: The data dimension received in the output data receiving unit is the same as the label, so the data is written as Z∈R G×F×P , where each slice matrix Z p ∈R G×F is an input feature map, 1≤p≤P.

[0021] S15: The output data receiving unit of the region plane receives the data output unit from the algorithm plane and transmits the received data to the data post-processing unit. The data processed by the data post-processing unit is used as the final result.

[0022] 4. An algorithm plane of the general neural network model according to claims 1 and 2, characterized in that it includes the following:

[0023] S21: The algorithm plane includes a data receiving unit (ADRU), a label receiving unit (ALRU), a data output unit (ADOU), a model storage unit (AMSU), a loss function storage unit (ALFSU), a model decomposition unit (AMDU) and a parameter computing unit (APCU), wherein the model decomposition unit includes a module resolver (AMR), a data resolver (ADR), a step memory (ASM), a module indicator (AMI), a zero fill memory (AZFM), a sigma storage (ASS) and a nonlinear function memory (ANFM).

[0024] S22: The data receiving unit ADRU receives data X from the area plane. The label receiving unit (ALRU) receives label Y from the area plane.

[0025] S23: The data output unit ADOU receives data from the control plane data output unit, and returns the final calculation result Y of the algorithm plane to the output data receiving unit (RODRU) of the regional plane.

[0026] S24: The model storage unit AMSU receives the entire input neural network model, which can be a fully connected neural network, a convolutional neural network, a recurrent neural network, a graph neural network, a generative adversarial network, or a fusion model thereof. The model storage unit has an interface, and model data is directly written via the external interface.

[0027] S25: The core function of the model decomposition unit AMDU is to decompose the neural network model in the model storage unit (AMSU). The model decomposition unit sequentially decomposes each functional layer according to the construction order of the neural network model, such as the convolutional layer, pooling layer, fully connected layer, recurrent network layer, graph neural network layer, normalization layer, summation formula, non-linear function, and zero-padding value, etc., and writes these modules or data into the module decomposer AMR in sequence. Encoding is performed on layer structures such as the convolutional layer, pooling layer, fully connected layer, recurrent network layer, graph neural network layer, normalization layer, etc., and the encoding is used to illustrate which type of layer this is. Different symbols can be used to encode the modules. For example, "001" is used to represent the convolutional algorithm, "002" is used to represent the pooling layer algorithm (max pooling or average pooling), and "003" is used to represent the fully connected network, etc.

[0028] S26: The module decomposer AMR further decomposes each module in sequence, clarifying the specific structure, number of layers, inter-layer parameters W, bias B, stride S, and summation formula of each module. The module decomposer writes the layer encodings of the convolutional layer, pooling layer, fully connected layer, recurrent network layer, graph neural network layer, normalization layer, etc. into the module indicator AMI, writes the inter-layer parameters and input data into the data decomposer ADR, writes the stride S into the stride memory ASM, writes the zero-padding value into the zero-padding memory AZFM, writes the summation formula into the summation memory ASS, and writes the non-linear function into the non-linear function memory ANFM.

[0029] S27: The loss function storage unit stores the loss functions required by the neural network model. The loss function storage unit has an external interface, and the data in the unit is input through the external interface. There are marks in the neural network model for which loss function to specifically call.

[0030] In the locally connected neural network, the parameter W ∈ R U×V×P×D is the length and width of one piece of convolution, where U < M and V < N. The parameter W p,d ∈ R U×V , where p is the number of output feature maps of the algorithm plane at the beginning, and d is the number of output feature maps of the algorithm plane at the end. In the fully connected neural network, during each convolution operation, all neurons in each layer participate in the operation to obtain an input value of one neuron in the lower layer. At this time, U = M and V = N.

[0031] Let B = (b 1 , b 2 , …, b p , …, b P ), T where b P is the scalar bias. Let the output feature of the convolution operation be Y p , then there is

[0032]

[0033] Y p =f(A p ) (2)

[0034] Among them, W p ∈R U×V×D is a three-dimensional convolution kernel, and f(·) is a nonlinear activation function. The calculation of and f(·) is performed on the control plane and the neural plane. According to the structure of the neural plane, the weight-feature multiplier of the neural plane controlled by each controller unit of the control plane can only realize one Convolution operation, for D The operation requires weight-feature multipliers of the neural plane controlled by D controller units.

[0035] S28: The data decomposer receives the parameters W and bias B from the module decomposer. The data decomposer decomposes the received data and sends the decomposed data to the data receiving unit of the control plane. The data decomposer needs to retain the parameters of the entire neural network in order to perform the backpropagation operation with the data in the data output unit. The decomposition rules here are as follows:

[0036] (1) For the parameters between layers in a module of a traditional neural network (such as a fully connected network and a locally connected neural network), they are decomposed and written into a matrix. In a fully connected neural network, the parameters between each two layers occupy one column of the matrix, ultimately forming a two-dimensional matrix. In a locally connected neural network, the parameters between each two layers are stored in the matrix in the form of convolution kernels, ultimately forming a three-dimensional matrix.

[0037] (2) For a two-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by column. For a three-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by convolution kernel. After the data receiving unit of the control plane receives the parameters, the control parameters and the input X are operated on the neural plane.

[0038] S29: The parameter calculation unit updates the parameters using the loss function stored in the loss function storage unit ALFSU. This update follows the principles of neural network backpropagation and requires combining the labels from the label receiving unit, the data Y returned from the control plane to the algorithm plane data output unit, and the parameters in the data decomposer. The updated parameters are returned to the parameter values stored in the data decomposer prior to the update.

[0039] 5. A control plane for a general neural network according to claims 1 and 2, comprising:

[0040] S31: The control plane includes a data storage unit (CDSU), a controller unit (CCU), and an algorithm receiving unit (CARU). The data storage unit CDSU includes a data receiving unit (CDRU), a data output unit (CDOU), a data decomposition unit (CDDU), and a controller selection unit (CCSU). The controller unit CCU includes a neuron selector (CNS) and a neuron controller (CNC).

[0041] S32: The data receiving unit CDRU of the control plane is responsible for receiving the input features X, B, W, zero padding value and step size S from each module of the algorithm plane.

[0042] S33: The data output unit CDOU of the control plane is responsible for receiving the output Y of the neural plane nonlinear function calculator.

[0043] S34: The algorithm receiving unit CARU of the control plane is responsible for receiving the AMR value from the algorithm plane module decomposer, including the summation formula, nonlinear function and algorithm structure coding, and transmitting the value to the neuron controller.

[0044] S35: The data decomposition unit CDDU of the control plane is responsible for decomposing X, B, and W in the data receiving unit CDRU into data of processable dimensions of neurons governed by each controller, storing the decomposed data in the buffer of the data decomposition unit, and transmitting the data to the controller selection unit.

[0045] S36: The controller selection unit CCSU is responsible for selecting a controller unit of a corresponding dimension according to the data dimension decomposed by the data decomposition unit CDDU, which includes a neuron selector and a neuron controller.

[0046] For example: X∈R M×N×D Decomposed into D X d ∈R M×N , B is decomposed into P b p , W∈R U×V×P×D Decomposed into D W p,d ∈R U×V and store it in the CDDU unit cache, and D controller units CCU need to be selected.

[0047] The control plane consists of multiple controller units (CCUs). Each controller unit controls only the weight-feature multipliers corresponding to the neural plane. Each controller unit does not cross-control the weight-feature multipliers controlled by other controller units. The weight-feature multipliers controlled by a controller unit can be arranged in a single column or in multiple rows and columns. A single column corresponds to a fully connected network, while multiple rows and columns correspond to a partially connected network.

[0048] The neuron selector CNS in the controller unit CCU selects a corresponding number of weight-feature multipliers based on the data dimensions in the data decomposition unit CDDU. The neuron controller CNC in the controller unit CCU receives the summation formula, nonlinear function, and algorithm structure encoding from the algorithm receiving unit. The controller unit operates as follows.

[0049] (1) The data receiving unit CDRU sends the zero padding value, step size S, input feature X, bias B, and parameter W to the data decomposition unit CDDU.

[0050] (2) The algorithm receiving unit CARU sends the summation formula and the nonlinear function to the neuron controller CNC respectively, and the neuron controller CNC sends them to the summation unit NSU and the nonlinear function calculator NNFC of the neural plane respectively.

[0051] (3) The data decomposition unit CDDU decomposes X, W into D X d ∈R M×N and W p,d , decompose B into P b p The data decomposition unit sends the decomposed data dimensions to the controller selection unit, neuron selector, and neuron controller. The controller selection unit CCSU selects D controller units to perform control.

[0052] (4) The controller selection unit selects D controller units, and the neuron selector in each controller unit selects M×N weight-feature multipliers WFM. The D neuron selectors CNS jointly select a summation unit and a nonlinear function calculation unit to perform summation and nonlinear calculations.

[0053] (5) The selected D neuron controller CNC is connected to the input X d The corresponding b p Send it to the selected summation unit, and send the nonlinear function to the selected nonlinear function calculation unit.

[0054] (6) The data decomposition unit CDDU transmits the zero filling value and step size S to the neuron controller CNC.

[0055] (7) According to the step size S and zero filling value in the neuron controller CNC, the neuron controller control parameter W p,d and X d The operation is performed in the weight-feature multiplier NWFM of the neural plane, and the operation result is temporarily stored in the temporary storage unit NTSU of the neural plane.

[0056] Subsequent operations are performed by the neural plane. The neural plane performs calculations according to the algorithm requirements of the CACU in the control unit CCU and returns the final results to the data output unit CDOU of the control plane.

[0057] 6. A neural plane of the general neural network model according to claims 1 and 2, characterized in that it comprises the following:

[0058] S41: The neural plane includes a weight-feature multiplier (NWFM), a summation unit (NSU), a nonlinear function computer (NNFC), and a temporary storage unit (NTSU).

[0059] S42: Let the i-th controller unit be NCU i Assuming that the number of weight-feature multipliers managed by the i-th controller unit is K, then the K NWFMs are expressed as Assume that the neural plane has L summation units NSU, which can be expressed as NSU1,…,NSU l ,…,NSU L , and L nonlinear function calculators NNFC, which can be expressed as NNFC1,…,NNFC l ,…,NNFC L .

[0060] Assume that the controller selection unit CCSU selects D controller units described as NCU1, ..., NCU i ,…,NCU D The neuron selector CNS of the i-th controller is based on X d ∈R M×N The size of the weight-feature multiplier (WFM) is selected in the neural plane, which is denoted as The selected one of the summing units is marked as NSU l .

[0061] S43: The neural plane operation rules are as follows:

[0062] (1) According to the step size S and zero filling value in the neuron controller CNC, in the weight-feature multiplier The weights and features are multiplied by their corresponding elements respectively. 1≤m≤M,1≤n≤N,1≤u≤U,1≤v≤V.

[0063] (2) After all NWFMs controlled by the M×N controller units have completed calculations, the results are stored in their respective temporary storage units. The temporary storage units send the calculation results to a summing unit for summing. The summing formula is:

[0064]

[0065] (3) The summation result obtained by the summation unit is sent to the nonlinear function calculator NNFC, in which the nonlinear operation is performed, and the operation result is sent to the data output unit CDOU of the control plane.

[0066] The present invention has the following beneficial effects: It constructs a universal neural network model that divides traditional neural networks into four planes: the regional plane, the algorithmic plane, the control plane, and the neural plane. This provides a solid theoretical foundation and innovative technical support for the standardization of neural network architectures. Universal neural network models with similar layered architectures are all within the scope of protection of this patent. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a detailed framework diagram of a general neural network model;

[0068] Figure 2 It is a hierarchical structure diagram of a general neural network model;

[0069] Figure 3 This is a neural plane architecture diagram of a general neural network model;

[0070] Figure 4 This is the control plane architecture diagram of a general neural network model;

[0071] Figure 5 This is the algorithm plane architecture diagram of the general neural network model;

[0072] Figure 6 This is a regional planar architecture diagram of a general neural network model. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0074] exist Figure 1 The detailed architecture diagram of the general neural network model is given in Figure 2 A hierarchical diagram of a general neural network is given in .

[0075] from Figure 3-6 It can be seen that the embodiment provides a general neural network model based on a layered architecture, comprising the following steps:

[0076] S1: Input the original input data and its corresponding labels into the original data receiving unit of the regional plane, such as Figure 6 shown.

[0077] The original data can be represented as a matrix X′∈R M′×N′×D′ , which is a three-dimensional tensor where each slice matrix X of X′ d′ ′∈R M′×N′ is an input feature map. The original data can be video, audio or text signals, etc. The label can be represented as Y′∈R G′×F′×P′ , where each slice matrix Y of Y′ p′ ∈R G′×F′ is an input feature map, 1≤p′≤P′.

[0078] After the original data is processed by the feature extractor in the regional plane preprocessing unit RPU, the data is converted to X∈R M×N×D , where each slice matrix X d ∈R M×N is an input feature map, 1≤d≤D. The dimension identifier in the preprocessing unit RPU can automatically identify the dimension of the original input data X′.

[0079] After the label is processed by the feature extractor in the preprocessing unit RPU of the regional plane, the data is converted to Y∈R G ×F×P , where each slice matrix Y p ∈R G×F is an input feature map, 1≤p≤P. The dimension identifier in the preprocessing unit RPU can automatically identify the dimension of the label.

[0080] The data dimension received in the output data receiving unit of the region plane is the same as the label, so the data is written as Z∈R G×F×P , where each slice matrix Z p ∈R G×F is an input feature map, 1≤p≤P.

[0081] The output data receiving unit of the region plane receives the data output unit from the algorithm plane and transmits the received data to the data post-processing unit. The data processed by the data post-processing unit is used as the final result.

[0082] S2: The algorithm plane data receiving unit ADRU receives data X from the regional plane. The algorithm plane label receiving unit ALRU receives the label Y from the regional plane, such as Figure 5 shown.

[0083] The data output unit ADOU of the algorithm plane receives data from the data output unit of the control plane and returns the final calculation result Y of the algorithm plane to the output data receiving unit (RODRU) of the regional plane.

[0084] The algorithmic plane's model storage unit (AMSU) receives the entire input neural network model. This model can be a fully connected neural network, convolutional neural network, recurrent neural network, graph neural network, generative adversarial network, or a fusion of these. The model storage unit has an interface, and model data is directly written via an external interface.

[0085] The core function of the model decomposition unit AMDU of the algorithm plane is to decompose the neural network model in the model storage unit (AMSU). The model decomposition unit decomposes each functional layer in sequence according to the construction order of the neural network model, such as the convolution layer, the pooling layer, the fully connected layer, the recurrent network layer, the graph neural network layer, the normalization layer, the summation formula, the nonlinear function and the zero-filling value, and writes these modules or data into the module decomposer AMR in sequence. The layer structures such as the convolution layer, the pooling layer, the fully connected layer, the recurrent network layer, the graph neural network layer, the normalization layer, etc. are encoded, and the encoding is used to indicate which situation the layer is in. Different symbols can be used to encode the modules, for example, "001" is used to represent the convolution algorithm, "002" is used to represent the pooling layer algorithm (maximum pooling or average pooling), and "003" is used to represent the fully connected network.

[0086] The module decomposer AMR of the algorithmic plane further decomposes each module in sequence, clarifying the specific structure, number of layers, inter-layer parameters W, bias B, step size S, and summation formula of each module. The module decomposer writes the layer encodings of the convolutional layer, pooling layer, fully connected layer, recurrent network layer, graph neural network layer, normalization layer, etc. into the module indicator AMI, writes the inter-layer parameters and input data into the data decomposer ADR, writes the step size S into the step size memory ASM, writes the zero padding value into the zero padding memory AZFM, writes the summation formula into the summation memory ASS, and writes the nonlinear function into the nonlinear function memory ANFM.

[0087] The loss function storage unit of the algorithm plane stores the loss function required by the neural network model. The loss function storage unit has an external interface, and the data in the unit is input through the external interface. There is a mark in the neural network model that specifically calls which loss function.

[0088] In a locally connected neural network, the parameter W∈R U×V×P×Dis the length and width of one of the convolutions, where U < M and V < N. The parameter W p,d ∈R U×V , where p is the number of initially output feature maps of the algorithm plane, and d is the number of finally output feature maps of the algorithm plane. In a fully connected neural network, during each convolution operation, all neurons in each layer participate in the operation to obtain an input value of one neuron in the lower layer. At this time, U = M and V = N.

[0089] Let B = (b 1 , b 2 , …, b p , …, b P ), T , where b P is a scalar bias. Let the output feature of the convolution operation be Y p , then there is

[0090]

[0091] Y p = f(A p )(4)

[0092] where, W p ∈R U×V×D is a three-dimensional convolution kernel, and f(·) is a non-linear activation function. Here and the calculation of f(·) are performed in the control plane and the neural plane. According to the structure of the neural plane, the weight-feature multipliers of the neural plane controlled by each controller unit in the control plane can only implement one convolution operation. For D operations, D controller units are required to control the weight-feature multipliers of the neural plane.

[0093] The data decomposer in the algorithm plane receives the parameter W and the bias B from the module decomposer. The data decomposer decomposes the received data and distributes the decomposed data to the data receiving unit in the control plane. The data decomposer needs to retain the parameters of the entire neural network to cooperate with the data in the data output unit to perform backpropagation operations. The decomposition rules are as follows:

[0094] (1) For the parameters between the inner layers of the module in a traditional neural network (such as a fully connected network, a locally connected neural network), decompose and write them into a matrix in sequence. In a fully connected neural network, the parameters between every two layers occupy one column of the matrix, and finally a two-dimensional matrix is formed. In a locally connected neural network, the parameters between every two layers are stored in the matrix in the form of a convolution kernel, and finally a three-dimensional matrix is formed.

[0095] (2) For a two-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by column. For a three-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by convolution kernel. After the data receiving unit of the control plane receives the parameters, the control parameters and the input X are operated on the neural plane.

[0096] The algorithmic plane's parameter calculation unit updates parameters using the loss function stored in the loss function storage unit (ALFSU). This update follows the principles of neural network backpropagation and combines the labels from the label receiving unit, the data Y returned from the control plane to the algorithmic plane's data output unit, and the parameters in the data decomposer. The updated parameters return to the values stored in the data decomposer prior to the update.

[0097] S3: The data receiving unit CDRU of the control plane is responsible for receiving the input features X, B, W, zero padding value and step size S from each module of the algorithm plane, such as Figure 4 The data output unit CDOU of the control plane is responsible for receiving the output Y of the nonlinear function calculator of the neural plane.

[0098] The algorithm receiving unit CARU of the control plane is responsible for receiving the AMR value from the algorithm plane module decomposer, including the summation formula, nonlinear function and algorithm structure coding, and transmitting the value to the neuron controller.

[0099] The data decomposition unit CDDU of the control plane is responsible for decomposing X, B, and W in the data receiving unit CDRU into data of processable dimensions of neurons under the jurisdiction of each controller, storing the decomposed data in the buffer of the data decomposition unit, and transmitting the data to the controller selection unit.

[0100] The controller selection unit CCSU is responsible for selecting the controller unit of the corresponding dimension according to the data dimension decomposed by the data decomposition unit CDDU, which includes a neuron selector and a neuron controller.

[0101] For example: X∈R M×N×D Decomposed into D X d ∈R M×N , B is decomposed into P b p , W∈R U×V×P×D Decomposed into D W p,d ∈R U×V and store it in the CDDU unit cache, and D controller units CCU need to be selected.

[0102] The control plane consists of many controller units (CCUs). Each CCU controls only the weight-feature multipliers corresponding to the neural plane. Each CCU does not cross-control the weight-feature multipliers controlled by other CCUs. The weight-feature multipliers controlled by a controller unit can be arranged in a single column or in multiple rows and columns. A single column corresponds to a fully connected network, while multiple rows and columns correspond to a partially connected network.

[0103] The neuron selector CNS in the controller unit CCU selects a corresponding number of weight-feature multipliers based on the data dimensions in the data decomposition unit CDDU. The neuron controller CNC in the controller unit CCU receives the summation formula, nonlinear function, and algorithm structure encoding from the algorithm receiving unit. The controller unit operates as follows.

[0104] (1) The data receiving unit CDRU sends the zero padding value, step size S, input feature X, bias B, and parameter W to the data decomposition unit CDDU.

[0105] (2) The algorithm receiving unit CARU sends the summation formula and the nonlinear function to the neuron controller CNC respectively, and the neuron controller CNC sends them to the summation unit NSU and the nonlinear function calculator NNFC of the neural plane respectively.

[0106] (3) The data decomposition unit CDDU decomposes X, W into D X d ∈R M×N and W p,d , decompose B into P b p The data decomposition unit sends the decomposed data dimensions to the controller selection unit, neuron selector, and neuron controller. The controller selection unit CCSU selects D controller units to perform control.

[0107] (4) The controller selection unit selects D controller units, and the neuron selector in each controller unit selects M×N weight-feature multipliers WFM. The D neuron selectors CNS jointly select a summation unit and a nonlinear function calculation unit to perform summation and nonlinear calculations.

[0108] (5) The selected D neuron controller CNC is connected to the input X d The corresponding b p Send it to the selected summation unit, and send the nonlinear function to the selected nonlinear function calculation unit.

[0109] (6) The data decomposition unit CDDU transmits the zero filling value and step size S to the neuron controller CNC.

[0110] (7) According to the step size S and zero filling value in the neuron controller CNC, the neuron controller control parameter W p,d and X d The operation is performed in the weight-feature multiplier NWFM of the neural plane, and the operation result is temporarily stored in the temporary storage unit NTSU of the neural plane.

[0111] Subsequent operations are performed by the neural plane. The neural plane performs calculations according to the algorithm requirements of the CACU in the control unit CCU and returns the final results to the data output unit CDOU of the control plane.

[0112] S4: Let the i-th controller unit be NCU i Assuming that the number of weight-feature multipliers managed by the i-th controller unit is K, then the K NWFMs are expressed as Assume that the neural plane has L summation units NSU, which can be expressed as NSU1,…,NSU l ,…,NSU L , and L nonlinear function calculators NNFC, which can be expressed as NNFC1,…,NNFC l ,…,NNFC L .

[0113] Assume that the controller selection unit CCSU selects D controller units described as NCU1, ..., NCU i ,…,NCU D The neuron selector CNS of the i-th controller is based on X d ∈R M×N The size of the weight-feature multiplier (WFM) is selected in the neural plane, which is denoted as The selected one of the summing units is marked as NSU l .

[0114] The neural plane operates as follows Figure 3 shown.

[0115] (1) According to the step size S and zero filling value in the neuron controller CNC, in the weight-feature multiplier The weights and features are multiplied by their corresponding elements respectively. 1≤m≤M,1≤n≤N,1≤u≤U,1≤v≤V.

[0116] (2) After all NWFMs controlled by the M×N controller units have completed calculations, the results are stored in their respective temporary storage units. The temporary storage units send the calculation results to a summing unit for summing. The summing formula is:

[0117]

[0118] (3) The summation result obtained by the summation unit is sent to the nonlinear function calculator NNFC, in which the nonlinear operation is performed, and the operation result is sent to the data output unit CDOU of the control plane.

Claims

1. A general neural network model based on a layered architecture, characterized in that: Including neural plane, control plane, algorithm plane, and regional plane; S1: The regional plane comprises a raw data receiving unit, a pre-processing unit, an output data receiving unit and a data post-processing unit, wherein the pre-processing unit comprises a dimension identifier and a feature extractor; S2: The algorithm plane includes a data receiving unit, a label receiving unit, a data output unit, a model storage unit, a loss function storage unit, a model decomposition unit and a parameter calculation unit, wherein the model decomposition unit is composed of a module decomposer, a data decomposer, a step memory and a zero filling memory, a sum memory, and a nonlinear function memory; S3: The control plane includes a data storage unit, a controller unit and an algorithm receiving unit, wherein the data storage unit is divided into a data receiving unit, a data output unit, a data decomposition unit and a controller selection unit; S4: The neural plane includes a weight-feature multiplier, a temporary storage unit, a summation unit and a nonlinear function calculation unit.

2. The general neural network model according to claim 1, characterized in that The general neural network standardized model is a general neural network model with scalability.

3. A regional plane of the general neural network model according to claim S1, characterized in that Includes the following: S11: The regional plane includes a raw data receiving unit (RRDRU), a preprocessing unit (RPU), an output data receiving unit (RODRU) and a data post-processing unit (RDPU), wherein the preprocessing unit includes a dimension identifier (RDI) and a feature extractor (RFE). S12: The raw data receiving unit receives the raw input data and its corresponding label. The raw data can be represented as a matrix X′∈R M×N×D′ , which is a three-dimensional tensor where each slice matrix X of X′ d′ ′∈R M′×N× is an input feature map. The original data can be video, audio or text signals, etc. The label can be represented as Y′∈R G×F×P′ , where each slice matrix Y of Y′ p′ ∈R G′×F′ is an input feature map, 1≤p′≤P′. S13: After the original data is processed by the feature extractor in the pre-processing unit RPU, the data is converted into X∈R M×N×D , where each slice matrix X d ∈R M×N is an input feature map, 1≤d≤D. The dimension identifier in the pre-processing unit RPU can automatically identify the dimension of the original input data X'. After the label is processed by the feature extractor in the preprocessing unit RPU, the data is converted to Y∈R G×F×P , where each slice matrix Y p ∈R G×F is an input feature map, 1≤p≤P. The dimension identifier in the pre-processing unit RPU can automatically identify the dimension of the label. The core functions of the preprocessing unit include a dimension identifier and a feature extractor. The dimension identifier automatically identifies the dimensions of the input data, while the feature extractor performs preliminary preprocessing and feature extraction on the input data. The dimension identification and feature extraction algorithms can be customized based on actual needs. This design allows the preprocessing unit to adaptively identify the dimensions of cross-modal data, preprocess it, and finally achieve dimensionality reduction using appropriate feature extraction methods. S14: The data dimension received in the output data receiving unit is the same as the label, so the data is written as Z∈R G ×F×P , where each slice matrix Z p ∈R G×F is an input feature map, 1≤p≤P. S15: The output data receiving unit of the region plane receives the data output unit from the algorithm plane and transmits the received data to the data post-processing unit. The data processed by the data post-processing unit is used as the final result.

4. An algorithm plane of the general neural network model according to claims 1 and 2, characterized in that: Includes the following: S21: The algorithm plane includes a data receiving unit (ADRU), a label receiving unit (ALRU), a data output unit (ADOU), a model storage unit (AMSU), a loss function storage unit (ALFSU), a model decomposition unit (AMDU) and a parameter computing unit (APCU), wherein the model decomposition unit includes a module resolver (AMR), a data resolver (ADR), a step memory (ASM), a module indicator (AMI), a zero fill memory (AZFM), a sigma storage (ASS) and a nonlinear function memory (ANFM). S22: The data receiving unit ADRU receives data X from the area plane. The label receiving unit (ALRU) receives label Y from the area plane. S23: The data output unit ADOU receives data from the control plane data output unit, and returns the final calculation result Y of the algorithm plane to the output data receiving unit (RODRU) of the regional plane. S24: The model storage unit AMSU receives the entire input neural network model, which can be a fully connected neural network, a convolutional neural network, a recurrent neural network, a graph neural network, a generative adversarial network, or a fusion model thereof. The model storage unit has an interface, and model data is directly written via the external interface. S25: The core function of the model decomposition unit AMDU is to decompose the neural network model in the model storage unit (AMSU). The model decomposition unit decomposes each functional layer in sequence according to the construction order of the neural network model, such as the convolution layer, the pooling layer, the fully connected layer, the recurrent network layer, the graph neural network layer, the normalization layer, the summation formula, the nonlinear function and the zero-filling value, and writes these modules or data into the module decomposer AMR in sequence. The layer structures such as the convolution layer, the pooling layer, the fully connected layer, the recurrent network layer, the graph neural network layer, the normalization layer are encoded, and the encoding is used to indicate which situation the layer is in. Different symbols can be used to encode the modules, for example, "001" represents the convolution algorithm, "002" represents the pooling layer algorithm (maximum pooling or average pooling), and "003" represents the fully connected network. S26: The module decomposer AMR further decomposes each module in sequence to clarify the specific structure, number of layers, inter-layer parameters W, bias B, step size S, and summation formula of each module. The module decomposer writes the layer codes of the convolution layer, pooling layer, fully connected layer, recurrent network layer, graph neural network layer, normalization layer, etc. into the module indicator AMI, writes the inter-layer parameters and input data into the data decomposer ADR, writes the step size S into the step size memory ASM, writes the zero padding value into the zero padding memory AZFM, writes the summation formula into the summation memory ASS, and writes the nonlinear function into the nonlinear function memory ANFM. S27: The loss function storage unit stores the loss function required by the neural network model. The loss function storage unit has an external interface, and data in the unit is input through the external interface. There is a mark in the neural network model that specifically calls which loss function. In a locally connected neural network, the parameter W ∈ R U×V×P×D is the length and width of one of the convolutions, where U < M and V < N. The parameter W p,d ∈ R U×V , where p is the number of initially output feature maps of the algorithm plane, and d is the number of finally output feature maps of the algorithm plane. In a fully connected neural network, during each convolution operation, all neurons in each layer participate in the operation to obtain an input value of one neuron in the lower layer. At this time, U = M and V = N. Let B = b 1 ,b 2 ,…,b p ,…,b P ) T , where b P Is a scalar bias. Let the convolution operation output feature be Y p , then Y p =f(A p ) (2) Among them, W p ∈R U×V×D is a three-dimensional convolution kernel, and f(·) is a nonlinear activation function. The calculation of and f(·) is performed on the control plane and the neural plane. According to the structure of the neural plane, the weight-feature multiplier of the neural plane controlled by each controller unit of the control plane can only realize one Convolution operation, for D The operation requires weight-feature multipliers of the neural plane controlled by D controller units. S28: The data decomposer receives the parameters W and bias B from the module decomposer. The data decomposer decomposes the received data and sends the decomposed data to the data receiving unit of the control plane. The data decomposer needs to retain the parameters of the entire neural network in order to perform the backpropagation operation with the data in the data output unit. The decomposition rules here are as follows: (1) For the parameters between layers in a module of a traditional neural network (such as a fully connected network and a locally connected neural network), they are decomposed and written into a matrix. In a fully connected neural network, the parameters between each two layers occupy one column of the matrix, ultimately forming a two-dimensional matrix. In a locally connected neural network, the parameters between each two layers are stored in the matrix in the form of convolution kernels, ultimately forming a three-dimensional matrix. (2) For a two-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by column. For a three-dimensional matrix, the parameters are sent to the data receiving unit of the control plane in order by convolution kernel. After the data receiving unit of the control plane receives the parameters, the control parameters and the input X are operated on the neural plane. S29: The parameter calculation unit updates the parameters using the loss function stored in the loss function storage unit ALFSU. This update follows the principles of neural network backpropagation and requires combining the labels from the label receiving unit, the data Y returned from the control plane to the algorithm plane data output unit, and the parameters in the data decomposer. The updated parameters are returned to the parameter values stored in the data decomposer prior to the update.

5. A control plane for a general neural network according to claims 1 and 2, characterized in that Includes the following: S31: The control plane includes a data storage unit (CDSU), a controller unit (CCU), and an algorithm receiving unit (CARU). The data storage unit CDSU includes a data receiving unit (CDRU), a data output unit (CDOU), a data decomposition unit (CDDU), and a controller selection unit (CCSU). The controller unit CCU includes a neuron selector (CNS) and a neuron controller (CNC). S32: The data receiving unit CDRU of the control plane is responsible for receiving the input features X, B, W, zero padding value and step size S from each module of the algorithm plane. S33: The data output unit CDOU of the control plane is responsible for receiving the output Y of the neural plane nonlinear function calculator. S34: The algorithm receiving unit CARU of the control plane is responsible for receiving the AMR value from the algorithm plane module decomposer, including the summation formula, nonlinear function and algorithm structure coding, and transmitting the value to the neuron controller. S35: The data decomposition unit CDDU of the control plane is responsible for decomposing X, B, and W in the data receiving unit CDRU into data of processable dimensions of neurons governed by each controller, storing the decomposed data in the buffer of the data decomposition unit, and transmitting the data to the controller selection unit. S36: The controller selection unit CCSU is responsible for selecting a controller unit of a corresponding dimension according to the data dimension decomposed by the data decomposition unit CDDU, which includes a neuron selector and a neuron controller. For example: X∈R M×N×D Decomposed into D X d ∈R M×N , B is decomposed into P b p , W∈R U×V×P×D Decomposed into D W p,d ∈R U×V and store it in the CDDU unit cache, and D controller units CCU need to be selected. The control plane consists of multiple controller units (CCUs). Each controller unit controls only the weight-feature multipliers corresponding to the neural plane. Each controller unit does not cross-control the weight-feature multipliers controlled by other controller units. The weight-feature multipliers controlled by a controller unit can be arranged in a single column or in multiple rows and columns. A single column corresponds to a fully connected network, while multiple rows and columns correspond to a partially connected network. The neuron selector CNS in the controller unit CCU selects a corresponding number of weight-feature multipliers based on the data dimensions in the data decomposition unit CDDU. The neuron controller CNC in the controller unit CCU receives the summation formula, nonlinear function, and algorithm structure encoding from the algorithm receiving unit. The controller unit operates as follows. (1) The data receiving unit CDRU sends the zero padding value, step size S, input feature X, bias B, and parameter W to the data decomposition unit CDDU. (2) The algorithm receiving unit CARU sends the summation formula and the nonlinear function to the neuron controller CNC respectively, and the neuron controller CNC sends them to the summation unit NSU and the nonlinear function calculator NNFC of the neural plane respectively. (3) The data decomposition unit CDDU decomposes X, W into D X d ∈R M×N and W p,d , decompose B into P b p The data decomposition unit sends the decomposed data dimensions to the controller selection unit, neuron selector, and neuron controller. The controller selection unit CCSU selects D controller units to perform control. (4) The controller selection unit selects D controller units, and the neuron selector in each controller unit selects M×N weight-feature multipliers WFM. The D neuron selectors CNS jointly select a summation unit and a nonlinear function calculation unit to perform summation and nonlinear calculations. (5) The selected D neuron controller CNC is connected to the input X d The corresponding b p Send it to the selected summation unit, and send the nonlinear function to the selected nonlinear function calculation unit. (6) The data decomposition unit CDDU transmits the zero filling value and step size S to the neuron controller CNC. (7) According to the step size S and zero filling value in the neuron controller CNC, the neuron controller control parameter W p,d and X d The operation is performed in the weight-feature multiplier NWFM of the neural plane, and the operation result is temporarily stored in the temporary storage unit NTSU of the neural plane. Subsequent operations are performed by the neural plane. The neural plane performs calculations according to the algorithm requirements of the CACU in the control unit CCU and returns the final results to the data output unit CDOU of the control plane.

6. A neural plane of the general neural network model according to claims 1 and 2, characterized in that Includes the following: S41: The neural plane includes a weight-feature multiplier (NWFM), a summation unit (NSU), a nonlinear function computer (NNFC), and a temporary storage unit (NTSU). S42: Let the i-th controller unit be NCU i Assuming that the number of weight-feature multipliers managed by the i-th controller unit is K, then the K NWFMs are expressed as Assume that the neural plane has L summation units NSU, which can be expressed as NSU1,…,NSU l ,…,NSU L , and L nonlinear function calculators NNFC, which can be expressed as NNFC1,…,NNFC l ,…,NNFC L . Assume that the controller selection unit CCSU selects D controller units described as NCU1, ..., NCU i ,…,NCU D The neuron selector CNS of the i-th controller is based on X d ∈R M×N The size of the weight-feature multiplier (WFM) is selected in the neural plane, which is denoted as The selected one of the summing units is marked as NSU l . S43: The neural plane operation rules are as follows: (1) According to the step size S and zero filling value in the neuron controller CNC, in the weight-feature multiplier The weights and features are multiplied by their corresponding elements respectively. 1≤m≤M,1≤n≤N,1≤u≤U,1≤v≤V. (2) After all NWFMs controlled by the M×N controller units have completed calculations, the results are stored in their respective temporary storage units. The temporary storage units send the calculation results to a summing unit for summing. The summing formula is: (3) The summation result obtained by the summation unit is sent to the nonlinear function calculator NNFC, in which the nonlinear operation is performed, and the operation result is sent to the data output unit CDOU of the control plane.

Images