Neural network reasoning method and related equipment based on storage and computing architecture

Abstract

The present invention discloses a neural network reasoning method and related equipment based on storage and computing architecture, which relates to the field of neural network model reasoning, and is mainly used to solve the problem that the current model reasoning method needs to frequently update weight data, which weakens the advantages of storage and computing integrated chips. Specifically, it includes: determining weight data and input vector groups; traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two weight data; reshaping the data matrix based on the output vector and the neural network calculation graph to forward it to the output device.

Description

Technical Field

[0001] The present invention relates to the field of neural network model reasoning, and in particular to a neural network reasoning method and related equipment based on a storage and computing architecture. Background Art

[0002] The storage and computing integrated chip technology integrates multiplication and accumulation computing units into the memory to give the memory computing capabilities. While reducing data movement in calculations, it utilizes the high parallelism of storage units to greatly improve computing performance. It is suitable for high-density computing scenarios such as reasoning of neural network models.

[0003] However, commonly used model inference methods often require frequent updates of weight data. In addition, the storage capacity of integrated storage chips is limited, which greatly weakens the advantages of integrated storage and computing chips when performing deep neural network model inference. Summary of the invention

[0004] In view of the above problems, the present invention provides a neural network reasoning method and related equipment based on a storage and computing architecture, the main purpose of which is to solve the problem that the current model reasoning method requires frequent updates of weight data, which weakens the advantages of storage and computing integrated chips.

[0005] In order to solve at least one of the above technical problems, in a first aspect, the present invention provides a neural network reasoning method based on a storage-computing architecture, the method comprising:

[0006] Determine weight data and input vector group;

[0007] Traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two of the weight data;

[0008] Reshape the data matrix based on the above output vector and neural network computation graph to forward to the output device.

[0009] Optionally, the above method further includes:

[0010] Traversing all storage computing units in the storage computing array to generate a storage computing array routing table, wherein the storage computing array routing table is used to determine the number of storage computing units and storage space information of the storage computing units;

[0011] The computation graph is traversed to expand the weight tensor of each node in the above computation graph into a weight vector.

[0012] Optionally, the above-mentioned determination of weight data and input vector group includes:

[0013] Based on the number of the above-mentioned storage and computing units, the storage space information of the above-mentioned storage and computing units, the size of the above-mentioned weight data and the data bit width of the above-mentioned weight data, the above-mentioned weight data is written into the cache area and the storage and computing array.

[0014] Optionally, the above-mentioned determination of weight data and input vector group includes:

[0015] Preprocessing the image data to obtain an input feature map, wherein the image data is obtained based on an image file and / or an acquisition device;

[0016] Expand the above input feature map into an input vector group;

[0017] The input vector group is forwarded to the cache area based on the embedded memory.

[0018] Optionally, traversing the input vector group and the weight data until the weight vector is updated to determine the output vector includes:

[0019] The input vector group and the weight data are traversed based on a preset stride and a preset unit length until the weight vector is updated to determine the output vector.

[0020] Optionally, the preset stride is determined based on the amplitude of the convolution, and the preset unit length is determined based on the size of the convolution kernel.

[0021] Optionally, the above method further includes:

[0022] In the case that the reshaped data matrix comes from the storage array, data type conversion is performed on the reshaped data matrix.

[0023] In a second aspect, an embodiment of the present invention further provides a neural network reasoning device based on a storage and computing architecture, comprising:

[0024] A determination unit, used for determining weight data and an input vector group;

[0025] A traversal unit, used for traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two weight data;

[0026] The reshaping unit is used to reshape the data matrix based on the output vector and the neural network calculation graph so as to forward it to the output device.

[0027] In order to achieve the above-mentioned purpose, according to the third aspect of the present invention, a computer-readable storage medium is provided, and the above-mentioned computer-readable storage medium includes a stored program, wherein when the above-mentioned program is executed by a processor, the steps of the above-mentioned neural network reasoning method based on the storage-computing architecture are implemented.

[0028] In order to achieve the above-mentioned purpose, according to the fourth aspect of the present invention, there is provided an electronic device, comprising at least one processor and at least one memory connected to the above-mentioned processor; wherein the above-mentioned processor is used to call the program instructions in the above-mentioned memory to execute the steps of the above-mentioned neural network reasoning method based on the storage-computing architecture.

[0029] Through the above technical scheme, the neural network reasoning method and related equipment based on the storage-computing architecture provided by the present invention can solve the problem that the current model reasoning method needs to frequently update the weight data, which weakens the advantages of the storage-computing integrated chip. The present invention determines the weight data and the input vector group; traverses the above input vector group and the above weight data until the weight vector is updated to determine the output vector, wherein the above weight vector is composed of at least two of the above weight data; and reshapes the data matrix based on the above output vector and the neural network calculation graph to forward it to the output device. In the above scheme, compared with the commonly used Im2Col method, the amount of data during transmission can be effectively reduced, so that there is no need to frequently update the weight data, and the effect of efficient reasoning of the neural network model is achieved.

[0030] Correspondingly, the neural network reasoning device, equipment and computer-readable storage medium based on the storage and computing architecture provided by the embodiments of the present invention also have the above-mentioned technical effects.

[0031] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0033] Figure 1 A schematic diagram of a process flow of a neural network reasoning method based on a storage-computing architecture provided by an embodiment of the present invention is shown;

[0034] Figure 2 A schematic diagram of weight data expansion provided by an embodiment of the present invention is shown;

[0035] Figure 3 A schematic diagram of expanding an input feature map provided by an embodiment of the present invention is shown;

[0036] Figure 4A schematic diagram of a process of traversing an input vector group and weight data provided by an embodiment of the present invention is shown;

[0037] Figure 5 A schematic diagram of the structure of a neural network reasoning system provided by an embodiment of the present invention is shown;

[0038] Figure 6 A data expansion schematic diagram provided by an embodiment of the present invention is shown;

[0039] Figure 7 A schematic block diagram of the composition of a neural network reasoning device based on a storage-computation architecture provided by an embodiment of the present invention is shown;

[0040] Figure 8 A schematic block diagram of the composition of a neural network reasoning electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0041] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0042] In order to solve the problem that the current model reasoning method requires frequent updates of weight data, which weakens the advantages of the integrated storage and computing chip, the embodiment of the present invention provides a neural network reasoning method based on the storage and computing architecture, such as Figure 1 As shown, the method includes:

[0043] S101, determining weight data and input vector group;

[0044] The above step S101 also includes S1011, S1012 and S1013:

[0045] S1011. Traverse all storage computing units in the storage computing array to generate a storage computing array routing table, wherein the storage computing array routing table is used to determine the number of storage computing units and the storage space information of the storage computing units; traverse the computation graph to expand the weight tensor of each node in the computation graph into a weight vector.

[0046] Initialize the system: The central processor reads the system configuration information, neural network calculation graph and weight data from the initial file, and the storage and computing controller traverses all storage and computing units in the storage and computing array to generate a storage and computing array routing table.

[0047] Exemplarily, the system configuration information includes display interface setting information, input source setting information, etc. The neural network calculation graph is used to indicate the input and output of the model, as well as the flow of data in the model. The weight data includes integer weights, data bit width, tensor dimension, quantization factor and quantization zero point, etc. The storage array routing table is used to record the address of each unit in the storage array, the size of storage space and other configurable information.

[0048] Further, such as Figure 2 As shown in the figure, the steps of expanding the weight tensor of each node in the computation graph into a weight vector are as follows: first, split the weight tensor into Ouc×Inc groups of k×k weight matrices, and then reshape the weight matrix into k 2 The weight vector of . That is, we get a vector containing Ouc×Inc with a capacity of k 2 Then, the weight data contained in the weight vector is written into the cache area, and the weight data is distributed to each unit by row by the storage controller. When the storage space required for the weight data is greater than the storage limit, the storage controller is responsible for updating the weight data from the cache area. The cache area is implemented using embedded memory (DDR4 RAM) to ensure fast reading and writing speed.

[0049] S1012. Based on the number of the above-mentioned storage and computing units, the storage space information of the above-mentioned storage and computing units, the size of the above-mentioned weight data and the data bit width of the above-mentioned weight data, the above-mentioned weight data is written into the cache area and the storage and computing array.

[0050] S1013, preprocessing the image data to obtain an input feature map, wherein the image data is obtained based on an image file and / or an acquisition device; expanding the input feature map into an input vector group; and forwarding the input vector group to the cache area based on an embedded memory.

[0051] Exemplarily, the preprocessing steps are: first, perform gamma transformation, normalization and other operations on the input, then use a scalar operator to quantize, use a data reshaping module to perform dimensional transformation, and obtain an input feature map, which is expanded as follows: Figure 3 shown.

[0052] Furthermore, the above method of expanding the input feature map into an input vector group mainly includes two steps of group expansion and interpolation, which are as follows:

[0053] Assume the input dimension is [1,c,h,h], and group it by channel. That is, divide it into c groups of h×h feature matrices. For each feature matrix, perform the following operations:

[0054] 1. Convert the matrix into a one-dimensional vector, containing a total of h×h=N elements;

[0055] 2. Insert (k-1) elements between two adjacent elements. The value of the elements is as follows:

[0056] Assuming that the index values ​​of adjacent elements are i and i+1, the subscript values ​​of inserted elements are: i+k, i+2k, ..., i+(k-1)h, where k is the side length of the convolution kernel.

[0057] 3. Repeat step 2 until the insertion is completed, obtain the input feature vector, and write it into the cache.

[0058] An input vector group, a total of c input vectors of M elements, where M is calculated as follows:

[0059]

[0060] S102, traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two of the weight data;

[0061] The above step S102 further includes S1021:

[0062] S1021, traversing the input vector group and the weight data based on a preset stride and a preset unit length until the weight vector is updated to determine an output vector. The preset stride is determined based on the amplitude of the convolution, and the preset unit length is determined based on the size of the convolution kernel.

[0063] The above steps of traversing the input vector group and weight data based on the preset stride and the preset unit length until the weight vector is updated are:

[0064] 1. Expand the input feature map according to the step S1013 to obtain an input vector group;

[0065] 2. Set pointer P to the first address of the input vector and offset S to the data bit width;

[0066] 3. Read k from the input vector group 2 elements, reuse them and input them into the storage array, fill the output into the corresponding position of the output matrix, and set the pointer to P+S·k;

[0067] 4. Repeat step 3 until all input vector groups and weight data are traversed, and the output matrix is ​​reshaped as the input feature map of the next layer;

[0068] 5. Repeat the above steps until the feature map of the network output layer, that is, the above output vector, is obtained.

[0069] For example, suppose a network contains one convolution, the number of input channels is 3, the number of output channels is 16, the sliding amplitude is 1, the padding value is 0, and the quantization width is 4 bits. The storage capacity of the storage array is 64×64=4096 bits, which can store 16 4-bit weight data with a capacity not exceeding 64 bits.

[0070] Expand the weights into 48 vectors, each with a dimension of 1×9, and each vector requires 36 bits of storage space. Write the weights into the cache and write the first 16 rows into the storage array. Assuming the dimension of the input tensor is [1,3,32,32], the inference steps are as follows:

[0071] 1. Expand the input feature map to obtain an input vector group, which contains 3 input vectors with 192 elements.

[0072] 2. Set pointer P to point to the first address of the feature vector and set the offset to 4;

[0073] 3. Traverse the input vectors from the buffer, read 9 at a time and reuse them (i.e. the preset unit length mentioned above), slide by 3 (i.e. the stride mentioned above), write the output to the buffer, and set the pointer to P+12;

[0074] 4. Update the weight data and repeat step 2 until the weight vector is updated. After the update is completed, the output vector is obtained and reshaped to [1, 16, 30, 30] as the input feature map of the next layer;

[0075] 5. Network inference is completed, and the output feature map of the output layer is obtained.

[0076] S103, reshape the data matrix based on the above output vector and neural network calculation graph to forward it to the output device.

[0077] The above step S103 further includes S1031:

[0078] S1031. When the reshaped data matrix comes from a storage and computation array, perform data type conversion on the reshaped data matrix.

[0079] For example, when the weight and input are both 4-bit integer data, the result of the multiplication and accumulation is 9-bit integer data, which needs to be converted into 32-bit integer data that can be recognized by the central processing unit.

[0080] Another expression of the above method implementation is shown below:

[0081] Traversing all storage computing units in the storage computing array to generate a storage computing array routing table, wherein the storage computing array routing table is used to determine the number of storage computing units and storage space information of the storage computing units;

[0082] Traversing a computation graph to expand a weight tensor of each node in the computation graph into a weight vector, wherein the weight vector is composed of a plurality of weight data;

[0083] Based on the number of the storage and computing units, the storage space information of the storage and computing units, the size of the weight data, and the data bit width of the weight data, the weight data is written into a cache area and a storage and computing array;

[0084] Preprocessing image data to obtain an input feature map, wherein the image data is obtained based on an image file and / or an acquisition device;

[0085] Expanding the input feature map into an input vector group;

[0086] forwarding the input vector group to the cache area based on the embedded memory;

[0087] Traversing the input vector group and the weight data based on a preset stride and a preset unit length until the weight vector is updated to determine an output vector, wherein the preset stride is determined based on the amplitude of the convolution, and the preset unit length is determined based on the size of the convolution kernel;

[0088] Reshape the data matrix based on the output vector and the neural network computation graph;

[0089] forwarding the reshaped data matrix to an output device;

[0090] In the case where the reshaped data matrix comes from a storage array, data type conversion is performed on the reshaped data matrix.

[0091] Furthermore, the neural network reasoning system is composed of at least one central processor, at least one data communication module, at least one scalar operator supporting high parallelism, multiple vector operators, at least one data reshaping module, an embedded memory for creating a cache area, a storage and computing controller, and a storage and computing array containing a plurality of storage and computing units. The storage and computing units are interconnected in the NoC manner, and the storage and computing controller implements access and mode control of each unit in the storage and computing array through row addresses (RAS) and column addresses (CAS).

[0092] The central processing unit is responsible for interacting with external devices of the system, such as receiving input from sensors, outputting data to display devices, and reading, writing and controlling data of modules connected to it;

[0093] Scalar operator, which performs scalar multiplication and addition operations and supports integer and floating-point data;

[0094] Vector operator, which performs activation, pooling, bias addition, etc., and supports integer and floating-point data;

[0095] The data reshaping module performs the expansion and folding of network weights and inputs, that is, expanding tensors into vectors and folding vectors into tensors;

[0096] The data communication module is responsible for reading and writing data between the CPU and the cache, including writing the expanded network weights and input vectors, and reading the output of the storage array;

[0097] Storage and computing controller, responsible for:

[0098] a) Read and write data to the cache;

[0099] b) Access and control any storage and computing unit in the storage and computing array.

[0100] Furthermore, a specific embodiment of the present application is shown below:

[0101] In order to verify the feasibility of the proposed neural network reasoning system and method based on the storage and computing integrated architecture, LeNet is used for MNIST test (handwritten digit recognition) and ResNet18 is used for Cifar100 test (color image classification) in the embodiment.

[0102] The MNIST data set is verified using LeNet, and the network structure is as follows:

[0103] Table 1 Mnist_LeNet network structure table

[0104]

[0105] The network has 44.426K parameters and 281.64K multiplication and accumulation calculations.

[0106] The Cifar100 dataset is verified using ResNet18, and the network structure is as follows:

[0107] Table 2 Cifar100_ResNet18 network structure table

[0108]

[0109]

[0110] Note: AdaptiveAvgPool is implemented using the CPU

[0111] The network has 11.181642M parameters and 555.4736M multiplication and accumulation calculations.

[0112] The network is trained using the quantization-aware-training method (QAT), and an 8-bit network model with an accuracy of 75.5% is obtained.

[0113] Then, according to the method, model inference is performed to obtain the test results shown in the following table:

[0114] Table 3 Network inference verification results of the proposed method

[0115]

[0116] Note: The tested storage and computing chip only supports 1 to 8 bit integer data.

[0117] Through the above technical scheme, the neural network reasoning method and related equipment based on the storage-computing architecture provided by the present invention can solve the problem that the current model reasoning method needs to frequently update the weight data, which weakens the advantages of the storage-computing integrated chip. The present invention determines the weight data and the input vector group; traverses the above input vector group and the above weight data until the weight vector is updated to determine the output vector, wherein the above weight vector is composed of at least two of the above weight data; and reshapes the data matrix based on the above output vector and the neural network calculation graph to forward it to the output device. In the above scheme, compared with the commonly used Im2Col method, the amount of data during transmission can be effectively reduced, so that there is no need to frequently update the weight data, and the effect of efficient reasoning of the neural network model is achieved.

[0118] The neural network reasoning method based on the storage-computing chip and the storage-computing architecture proposed in the embodiment of the present application can effectively reduce the amount of data during transmission compared with the commonly used Im2Col method, and the reduction ratio is:

[0119]

[0120] In particular, when S=1 and Padding=0, the amount of data can be reduced as follows:

[0121]

[0122] For example: Figure 6 As shown, matrix A is the weight matrix, the convolution kernel size is 2×2, and the convolution stride is 1, and B is the feature matrix, the size is 3×3. Then A is expanded into a one-dimensional vector containing 4 elements, and B is expanded and the value is inserted to obtain an input feature vector with a capacity of 12.

[0123] Compared to Im2Col, it can reduce:

[0124]

[0125] Furthermore, as a response to the above Figure 1 In order to realize the method shown in the figure, the embodiment of the present invention also provides a neural network reasoning device based on a storage and computing architecture, which is used to Figure 1 The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. Figure 7 As shown, the device includes: a determination unit 21, a traversal unit 22 and a reshaping unit 23, wherein

[0126] A determination unit 21, used to determine weight data and an input vector group;

[0127] A traversal unit 22, used for traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two weight data;

[0128] The reshaping unit 23 is used to reshape the data matrix based on the above output vector and the neural network calculation graph so as to forward it to the output device.

[0129] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and a neural network reasoning method based on storage and computing architecture can be implemented by adjusting kernel parameters, which can solve the problem that the current model reasoning method requires frequent updates of weight data, thus weakening the advantages of storage and computing integrated chips.

[0130] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, and when the program is executed by a processor, it implements the above-mentioned neural network reasoning method based on the storage and computing architecture.

[0131] An embodiment of the present invention provides a processor, which is used to run a program, wherein the neural network reasoning method based on the storage and computing architecture is executed when the program is running.

[0132] An embodiment of the present invention provides an electronic device, the electronic device comprising at least one processor and at least one memory connected to the processor; wherein the processor is used to call program instructions in the memory to execute the neural network reasoning method based on the storage-computing architecture as described above

[0133] An embodiment of the present invention provides an electronic device 30, such as Figure 8As shown, the electronic device includes at least one processor 301, and at least one memory 302 and a bus 303 connected to the processor; wherein the processor 301 and the memory 302 communicate with each other through the bus 303; the processor 301 is used to call the program instructions in the memory to execute the above-mentioned neural network reasoning method based on the storage-computing architecture.

[0134] The intelligent electronic devices in this article can be PCs, PADs, mobile phones, etc.

[0135] The present application also provides a computer program product, which, when executed on a process management electronic device, is suitable for executing a program that initializes the steps of the above-mentioned neural network reasoning method based on the storage and computing architecture.

[0136] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device is caused to execute the following Figure 1 This corresponds to the flow of control of the memory in the embodiment.

[0142] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0148] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A neural network reasoning method based on a storage and computing architecture, characterized in that: include: Determine weight data and input vector group; Traversing the input vector group and the weight data until the weight vector is updated to determine the output vector, wherein the weight vector is composed of at least two of the weight data; reshape the data matrix based on the output vector and the neural network computation graph to forward to the output device; Traversing all storage computing units in the storage computing array to generate a storage computing array routing table, wherein the storage computing array routing table is used to determine the number of storage computing units and storage space information of the storage computing units; Traversing a computation graph to expand a weight tensor of each node in the computation graph into a weight vector; The step of determining weight data and inputting a vector group comprises: Based on the number of the storage and calculation units, the storage space information of the storage and calculation units, the size of the weight data and the data bit width of the weight data, the weight data is written into the cache area and the storage and calculation array: Preprocessing image data to obtain an input feature map, wherein the image data is obtained based on an image file and / or an acquisition device; Expanding the input feature map into an input vector group; The input vector group is forwarded to a cache area based on the embedded memory.

2. The method according to claim 1, characterized in that The traversing the input vector group and the weight data until the weight vector is updated to determine the output vector includes: The input vector group and the weight data are traversed based on a preset stride and a preset unit length until the weight vector is updated to determine an output vector.

3. The method according to claim 2, characterized in that The preset stride is determined based on the amplitude of the convolution, and the preset unit length is determined based on the size of the convolution kernel.

4. The method according to claim 1, characterized in that: Also includes: In the case where the reshaped data matrix comes from a storage array, data type conversion is performed on the reshaped data matrix.

5. A neural network reasoning device based on a storage and computing architecture, characterized in that: include: A determination unit, used for determining weight data and an input vector group; A traversal unit, used for traversing the input vector group and the weight data until the weight vector is updated to determine an output vector, wherein the weight vector is composed of at least two weight data; A reshaping unit, for reshaping a data matrix based on the output vector and the neural network computation graph to forward it to an output device; Traversing all storage computing units in the storage computing array to generate a storage computing array routing table, wherein the storage computing array routing table is used to determine the number of storage computing units and storage space information of the storage computing units; Traversing a computation graph to expand a weight tensor of each node in the computation graph into a weight vector; The step of determining weight data and inputting a vector group comprises: Based on the number of the storage and calculation units, the storage space information of the storage and calculation units, the size of the weight data and the data bit width of the weight data, the weight data is written into the cache area and the storage and calculation array: Preprocessing image data to obtain an input feature map, wherein the image data is obtained based on an image file and / or an acquisition device; Expanding the input feature map into an input vector group; The input vector group is forwarded to a cache area based on the embedded memory.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed by a processor, a neural network reasoning method based on a storage-computing architecture as described in any one of claims 1 to 5 is implemented.

7. An electronic device, characterized in that: The electronic device includes at least one processor and at least one memory connected to the processor; wherein the processor is used to call program instructions in the memory to execute the neural network reasoning method based on the storage-computing architecture as described in any one of claims 1 to 5.

Images