Data processing method, device and equipment applied to in-memory computing chip

Abstract

The present disclosure provides a data processing method, device and equipment applied to an in-memory computing chip. The data processing method comprises: converting current input data into current target data; in response to the current target data, performing weight distribution on a plurality of convolution layers corresponding to the in-memory computing chip based on a standard convolution layer in accordance with a set weight distribution rule; and in response to the weight distribution, performing convolution processing on the current target data in parallel through the plurality of convolution layers. Therefore, based on the above data processing, the time utilization rate of the non-volatile memory can be significantly improved, thereby realizing an in-memory computing chip with a super-pipeline design in-memory computing architecture, so as to greatly improve the throughput of the in-memory computing accelerator based on the non-volatile memory.

Description

Technical Field

[0001] This disclosure relates to the field of integrated circuit technology, and in particular to a data processing method, apparatus and device for in-memory computing chips. Background Technology

[0002] Traditional deep neural network processors face significant challenges to energy efficiency and throughput due to the slowdown of Moore's Law and the von Neumann bottleneck. In-memory computing accelerators based on non-volatile memory (NVM) can efficiently perform vector-matrix multiplication operations based on Ohm's and Kirchhoff's laws, which aligns with the requirements of deep neural network inference tasks, where over 90% of operations are vector-matrix multiplications. Therefore, NVM-based NVM accelerators can improve the energy efficiency and throughput of deep neural network inference. A key feature of NVM-based NVM accelerators compared to other hardware accelerators is weight pre-storage. For convolutional neural network inference tasks, all model weights are pre-programmed into the conductance state of non-volatile memory. In deep convolutional neural networks, the output feature map of the current layer also serves as the input feature map for the next layer. If the computation of the next layer doesn't begin until the entire output feature map of the current layer is complete, most hardware resources will remain idle for extended periods. This leads to unnecessary waste of hardware resources and reduces the throughput of NVM-based NVM accelerators. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] To address at least one of the technical problems caused by the low utilization of hardware resources in existing deep convolutional neural networks, this disclosure provides a data processing method, apparatus, and device for in-memory computing chips, aiming to significantly improve the time utilization of non-volatile memory, thereby realizing an in-memory computing chip with a super pipelined in-memory computing architecture, and greatly improving the throughput of in-memory computing accelerators based on non-volatile memory.

[0005] (II) Technical Solution

[0006] One aspect of this disclosure provides a data processing method for an in-memory computing chip, comprising: converting current input data into current target data; in response to the current target data, assigning weights to multiple convolutional layers corresponding to the in-memory computing chip based on standard convolutional layers in accordance with a set weight allocation rule; and in response to the weight allocation, performing convolution processing on the current target data through the multiple convolutional layers.

[0007] According to an embodiment of this disclosure, the conversion of current input data into current target data includes: converting the three-dimensional current input data into the one-dimensional current target data; wherein the current target data includes one-dimensional input data and one-dimensional output data.

[0008] According to an embodiment of this disclosure, the process of converting the current input data into the current target data further includes: converting the four-dimensional weight data of the in-memory computing chip into two-dimensional weight matrix data based on a rectangular weight mapping rule, wherein the two-dimensional weight matrix data is used to perform convolution processing on the current target data.

[0009] According to embodiments of this disclosure, before assigning weights to multiple convolutional layers corresponding to the in-memory computing chip in response to the current target data according to a set weight allocation rule, the method further includes: selecting any one of the multiple convolutional layers of the in-memory computing chip as a first reference convolutional layer; performing operation delay traversal on the first reference convolutional layer and other convolutional layers other than the first reference convolutional layer according to the set weight allocation rule; selecting any one of the multiple convolutional layers other than the first reference convolutional layer as a second reference convolutional layer; performing operation delay traversal on the second reference convolutional layer and other convolutional layers other than the second reference convolutional layer according to the set weight allocation rule, and repeating this process until the reference convolutional layer corresponding to the one with the smallest product of the total operation delay and the total area of ​​the convolutional layer slice units is determined as the standard convolutional layer.

[0010] According to an embodiment of this disclosure, in responding to the current target data, the weight allocation of multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layer in accordance with the set weight allocation rules includes: based on the weight allocation rules, copying the weights of the corresponding convolutional layers whose total operation latency is greater than the total operation latency of the standard convolutional layer, and allocating them to at least one slice unit to be executed in the corresponding convolutional layer.

[0011] According to embodiments of this disclosure, each of the at least one executable slice unit includes at least two executable macro units, each corresponding to at least one matrix multiplication and addition operation, and a second executable macro unit corresponding to at least one matrix multiplication and addition operation. In the process of performing convolution processing on the current target data through the plurality of convolutional layers in response to the weight allocation, the process includes: executing at least one matrix multiplication and addition operation of the first executable macro unit and at least one matrix multiplication and addition operation of the second executable macro unit in parallel within the same execution cycle.

[0012] According to embodiments of this disclosure, the step of performing convolution processing on the current target data through the plurality of convolutional layers in response to the weight allocation includes: performing convolution processing on the first input data of the current target data through a first convolutional layer of the plurality of convolutional layers in response to the weight allocation to generate first convolution result data; and performing convolution processing on the first convolution result data through a second convolutional layer of the plurality of convolutional layers in response to the first convolution result data to generate second convolution result data, while the first convolutional layer performs convolution processing on the second input data of the current target data.

[0013] According to an embodiment of this disclosure, in response to the weight allocation, the first convolutional layer of the plurality of convolutional layers performs convolution processing on the first input data of the current target data to generate the first convolution result data, including: loading the first input data, and performing at least one matrix multiplication and addition operation corresponding to each macrounit to be executed in each of the in-memory computing chips corresponding to the first convolutional layer; generating the sum of the multiplication and addition operation results of the at least one matrix multiplication and addition operation as the first convolution result data.

[0014] Another aspect of this disclosure provides a data processing apparatus for an in-memory computing chip, comprising a data conversion module, a weight allocation module, and a convolution processing module. The data conversion module converts current input data into current target data; the weight allocation module, in response to the current target data, allocates weights to multiple convolutional layers corresponding to the in-memory computing chip according to a set weight allocation rule based on standard convolutional layers; and the convolution processing module, in response to the weight allocation, performs convolution processing on the current target data in parallel through the multiple convolutional layers.

[0015] Another aspect of this disclosure provides an electronic device including one or more processors and a memory; the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the methods of embodiments of this disclosure.

[0016] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods of embodiments of this disclosure.

[0017] Another aspect of this disclosure provides a computer program including computer-executable instructions that, when executed, implement the methods of embodiments of this disclosure.

[0018] (III) Beneficial Effects

[0019] A data processing method and apparatus for in-memory computing chips based on embodiments of this disclosure can realize an in-memory computing architecture with a super pipeline design. This architecture can effectively and greatly improve the hardware resource utilization of existing in-memory computing architectures, realize true parallel multiply-accumulate operations, and greatly improve the throughput of in-memory computing neural network accelerators based on non-volatile memory. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a data processing method applied to an in-memory computing chip according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 2 The schematic diagram illustrates the architectural composition of an in-memory computing chip according to an embodiment of the present disclosure;

[0022] Figure 3 A data conversion diagram according to an embodiment of the present disclosure is illustrated schematically;

[0023] Figure 4A A schematic diagram illustrating a convolutional neural network data processing pipeline according to an embodiment of the present disclosure is shown.

[0024] Figure 4B A timing diagram of a data processing pipeline for a convolutional neural network according to an embodiment of the present disclosure is illustrated schematically.

[0025] Figure 5 The illustration shows the corresponding embodiment according to this disclosure. Figure 4A and Figure 4B The timing diagram shown is for input data access and vector-matrix multiplication calculations;

[0026] Figure 6 The illustration shows the corresponding embodiment according to this disclosure. Figure 4A and Figure 4B The diagram shows the mixed intra-chip / inter-chip allocation in the data processing pipeline.

[0027] Figure 7 The illustration shows the corresponding embodiment according to this disclosure. Figure 4A and Figure 4B The diagram shows a matrix multiplication and addition operation of a macrocell in an in-memory computing chip within a data processing pipeline.

[0028] Figure 8A The schematic illustration shows a portion of the instruction flow for corresponding input data access and vector-matrix multiplication calculations according to embodiments of the present disclosure;

[0029] Figure 8B The diagram schematically illustrates the delay results of a pipeline optimized with ResNet18 according to an embodiment of the present disclosure.

[0030] Figure 9 This schematically illustrates an architecture diagram of a data processing apparatus applied to an in-memory computing chip according to an embodiment of the present disclosure;

[0031] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0033] It should be noted that implementations not illustrated or described in the accompanying drawings or the main text of the specification are all forms known to those skilled in the art and are not described in detail. Furthermore, the definitions of the various elements and methods described above are not limited to the specific structures, shapes, or methods mentioned in the embodiments, and those skilled in the art can easily modify or substitute them.

[0034] It should also be noted that the directional terms mentioned in the embodiments, such as "up," "down," "front," "back," "left," and "right," are only for reference to the directions in the accompanying drawings and are not intended to limit the scope of protection of this disclosure. Throughout the drawings, the same elements are represented by the same or similar reference numerals. Conventional structures or constructions will be omitted where they may cause confusion in understanding this disclosure.

[0035] Furthermore, the shapes and dimensions of the components in the figures do not reflect actual size and proportion, but are merely illustrative of embodiments of this disclosure. Additionally, any reference numerals placed between parentheses in the claims should not be construed as limiting the scope of the claims.

[0036] Furthermore, the word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0037] The use of ordinal numbers such as "first," "second," "third," etc., in the specification and claims to modify the corresponding elements does not imply that the element has any ordinal number, nor does it represent the order of one element with another element or the order of manufacturing methods. The use of these ordinal numbers is only to enable a named element to be clearly distinguished from another element with the same name.

[0038] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose. Furthermore, in the unit claims enumerating several means, several of these means may be embodied by the same hardware item.

[0039] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the disclosure, in the foregoing description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the aspects of the disclosure consist of fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the disclosure.

[0040] To address at least one of the technical problems caused by the low utilization of hardware resources in existing deep convolutional neural networks, this disclosure provides a data processing method, apparatus, and device for in-memory computing chips, aiming to significantly improve the time utilization of non-volatile memory, thereby realizing an in-memory computing chip with a super pipelined in-memory computing architecture, and greatly improving the throughput of in-memory computing accelerators based on non-volatile memory.

[0041] like Figure 1 and Figure 2 As shown, one aspect of this disclosure provides a data processing method applied to an in-memory computing chip, which includes steps S101-S103.

[0042] In step S101, the current input data is converted into the current target data;

[0043] In step S102, in response to the current target data, weights are allocated to multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layers, conforming to a set weight allocation rule; and

[0044] In step S103, in response to the weight assignment, convolution processing is performed on the current target data in parallel through the multiple convolutional layers.

[0045] like Figure 2 As shown in this embodiment, the overall architecture of the in-memory computing chip can mainly include peripheral main memory (i.e., main memory), a chip array module composed of multiple chip units, a control unit, a special function unit, and an L2 buffer (i.e., a cache buffer). Specifically, the connection of the above components can be achieved through a chip mesh network distribution rule. Each chip unit typically consists of multiple macrocells, an L1 buffer, and an adder, which can be connected based on an H-tree topology distribution rule. Further, each macrocell mainly includes a non-volatile memory cross array, an input register, an output register, a row multiplexer, a column multiplexer, an analog-to-digital converter, and an adder. The non-volatile memory cross array can be composed of multiple non-volatile memory transistors arranged in parallel and cross configurations. It should be noted that the above... Figure 2 The in-memory computing chip architecture shown is merely a preferred architecture for an embodiment of this disclosure. Those skilled in the art should understand that the in-memory computing chip described in this embodiment is not based on this preferred architecture. Figure 2 The architecture shown is limited.

[0046] The current input data can be the input feature map data received by the aforementioned in-memory computing chip, and the current target data can be the two-dimensional transformation data of the input feature map data. This can significantly reduce the convolution latency of in-memory computing and speed up the computing speed. For details, please refer to the following description.

[0047] This invention implements convolution processing using an in-memory computing chip with multiple convolutional layers. Each convolutional layer performs vector multiplication and addition operations based on set weights and input data to achieve convolution. Unlike existing technologies where each convolutional layer has fixed weights, the weights of these convolutional layers in this invention are assigned different weights based on a set weight allocation rule. This set weight allocation rule refers to the correspondence between each convolutional layer and its corresponding weights within the in-memory computing chip; it can be understood as "how to allocate weights to each convolutional layer (specifically, this may include how many times weights are allocated, what type of weights are allocated, and what range of weights are allocated)," and can be defined based on the total convolutional latency of each convolutional layer. The allocation can be implemented based on data mapping. A standard convolutional layer is a reference convolutional layer corresponding to the set weight allocation rule; in other words, weights can be allocated to each convolutional layer in the in-memory computing chip based on the total convolutional latency corresponding to the standard convolutional layer.

[0048] Specifically, in this embodiment, given that the convolutional latency of each convolutional layer differs in the parallel computing neural network corresponding to the aforementioned in-memory computing chip, based on the aforementioned weight allocation rules, the weights of layers with larger latency (which can be understood as bottleneck layers) can be copied and reasonably allocated to multiple slices (i.e., slice units) for parallel computing to accelerate the computation process of the bottleneck layer. The latency of each convolutional layer can be determined based on the latency of the standard convolutional layer. For example, if the latency of a convolutional layer is greater than that of the standard convolutional layer, the aforementioned weight allocation rules are met, and corresponding weight allocation is performed; otherwise, no weight allocation is performed. The aforementioned weight allocation rules, as one of the improvements to the data processing method in this embodiment, can achieve optimal weight allocation for each convolutional layer, ultimately reducing hardware idle time and achieving a more balanced pipeline design.

[0049] Therefore, based on the convolutional neural network convolutional layer corresponding to the in-memory computing chip with the weight allocation completed above, the speed of convolution processing can be further improved on the basis of the current target data transformation, ensuring that a true pipelined processing process (i.e., super pipeline design) can be realized, greatly improving the utilization of hardware resources and accelerating the parallel processing speed of in-memory computing.

[0050] In other words, the data processing method applied to in-memory computing chips based on the above-described embodiments of this disclosure can realize an in-memory computing architecture with a super pipeline design. This architecture can effectively and greatly improve the hardware resource utilization of existing in-memory computing architectures, realize true parallel multiply-accumulate operations, and greatly improve the throughput of in-memory computing neural network accelerators based on non-volatile memory.

[0051] like Figures 1-3As shown, according to an embodiment of this disclosure, the conversion of current input data into current target data includes: converting the three-dimensional current input data into the one-dimensional current target data; wherein the current target data includes one-dimensional input data and one-dimensional output data.

[0052] According to an embodiment of this disclosure, the process of converting the current input data into the current target data further includes: converting the four-dimensional weight data of the in-memory computing chip into two-dimensional weight matrix data based on a rectangular weight mapping rule, wherein the two-dimensional weight matrix data is used to perform convolution processing on the current target data.

[0053] like Figure 3 The diagram shown illustrates the convolution weight mapping transformation and input data transformation, where C... in It is the number of input channels, C out It is the number of output channels, k is the kernel size, and F is the number of output channels. i F is the size of the input feature map of the current input data. o This represents the size of the output feature map corresponding to the current output data of the current input data. Using data transformation rules based on matrix-vector multiplication, the three-dimensional current input and output data can be transformed into one-dimensional input and output vector data as the current target data. This transformation process can be specifically implemented based on matrix-vector multiplication calculations corresponding to the current input and output data.

[0054] Furthermore, the rectangular weight mapping rule is a transformation rule for mapping four-dimensional weight data to two-dimensional weight matrix data, used to achieve weight data transformation. Based on the above transformation for the current input data, such as... Figure 3 As shown, it enables C out Three-dimensional convolutional kernels (k, k, C) in Mapped in matrix form to, for example Figure 2 The non-volatile memory cross array (k×C) shown out , k×C in Therefore, in the specific multiplication and addition operation, each time the window slides, k×C in The input feature map of size is loaded from the buffer and input into the cross array, with each input feature point corresponding to a k×C line in the same row. out The size weights are multiplied, and k×C is calculated for the same column. in The sum of the multiplication results is obtained by adding the partial sums, and the sums of the partial sums obtained by sliding the window k times are added together to obtain the final output feature map.

[0055] Therefore, by leveraging the aforementioned convolution weight mapping and the transformation process of the current input data, the transformation of the current target data can be achieved. This reduces the number of feature points that need to be read from the cache each time to 1 / k of the original number, thereby significantly reducing cache access time. In other words, the amount of data read from the cache in a single computation is greatly reduced.

[0056] Therefore, the above-described new convolution weight mapping design scheme, as one of the improvements in the embodiments of this disclosure, can effectively achieve the following: Figure 4A and Figure 4B The amount of data access is significantly reduced, thereby reducing the latency of each operator (convolution).

[0057] like Figures 1-3 As shown, according to an embodiment of this disclosure, before step S102, in response to the current target data, performing weight allocation on the multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layer according to a set weight allocation rule, the method further includes:

[0058] Select any one of the multiple convolutional layers in the in-memory computing chip as the first reference convolutional layer; perform delayed traversal of the first reference convolutional layer and other convolutional layers other than the first reference convolutional layer one by one based on the set weight allocation rules;

[0059] Select any one of the convolutional layers in the in-memory computing chip other than the first reference convolutional layer as the second reference convolutional layer; based on the set weight allocation rules, perform operation delay traversal on the second reference convolutional layer and the other convolutional layers in the multiple convolutional layers other than the second reference convolutional layer one by one, and repeat this process until the reference convolutional layer corresponding to the minimum product of the total operation delay and the total area of ​​the convolutional layer slice unit is determined as the standard convolutional layer.

[0060] In determining the standard convolutional layer, a reference convolutional layer needs to be predefined. Based on this reference convolutional layer, the operation latency of each of the remaining convolutional layers in the in-memory computing chip is calculated, thereby determining the total operation latency of each convolutional layer in the in-memory computing chip. For each convolutional layer, the total operation latency can be the sum of the operation latency of data readout (IOAD), write-back (WORK), and computation (execute, i.e., EXE execution) for each macrocell of the corresponding convolutional layer. This total operation latency is also related to the area of ​​the slice cell in the corresponding convolutional layer (the larger the area of ​​the convolutional layer, the smaller its corresponding latency).

[0061] The reference convolutional layer is any selected convolutional layer among the multiple convolutional layers in the aforementioned in-memory computing chip. It is used to iterate through the operation latencies of the other multiple convolutional layers based on the aforementioned set weight allocation rules to obtain the total operation latency of each convolutional layer. Specifically, the total operation latencies of the other multiple convolutional layers can be compared one by one with the reference convolutional layer. If the total operation latency of one of the multiple convolutional layers is greater than or equal to the total operation latency of the reference convolutional layer, then the corresponding set weight allocation rules are satisfied. For example, given three convolutional layers A, B, and C, one convolutional layer A can be chosen as the first reference convolutional layer. Operation delay traversal is then performed on each of the three convolutional layers A, B, and C sequentially. Next, another convolutional layer C is chosen (it cannot be A again), and operation delay traversal is performed on each of the three convolutional layers A, B, and C sequentially again. Finally, a convolutional layer B is selected, and operation delay traversal is performed on each of the three convolutional layers A, B, and C sequentially. From the above three traversals, the reference convolutional layer corresponding to the minimum product of the total operation delay and the total area of ​​the convolutional layer's unit slices is selected as the standard convolutional layer. In this way, the reference convolutional layer corresponding to the minimum product of the total operation delay and the total area of ​​the convolutional layer's unit slices among multiple convolutional layers, including the reference convolutional layer, can be determined, and this convolutional layer is used as the standard convolutional layer.

[0062] like Figures 1-4B As shown, according to an embodiment of this disclosure, in step S102, in response to the current target data, weight allocation is performed on the multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layers, conforming to a set weight allocation rule, including:

[0063] Based on the set weight allocation rules, the weights of the corresponding convolutional layers whose total operation latency is greater than that of the standard convolutional layer are copied and assigned to at least one slice unit to be executed in the corresponding convolutional layer.

[0064] Specifically, the weight allocation rules for multiple convolutional layers can be defined based on the following hybrid on-chip / inter-chip pipeline allocation algorithm for convolutional neural network in-memory computing chips, and the execution of this weight allocation can also be implemented based on the following algorithm:

[0065]

[0066]

[0067] Where, N p N represents the number of weight parameters mapped to a single chip; i N represents the number of slice units required for a single layer. s The total number of weights in a single layer; k is the size of the neural network convolution kernel; F is the size of the neural network feature map. min Feature map size of a standard convolutional layer; M cH is the number of times the weights are copied; H is the length of the expanded weights; W is the width of the expanded weights. In the above convolutional neural network memory computing chip hybrid on-chip / inter-chip pipeline allocation algorithm, F is defined as... map Function, M c That is, the number of times the weights are copied, (converting four-dimensional to two-dimensional) H×W is the number of weights corresponding to this convolutional layer, N s It is the maximum number of weights for that convolutional layer after copying, i.e., the total number of weights in a single convolutional layer; in addition, H t ×W t It is the maximum weight of a single slice unit; N p The maximum number of weights in a single slice unit of this layer, i.e., the number of weight parameters mapped to a single slice; N i N is the number of slice units required for a single convolutional layer. s Let F be the total number of weights in this convolutional layer. decouple This function is used to read input data, write output data back, and calculate the delay T. load T wirte T exe Evaluate the operational latency of a single chip unit.

[0068] Based on the above hybrid intra-chip / inter-chip pipeline allocation algorithm, the delay of any layer of the convolutional neural network (L) is used as the delay (T) of the overall system. min When ), evaluate the product of the delayed areas P at this time. LA (System delay T) min With area cost ∑N i (l) product). If the product of the delay areas of layers L is P LA If the delay T of layer L is minimized, then... min To optimize the latency of the pipeline, it can be used as the standard convolutional layer mentioned above. When evaluating the latency and area of ​​each layer in the network, firstly, according to the rectangular weight mapping scheme F... map The convolutional kernels are mapped onto a non-volatile memory cross array, and then the timing F with inter-layer access and execution decoupling is computed. decouple The corresponding operating latency T(l) of this network layer. If T(l) is longer than T... min If the value is large, then calculate the acceleration factor required for that layer in the pipeline design, i.e., the number of weights that need to be replicated (M). c Thus, the aforementioned weight allocation rules can be defined. Then, based on these rules, the number of weight replications for each convolutional layer, the mapped slice units, and their corresponding quantities can be determined, thereby optimizing the number of weights (N) mapped into the slice. p (l) and inter-piece weights (N) i (l)) such that the delay T(l) corresponding to the convolutional layer is less than T min .

[0069] Therefore, by combining the above-described hybrid intra-chip / inter-chip allocation algorithm, a set weight allocation rule can be defined for the embodiments of this disclosure. In this way, the set weight allocation rule can be used to further optimize the "obtaining greater throughput benefits with smaller area costs".

[0070] like Figure 4A The convolutional neural network pipeline shown includes convolutional layers L1, L2, and L3. Given the latency differences between layers in the parallel computing network, the weights of the layer with the larger latency (bottleneck layer) are copied and rationally distributed across multiple slices, thereby achieving parallel computation of matrix multiplication and addition to accelerate the bottleneck layer. Specifically, as shown... Figure 4A As shown, the weights of convolutional layer L1 are copied 16 times and mapped to slice units 1-4. Furthermore, the weights of convolutional layer L2 are copied 4 times and mapped to slice units 5-6. Through the above weight allocation design, the idle time of the hardware can be significantly reduced, the utilization of hardware resources can be greatly improved, and a more balanced pipeline design can be achieved.

[0071] By using the weight allocation design based on the set weight allocation rules, the operation latency of each convolutional layer can be well balanced, thereby realizing a super pipeline of multiple convolutional layers. Ultimately, this can greatly reduce the idle time of the hardware and achieve a more balanced pipeline design.

[0072] Specifically, such as Figure 6 The system design diagram shown is a hybrid intra-chip / inter-chip pipeline design that comprehensively considers throughput efficiency and area cost, and balances latency differences between different convolutional neural network layers through a suitable intra-chip / inter-chip allocation scheme. For example, even if each chip can accommodate four vector matrix multiplication units VMM1, VMM2, VMM3, and VMM4, the weight allocation design scheme of the present disclosure still maps the four vector matrix multiplication units VMM1, VMM2, VMM3, and VMM4 to two chip units (i.e., chip 1 and chip 2) respectively. Each chip can map two vector matrix multiplication units simultaneously, such as chip 1 mapping vector matrix multiplication units VMM1 and VMM2, and chip 2 mapping vector matrix multiplication units VMM3 and VMM4. This reduces the amount of computation required for a single slice unit, thereby reducing its latency. Consequently, the total latency of operations in the convolutional layer allocated to that slice unit is no greater than the latency of a standard convolutional layer, truly achieving parallel computation of vector-matrix multiplication. Mapping more vector-matrix multiplication units within a single slice unit significantly increases intra-slice latency, while a larger number of slice units significantly increases area. Therefore, there can be an optimal solution for the specific hybrid intra-slice / inter-slice allocation, achieving the maximum throughput gain with the minimum area cost.

[0073] Therefore, based on this weight allocation design, a true super pipeline for convolutional neural networks can be established, thereby achieving parallel computing of each convolutional layer of the network to significantly improve the time utilization of hardware resources and reduce the overall system latency.

[0074] like Figures 1-4B As shown, according to an embodiment of the present disclosure, each of the at least one executable slice unit includes at least two executable macro units, including a first executable macro unit corresponding to at least one matrix multiplication and addition operation and a second executable macro unit corresponding to at least one matrix multiplication and addition operation. In the process of performing convolution processing on the current target data through the plurality of convolutional layers in response to the weight allocation, the process includes: executing at least one matrix multiplication and addition operation of the first executable macro unit and at least one matrix multiplication and addition operation of the second executable macro unit in parallel within the same execution cycle.

[0075] like Figure 8A The diagram illustrates the instruction flow for parallel inter-layer data access and computation according to an embodiment of this disclosure. This instruction flow is specifically implemented using a single instruction multiple data (SIM) instruction. Here, VMM stands for vector-matrix multiplication, ld.perform loads data from the cache into the macrocell, ex.perform performs vector-matrix multiplication in the macrocell, and wr.perform writes the macrocell output data back to the cache. Specifically, first, an instruction is sent to write the output result of macrocell 1 back to the cache. Since the output result has not yet been calculated initially, this instruction can be ignored initially. Next, an instruction is sent to read the input data X1 for the next computation cycle, i.e., ld.perform(X1). Then, an instruction is sent to execute VMM1, i.e., ex.perform(VMM1). During the execution of VMM1, instructions can continue to be sent to write back the output data of macrocell 2, then read the input data X2 for the next cycle, i.e., ld.perform(X2), and then execute VMM2, i.e., ex.perform(VMM2). Therefore, for the same convolutional layer L1, each macrounit to be executed can correspond to a different vector matrix multiplication unit (VMM) at different execution cycles, and each VMM is used to perform the corresponding vector matrix multiplication operation. Thus, instructions can generally be sent sequentially to write back the output data of each macrounit, read out the input data, and then execute the operation again. Furthermore, since multiple macrounits to be executed can be listed side-by-side and each macrounit corresponds to multiple VMMs, this instruction design can also be extended to a single-instruction multithreaded mode, controlling the operation of multiple slice units simultaneously with a single instruction.

[0076] like Figure 5The data access and computation timing diagram shown in this embodiment of the present disclosure illustrates that, within the same computation cycle, different macrounits can correspond to different vector matrix multiplication units, thereby implementing different vector matrix multiplications. For example, within cycle 1, the vector matrix multiplication unit VMM1 of the corresponding macrounit 1 can be based on reading (T) from the L1 cache of the convolutional layer. load The input data X1 is used to perform vector-matrix multiplication (T). exe ), and write back the output data Y1(T) write Meanwhile, during the execution of the aforementioned vector matrix multiplication unit VMM1, the vector matrix multiplication unit VMM2 of macrounit 2 can already perform vector matrix multiplication operations based on the input data X2 from the convolutional layer L1 cache, and write back the output data Y2 in cycle 2. Therefore, through the above design of decoupling data access and computation, data access and computation execution are allowed to overlap in time. For example, reading data X2 from the cache and macrounit 1 performing VMM1 (vector matrix multiplication) computation are executed in parallel.

[0077] When the macrocell computation delay (T) exe ) is greater than or equal to the total buffer access latency (N) p -1)×(T load +T write (N) p When the number of macrocells is N, the computational latency will account for the majority of the total latency. p The total delay (T) of each macrounit over M convolution cycles total ) is (T exe +T load +T write )×M+(T load +T write )×(N p -1). Conversely, buffer access latency will dominate the total latency, N per slice unit. p The total delay (T) of a macrounit over M convolution cycles total ) is (T load +T write )×N p ×M+(T load +T write )×(Np-1), specifically corresponding to the following algorithm:

[0078]

[0079] Therefore, as another improvement of the embodiments of this disclosure, by decoupling data access and vector matrix multiplication calculation as described above, data reading, data writing back and vector matrix multiplication calculation can be highly overlapped in time (operated simultaneously), thereby greatly reducing the latency of each chip.

[0080] Furthermore, such as Figure 4B The timing diagram of the convolutional neural network pipeline shown demonstrates that for different convolutional layers L1, L2, and L3, slice unit 1 can perform at least two operations in parallel within the same execution cycle: data readout, data writeback, and vector-matrix multiplication. This significantly reduces the idle time of each slice unit and greatly improves the efficiency of hardware resource utilization.

[0081] Therefore, based on the weight allocation design defined by the above algorithm, a true super pipeline for convolutional neural networks can be established, thereby achieving parallel computing of each convolutional layer of the network to significantly improve the time utilization of hardware resources and reduce the overall system latency.

[0082] like Figures 1-4B As shown, according to an embodiment of this disclosure, in step S103, in response to the weight allocation, performing convolution processing on the current target data through the plurality of convolutional layers includes:

[0083] In response to the weight assignment, the first convolutional layer of the plurality of convolutional layers performs convolution processing on the first input data of the current target data to generate the first convolution result data;

[0084] In response to the first convolution result data, a second convolutional layer through the plurality of convolutional layers performs convolution processing on the first convolution result data to generate a second convolution result data, while the first convolutional layer performs convolution processing on the second input data of the current target data.

[0085] like Figure 4A As shown, for convolutional layers L1, L2, and L3 that have completed weight allocation, after the first convolutional layer L1 completes the convolution operation on the first input data of the current target data, the first convolution result data generated by the convolution operation is used as the input data of the second convolutional layer L2 and the convolution process continues through the second convolutional layer L2. Furthermore, the second convolution result data generated by the convolution operation is further used as the input data of the third convolutional layer L3 and the convolution process continues.

[0086] Meanwhile, during the process of the second convolutional layer L2 performing convolution processing on the first convolution result data, the first convolutional layer L1 can continue to perform parallel convolution processing with the second input data of the current target data. Similarly, during the process of the third convolutional layer L3 performing convolution operations, the second convolutional layer L2 also performs convolution operations in parallel at the same time... In this way, it can be ensured that the super pipeline of the convolutional neural network including the above convolutional layers L1, L2 and L3 can be built in the convolutional neural network of the corresponding in-memory computing chip, and each convolutional layer can perform parallel convolution operations relative to the adjacent convolutional layers.

[0087] In other words, by performing convolution processing on the current target data based on the above multiple convolutional layers, it is possible to truly realize multi-threaded parallel super pipeline convolution operations, ensuring that almost all convolutional layers are in the operation execution process during convolution processing, thereby greatly improving the resource utilization of convolutional layers, significantly improving the execution efficiency of parallel operations, greatly increasing the throughput of the entire system, and reducing the computation time.

[0088] like Figures 1-4B As shown, according to an embodiment of this disclosure, in response to the weight allocation, the first convolutional layer of the plurality of convolutional layers performs convolution processing on the first input data of the current target data to generate the first convolution result data, including:

[0089] Load the first input data, and perform at least one matrix multiplication and addition operation corresponding to each macrounit to be executed in each of the in-memory computing chips corresponding to the first convolutional layer;

[0090] The sum of the results of the multiplication and addition operations of at least one matrix is ​​generated as the first convolution result data.

[0091] To more clearly illustrate the implementation process of convolution, such as Figure 7 The circuit implementation scheme of the convolution weight mapping method shown assumes that the convolution kernel size is k = 3 × 3, and represents an output value (Y) in one output channel. 11 (This means that k×C can be obtained simultaneously with each window swipe). out The size of the output feature points. In period 1, (X 11 X 21 X 31 The data is loaded from the buffer and input into the non-volatile memory cross array 701, and then compared with the weights (w) of the cross array 701. 11 w 21 w 31 Perform multiplication and addition calculations, where the output current I... 11 1 Converted from analog-to-digital (ADC) to digital signal Y 11 1 And store it in register 1. In cycle 2, load (X) 12 X 22 X 32 ), and with weights (w) 12 w 22 w 32 Perform multiplication and addition calculations, then convert the analog-to-digital signal Y to obtain the digital signal. 11 2 With Y stored in register 1 11 1Add them together, and Y 11 1+2 Stored in register 2. Similarly, in cycle 3, by adding Y... 11 3 and Y 11 1+2 The final output Y is obtained. 11 1+2+3 The results are stored in register 3. In other words, each output result can generally be obtained by adding the results of k consecutive multiplication and addition operations on adjacent k columns.

[0092] Therefore, the data processing method applied to in-memory computing chips described in the embodiments of this disclosure can construct an in-memory computing chip architecture with a super pipeline design, thereby greatly improving the time utilization of hardware and increasing the throughput of the entire system.

[0093] like Figure 8B The comparison chart of pipeline latency results after ResNet18 optimization shows that by gradually introducing matrix mapping methods, access and computation decoupling methods, and hybrid intra-chip / inter-chip design, the latency of each layer conv1, conv2_x, conv3_x, conv4_x, and conv5_x gradually decreases, and the latency between each layer gradually becomes more balanced, thus demonstrating a significant improvement in system throughput.

[0094] like Figure 9 As shown, another aspect of this disclosure provides a data processing apparatus 900 applied to an in-memory computing chip, which includes a data conversion module 910, a weight allocation module 920, and a convolution processing module 930.

[0095] The data conversion module 910 is used to convert the current input data into the current target data. Specifically, the data conversion module 910 can be used to achieve, for example... Figure 1 The method of step S101 in the process shown will not be described in detail here.

[0096] The weight allocation module 920 is used to respond to the current target data and, based on the standard convolutional layers, allocate weights to multiple convolutional layers corresponding to the in-memory computing chip according to a set weight allocation rule. The weight allocation module 920 can be used to implement, for example... Figure 1 The method of step S102 in the process shown will not be described in detail here.

[0097] The convolution processing module 930 is used to perform convolution processing on the current target data through the multiple convolutional layers in response to the weight assignment. The convolution processing module 930 can be used to implement, for example... Figure 1 The method of step S103 in the process shown will not be described in detail here.

[0098] It is important to note that Figure 9 The examples shown are merely examples of data processing apparatus 900 applied to an in-memory computing chip that can be applied to embodiments of this disclosure, to help those skilled in the art understand the technical content of this disclosure, but do not mean that embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. Furthermore, the embodiments of the data processing apparatus 900 applied to an in-memory computing chip are similar in manner to the embodiments of the data processing method applied to an in-memory computing chip, and the achieved technical effects are also similar, and will not be repeated here.

[0099] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown schematically.

[0100] Another aspect of this disclosure provides an electronic device including one or more processors and a memory; the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the methods of embodiments of this disclosure.

[0101] Figure 10 A block diagram of an electronic device according to an embodiment of the present disclosure is shown schematically. Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0102] like Figure 10 As shown, a computer system 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage portion 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0103] RAM 1003 stores various programs and data required for the operation of system 1000. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Processor 1001 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1002 and / or RAM 1003. It should be noted that the program may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in such one or more memories.

[0104] According to embodiments of this disclosure, system 1000 may further include an input / output (I / O) interface 1005, which is also connected to bus 1004. System 1000 may also include one or more of the following components connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1008 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0105] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by processor 1001, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0106] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0107] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1002 and / or RAM 1003 and / or one or more memories other than ROM 1002 and RAM 1003 described above.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods of embodiments of this disclosure.

[0110] Specifically, the computer-readable storage medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the data processing method applied to an in-memory computing chip according to the embodiments of this disclosure.

[0111] Alternatively, the computer-readable storage medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0112] Another aspect of this disclosure provides a computer program including computer-executable instructions that, when executed, implement the data processing method applied to an in-memory computing chip according to embodiments of this disclosure.

[0113] The embodiments of this disclosure have now been described in detail with reference to the accompanying drawings.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] Those skilled in the art will understand that although this disclosure has been shown and described with reference to specific exemplary embodiments thereof, they should understand that various changes in form and detail may be made to this disclosure without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents. Therefore, the scope of this disclosure should not be limited to the above embodiments, but should be determined not only by the appended claims, but also by their equivalents.

[0116] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of this disclosure. It should be understood that the above are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A data processing method applied to an in-memory computing chip, wherein, include: Transform the current input data into the current target data; In response to the current target data, weights are allocated to multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layers, in accordance with the set weight allocation rules. as well as In response to the weight allocation, convolution processing is performed on the current target data in parallel through the multiple convolutional layers; Before assigning weights to the multiple convolutional layers corresponding to the in-memory computing chip according to a set weight allocation rule based on the standard convolutional layer in response to the current target data, the method further includes: Select any one of the multiple convolutional layers in the in-memory computing chip as the first reference convolutional layer; perform delayed traversal on the first reference convolutional layer and the other convolutional layers in the multiple convolutional layers except the first reference convolutional layer one by one based on the set weight allocation rule; Select any one of the convolutional layers in the in-memory computing chip other than the first reference convolutional layer as the second reference convolutional layer; based on the set weight allocation rule, perform operation delay traversal on the second reference convolutional layer and the other convolutional layers in the multiple convolutional layers other than the second reference convolutional layer one by one, and repeat this process until the reference convolutional layer corresponding to the minimum product of the total operation delay and the total area of ​​the convolutional layer slice unit is determined as the standard convolutional layer.

2. The method of claim 1, wherein, The process of converting the current input data into the current target data includes: The current input data in three dimensions is converted into the current target data in one dimension; The current target data includes one-dimensional input data and one-dimensional output data.

3. The method of claim 1, wherein, The process of converting the current input data into the current target data also includes: Based on the rectangular weight mapping rule, the four-dimensional weight data of the in-memory computing chip is converted into two-dimensional weight matrix data, which is used to perform convolution processing on the current target data.

4. The method of claim 1, wherein, In the process of responding to the current target data and assigning weights to multiple convolutional layers corresponding to the in-memory computing chip based on standard convolutional layers in accordance with a set weight allocation rule, the following is included: Based on the set weight allocation rules, the weights of the corresponding convolutional layers whose total operation latency is greater than that of the standard convolutional layer are copied and assigned to at least one slice unit to be executed in the corresponding convolutional layer.

5. The method of claim 4, wherein, In the at least one executable slice unit, each executable slice unit includes at least two executable macro units, comprising a first executable macro unit corresponding to at least one matrix multiplication-addition operation and a second executable macro unit corresponding to at least one matrix multiplication-addition operation, wherein the process of performing convolution processing on the current target data through the plurality of convolutional layers in response to the weight allocation includes: Within the same execution cycle, at least one matrix multiplication and addition operation of the first macro unit to be executed and at least one matrix multiplication and addition operation of the second macro unit to be executed are executed in parallel.

6. The method of claim 1, wherein, In the step of performing convolution processing on the current target data through the plurality of convolutional layers in response to the weight assignment, the following is included: In response to the weight allocation, a convolution process is performed on the first input data of the current target data through the first convolutional layer of the plurality of convolutional layers to generate the first convolution result data; In response to the first convolution result data, a second convolution layer of the plurality of convolutional layers performs convolution processing on the first convolution result data to generate second convolution result data, while the first convolutional layer performs convolution processing on the second input data of the current target data.

7. The method of claim 6, wherein, In the process of performing convolution processing on the first input data of the current target data through the first convolutional layer of the plurality of convolutional layers in response to the weight allocation, to generate the first convolution result data, the following is included: Load the first input data, and based on the corresponding weights in each executable chip unit of the in-memory computing chip corresponding to the first convolutional layer, perform at least one matrix multiplication and addition operation corresponding to each executable macrounit in each executable chip unit; The sum of the results of the multiplication and addition operations of the at least one matrix is ​​generated as the first convolution result data.

8. A data processing device applied to an in-memory computing chip, wherein, include: The data conversion module is used to convert the current input data into the current target data. The weight allocation module is used to, in response to the current target data, allocate weights to multiple convolutional layers corresponding to the in-memory computing chip based on the standard convolutional layer, in accordance with the set weight allocation rules. as well as The convolution processing module is used to perform convolution processing on the current target data through the multiple convolutional layers in response to the weight allocation; The weight allocation module is also used for: Select any one of the multiple convolutional layers in the in-memory computing chip as the first reference convolutional layer; perform delayed traversal on the first reference convolutional layer and the other convolutional layers in the multiple convolutional layers except the first reference convolutional layer one by one based on the set weight allocation rule; Select any one of the convolutional layers in the in-memory computing chip other than the first reference convolutional layer as the second reference convolutional layer; Based on the set weight allocation rule, the operation delay is traversed one by one for the second reference convolutional layer and the other convolutional layers among the plurality of convolutional layers except for the second reference convolutional layer. This process is repeated until the reference convolutional layer corresponding to the minimum product of the total operation delay and the total area of ​​the convolutional layer slice unit is determined as the standard convolutional layer.

9. An electronic device, wherein, include: One or more processors; as well as A memory for storing one or more programs; wherein, when the one or more programs are executed by one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the method of any one of claims 1-7.

Images