Storage and computing integrated computing device and neural network training system

Abstract

The present application relates to a storage-computation integrated computing device and a neural network training system, wherein the storage-computation integrated computing device includes: a preloading module, a mixed precision input module, an input control module and a storage-computation integrated array module; the preloading module is used to preload the input data and output the to-be-processed data to the mixed precision input module; the mixed precision input module is used to serially output the to-be-processed data output by the preloading module according to the preset precision requirement; the input control module is used to output the row selection signal and the column selection signal to the storage-computation integrated array module; the storage-computation integrated array module is used to read, write and perform calculation processing on the data serially output by the mixed precision input module based on the row selection signal and the column selection signal output by the input control module. It can provide more precision options for different neural network algorithms, and improve the flexibility and applicability of the storage-computation integrated device.

Description

Technical Field

[0001] The present application relates to the field of intelligent computing, and in particular to a storage-computing integrated computing device and a neural network training system. Background Art

[0002] With the development of semiconductor technology, the integration density of chips has also increased rapidly. Miniaturizing the chip size is difficult to meet the demand for increasing data volume in the context of big data. The performance gap between storage units and computing units in the von Neumann architecture is also increasing, and the problem of "storage wall" is becoming increasingly prominent. At present, computing systems based on storage-computing integrated architecture are often used to minimize the time and energy consumption of data transmission. Non-volatile memory can perform multiplication and accumulation calculations on the input data of the storage-computing integrated array and the stored data to achieve highly parallel and energy-efficient convolution calculations.

[0003] The core of the current storage-computing technology lies in the multiplication-accumulation engine. The storage capacity of the storage-computing array unit can store weights and perform highly parallel multiplication-accumulation calculations on the stored weights and inputs. Since the current storage-computing technology is mostly in the form of fixed input precision to match the neural network calculation applicable to a single study, it lacks flexibility and applicability in terms of precision input.

[0004] There is currently no effective solution to the problem of insufficient flexibility and applicability of precision input in related technologies. Summary of the invention

[0005] In this embodiment, a storage-computing integrated computing device and a neural network training system are provided to solve the problem of insufficient flexibility and applicability in precision input in related technologies.

[0006] In a first aspect, a storage-computation integrated computing device is provided in this embodiment, and the storage-computation integrated computing device is used for neural network computing; the storage-computation integrated computing device comprises: a preloading module, a mixed precision input module, an input control module, and a storage-computation integrated array module; wherein the preloading module, the mixed precision input module, the input control module, and the storage-computation integrated array module are connected in sequence; the input end of the preloading module is connected to the output end of the associated data storage device;

[0007] The preloading module is used to perform preloading processing on the input data and output the data to be processed to the mixed precision input module;

[0008] The mixed precision input module is used to serially output the to-be-processed data output by the preload module according to a preset precision requirement;

[0009] The input control module is used to output a row selection signal and a column selection signal to the storage and computing integrated array module;

[0010] The integrated storage and computing array module is used to perform reading, writing and computing processing on the data serially output by the mixed precision input module based on the row selection signal and column selection signal output by the input control module.

[0011] In some of the embodiments, the preloading module includes a first configuration register and a plurality of preloading units; each of the preloading units includes a first register group;

[0012] The first configuration register is used to configure a gating parameter for the preload unit;

[0013] Each of the preloading units is used to temporarily store single-cycle data of the data storage device; the preloading unit has a preset data loading mode and a preset data shifting mode; wherein, in the data loading mode, the preloading unit is used to load data based on the selection parameters for the preloading unit, and perform segmented selection and temporary storage on the loaded data; in the data shifting mode, the preloading unit is used to perform an overall shift operation of data on the first register group.

[0014] In some of the embodiments, the input data bit width of each of the preloading units is the same as the output data bit width of the data storage; the input data bit width of a single register in the preloading unit is determined based on the input data bit width of the preloading unit and the number of registers in the preloading unit.

[0015] In some of the embodiments, the mixed precision input module includes a second configuration register, a row strobe unit, an address decoding unit, and a plurality of parallel-input and serial-output units;

[0016] The second configuration register is at least used to configure a precision mode parameter, an address start bit parameter, and a channel selection parameter;

[0017] The address decoding unit is used to determine the selected parallel-input and serial-output units for data output according to the address start bit parameter and the channel selection parameter;

[0018] The selected parallel-input and serial-output unit is used to output the to-be-processed data received from the preload module in a corresponding precision mode according to the precision mode parameter after precision selection processing in a periodic serial manner; different precision modes correspond to different serial output periods;

[0019] The row strobe unit is used for parallel-serializing the output data of the parallel-input-serial-output unit.

[0020] In some of the embodiments, the parallel-in serial-out unit includes a second register group and a high-order zero detection subunit; the high-order zero detection subunit is used to detect the number of high-order zeros on the output data of the second register group according to the precision mode, and generate a jump signal according to the detection result to perform a jump operation on the output data of the second register group.

[0021] In some embodiments, the mixed precision input module has an INT2 precision mode, an INT4 precision mode, and an INT8 precision mode; wherein:

[0022] In the INT2 precision mode, each register in the single parallel input and serial output unit outputs 2 bits of data in series;

[0023] In the INT4 precision mode, every two registers in a single parallel-input and serial-output unit are connected to output 4 bits of data;

[0024] In the INT8 precision mode, every four registers in a single serial input and output unit are connected to output 8 bits of data.

[0025] In some of the embodiments: the high-order zero detection subunit is further used to perform the high-order zero number detection on the output data of each register in the single parallel-input and serial-output unit in the INT2 precision mode;

[0026] The high-order zero detection subunit is also used to detect the number of high-order zeros in the output data of one of the two registers in the single parallel-input and serial-output unit that has high-order zero detection enabled, in the INT4 precision mode;

[0027] The high-order zero detection subunit is also used to detect the number of high-order zeros in the output data of one of the four registers in the single parallel-input and serial-output unit that has high-order zero detection enabled in the INT8 precision mode.

[0028] In some embodiments, the input control module includes a read-write input control unit, a calculation input control unit, and a row and column address decoding unit; wherein:

[0029] The read-write input control unit is used to provide the storage-computation integrated array module with row and column address selection signals corresponding to the read-write weights;

[0030] The calculation input control unit is used to provide the storage and calculation integrated array module with row and column address selection signals corresponding to the calculation;

[0031] The address decoding unit is used to generate row and column address selection signals corresponding to the storage-computation-in-one array module according to a preset address start signal and a row and column address channel signal generated by the read-write input control unit.

[0032] In some of the embodiments, the integrated storage and computing array module is formed by a plurality of memory units arranged crosswise; the memory units include non-volatile memory devices.

[0033] In a second aspect, a neural network training system is provided in this embodiment, comprising the storage-computing integrated computing device described in the first aspect above; a target neural network model to be trained is deployed on the storage-computing integrated computing device.

[0034] Compared with the related art, a storage-computation integrated computing device and a neural network training system are provided in this embodiment. The storage-computation integrated computing device includes: a preloading module, a mixed precision input module, an input control module and a storage-computation integrated array module; wherein the preloading module, the mixed precision input module, the input control module and the storage-computation integrated array module are connected in sequence; the input end of the preloading module is connected to the output end of the associated data storage device; the preloading module is used to preload the input data and output the data to be processed to the mixed precision input module; the mixed precision input module is used to serially output the data to be processed output by the preloading module according to the preset precision requirement; the input control module is used to output the row selection signal and the column selection signal to the storage-computation integrated array module; the storage-computation integrated array module is used to perform reading, writing and calculation processing on the data serially output by the mixed precision input module based on the row selection signal and the column selection signal output by the input control module. It can provide more precision options for different neural network algorithms, and improve the flexibility and applicability of the storage-computation integrated device.

[0035] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0037] Figure 1 It is a structural block diagram of a storage-computing integrated computing device according to an embodiment of the present application;

[0038] Figure 2 is a structural schematic diagram of a preload module in an embodiment of the present application;

[0039] Figure 3 is a schematic diagram of a shift operation in an embodiment of the present application;

[0040] Figure 4 is a schematic diagram of the structure of a mixed precision input module according to an embodiment of the present application;

[0041] Figure 5 1 is a schematic diagram of data output of the serial-input and serial-output unit in different precision modes according to an embodiment of the present application;

[0042] Figure 6 is a schematic diagram of the output of a row gating unit in different precision modes according to an embodiment of the present application;

[0043] Figure 7a is a schematic diagram of input of the mixed precision input module in the INT2 precision mode of an embodiment of the present application;

[0044] Figure 7b 1 is a schematic diagram of input of the mixed precision input module in the INT4 precision mode of an embodiment of the present application;

[0045] Figure 7c 1 is a schematic diagram of input of the mixed precision input module in the INT8 precision mode of an embodiment of the present application;

[0046] Figure 8a 1 is a schematic diagram of detection of the high-bit zero detection subunit in the embodiment of the present application for the INT2 precision mode;

[0047] Figure 8b This is a detection schematic diagram of the high-bit zero detection subunit for the INT4 precision mode according to an embodiment of the present application;

[0048] Figure 8c This is a detection schematic diagram of the high-bit zero detection subunit for the INT8 precision mode according to an embodiment of the present application;

[0049] Fig. 9 is a structural diagram of an input control module in an embodiment of the present application;

[0050] Fig.10 It is a structural schematic diagram of a storage and computing integrated array module according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0053] In this embodiment, a storage and computing integrated computing device is provided. Figure 1 is a structural block diagram of the storage and computing integrated computing device 100 of this embodiment. Figure 1 As shown, the storage-computation integrated computing device 100 is used for neural network computing, and the storage-computation integrated computing device 100 includes: a preloading module 102, a mixed precision input module 104, an input control module 106, and a storage-computation integrated array module 108; wherein the preloading module 102, the mixed precision input module 104, the input control module 106, and the storage-computation integrated array module 108 are connected in sequence; the input end of the preloading module 102 is connected to the output end of the associated data storage device;

[0054] The preloading module 102 is used to preload the input data and output the data to be processed to the mixed precision input module 104; the mixed precision input module 104 is used to serially output the data to be processed output by the preloading module 102 according to the preset precision requirements; the input control module 106 is used to output row selection signals and column selection signals to the integrated storage and computing array module 108; the integrated storage and computing array module 108 is used to perform reading, writing and calculation processing on the data serially output by the mixed precision input module 104 based on the row selection signals and column selection signals output by the input control module 106.

[0055] The storage and computing integrated computing device 100 provided in this embodiment can adapt to neural networks of different precisions. For example, the black and white image recognition of license plates can be calculated based on a neural network with lower precision. At this time, the storage and computing integrated computing device 100 of this embodiment can be used to implement neural network calculations with lower precision. For the recognition or classification of color images, a neural network calculation with higher precision is required. Similarly, the storage and computing integrated computing device 100 of this embodiment can be used to implement neural network calculations with higher precision. The preloading module 102 connects the associated data storage device and the mixed precision input module 104. The input end of the preloading module 102 receives the data in the data storage device and preloads the received input data. The output end of the preloading module 102 is connected to the mixed precision input module 104.

[0056] The mixed precision input module 104 can output the data in series according to different precision selection requirements, so as to input data matching the precision selection requirements to the subsequent modules. The input control module 106 can input corresponding row and column selection signals to the storage and computing integrated array module 108 according to the data reading and writing requirements and calculation requirements. Based on the row and column selection signals output by the input control module 106, the storage and computing integrated array module 108 performs processing such as reading and writing weights of the neural network and performing inference calculations of the neural network on the data output by the mixed precision input module 104.

[0057] The storage-and-computing integrated computing device 100 provided in this embodiment can realize a mixed-precision input function based on storage-and-computing integrated computing, and has strong flexibility and versatility, thereby providing more precision options for different neural network algorithms and being flexibly used in different computing application scenarios.

[0058] In one embodiment, the above-mentioned preloading module 102 includes a first configuration register and multiple preloading units; each preloading unit includes a first register group; the first configuration register is used to configure the selection parameters for the preloading unit; each preloading unit is used to temporarily store single-cycle data of the data storage; the preloading unit has a preset data loading mode and a preset data shifting mode; wherein, in the data loading mode, the preloading unit is used to load data based on the selection parameters for the preloading unit, and perform segmented selection and temporary storage on the loaded data; in the data shifting mode, the preloading unit is used to perform an overall shift operation of the data on the first register group.

[0059] Figure 2 FIG. 1 is a schematic diagram of the structure of a preload module 102 of this embodiment. Figure 2As shown, the preloading module 102 may include a first configuration register and a plurality of preloading units. The first configuration register may set a gating parameter of the preloading unit for gating the preloading unit, and may also include a register gating parameter for gating the register in the preloading unit. Each preloading unit may temporarily store single-cycle data from the data storage device. A single cycle refers to a period of time (i.e., the time between two rising edges) from the rising edge (1) to the falling edge (0) of the clock in a digital circuit controlled by a clock domain, which is a cycle. The specific working modes of the preloading module 102 are divided into a data loading mode and a data shifting mode. In the data loading mode, the preloading module 102 selects to open the corresponding preloading unit for data loading according to the gating parameter for the preloading unit set in the first configuration register, receives data from the memory, and performs cyclic loading in multiple cycles, and at the same time, performs segmented gating and temporary storage of the received data according to the gating parameter for the preloading unit.

[0060] The above-mentioned segmented gating is only used in the loading mode. Assume that the data received from the memory is 32 bits (bit), and a preload unit is selected to receive the 32 bits. The preload unit has 8 registers, and the 32-bit data is split into 8 4-bit data and temporarily stored in 8 registers. In the storage process, not all of them need to be stored, and only a few registers can be selected to store the corresponding 4-bit data. This storage method is the above-mentioned segmented gating. Segmented gating is applicable to the following situations: after several shift modes, some registers will become empty. At this time, the data loading mode can only select these empty registers, and store part of the new 32-bit data in these empty registers, thereby avoiding the new data from covering the non-empty registers. It can be understood that the gating for the preload unit in the data loading mode is not unique, and multiple preload units and the first register group are usually selected together.

[0061] In the data shift mode, the preloading module 102 no longer receives data from the memory, and the data of the first register group in the selected preloading unit is subjected to an overall upward shift operation. Figure 3 is a schematic diagram of a shift operation in this embodiment, such as Figure 3 As shown, the squares from top to bottom are registers 0 to 7 in the first register group. In the data shift mode, data 0 in register 0 is removed, data 1 in register 1 is shifted to register 0, and so on, and the data in register 7 is cleared after the up-shift operation.

[0062] The preloading module provided in this embodiment can be used as a front input module of the storage-computing integrated computing device to implement data loading and data shifting operations.

[0063] In particular, in one embodiment, the input data bit width of each preloading unit is the same as the output data bit width of the data storage device; the input data bit width of a single register in the preloading unit is determined based on the input data bit width of the preloading unit and the number of registers in the preloading unit. In this embodiment, the input data bit width of a single register can be: the input data bit width of the preloading unit ÷ the number of registers contained in the preloading unit.

[0064] In addition, in one embodiment, the mixed precision input module 104 includes a second configuration register, a row selection unit, an address decoding unit, and a plurality of parallel-input and serial-output units; the second configuration register is at least used to configure the precision mode parameters, the address start bit parameters and the channel selection parameters; the address decoding unit is used to determine the selected parallel-input and serial-output units for data output according to the address start bit parameters and the channel selection parameters; the selected parallel-input and serial-output units are used to output the to-be-processed data received from the preload module in a corresponding precision mode according to the precision mode parameters after precision selection processing in a periodic serial manner; different precision modes correspond to different serial output periods; the row selection unit is used to parallelly output the output data of the parallel-input and serial-output units.

[0065] in, Figure 4 is a schematic diagram of the structure of the mixed precision input module 104 of this embodiment. Figure 4 As shown, the mixed precision input module 104 includes a second configuration register, a row selection unit, an address decoding unit and a plurality of parallel-input and serial-output units. The parallel-input and serial-output unit will output the data signal received from the preload module 102 to the input control module 106 in serial in multiple cycles after precision selection processing. The row selection unit is used to perform multiple rows of parallel serial output of the output data of the parallel-input and serial-output unit. The address decoding unit is used to specify the selected parallel-input and serial-output unit for data output according to the address start bit parameter and the channel selection parameter. The number of parallel-input and serial-output units = the row address bit width of the storage and computing integrated array module ÷ the output data bit width of the parallel-input and serial-output unit. As shown in Figure 4 As shown, the second configuration register can also configure the row selection parameter of the parallel-input-serial-output unit. In this embodiment, the parallel-input-serial-output unit can make the second register perform corresponding serial output according to different precision modes.

[0066] In one embodiment, the parallel-in serial-out unit includes a second register group and a high-order zero detection subunit; the high-order zero detection subunit is used to detect the number of high-order zeros in the output data of the second register group according to the precision mode, and generate a jump signal according to the detection result to perform a jump operation on the output data of the second register group.

[0067] The mixed precision input module 104 may include multiple parallel-input and serial-output units; a parallel-input and serial-output unit may be provided with multiple second registers to form a second register group. The high-order zero detection subunit is used to detect the number of high-order zeros of the temporarily stored data in the second register group, starting from the highest bit and detecting 0 one by one until 1 is detected and stops, and taking the minimum value in the detection result, and generating a jump signal for performing a jump operation during input to optimize the redundant cycle caused by the invalid bit and improve the calculation efficiency.

[0068] Assume that in the INT8 mode of integer type with a precision mode of 8 bytes, two parallel input and output units are selected, and two lines of data are output serially. The data output by the first parallel input and output unit is 00011001, and the serial output is 1-0-0-1-1-0-0-0; the data output by the second parallel input and output unit is 00001111, and the serial output is 1-1-1-1-0-0-0-0. In the data output by these two parallel input and output units, the 5th to 7th bits are all 0. At this time, these three 0 bits are invalid output bits. Normally, three cycles are required to serially output these three 0 bits. These three cycles are redundant cycles, which are meaningless in array calculations and only increase the calculation time. Therefore, based on the high-order zero detection subunit of this embodiment, it is possible to detect invalid output bits in the output data, so as to facilitate the jump operation, reduce the calculation time, and improve the calculation efficiency.

[0069] Assume that in INT8 mode, in all the enabled parallel-input and parallel-output units, the minimum value obtained by the high-bit 0 detection is 2, that is, all the enabled parallel-input and parallel-output units will perform serial output xxxxxx-0-0, and the last two bits will output 0. The two bits of 0 here are two invalid bits. At this time, if the output process without skipping is followed, there will be two more cycles to output 0-0 (all the enabled parallel-input and parallel-output units will output 0-0). These two cycles are called redundant cycles, which have no meaning in the calculation of the array and increase the calculation time. In this embodiment, based on the high-bit zero detection of the high-bit zero detection subunit, when a skipping signal is provided, the skipping operation will cut off the two bits of 0-0 when performing serial output, thereby optimizing the calculation efficiency of the array.

[0070] In one embodiment, the mixed precision input module has INT2 precision mode, INT4 precision mode and INT8 precision mode; wherein: in the INT2 precision mode, each register in a single parallel input serial output unit serially outputs 2 bits of data; in the INT4 precision mode, every two registers in a single parallel input serial output unit are connected to output 4 bits of data; in the INT8 precision mode, every four registers in a single parallel input serial output unit are connected to output 8 bits of data.

[0071] Among them, INT2 is a 2-byte integer type; INT4 is a 4-byte integer type; INT8 is an 8-byte integer type. Figure 5 Schematic diagram of data output of the serial-input and serial-output unit in different precision modes of this embodiment. Figure 5 As shown, the input data of a single parallel input and output unit can be set to 8 bits. The 8-bit data is segmented and stored in four 2-bit wide second registers ( Figure 5 In the second register 1 to the second register 4). In the INT2 precision mode, the four second registers respectively output four groups of 2-bit data in series, and the output is completed in two cycles; in the INT4 precision mode, the second registers are connected in pairs, for example, the second register 1 is connected to the second register 2, and the second register 3 is connected to the second register 4, and two groups of 4-bit data are output respectively, and the output is completed in four cycles; in the INT8 precision mode, the four second registers 1, 2, 3, and 4 are connected to output a group of 8-bit data, and the output is completed in eight cycles. Therefore, this embodiment can realize data output of different precisions by controlling the serial output unit to perform different serial outputs.

[0072] Figure 6 Schematic diagram of the output of the row strobe unit in different precision modes of this embodiment. Figure 6 As shown in the figure, for a single parallel-input and serial-output unit, the row selection unit has different row selection output modes in different precision modes. In the INT2 precision mode, different output data can appear in the 4 output channels, and the specified channel output is selected according to the channel selection parameters. In the INT4 precision mode, the data in the 4 output channels are the same, and the specified channel output can be selected according to the channel selection parameters. In the INT8 precision mode, the data in the 4 output channels are consistent, and a single or a single channel parallel serial output is selected according to the channel selection parameters.

[0073] Figure 7a This is a schematic diagram of the input of the mixed precision input module in INT2 precision mode; Figure 7b This is a schematic diagram of the input of the mixed precision input module in INT4 precision mode; Figure 7c This is a schematic diagram of the mixed precision input module in INT8 precision mode. Figures 7a to 7c , when the input data of a single parallel input and output unit is 10010111, the input data is split into 10, 01, 01, 11 and stored in the 4 second registers of the single parallel input and output unit. Each second register has its own serial output channel. In INT2 precision mode, the output channels of the 4 second registers ( Figure 7aChannels 1 to 4 in the serial output unit are all turned on and output in two cycles, where channel 1 sequentially outputs data 0-1, channel 2 sequentially outputs data 1-0, channel 3 sequentially outputs data 1-0, and channel 4 sequentially outputs data 1-1. Based on the row selection parameter of the parallel-in serial output unit, for example, the row selection parameter is 1100, that is, row 1 and row 2 are selected, and row 3 and row 4 are not selected. Then under this row selection parameter, row 1 sequentially outputs data 0-1, row 2 sequentially outputs data 1-0, and row 3 and row 4 both output 0.

[0074] In INT4 precision mode, the output channels of the second register 2 and the second register 4 (corresponding to Figure 7b Channel 2 and channel 4 in the second register are turned on and output in 4 cycles. Channel 2 outputs 1-0-0-1 in sequence, and channel 4 outputs 1-1-1-0 in sequence. After row selection, row 1 and row 2 output the same data, with this data 1-0-0-1, and row 3 and row 4 both output 0. In INT8 precision mode, the output channel of the second register 4 (corresponding to Figure 7c Channel 4 in the output is turned on and outputs in 8 cycles. Channel 4 outputs 1-1-1-0-1-0-0-1 in sequence. After row selection, rows 1 and 2 output 1-1-1-0-1-0-0-1 in sequence, and rows 3 and 4 output 0.

[0075] In addition, in one embodiment, the high-order zero detection subunit is also used to perform high-order zero number detection on the output data of each register in a single parallel-input serial-output unit in the INT2 precision mode; the high-order zero detection subunit is also used to perform high-order zero number detection on the output data of one of every two registers in the single parallel-input serial-output unit with high-order zero detection turned on in the INT4 precision mode; the high-order zero detection subunit is also used to perform high-order zero number detection on the output data of one of every four registers in the single parallel-input serial-output unit with high-order zero detection turned on in the INT8 precision mode.

[0076] Figure 8a This is a detection schematic diagram of the high-bit zero detection subunit of this embodiment for the INT2 precision mode; Figure 8b This is a detection schematic diagram of the high-bit zero detection subunit of this embodiment for the INT4 precision mode; Figure 8c 8a to 8b are schematic diagrams of the detection of the high-order zero detection subunit in this embodiment for the INT8 precision mode. Figure 8cThe high-order zero detection subunit is used to detect the number of high-order zeros in the temporarily stored data in the second register group, starting from the highest bit and detecting 0 one by one until 1 is detected, and taking the minimum value in the detection result, and generating a jump signal based on the minimum value for performing jump operations on the subsequent module inputs, so as to optimize the redundant cycles caused by invalid bits and improve computing efficiency. For a single parallel-input serial-output unit, for example, for parallel-input serial-output unit 1, each of the four second registers will generate a zero detection signal sf. In the INT2 precision mode, each second register will start zero detection from the high bit and output 4 sf signals ( Figure 8a sf1 to sf4 in the 4 sf signals). The minimum value is taken from the 4 sf signals to obtain the number of redundant cycles that can be optimized for a single unit. In the INT4 precision mode, the high-order zero detection operation is started from 2 of the 4 second registers, for example, from the second register 1 and the second register 3, two valid sf signals are output ( Figure 8b sf1 and sf3 in the INT8 precision mode), the second register that does not enable the high-bit zero detection operation separately, the corresponding input sf signal can set the default bit width maximum value. In INT8 precision mode, the high-bit zero detection operation is enabled from one of the four second registers, for example, from the second register 1, and a valid sf signal is input ( Figure 8c sf1 in ). Finally, the minimum value is taken from all the parallel-input and serial-output units to obtain a skip signal, that is, SF=min{SF1, ..., SFn}, where SF1 to SFn are SF signals output by the first parallel-input and serial-output unit to the nth parallel-input and serial-output unit, and the skip signal SFx of the non-selected parallel-input and serial-output unit x is set to the maximum bit width by default. In this way, the present embodiment can implement the corresponding high-bit zero detection operation based on different precision modes, thereby optimizing the redundant cycle caused by the invalid bit and improving the calculation efficiency.

[0077] In one embodiment, the input control module 106 includes a read-write input control unit, a calculation input control unit and a row and column address decoding unit; wherein: the read-write input control unit is used to provide the storage and computing integrated array module with row and column address selection signals corresponding to the read-write weight; the calculation input control unit is used to provide the storage and computing integrated array module with row and column address selection signals corresponding to the calculation; the address decoding unit generates a row and column address selection signal corresponding to the storage and computing integrated array module according to a preset address start signal and a row and column address channel signal generated by the read-write input control unit.

[0078] Fig. 9 is a schematic diagram of the structure of an input control module 106 of this embodiment, as shown in Fig. 9As shown, the input control module can be composed of a read-write input control unit, a calculation input control unit, and a row and column address decoding unit. Among them, the read-write input control unit can give the storage-computation integrated array module a corresponding row and column address selection signal when reading and writing weights; the calculation input control unit can generate a corresponding row and column address selection signal when giving the storage-computation integrated array module to perform reasoning calculations of the neural network; the row and column address decoding unit generates a row and column address selection signal directly corresponding to the storage-computation integrated array module according to the address start signal and the channel signal generated by the read-write control unit. Since the input data bit width of the calculation input control unit corresponds to the row address bit width of the storage-computation integrated array module, row address decoding is not required, and the column address selection is determined by the calculation characteristics of the storage-computation integrated array module, and multiple column selection signals are generated synchronously to the storage-computation integrated array module.

[0079] In one embodiment, the integrated storage and computing array module 108 is formed by a plurality of memory units arranged crosswise; the memory units include non-volatile memory devices. Fig.10 FIG. 1 is a schematic diagram of a storage-computing integrated array module 108 of this embodiment. Fig.10 As shown, the storage-computing integrated array module 108 includes a plurality of memory units ( Fig.10 ), multiple memory cells are arranged crosswise. The word line WL is controlled by the row selection address signal output by the input control module 106, and the bit line BL and the source line SL are controlled by the column selection address signal generated by the input control module 106. In the calculation mode, the data output by the mixed precision input module will be processed into the WL selection signal of the storage and calculation integrated array module for calculation. The memory cells constituting the storage and calculation integrated array module include but are not limited to resistive random access memory (RRAM), magnetic random access memory (MRAM), and FeRAM that uses the characteristics of ferroelectric materials for data storage.

[0080] In this embodiment, a neural network training system is also provided, including the storage-computing integrated computing device 100 provided in any of the above embodiments; a target neural network model with training is deployed on the storage-computing integrated computing device 100.

[0081] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0082] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0083] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0084] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0085] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0086] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A storage and computing integrated computing device, characterized in that: The storage-computation integrated computing device is used for neural network computing; the storage-computation integrated computing device comprises: a preloading module, a mixed precision input module, an input control module and a storage-computation integrated array module; wherein the preloading module, the mixed precision input module, the input control module and the storage-computation integrated array module are connected in sequence; the input end of the preloading module is connected to the output end of the associated data storage device; The preloading module is used to perform preloading processing on the input data and output the data to be processed to the mixed precision input module; The mixed precision input module is used to serially output the to-be-processed data output by the preload module according to a preset precision requirement; The input control module is used to output a row selection signal and a column selection signal to the storage and computing integrated array module; The storage-computation integrated array module is used to perform reading, writing and calculation processing on the data serially output by the mixed precision input module based on the row selection signal and the column selection signal output by the input control module; wherein: The mixed precision input module includes a second configuration register, a row strobe unit, an address decoding unit, and a plurality of parallel-input and serial-output units; The second configuration register is at least used to configure a precision mode parameter, an address start bit parameter, and a channel selection parameter; The address decoding unit is used to determine the selected parallel-input and serial-output units for data output according to the address start bit parameter and the channel selection parameter; The selected parallel-input and serial-output unit is used to output the to-be-processed data received from the preload module in a corresponding precision mode according to the precision mode parameter after precision selection processing in a periodic serial manner; different precision modes correspond to different serial output periods; The row strobe unit is used for parallel-serializing the output data of the parallel-input-serial-output unit.

2. The storage and computing integrated computing device according to claim 1, characterized in that: The preloading module includes a first configuration register and a plurality of preloading units; each of the preloading units includes a first register group; The first configuration register is used to configure a gating parameter for the preload unit; Each of the preloading units is used to temporarily store single-cycle data of the data storage device; the preloading unit has a preset data loading mode and a preset data shifting mode; wherein, in the data loading mode, the preloading unit is used to load data based on the selection parameters for the preloading unit, and perform segmented selection and temporary storage on the loaded data; in the data shifting mode, the preloading unit is used to perform an overall shift operation of data on the first register group.

3. The storage and computing integrated computing device according to claim 2, characterized in that: The input data bit width of each preloading unit is the same as the output data bit width of the data storage; the input data bit width of a single register in the preloading unit is determined based on the input data bit width of the preloading unit and the number of registers in the preloading unit.

4. The storage and computing integrated computing device according to claim 1, characterized in that: The parallel-in serial-out unit includes a second register group and a high-order zero detection subunit; the high-order zero detection subunit is used to detect the number of high-order zeros in the output data of the second register group according to the precision mode, and generate a jump signal according to the detection result to perform a jump operation on the output data of the second register group.

5. The storage and computing integrated computing device according to claim 4, characterized in that: The mixed precision input module has an INT2 precision mode, an INT4 precision mode and an INT8 precision mode; wherein: In the INT2 precision mode, each register in the single parallel input and serial output unit outputs 2 bits of data in series; In the INT4 precision mode, every two registers in a single parallel-input and serial-output unit are connected to output 4 bits of data; In the INT8 precision mode, every four registers in a single serial input and output unit are connected to output 8 bits of data.

6. The storage and computing integrated computing device according to claim 5, characterized in that: The high-order zero detection subunit is also used to perform the high-order zero number detection on the output data of each register in the single parallel-input and serial-output unit in the INT2 precision mode; The high-order zero detection subunit is also used to detect the number of high-order zeros in the output data of one of the two registers in the single parallel-input and serial-output unit that has high-order zero detection enabled, in the INT4 precision mode; The high-order zero detection subunit is also used to detect the number of high-order zeros in the output data of one of the four registers in the single parallel-input and serial-output unit that has high-order zero detection enabled in the INT8 precision mode.

7. The storage and computing integrated computing device according to claim 1, characterized in that: The input control module includes a read-write input control unit, a calculation input control unit and a row and column address decoding unit; wherein: The read-write input control unit is used to provide the storage-computation integrated array module with row and column address selection signals corresponding to the read-write weights; The calculation input control unit is used to provide the storage and calculation integrated array module with row and column address selection signals corresponding to the calculation; The address decoding unit is used to generate row and column address selection signals corresponding to the storage-computation-in-one array module according to a preset address start signal and a row and column address channel signal generated by the read-write input control unit.

8. The storage-computing integrated computing device according to any one of claims 1 to 7, characterized in that: The storage-computing integrated array module is formed by a plurality of memory units arranged crosswise; the memory units include non-volatile memory devices.

9. A neural network training system, characterized in that: It comprises a storage-computing integrated computing device as described in any one of claims 1 to 8; a target neural network model to be trained is deployed on the storage-computing integrated computing device.

Images