In-memory computing deep convolutional computing circuit based on a separable word line SRAM

Abstract

This invention discloses an in-memory deep convolution computation circuit based on separable word-line SRAM, comprising: a separable word-line SRAM near-memory computation array, used to replace part of the data by switching the word line conduction state during convolution kernel shifting, and to perform multiplication and addition operations on single-bit data and weight data stored in the array to generate a partial product; a weight data processing module, used to encode the weight data and rearrange the flow direction of the encoded weight data according to a shift control signal; a shift control module, used to control the conduction state of the word lines inside the separable word-line SRAM near-memory computation array to realize the shifting of the convolution kernel on the feature map, and to reload part of the data during the shifting process; and a result aggregation module, used to aggregate the partial product and output the convolution computation result. This invention reduces memory access overhead and power consumption with controllable circuit complexity, and improves the computational efficiency of deep convolution operations.

Description

Technical Field

[0001] This invention belongs to the field of in-memory computing technology, and in particular relates to an in-memory deep convolution computing circuit based on separable word line SRAM. Background Technology

[0002] Deep convolutional neural networks (CNNs) are widely used in computer vision, image recognition, and speech processing. As the core operation of neural networks, deep convolution operations are computationally intensive and involve frequent memory accesses, becoming a bottleneck restricting the speed and power consumption of neural network operations. Traditional convolution calculations employ a von Neumann in-memory architecture, storing data in memory units such as SRAM. During computation, data needs to be moved from memory units to computation units. This large-scale data movement not only increases power consumption but also introduces memory access latency. With the increasing amount of data required for neural network computations, this approach has become insufficient to meet the demands of real-time neural network computation.

[0003] In-memory computing technology integrates computing and storage units, enabling in-situ computation, reducing data movement, lowering power consumption, and improving computational efficiency. SRAM, as a high-speed storage device, is one of the carriers of in-memory computing architecture. Existing SRAM in-memory computing circuits mostly adopt a traditional 6-transistor cell structure with fixed word and bit line driving methods, making it impossible to achieve synchronous data reading from a single column of SRAM cells. This makes it difficult to support the multi-channel parallel multiplication and accumulation operations required for depth convolution, resulting in reduced computational density and parallelism of in-memory computing. Furthermore, existing SRAM in-memory computing architectures still require frequent data reloading and on-chip data scheduling during convolution kernel sliding. Improving the speed of depth convolution operations and reducing power consumption while controlling circuit complexity has become a key technical challenge for current in-memory computing architectures. Summary of the Invention

[0004] To address the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide an in-memory deep convolution calculation circuit based on separable word line SRAM.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: An in-memory depthwise convolution computation circuit based on separable word-line SRAM includes: A separable word-line SRAM near-memory computing array is used to replace part of the data by switching the word-line conduction state when the convolution kernel is shifted, and to perform multiplication and addition operations on the single-bit data stored in the array and the encoded weight data to generate a partial product; The weight data processing module is used to encode the weight data and rearrange the flow direction of the encoded weight data according to the shift control signal. The shift control module is used to output a shift control signal to control the conduction state of the word lines inside the separable word line SRAM near-memory computing array, thereby shifting the convolution kernel on the feature map and reloading some data during the shifting process. The result aggregation module is used to aggregate the partial products and output the convolution calculation results.

[0006] Furthermore, the separable word-line SRAM near-memory computing array includes a separable word-line 6-transistor SRAM array, a decoding and storage value processing module, and a partial product shift and accumulation module; The separable word-line 6-transistor SRAM array is composed of multiple separable word-line SRAM subarrays. Each separable word-line SRAM subarray corresponds to one bit of int8 format feature map data, and each separable word-line SRAM subarray stores bits 0 to bits 7 of the feature map data respectively. The int8 format n-channel feature map data is split into single-bit data according to bits 0 to bits 7 and then written into each separable word-line SRAM subarray in a vertical arrangement. Each separable word-line SRAM subarray includes multiple 6-transistor SRAM cells arranged in an array, and each 6-transistor SRAM cell has an independent left word line WLL and a right word line WLR. The decoding and storage value processing module is configured as multiple, each corresponding to a separable word line SRAM subarray. The decoding and storage value processing module is used to receive single-bit data read from the corresponding separable word line SRAM subarray, and perform decoding operation on the single-bit data according to the encoded weight data to generate a partial product. There are multiple partial product shift accumulation modules, each corresponding to a separable word line SRAM subarray. The partial product shift accumulation module is used to shift and accumulate the partial products generated by the corresponding separable word line SRAM subarray in multiple cycles.

[0007] Furthermore, the 6-transistor SRAM cell includes a first inverter, a second inverter, a first NMOS transistor, and a second NMOS transistor; The first inverter and the second inverter are cross-coupled to form a data latch structure, the output node of the first inverter is the first storage node, and the output node of the second inverter is the second storage node; The gate of the first NMOS transistor is connected to the left word line WLL, the first end is connected to the first bit line BL, and the second end is connected to the first memory node; The gate of the second NMOS transistor is connected to the right word line WLR, the first end is connected to the second bit line BLB, and the second end is connected to the second memory node; The left word line WLL and the right word line WLR are respectively connected to the corresponding word line strobe signals to independently control the conduction of the left and right halves of each 6-transistor SRAM cell.

[0008] Furthermore, in storage mode, the word line strobe signal WL[i] connects both the left word line WLL[i] and the right word line WLR[i]; in in-memory computing mode, the word line strobe signal WL[2i] connects the left word line WLL[2i+1] and the right word line WLR[2i], and the word line strobe signal WL[2i+1] connects the left word line WLL[2i+1] and the right word line WLR[2i+2]; where i is a non-negative integer representing the row index.

[0009] Furthermore, the weighted data processing module includes: The weight data reloading submodule is used to rearrange the arrangement order of each weight value of the convolution kernel according to the horizontal shift control signal in the shift control signal; The encoding submodule is used to perform radix-4 Booth encoding on the weighted data and generate an encoding control signal to be output to the decoding and storage value processing module.

[0010] Furthermore, the decoding and storage value processing module contains multiple parallel decoding sub-units, each of which corresponds to a weight value in the convolution kernel. The input of each decoding sub-unit is connected to the encoding control signal line, the first bit line BL of the corresponding separable word line SRAM sub-array, and the second bit line BLB of the corresponding separable word line SRAM sub-array, respectively. It is used to receive the encoding control signal and the single bit data read from the corresponding separable word line SRAM sub-array as the multiplicand, and to perform a decoding operation on the multiplicand according to the encoding control signal, and output the corresponding partial product in one cycle.

[0011] Furthermore, the radix-4 Booth encoding method is as follows: the least significant bit of the weight data is padded with 0 to expand it to 9 bits; the data is then grouped four times in a loop, with each group consisting of three bits and an overlap of one bit between adjacent groups, resulting in four intermediate values; each intermediate value corresponds to a three-bit encoded value output. When the intermediate value is 000 or 111, the value of the first encoding bit is "1"; at this time, the second and third encoding bits are invalid. When the middle value is 001 or 010, the values ​​of the first, second, and third coding bits are all "0"; When the middle value is 011, the values ​​of the first and second coding bits are "0", and the value of the third coding bit is "1". When the median value is 100, the value of the first encoding bit is "0", and the values ​​of the second and third encoding bits are "1". When the middle value is 101 or 110, the values ​​of the first and third coding bits are "0", and the value of the second coding bit is "1". BoothT, BoothP, and BoothS signals correspond to the first, second, and third encoded bits of the three-bit encoded value, respectively, thereby representing the output of the three-bit encoded value to the decoding subunit of the decoding and storage value processing module.

[0012] Furthermore, the decoding subunit of the decoding and storage value processing module performs the decoding operation on the multiplicand according to the encoding control signal as follows: When the value of the first encoded bit is "1", the multiplicand is set to zero; When the values ​​of the first, second, and third coding bits are all "0", the multiplicand remains unchanged; When the values ​​of the first and second coding bits are "0" and the value of the third coding bit is "1", the multiplier is shifted left by one bit. When the value of the first encoding bit is "0" and the values ​​of the second and third encoding bits are "1", the multiplicand is taken in two's complement and then shifted left by one bit. When the values ​​of the first and third encoding bits are "0" and the value of the second encoding bit is "1", the multiplicand is taken in two's complement.

[0013] Furthermore, the shift control signal includes a horizontal shift control signal hk_ctrl and a vertical shift control signal vk_ctrl; both the horizontal shift control signal hk_ctrl and the vertical shift control signal vk_ctrl are multi-bit control signals, and the convolution kernel is a K×K convolution kernel, where K≥2; During horizontal shifting, the separable word line 6-tube SRAM array slides the convolution kernel in the horizontal direction according to the different states of hk_ctrl and replaces the feature data of some columns located at the edge of the convolution window accordingly, so as to realize the horizontal shifting control of the K×K convolution kernel; During vertical shifting, the separable word-line 6-transistor SRAM array switches the combination of active word lines according to different states of vk_ctrl, so as to realize the vertical shift control of the K×K convolution kernel without reloading all feature data.

[0014] Furthermore, the result aggregation module is used to shift and align the partial products of different bits according to their bit weights, perform shift accumulation on the numerical bits from bit 0 to bit 6, perform shift subtraction on the sign bit of bit 7, and use a Wallace tree structure to perform multi-level compression accumulation on the multiple partial products, output the convolution calculation result, and output the total_done signal after completion to trigger the next shift convolution.

[0015] This invention employs a separable word-line 6-transistor SRAM array. By utilizing different connection relationships of word-line strobe signals in storage mode and in-memory computation mode, data storage and convolution computation are completed within the same array, reducing data handling and memory access overhead, and lowering power consumption. Feature map data is split into single-bit data and written to each separable word-line SRAM subarray in a vertical arrangement. Each bit is computed in parallel. Combined with radix-4 Booth encoding of the weight data, the number of partial products is reduced from 8 to 4, lowering computational latency. When the convolution kernel is shifted, only the feature data in a portion of the columns located at the edge of the convolution window is replaced. During vertical shift, the read feature data is logically shifted vertically by switching the combination of activated word lines, eliminating the need for a full reload of the feature data. This invention achieves a balance between in-memory computation speed and computational efficiency with controllable area overhead and circuit complexity, and is suitable for fixed-point depthwise convolution operations on int8 data types. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is an architecture diagram of an embodiment of the in-memory deep convolution calculation circuit based on separable word line SRAM according to the present invention.

[0017] Figure 2 This is a schematic diagram of a Booth encoding strategy in one embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the internal circuit structure of a near-memory computing array in one embodiment of the present invention.

[0019] Figure 4 This is a schematic diagram of the dual-mode word line connection relationship of a separable word line 6-transistor SRAM array in one embodiment of the present invention.

[0020] Figure 5 This is a schematic diagram of a separable word line 6-transistor SRAM cell in one embodiment of the present invention.

[0021] Figure 6 This is a schematic diagram of the vertical and horizontal shifting strategy of the convolution kernel in one embodiment of the present invention.

[0022] The diagrams in the instruction manual are labeled as follows: Separable word-line SRAM near-memory computing array 1; weight data processing module 2; shift control module 3; result aggregation module 4; separable word-line SRAM subarray 110; 6-tube SRAM cell 111; decoding and storage value processing module 120; partial product shift accumulation module 130; weight data reloading submodule 210; encoding submodule 220. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. The illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0024] Please see Figure 1 , Figure 1 This is an architectural diagram of an embodiment of the in-memory deep convolution calculation circuit based on separable word-line SRAM according to the present invention. The in-memory deep convolution calculation circuit based on separable word-line SRAM in this embodiment includes a separable word-line SRAM near-memory calculation array 1, a weight data processing module 2, a shift control module 3, and a result aggregation module 4.

[0025] Each storage cell of the separable word-line SRAM near-memory computing array 1 has independent left and right word lines. These are used to replace only a portion of the data during kernel shifting by switching the conduction states of the word lines (including the left and right word lines). The single-bit data stored in the array is then multiplied and added with the encoded weight data to generate a partial product. The weight data processing module 2 encodes the weight data and rearranges its flow according to the shift control signal. The shift control module 3 outputs a shift control signal to control the conduction states of the word lines within the separable word-line SRAM near-memory computing array 1, enabling kernel shifting on the feature map while reloading only a portion of the data during the shift process. The result aggregation module 4 aggregates the partial products and outputs the convolution calculation result.

[0026] In this process, feature map data is input to the separable word-line SRAM near-memory computation array 1, weight data is input to the weight data processing module 2, the weight data processing module 2 sends the encoded weight data to the separable word-line SRAM near-memory computation array 1, the shift control module 3 outputs a shift control signal to the separable word-line SRAM near-memory computation array 1, the partial product output by the separable word-line SRAM near-memory computation array 1 is sent to the result aggregation module 4, and the result aggregation module 4 outputs the convolution calculation result. The shift control signal includes a horizontal shift control signal hk_ctrl and a vertical shift control signal vk_ctrl, both of which are multi-bit control signals, for example, they can be 3-bit control signals.

[0027] The weight data processing module 2 may include a weight data reloading submodule and an encoding submodule. The weight data reloading submodule is used to rearrange the arrangement order of the weight values ​​of the convolution kernel according to the horizontal shift control signal hk_ctrl in the shift control signal. The encoding submodule is used to perform radix-4 Booth encoding on the weight data and generate an encoding control signal to be output to the decoding and storage value processing module 120.

[0028] Please see Figure 2 The radix-4 Booth encoding method is as follows: the least significant bit of the weight data is padded with 0 to expand it to 9 bits, and then the data is cyclically grouped four times according to the rule of grouping three bits together and overlapping one bit between adjacent groups to obtain four intermediate values, namely: cycle0-cycle3. Each intermediate value corresponds to a three-bit encoded value output.

[0029] The Booth encoding rules are as follows: When the middle value is 000 or 111, the value of the first encoding bit is "1"; at this time, the second and third encoding bits are invalid (the values ​​of the second and third encoding bits will not affect the decoding and can be set arbitrarily, for example, they can all be set to "0"); when the middle value is 001 or 010, the values ​​of the first, second, and third encoding bits are all "0"; when the middle value is 011, the values ​​of the first and second encoding bits are "0", and the value of the third encoding bit is "1"; when the middle value is 100, the value of the first encoding bit is "0", and the values ​​of the second and third encoding bits are "1"; when the middle value is 101 or 110, the values ​​of the first and third encoding bits are "0", and the value of the second encoding bit is "1". See Table 1 for details.

[0030] Table 1 In this embodiment, BoothT, BoothP, and BoothS signals are used as encoding signals, corresponding to the first, second, and third encoding bits of the three-bit encoded value, respectively, thereby representing the output of the three-bit encoded value to the decoding subunit of the decoding and storage value processing module 120. When the value of the encoding bit is "1", the corresponding encoding control signal is at a high level; when the value of the encoding bit is "0", the corresponding encoding control signal is at a low level.

[0031] Please see Figure 3 and Figure 4The separable word-line SRAM near-memory computing array 1 includes a separable word-line 6-transistor SRAM array, a decoding and storage value processing module 120, and a partial product shift and accumulation module 130. The separable word-line 6-transistor SRAM array is composed of multiple separable word-line SRAM sub-arrays 110. Each separable word-line SRAM sub-array 110 corresponds to one bit of int8 format feature map data, and each separable word-line SRAM sub-array 110 stores bits 0 to bits 7 of the feature map data respectively. The int8 format feature map data is split into single-bit data according to bits 0 to bits 7 and written to each separable word-line SRAM sub-array 110 in a vertical arrangement. Each separable word-line SRAM sub-array 110 has a multi-row × multi-column structure and is composed of multiple 6-transistor SRAM cells 111 (i.e., memory cells). In this embodiment, the separable word-line 6-transistor SRAM array consists of eight separable word-line SRAM subarrays 110, from bit 0 to bit 7, each with 16 rows × 36 columns. Each separable word-line SRAM subarray 110 is connected to global bit lines GB and GLB, which are arranged along the vertical direction of the array and used to import feature value data. Each separable word-line SRAM subarray 110 receives a mode signal, thereby controlling the switching between storage mode and in-memory computation mode according to the mode signal.

[0032] Please see Figure 5 Each 6-transistor SRAM cell 111 has an independent left word line WLL and a right word line WLR. Specifically, the 6-transistor SRAM cell 111 includes a first inverter N1, a second inverter N2, a first NMOS transistor NM1, and a second NMOS transistor NM2. The first inverter N1 and the second inverter N2 are cross-coupled to form a data latch structure. The output node of the first inverter N1 is the first storage node, and the output node of the second inverter N2 is the second storage node. The gate of the first NMOS transistor NM1 is connected to the left word line WLL, the first end is connected to the first bit line BL, and the second end is connected to the first storage node. The gate of the second NMOS transistor NM2 is connected to the right word line WLR, the first end is connected to the second bit line BLB, and the second end is connected to the second storage node. The left word line WLL and the right word line WLR are respectively connected to the corresponding word line strobe signals to independently control the conduction of the left and right halves of each 6-transistor SRAM cell 111.

[0033] Separable word-line 6-transistor SRAM array has two operating modes: memory mode and in-memory computing mode. In storage mode, under the control of the mode signal, the first bit line BL is connected to the first global bit line GB (not shown in the figure), and the second bit line BLB is connected to the second global bit line GLB (not shown in the figure). Feature value data is imported into each separable word line SRAM subarray 110 through the first global bit line GB and the second global bit line GLB. At this time, the word line strobe signal WL[i] is connected to both the left word line WLL[i] and the right word line WLR[i]. Taking i=0 as an example, WL[0] is connected to both WLL[0] and WLR[0], controlling the write operation of the 0th row memory cell. Here, i is a non-negative integer, representing the row index.

[0034] In in-memory computing mode, the word line strobe signal WL[2i] connects the left word line WLL[2i+1] and the right word line WLR[2i], and the word line strobe signal WL[2i+1] connects the left word line WLL[2i+1] and the right word line WLR[2i+2]. Taking i=0 as an example, WL[0] connects WLL[1] and WLR[0], and WL[1] connects WLL[1] and WLR[2].

[0035] In this embodiment, the convolution kernel is a K×K convolution kernel, where K ≥ 2. Taking a 3×3 convolution kernel as an example, the first two rows of SRAM cells have four word lines: wll0, wllr0, wll1, and wllr1. In in-memory computation mode, wll0 and wllr0 word lines correspond to a single bit value of the first column of feature data (feature0 to feature8), and wll1 and wllr1 correspond to a single bit value of the second column of feature data (feature9 to feature17).

[0036] Multiple decoding and storage value processing modules 120 can be configured, each corresponding to a separable word-line SRAM subarray 110. The decoding and storage value processing module 120 receives single-bit data read from the corresponding separable word-line SRAM subarray 110 and performs a decoding operation on the single-bit data according to the encoded weight data to generate a partial product. In this embodiment, the decoding and storage value processing module 120 receives encoding control signals boothT0~boothT8, boothP0~boothP8, and boothS0~boothS8, as well as single-bit data read from the separable word-line SRAM subarray 110. Specifically, boothT0~boothT8, boothP0~boothP8, and boothS0~boothS8 are the BoothT, BoothP, and BoothS signals representing the Booth encoding results of the nine weight values ​​in the 3×3 convolution kernel, respectively. The decoding subunit of the decoding and storage value processing module 120 performs the decoding operation on the multiplicand according to the encoding control signal as follows: When the value of the first encoding bit is "1" (i.e., when the BoothT signal is valid; at this time, the values ​​of the BoothP and BoothS signals will no longer be detected), the multiplicand is set to zero; when the values ​​of the first, second, and third encoding bits are all "0" (i.e., when the BoothT, BoothP, and BoothS signals are all invalid), the multiplicand remains unchanged; when the values ​​of the first and second encoding bits are "0" and the value of the third encoding bit is "1" (i.e., when the BoothS signal is valid and the BoothP and BoothS signals are invalid), the multiplicand remains unchanged; When both the Booth P and Booth T signals are invalid, the multiplicand is shifted left by one bit; when the value of the first encoding bit is "0" and the values ​​of the second and third encoding bits are "1" (i.e., when the Booth P and Booth S signals are valid and the Booth T signal is invalid), the multiplicand is taken in two's complement and then shifted left by one bit; when the values ​​of the first and third encoding bits are "0" and the value of the second encoding bit is "1" (i.e., when the Booth P signal is valid and the Booth T and Booth S signals are invalid), the multiplicand is taken in two's complement.

[0037] The decoding and storage value processing module 120 can contain multiple parallel decoding sub-units, each corresponding to a weight value in the convolution kernel. For a 3×3 convolution kernel, each decoding and storage value processing module 120 contains 9 decoding sub-units.

[0038] The input of each decoding sub-unit is connected to the encoding control signal line, the first bit line BL of the corresponding separable word line SRAM sub-array 110, and the second bit line BLB of the corresponding separable word line SRAM sub-array 110, respectively. It is used to receive the encoding control signal and the single bit data read from the corresponding separable word line SRAM sub-array 110 as the multiplicand, and to perform decoding operation on the multiplicand according to the encoding control signal, and output the corresponding partial product in one cycle.

[0039] The decoding subunit may include a zero-set selector, a left shifter, and a two's complement circuit. The control terminal of the zero-set selector is connected to the BoothT signal, used to set the multiplier to zero when the BoothT signal is valid. The input terminal of the left shifter is connected to the output terminal of the zero-set selector, and its control terminal is connected to the BoothS signal, used to shift the output of the zero-set selector one bit to the left when the BoothS signal is valid. The input terminal of the two's complement circuit is connected to the output terminal of the left shifter, and its control terminal is connected to the BoothP signal, used to extract the two's complement of the output of the left shifter when the BoothP signal is valid. The output terminal of the zero-set selector is connected to the input terminal of the left shifter, and the output terminal of the left shifter is connected to the input terminal of the two's complement circuit, forming a decoding channel cascaded according to the priority order of the BoothT, BoothS, and BoothP signals.

[0040] Multiple partial product shift-accumulation modules 130 can be configured, each corresponding to a separable word-line SRAM subarray 110. The partial product shift-accumulation module 130 is used to shift and accumulate the partial products generated by the corresponding separable word-line SRAM subarray 110 in multiple cycles.

[0041] Taking a 3×3 convolution kernel as an example, after receiving the feature value data from the independent bit lines BL and BLB of the separable word line SRAM subarray 110, the decoding and storage value processing module 120 controls the processing of the feature value reading from the separable word line SRAM subarray 110 according to the flag bits of the nine encoded 3-bit booth signals (boothP, boothT, and boothS respectively). After four cycles, the encoded weight value is processed for the single-bit feature value. The partial products pdt0 to pdt8 are also sequentially passed to the storage value partial accumulation module. In the storage value partial accumulation module, the four pdt0 to pdt8 obtained from the four cycles are shifted and accumulated to obtain the processing result of the single-bit weight value and feature value.

[0042] During horizontal shifting, the separable word-line 6-tube SRAM array slides the convolution kernel horizontally according to the different states of hk_ctrl, and replaces the feature data of some columns located at the edge of the convolution window accordingly, so as to achieve horizontal shift control of the K×K convolution kernel. Taking a 3×3 convolution kernel with a stride of 1 as an example, hk_ctrl has four states.

[0043] Please see Figure 6 When hk_ctrl is 000, it is state (1), and the convolution kernel covers feature values ​​from feature0 to feature8. Figure 6 In Chinese, feature is abbreviated as "F".

[0044] After one convolution calculation is completed, hk_ctrl becomes 010, which is state (2). At this time, the feature values ​​covered by the convolution kernel become feature9, feature1, feature2, feature10, feature4, feature5, feature11, feature7, and feature8. Compared with state (1), only feature0, feature3, and feature6 are replaced by feature9, feature10, and feature11.

[0045] After the second convolution calculation is completed, hk_ctrl becomes 100, and the state becomes (3). At this time, the feature values ​​covered by the convolution kernel are feature9, feature12, feature2, feature10, feature13, feature5, feature11, feature14, and feature8. Compared with state (2), only feature1, feature4, and feature7 are replaced by feature12, feature13, and feature14.

[0046] After the third convolution calculation is completed, hk_ctrl becomes 001, which is state (4). At this time, the feature values ​​covered by the convolution kernel are feature9, feature12, feature15, feature10, feature13, feature16, feature11, feature14, and feature17. Compared with state (3), only feature2, feature5, and feature8 are replaced with feature15, feature16, and feature17.

[0047] During vertical shifting, the separable word-line 6-transistor SRAM array can switch the combination of active word lines according to different states of vk_ctrl to make the read feature data logically vertically shifted, so as to realize the shift control of the K×K convolution kernel in the vertical direction without reloading all feature data.

[0048] Taking the initial state (i.e., state ①) with vk_ctrl set to 000 as an example, the read feature values ​​correspond to features 0 through 8. When vk_ctrl is 001 (i.e., state ②), the first three columns read the bit line values ​​of wll1, and the last six columns read the bit line values ​​of wll0, i.e., features 9 through 11 and features 3 through 8. When vk_ctrl is 011 (i.e., state ③), the read values ​​are features 9 through 14 and features 6 through 8. When vk_ctrl is 111 (i.e., state ④), the read values ​​are features 9 through 17.

[0049] The weight data reloading submodule rearranges the order of the weights in the convolution kernel according to the horizontal shift control signal hk_ctrl. If the hk_ctrl signal is 000 or 111, the weight flow remains unchanged, maintaining the arrangement of weights 0 to 8. Figure 6In this context, weight is abbreviated as "W". When the hk_ctrl signal is 001, the weight values ​​are arranged as weight6~weight8, weight0~weight5. When the hk_ctrl signal is 011, the weight values ​​are arranged as weight3~weight8, weight0~weight2. Changing the direction of the weight value arrangement is to ensure that the convolution shape remains correct after subsequent vertical shifting and feature value processing.

[0050] As a result, aggregation module 4 shifts and aligns the partial products of different bits according to their bit weights, performs shift accumulation on the numerical bits from bit 0 to bit 6, and performs shift subtraction on the sign bit of bit 7. Using a Wallace tree structure, a 3-to-2 compressor and a 4-to-2 compressor are used to perform multi-level compression accumulation on the multiple partial products, output the convolution calculation result, and output the total_done signal to trigger the next shift convolution.

[0051] The in-memory deep convolution calculation circuit based on separable word line SRAM of the present invention operates according to the following process during operation.

[0052] During the data writing phase, feature map data in int8 format is input into the separable word-line SRAM near-memory computing array 1. In a preferred embodiment of the invention, the feature map size is consistent with the size of the separable word-line 6-transistor SRAM array, which can both be 128×128, matching the size of the separable word-line 6-transistor SRAM array. Each separable word-line SRAM subarray 110 has 128 columns, and data from 32 channels undergoes multiplication and addition operations simultaneously. The feature map data is split into 8 single-bit data planes according to bit 0 to bit 7, and each bit plane is written to each separable word-line SRAM subarray 110 in a vertical arrangement, allowing each memory cell in the same column to synchronously read the feature data of each bit during subsequent calculations. During writing, the separable word-line 6-transistor SRAM array operates in memory mode, and the word line strobe signal simultaneously connects the left and right word lines of each row of memory cells. Data is imported into each memory cell via the first global bit line GB and the second global bit line GLB. Each separable word line SRAM subarray 110 corresponds to one bit of feature map data, and the memory cells in the same column are read out synchronously in subsequent calculations.

[0053] After entering the computation phase, the separable word-line 6-transistor SRAM array switches to in-memory computation mode. Shift control module 3 outputs horizontal shift control signal hk_ctrl and vertical shift control signal vk_ctrl to control the conduction state of the word lines inside the array. In in-memory computation mode, the connection relationship of the word line strobe signals is different from that in memory mode. By switching the combination of word line strobe signals, single-bit feature data is read synchronously from each memory cell in the same column.

[0054] The single-bit feature data read from each separable word-line SRAM subarray 110 is sent to the corresponding decoding and storage value processing module 120. The decoding subunit within this module receives the encoding control signal and the single-bit feature data read from the corresponding separable word-line SRAM subarray 110 as the multiplicand, performs decoding on the multiplicand according to the encoding control signal, and outputs the corresponding partial product within one cycle. A weight value, after Booth encoding, generates four sets of encoded values. The decoding and storage value processing module 120 completes the processing of one set of weight value and single-bit feature value within four cycles, sequentially outputting the four sets of partial products to the corresponding partial product shift and accumulation module 130. The partial product shift and accumulation module 130 shifts and accumulates the four sets of partial products according to their respective bit weights to obtain the multiplication result of the weight value and the single-bit feature value.

[0055] The single-bit results processed by each separable word-line SRAM subarray 110 are sent to the result aggregation module 4. The result aggregation module 4 first shifts and aligns the partial products of different bits according to their bit weights, performs shift accumulation on the numerical bits from bit 0 to bit 6, and performs shift subtraction on the sign bit of bit 7. Then, the result aggregation module 4 performs multi-level compression accumulation on the shifted and aligned multi-path partial products, outputs the convolution calculation result, and outputs the total_done signal upon completion.

[0056] After the current convolution window calculation is completed, the shift control module 3 outputs a shift control signal to shift the convolution kernel. During horizontal shifting, the state is switched via the hk_ctrl signal, and the array slides the convolution kernel horizontally, replacing the feature data of some columns located at the edge of the convolution window. During vertical shifting, the state is switched via the vk_ctrl signal, and the read feature data is logically shifted vertically by changing the combination of the activated word line gating signals, without needing to reload all feature data. After the shift is completed, the next round of convolution operation is started until all feature map data has completed convolution calculation.

[0057] Multiplication and addition operations on each bit plane are performed in parallel, and convolution operations on each output channel are performed in parallel. Feature map data can participate in multiple convolution calculations after being written once. During the shifting process, only the new data column entering the convolution window needs to be replaced, without reloading all the data.

[0058] This invention achieves a balance between in-memory computation speed and computational efficiency with controllable area overhead and circuit complexity, and is suitable for fixed-point depthwise convolution operations on int8 data types. Compared to traditional in-memory computation architectures and conventional SRAM-based computation schemes, this invention has the following advantages: First, a separable word-line 6-transistor SRAM array is adopted. Each 6-transistor SRAM cell 111 has an independent left word line and a right word line. By using the word line strobe signal to switch between storage mode and in-memory computing mode in different modes, the overhead of data transfer and memory access is reduced, and power consumption is reduced.

[0059] Second, the feature map data is split into single-bit data from bit0 to bit7 and written into each separable word line SRAM subarray 110 in a vertical arrangement, and each bit is calculated in parallel; after the weight data is encoded by radix 4 Booth, the number of partial products is reduced from 8 to 4, which reduces the hardware overhead and operation latency of the subsequent addition tree.

[0060] Third, the decoding subunit of the decoding and storage value processing module 120 adopts a three-level cascaded structure of a zero-set selector, a left shifter, and a complement code circuit, which completes the decoding processing of the multiplicand within one cycle. The three-level circuit is cascaded according to the priority order of BoothT signal, BoothS signal, and BoothP signal, which matches the Booth encoding rules and realizes the fast switching of various decoding operations.

[0061] Fourth, during horizontal shifting, only the feature data of some columns located at the edge of the convolution window is replaced. During vertical shifting, the read feature data is logically shifted vertically by switching the combination of the connected word lines, without the need to reload all feature data, thus reducing hardware resource consumption and circuit complexity.

[0062] The above embodiments merely illustrate preferred embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A memory-based depthwise convolution calculation circuit based on separable word-line SRAM, characterized in that, include: A separable word-line SRAM near-memory computing array is used to replace part of the data by switching the word-line conduction state when the convolution kernel is shifted, and to perform multiplication and addition operations on the single-bit data stored in the array and the encoded weight data to generate a partial product; The weight data processing module is used to encode the weight data and rearrange the flow direction of the encoded weight data according to the shift control signal. The shift control module is used to output a shift control signal to control the conduction state of the word lines inside the separable word line SRAM near-memory computing array, thereby shifting the convolution kernel on the feature map and reloading some data during the shifting process. The result aggregation module is used to aggregate the partial products and output the convolution calculation results.

2. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 1, characterized in that, The separable word-line SRAM near-memory computing array includes a separable word-line 6-transistor SRAM array, a decoding and storage value processing module, and a partial product shift accumulation module. The separable word-line 6-transistor SRAM array is composed of multiple separable word-line SRAM subarrays. Each separable word-line SRAM subarray corresponds to one bit of int8 format feature map data, and each separable word-line SRAM subarray stores bits 0 to bits 7 of the feature map data respectively. The int8 format n-channel feature map data is split into single-bit data according to bits 0 to bits 7 and then written into each separable word-line SRAM subarray in a vertical arrangement. Each separable word-line SRAM subarray includes multiple 6-transistor SRAM cells arranged in an array, and each 6-transistor SRAM cell has an independent left word line WLL and a right word line WLR. The decoding and storage value processing module is configured as multiple, each corresponding to a separable word line SRAM subarray. The decoding and storage value processing module is used to receive single-bit data read from the corresponding separable word line SRAM subarray, and perform decoding operation on the single-bit data according to the encoded weight data to generate a partial product. There are multiple partial product shift accumulation modules, each corresponding to a separable word line SRAM subarray. The partial product shift accumulation module is used to shift and accumulate the partial products generated by the corresponding separable word line SRAM subarray in multiple cycles.

3. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 2, characterized in that, The 6-transistor SRAM cell includes a first inverter, a second inverter, a first NMOS transistor, and a second NMOS transistor; The first inverter and the second inverter are cross-coupled to form a data latch structure, the output node of the first inverter is the first storage node, and the output node of the second inverter is the second storage node; The gate of the first NMOS transistor is connected to the left word line WLL, the first end is connected to the first bit line BL, and the second end is connected to the first memory node; The gate of the second NMOS transistor is connected to the right word line WLR, the first end is connected to the second bit line BLB, and the second end is connected to the second memory node; The left word line WLL and the right word line WLR are respectively connected to the corresponding word line strobe signals to independently control the conduction of the left and right halves of each 6-transistor SRAM cell.

4. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 3, characterized in that, In storage mode, the word line strobe signal WL[i] connects both the left word line WLL[i] and the right word line WLR[i]. In in-memory computing mode, the word line strobe signal WL[2i] connects the left word line WLL[2i+1] and the right word line WLR[2i], and the word line strobe signal WL[2i+1] connects the left word line WLL[2i+1] and the right word line WLR[2i+2]. Here, i is a non-negative integer representing the row index.

5. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 2, characterized in that, The weighted data processing module includes: The weight data reloading submodule is used to rearrange the arrangement order of each weight value of the convolution kernel according to the horizontal shift control signal in the shift control signal; The encoding submodule is used to perform radix-4 Booth encoding on the weighted data and generate an encoding control signal to be output to the decoding and storage value processing module.

6. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 5, characterized in that, The decoding and storage value processing module contains multiple parallel decoding sub-units, each corresponding to a weight value in the convolution kernel. Each of the decoding sub-units is connected to an encoding control signal line, a first bit line BL, and a second bit line BLB at its input. It is used to receive the encoding control signal and a single bit of data read from the corresponding separable word line SRAM sub-array as the multiplicand, and to perform a decoding operation on the multiplicand according to the encoding control signal, and output the corresponding partial product in one cycle.

7. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 6, characterized in that, The radix-4 Booth encoding method is as follows: The least significant bit of the weight data is padded with 0 to expand it to 9 bits. The data is then grouped four times in a loop, with each group consisting of three bits and an overlap of one bit between adjacent groups, resulting in four intermediate value groups. Each intermediate value group corresponds to a three-bit encoded output value. When the intermediate value is 000 or 111, the value of the first encoding bit is "1"; at this time, the second and third encoding bits are invalid. When the middle value is 001 or 010, the values ​​of the first, second, and third coding bits are all "0"; When the middle value is 011, the values ​​of the first and second coding bits are "0", and the value of the third coding bit is "1". When the median value is 100, the value of the first encoding bit is "0", and the values ​​of the second and third encoding bits are "1". When the middle value is 101 or 110, the values ​​of the first and third coding bits are "0", and the value of the second coding bit is "1". BoothT, BoothP, and BoothS signals correspond to the first, second, and third encoded bits of the three-bit encoded value, respectively, thereby representing the output of the three-bit encoded value to the decoding subunit of the decoding and storage value processing module.

8. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 7, characterized in that, The decoding subunit of the decoding and storage value processing module performs a decoding operation on the multiplicand according to the encoding control signal as follows: When the value of the first encoded bit is "1", the multiplicand is set to zero; When the values ​​of the first, second, and third coding bits are all "0", the multiplicand remains unchanged; When the value of the first and second coding bits is "0" and the value of the third coding bit is "1", the multiplier is shifted left by one bit; When the value of the first encoding bit is "0" and the values ​​of the second and third encoding bits are "1", the multiplicand is taken in two's complement and then shifted left by one bit. When the values ​​of the first and third encoding bits are "0" and the value of the second encoding bit is "1", the multiplicand is taken in two's complement.

9. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 1, characterized in that, The shift control signal includes a horizontal shift control signal hk_ctrl and a vertical shift control signal vk_ctrl; both the horizontal shift control signal hk_ctrl and the vertical shift control signal vk_ctrl are multi-bit control signals, and the convolution kernel is a K×K convolution kernel, where K≥2; During horizontal shifting, the separable word line 6-tube SRAM array slides the convolution kernel in the horizontal direction according to the different states of hk_ctrl and replaces the feature data of some columns located at the edge of the convolution window accordingly, so as to realize the horizontal shifting control of the K×K convolution kernel; During vertical shifting, the separable word-line 6-transistor SRAM array switches the combination of active word lines according to different states of vk_ctrl, so as to realize the vertical shift control of the K×K convolution kernel without reloading all feature data.

10. The in-memory depthwise convolution calculation circuit based on separable word-line SRAM according to claim 1, characterized in that, The result aggregation module is used to shift and align the partial products of different bits according to their bit weights, perform shift accumulation on the numerical bits from bit 0 to bit 6, perform shift subtraction on the sign bit of bit 7, and use a Wallace tree structure to perform multi-level compression accumulation on the multiple partial products, output the convolution calculation result, and output the total_done signal to trigger the next shift convolution.

Images