Two-dimensional signal transformation in-memory computing architecture, electronic device and method

Abstract

The invention relates to the technical field of signals, in particular to a two-dimensional signal transformation in-memory computing architecture, electronic equipment and a method. The cross array is used for executing matrix vector multiplication operation between the input vector and the parameter matrix to obtain a one-dimensional signal conversion result; the row driving module is used for converting the column data and the first-time one-dimensional signal conversion result of the cross array into an input vector of the cross array; the column driving module is used for outputting a cross array calculation result; the cache module is used for storing all the first-time one-dimensional signal conversion results calculated by the cross array and performing transposition operation on all the first-time one-dimensional signal conversion results calculated by the cross array so as to realize two-dimensional signal conversion in-memory calculation; and the data output module is used for outputting a calculation result in the two-dimensional signal conversion memory. Therefore, the problems that in the prior art, complete two-dimensional signal conversion cannot be executed, and the calculation efficiency is reduced and the hardware overhead is large due to the fact that data are repeatedly carried inside and outside a core are solved.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to a two-dimensional signal conversion in-memory computing architecture, electronic equipment and method. Background Art

[0002] Two-dimensional signal transformation is a widely used image processing method, including two-dimensional discrete Fourier transform (2D-DFT), two-dimensional discrete cosine transform (2D-DCT), two-dimensional discrete wavelet transform (2D-DWT), etc. The calculation process of the two-dimensional signal transformation algorithm can be roughly divided into three steps: the first one-dimensional signal transformation, intermediate result caching, reorganization and transposition, and the second signal transformation. In recent years, with the continuous increase in image size and video frame rate, traditional computing hardware will cause serious energy consumption and computing delay when executing two-dimensional signal transformation algorithms due to the bottleneck of the von Neumann architecture.

[0003] One-dimensional signal transformation algorithms can be efficiently implemented on computing cores based on crossbar arrays (including but not limited to resistive random access memory (RRAM), phase change memory (PCM), and magnetic random access memory (MRAM), with energy efficiency improved by about 2 orders of magnitude [1-3]. This is because the one-dimensional signal transformation algorithm can be equivalent to a matrix-vector multiplication operation between the input vector and the transformation matrix, and the crossbar array computing core can efficiently complete the matrix-vector multiplication operation based on the laws of physics.

[0004] However, the current cross-array computing core can only complete one-dimensional signal transformation, but cannot perform a complete two-dimensional signal transformation. The intermediate result data can only be moved outside the computing core for caching and reorganization, and then input into the cross-array computing core to complete the second one-dimensional signal transformation after the digital processing unit (CPU, FPGA, MCU, etc.) completes the transposition operation. This repeated movement of data will cause additional hardware overhead. Summary of the invention

[0005] The present application provides a two-dimensional signal conversion in-memory computing architecture, electronic device and method to solve the problems in the related art that the complete two-dimensional signal conversion cannot be performed, the data is repeatedly transferred in and out of the core, resulting in reduced computing efficiency and large hardware overhead.

[0006] The first aspect of the present application provides an in-memory computing architecture for two-dimensional signal transformation, including: a data input module, used to divide an externally input two-dimensional signal into column data by column; a cross array, used to store a parameter matrix of a one-dimensional signal transformation, and perform a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; a row driving module, used to drive the row lines of the cross array, and convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; the column driving module, used to drive the column lines of the cross array, and output the calculation result of the cross array; a cache module, used to store all the first one-dimensional signal transformation results calculated by the cross array, and perform a transposition operation on all the first one-dimensional signal transformation results calculated by the cross array to realize the in-memory computing of the two-dimensional signal transformation; a data output module, used to determine the result of the in-memory computing of the two-dimensional signal transformation according to the second one-dimensional signal transformation result of the cross array, and output the result of the in-memory computing of the two-dimensional signal transformation.

[0007] Optionally, it also includes: a data conversion module, used to obtain the output data of the column driving module, convert the first one-dimensional signal transformation result into a first digital signal vector, store the first one-dimensional signal transformation result into a cache module based on the first digital signal vector, and convert all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and output the second digital signal vector to the data output module.

[0008] Optionally, it also includes: a first encoding module, used to perform encoding operation on the first digital signal vector, and write the output data of the first encoding module into the cache module in parallel by row lines.

[0009] Optionally, if the data input module performs a multi-bit data batch input operation, the first encoding module performs a shift-and-add operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces an inverse code or a complement code, the first encoding module re-encodes the first digital signal vector into the original code form; after the first encoding module performs the shift-and-add operation or re-encoding, the first digital signal vector is sorted according to the cache order format of the cache module.

[0010] Optionally, it also includes: a second encoding module, which is used to perform at least one encoding operation of an accumulation operation, a shift operation and an original code and complement code exchange operation on the data cached by the cache module, and input the data after the encoding operation into the row driving module.

[0011] Optionally, the data input module allows buffering of partial data of the two-dimensional signal each time.

[0012] Optionally, the parameter matrix includes at least one of a discrete Fourier transform matrix, a discrete cosine transform matrix, and a discrete wavelet transform matrix.

[0013] Optionally, the computing core of the crossbar array includes: different types of memristors, storage-computing integrated architectures of different structures, or storage-computing integrated units of different structures.

[0014] A second aspect of the present application provides an electronic device, comprising the two-dimensional signal conversion in-memory computing architecture of the above embodiment.

[0015] The second aspect of the present application provides a two-dimensional signal transformation in-memory calculation method, including the following steps: dividing an externally input two-dimensional signal into column data by column; using a cross array to store a parameter matrix of a one-dimensional signal transformation, and performing a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; driving the row lines of the cross array to convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result based on the column data, and the cross array calculates the second one-dimensional signal transformation result based on the first one-dimensional signal transformation result; driving the column lines of the cross array to output the cross array calculation result; storing all the first one-dimensional signal transformation results calculated by the cross array, and transposing all the first one-dimensional signal transformation results calculated by the cross array to realize the two-dimensional signal transformation in-memory calculation; determining the result of the two-dimensional signal transformation in-memory calculation based on the second one-dimensional signal transformation result of the cross array, and outputting the result of the two-dimensional signal transformation in-memory calculation.

[0016] Therefore, this application includes the following beneficial effects:

[0017] Since the two-dimensional signal conversion needs to go through two conversions, the first one-dimensional conversion and the second one-dimensional conversion, the final two-dimensional signal conversion in-memory calculation result is output, but the existing cross array computing core can only complete one-dimensional signal conversion once, and cannot perform a complete two-dimensional signal conversion. Therefore, the two-dimensional signal conversion in-memory computing architecture of the embodiment of the present application, the two-dimensional signal first passes through the data input module, the cross array, the row driver module, and the column driver module for the first one-dimensional signal conversion, and a cache module is added to store the first one-dimensional signal conversion result, so that the first one-dimensional signal conversion result can be directly cached, transposed, and other operations in the cache module to achieve the reorganization of the intermediate result, and then the second one-dimensional signal conversion is performed in situ, and the final data output module determines the result of the two-dimensional signal conversion in-memory calculation according to the second one-dimensional signal conversion result, so as to output the result of the two-dimensional signal conversion in-memory calculation, so that the intermediate result can be directly cached, transposed, and other operations in the computing core, avoiding the repeated transfer of data inside and outside the core, thereby significantly improving the computing efficiency and reducing the hardware overhead. Thus, the technical problems that the complete two-dimensional signal conversion cannot be performed in the related art, and the repeated transfer of data inside and outside the core leads to reduced computing efficiency and large hardware overhead are solved.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 It is a schematic diagram of the calculation flow of the two-dimensional signal transformation algorithm;

[0021] Figure 2 A schematic diagram of matrix-vector multiplication operation based on a cross-array computing kernel;

[0022] Figure 3 A schematic diagram of a crossbar array computing core architecture in the related art;

[0023] Figure 4 A schematic diagram of an in-memory computing architecture for two-dimensional signal transformation provided according to an embodiment of the present application;

[0024] Figure 5 A specific structural diagram of the in-memory computing architecture for two-dimensional signal transformation provided according to an embodiment of the present application;

[0025] Figure 6 A schematic diagram of data input according to an embodiment of the present application;

[0026] Figure 7A schematic diagram of multi-bit data splitting input according to an embodiment of the present application;

[0027] Figure 8 A schematic diagram of a first one-dimensional signal transformation provided according to an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of data caching and reorganization provided according to an embodiment of the present application;

[0029] Fig.10 A schematic diagram of a second one-dimensional signal transformation provided according to an embodiment of the present application;

[0030] Fig.11 A schematic diagram of data output provided according to an embodiment of the present application;

[0031] Fig.12 A schematic diagram of a two-dimensional signal conversion in-memory computing architecture based on a memristor and SRAM according to an embodiment of the present application;

[0032] Fig.13 The present invention is a flowchart of an in-memory calculation method for two-dimensional signal transformation provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] Before describing the technical solution of the present application, the following current related technologies are described in the form of text and pictures to assist in understanding the technical solution of the present application.

[0035] like Figure 1 As shown in the figure, the calculation process of the two-dimensional signal transformation algorithm mainly includes the first one-dimensional signal transformation, intermediate result caching, reorganization and transposition, and the second signal transformation. The one-dimensional signal transformation algorithm can be efficiently implemented on the cross-array-based computing core. The one-dimensional signal transformation algorithm can be equivalent to the matrix-vector multiplication between the input vector and the transformation matrix, and the cross-array computing core can efficiently complete the matrix-vector multiplication operation based on the laws of physics, such as Figure 2 shown.

[0036] Current cross-array computing cores, such as Figure 3As shown, only one-dimensional signal transformation can be completed, but the complete two-dimensional signal transformation cannot be performed. This is due to two reasons. First, there is a lack of cache space in the core, and the intermediate result matrix cannot be cached; second, there is a lack of transposition operation unit in the core, and the transposition operation cannot be performed on the intermediate result matrix. Both of these lead to the intermediate result data can only be moved outside the computing core for caching and reorganization, and the digital processing unit completes the transposition operation and then inputs it to the cross array computing core to complete the second one-dimensional signal transformation. This repeated movement of data will cause additional hardware overhead.

[0037] To this end, the present application provides a two-dimensional signal conversion in-memory computing architecture, in which the two-dimensional signal first undergoes a first one-dimensional signal conversion through a data input module, a cross array, a row driver module, and a column driver module, and a cache module is added to store the first one-dimensional signal conversion result, so that the first one-dimensional signal conversion result can be directly cached, transposed, and other operations in the cache module to achieve the reorganization of the intermediate result, and then the second one-dimensional signal conversion is performed, and the final data output module determines the result of the two-dimensional signal conversion in-memory calculation according to the second one-dimensional signal conversion result to output the result of the two-dimensional signal conversion in-memory calculation, so that the intermediate result can be directly cached, transposed, and other operations in the computing core, avoiding the repeated transfer of data inside and outside the core, thereby significantly improving the computing efficiency and reducing the hardware overhead. Thus, the problems that the related art cannot perform a complete two-dimensional signal conversion, and the repeated transfer of data inside and outside the core leads to reduced computing efficiency and large hardware overhead are solved.

[0038] Specifically, Figure 4 A schematic diagram of an in-memory computing architecture for two-dimensional signal transformation provided in an embodiment of the present application.

[0039] like Figure 4 As shown, the two-dimensional signal conversion in-memory computing architecture 10 includes: a data input module 11, a cross array 12, a row driver module 13, a column driver module 14, a cache module 15 and a data output module 16.

[0040] Among them, the data input module 11 is used to divide the externally input two-dimensional signal into column data by column; the cross array 12 is used to store the parameter matrix of the one-dimensional signal transformation, and perform the matrix-vector multiplication operation between the input vector and the parameter matrix to obtain the one-dimensional signal transformation result; the row driving module 13 is used to drive the row lines of the cross array, and convert the column data and the first one-dimensional signal transformation result of the cross array into the input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; the column driving module 14 is used to drive the column lines of the cross array and output the calculation result of the cross array; the cache module 15 is used to store all the first one-dimensional signal transformation results calculated by the cross array, and transpose all the first one-dimensional signal transformation results calculated by the cross array to realize the in-memory calculation of the two-dimensional signal transformation; the data output module 16 is used to determine the result of the in-memory calculation of the two-dimensional signal transformation according to the second one-dimensional signal transformation result of the cross array, and output the result of the in-memory calculation of the two-dimensional signal transformation.

[0041] Since the two-dimensional signal transformation needs to undergo two transformations, the first one-dimensional transformation and the second one-dimensional transformation, to output the final result of the two-dimensional signal transformation in-memory calculation, but the existing cross array computing core can only complete one-dimensional signal transformation once, and cannot perform a complete two-dimensional signal transformation, therefore, in the two-dimensional signal transformation in-memory computing architecture 10 of the embodiment of the present application, the two-dimensional signal first passes through the data input module 11, the cross array 12, the row driver module 13, and the column driver module 14 for the first one-dimensional signal transformation, and a cache module 15 is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can be directly cached, transposed and other operations in the cache module 15 to achieve the reorganization of the intermediate result, and then the second one-dimensional signal transformation is performed in situ, and the final data output module 16 determines the result of the two-dimensional signal transformation in-memory calculation according to the second one-dimensional signal transformation result to output the result of the two-dimensional signal transformation in-memory calculation, so that the intermediate result can be cached, transposed and other operations can be completed directly in the computing core, avoiding the repeated transfer of data in and out of the core, thereby significantly improving the computing efficiency and reducing the hardware overhead.

[0042] Specifically, the two-dimensional signal transformation in-memory computing architecture 10 of the embodiment of the present application includes:

[0043] The data input module 11 divides the externally input two-dimensional signal into column data by column, and caches and encodes the data into a data format supported by the row driving module 13, which is usually composed of a storage unit and a decoder, wherein the two-dimensional signal is, for example, picture data;

[0044] A crossbar array 12, used to store a parameter matrix of a one-dimensional signal transformation, and to perform a matrix multiplication operation between an input matrix and a parameter matrix to obtain a one-dimensional signal transformation result. The crossbar array may be a memristor 1T1R array, a 2T2R array, etc.;

[0045] The row driving module 13 is used to drive the row lines of the cross array, convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result;

[0046] A column driving module 14, used to drive the column lines of the crossbar array;

[0047] The cache module 15 is used to store all the first one-dimensional signal transformation results of the cross array calculation, reorganize and transpose them, so as to realize the in-memory calculation of the two-dimensional signal transformation. It should be noted that the input of the cache module 15 is completed on the row line and the output is completed on the column line, so that the transposition operation of the intermediate result matrix can be completed directly in the memory to realize the in-memory calculation;

[0048] The data output module 16 is used to store the second one-dimensional signal transformation result, sort it out and output it to the outside of the core.

[0049] In an embodiment of the present application, the parameter matrix includes at least one of a discrete Fourier transform matrix, a discrete cosine transform matrix, and a discrete wavelet transform matrix.

[0050] In the embodiment of the present application, the architecture 10 of the embodiment of the present application also includes: a data conversion module.

[0051] Among them, the data conversion module is used to obtain the output data of the column driving module 14, convert the first one-dimensional signal transformation result into a first digital signal vector, store the first one-dimensional signal transformation result into the cache module 15 based on the first digital signal vector, and convert all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and output the second digital signal vector to the data output module 16.

[0052] The first digital signal vector is used to represent the result of the first one-dimensional signal transformation, and the second digital signal vector is used to represent the result of the second one-dimensional signal transformation.

[0053] It can be understood that the architecture 10 of the embodiment of the present application also includes a data conversion module, which is used to convert the first one-dimensional signal transformation result into a first data signal vector, and store the first one-dimensional signal transformation result into a cache module 15, and convert the second one-dimensional signal transformation result into a second array signal vector, and output the second digital signal vector to the data output module 16 for subsequent output of the in-memory calculation results.

[0054] In the embodiment of the present application, the architecture 10 of the embodiment of the present application also includes: a first encoding module.

[0055] The first encoding module is used to perform encoding operation on the first digital signal vector, and write the output data of the first encoding module into the cache module in parallel by row lines.

[0056] It can be understood that the structure 10 of the embodiment of the present application also includes a first encoding module, which is used to perform an encoding operation on the first digital signal vector and write the encoded data into the cache module 15 in parallel by row lines.

[0057] In an embodiment of the present application, if the data input module 11 performs a multi-bit data batch input operation, the first encoding module performs a shift-and-add operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces an inverse code or a complement code, the first encoding module re-encodes the first digital signal vector into the original code form; after the first encoding module performs the shift-and-add operation or re-encoding, the first digital signal vector is sorted according to the cache sequence format of the cache module 15.

[0058] It can be understood that the architecture 10 of the embodiment of the present application also supports multi-bit data batch input operations, so as to be able to flexibly process data of different formats, thereby enhancing the flexibility of the architecture, and performing a shift-and-add operation on the first digital signal vector to ensure that the data input in batches still maintains the corresponding weights after the operation. If the first one-dimensional signal transformation result introduces an inverse code or a complement code, the first encoding module will re-encode the first digital signal vector into the original code form. After the first encoding module performs a shift-and-add operation or re-encoding, the first digital signal vector is sorted according to the cache sequence format of the cache module 15.

[0059] In the embodiment of the present application, the architecture 10 of the embodiment of the present application also includes: a second encoding module.

[0060] The second encoding module is used to perform at least one encoding operation of accumulation operation, shift operation and original code complement code exchange operation on the data cached by the cache module 15 , and input the data after the encoding operation into the row driving module 13 .

[0061] It can be understood that the architecture 10 of the embodiment of the present application also includes a second encoding module, which is used to perform at least one encoding operation of addition operation, shift operation and original code complement exchange operation on the data cached by the cache module, and input the data after the encoding operation into the row driving module 13, and realize the subsequent two-dimensional signal transformation calculation in the memory by encoding the data of the cache module 15.

[0062] In the embodiment of the present application, the data input module 11 allows partial data of the two-dimensional signal to be cached each time.

[0063] It can be understood that the data input module 11 of the embodiment of the present application allows partial data of the two-dimensional signal to be cached each time, thereby reducing the demand for the capacity of the cache module.

[0064] In the embodiment of the present application, the computing core of the crossbar array 12 includes: different types of memristors, storage-computing integrated architectures of different structures, or storage-computing integrated units of different structures.

[0065] It can be understood that the computing core of the cross array 13 of the embodiment of the present application includes: (such as resistive memory, phase change memory, magnetic random access memory, etc.), storage and computing integrated architectures of different structures or storage and computing integrated units of different structures (such as 1T1R, 2T2R, 2T1R, etc.) to adapt to different application requirements and technical conditions.

[0066] In summary, the specific structure of the two-dimensional signal conversion in-memory computing architecture of the embodiment of the present application is as follows: Figure 5 As shown, it contains nine modules, namely:

[0067] 1. Data input module: receives input data from outside the core, caches the data and encodes it into a data format supported by the subsequent row driver module; usually consists of a storage unit and a decoder.

[0068] 2. Row driver module: receives data from the data input module or the second encoding module, and drives the row lines of the crossbar array based on the data. For example, the BL driver of the memristor storage and computing integrated chip.

[0069] 3. Cross array: stores the parameter matrix of one-dimensional signal transformation (e.g., discrete Fourier transform matrix, discrete cosine transform matrix, discrete wavelet transform matrix, etc.), and performs matrix-vector multiplication between the input vector and the one-dimensional signal transformation matrix. For example, the cross array can be a memristor 1T1R array or a 2T2R array.

[0070] 4. Column driver module: drives the column lines of the crossbar array. For example, the SL driver of the memristor storage and computing chip.

[0071] 5. Data conversion module: converts the calculation results of the crossbar array into a data form that is convenient for storage, transmission or subsequent calculation. For example, in a memristor storage and computing integrated chip, the "data conversion module" is usually a transimpedance amplifier (TIA) and an analog-to-digital converter (ADC).

[0072] 6. The first encoding module: encodes the cross array calculation results so that the data can be directly stored in the cache module later; usually this module includes operations such as accumulation, shift, and original code and complement code interchange.

[0073] 7. Cache module: cache, reorganize and transpose the calculation results of the first one-dimensional signal transformation; it should be noted that the input of the cache module is completed on the row line and the output is completed on the column line, so that the transposition operation of the intermediate result matrix can be completed directly in the memory to realize in-memory calculation.

[0074] 8. Second encoding module: encodes the output data of the cache module into a data format supported by the row driver module.

[0075] 9. Data output module: cache, organize and output the calculation results of the second one-dimensional signal transformation to the outside of the core.

[0076] based on Figure 5 The two-dimensional signal transformation in-memory computing architecture shown in the figure specifically describes the data flow during the two-dimensional signal transformation operation, including the following steps:

[0077] 1. Data input, such as Figure 6 shown.

[0078] Data input: The input two-dimensional signal (such as image data) is input from outside the core to the data input module. The data input module completes data caching and preprocessing, and then divides the two-dimensional signal by column. After each column of data is encoded, it will be input into the row driver module in units of column vectors. It should be noted that if each pixel is represented by multi-bit data, it needs to be input in batches according to the accuracy that the row driver module can support; for example, if each pixel is represented by 8-bit data, and the DAC (digital-to-analog converter) set in the row driver module has an accuracy of 2 bits, then the data of a column vector needs to be input into the row driver module in 4 times, such as Figure 7 In addition, it should be noted that in some implementation methods, the data input module does not need to cache the complete two-dimensional signal (for example, the entire image), but can only receive one column of input data from the outside of the computing core each time, encode this column of input data and send it to the row driver module, and then receive and process the second column of input data from the outside of the computing core.

[0079] Second, the first one-dimensional signal transformation, such as Figure 8 shown.

[0080] After receiving a column of data processed by the data input module, the row driving module converts the data into a signal form suitable for the cross array to perform operations (for example, if the cross array is a memristor array, the signal form is "voltage"), and then applies it to the cross array to complete the matrix-vector multiplication operation. The calculation result is converted into a column of digital signals by the data conversion module, which represents the result of the one-dimensional signal transformation corresponding to a column of input data.

[0081] 3. Data caching and reorganization, such as Fig. 9 shown.

[0082] A column of digital signals output by the data conversion module is encoded in the first encoding module. Specifically, three operations need to be completed: first, if the aforementioned data input step adopts a method of batch input of multi-bit data, it is necessary to complete the shift accumulation operation in the first encoding module to ensure that the batch input data still maintains the corresponding weight after the operation; second, if the inverse code or complement code is introduced in the first one-dimensional signal transformation step, it is necessary to re-encode it into the original code form; third, after obtaining a column of one-dimensional signal transformation calculation results, the order of the calculation results needs to be rearranged according to the data input requirements of the cache module. After completing the above three steps of data encoding operations, a column of intermediate result data cached in the data conversion module will be written into the cache module in parallel by the row line. At this time, a column of storage units in the cache module corresponds to a column of intermediate result data. Subsequently, the first one-dimensional signal transformation and the first half of the data caching and reorganization steps are repeated until all column vectors cached in the data input module are calculated through the cross array to complete the one-dimensional signal transformation and stored in the cache module.

[0083] Fourth, the second one-dimensional signal transformation, such as Fig.10 shown.

[0084] The cache module is full of intermediate result data after the first one-dimensional signal transformation. Next, the intermediate result data will be read out in parallel by the column lines of the cache module in rows and sent to the second encoding module. In the second encoding module, a row of intermediate result data will be encoded into a data form supported by the subsequent row driver module. Subsequently, after the row driver module receives the data processed by the second encoding module, it converts the data into a signal form suitable for the cross array to perform operations (for example, if the cross array is a memristor array, the signal form is voltage), and then applies it to the cross array to complete the matrix-vector multiplication operation. The calculation result is converted into a digital signal vector by the data conversion module. The digital signal vector represents the result of a row of intermediate result data after the second one-dimensional signal transformation is performed after transposition, and is then cached in the data output module.

[0085] 5. Data output, such as Fig.11 shown.

[0086] The data output module performs the last encoding and post-processing on the data stored therein to achieve interface matching with the external circuit and output the final two-dimensional signal transformation calculation result.

[0087] According to the two-dimensional signal transformation in-memory computing architecture proposed in the embodiment of the present application, since the two-dimensional signal transformation needs to undergo two transformations, the first one-dimensional transformation and the second one-dimensional transformation, to output the final result of the two-dimensional signal transformation in-memory computing, but the existing cross array computing core can only complete one-dimensional signal transformation once, and cannot perform a complete two-dimensional signal transformation, therefore, in the two-dimensional signal transformation in-memory computing architecture of the embodiment of the present application, the two-dimensional signal first passes through the data input module, the cross array, the row driver module, and the column driver module to perform the first one-dimensional signal transformation, and a cache module is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can be directly cached, transposed and other operations in the cache module to achieve the reorganization of the intermediate result, and then the second one-dimensional signal transformation is performed in situ, and the final data output module determines the result of the two-dimensional signal transformation in-memory computing according to the second one-dimensional signal transformation result to output the result of the two-dimensional signal transformation in-memory computing, so that the intermediate result can be cached, transposed and other operations can be completed directly in the computing core, avoiding the repeated transfer of data in and out of the core, thereby significantly improving the computing efficiency and reducing the hardware overhead.

[0088] The following describes the two-dimensional signal conversion memory computing architecture of the present application through a specific embodiment, taking the two-dimensional signal conversion memory computing architecture based on memristor and SRAM as an example. Fig.12 As shown, including:

[0089] 1. BL_R<1023:0>: 1024-bit register, corresponding to Figure 5 The data input module shown is used to organize and cache input data; the input data is input by bus BL_DATAIN<63:0>, and 64 bits of data are input per cycle. The 64 bits of data need to be cached to a specific location in BL_R<1023:0> according to the address signal BLROW_REGA<4:0>; here, at least 16 cycles are required to complete the input of 1024 bits of data.

[0090] 2. BL_IN_MUX: used to determine whether the data input to the subsequent driver module BL_DRV is BL_IN<1023:0> from BL_R<1023:0>, or RTN_DATA<1023:0> from the DECODER module (corresponding to Figure 5 The second transform input data in ).

[0091] 3. BL_DRV: corresponding Figure 5The “row driver module” shown in FIG. 1 receives the 1024-bit data passed in by BL_IN_MUX, encodes it into BL_POS<575:0> and BL_NEG<575:0>, and applies it to the memristor array (ARRAY) in the form of a voltage signal.

[0092] 4. ARRAY: memristor array, corresponding to Figure 5 The cross array in .

[0093] 5. ADC<63:0>: 64-channel analog-to-digital converter, corresponding to Figure 5 The data conversion module in the memory resistor array receives the current of the source line (Source line, SL) SL<511:0> and converts it into a digital signal <*64>ADC_OUT<7:0>. Note that for simplicity, the Figure 5 The column driver module shown in the figure needs to be retained during actual chip design.

[0094] 6. ADD_ENCODER and <*2>ADD_12BIT<255:0>: corresponding Figure 5 The first encoding module shown can realize the functions of shift accumulation, encoding, and data caching; receive the signal <*64>ADC_OUT<7:0> output by ADC<63:0>, and finally organize the data into <*256>ADD_OUT<7:0>; the module is controlled by signals WLGRP_SEL<2:0>, ADD_EN, ADD_MD, and ADD_CLR.

[0095] 7. SRAM: corresponding Figure 5 In the cache module shown, the SRAM array data is written in parallel by row lines and read out in parallel by column lines, so the in-situ transposition in-memory computing function can be realized; the module is controlled by signals STATE, REG_SCLK, and SRAM_ADDR<10:0>.

[0096] 8. SIGN_BUF and DECODER: corresponding Figure 5 The second encoding module shown is used to encode the buffered intermediate result data to generate the second transformation input data RTN_DATA<1023:0>; the module is controlled by the signal SIGN_EN.

[0097] 9. OUTPUT: Correspondence Figure 5 The data output module shown is used to output the calculation result of the two-dimensional signal transformation to the outside of the computing core. The data bus is DO<7:0> and is controlled by the signal DO_SEL<7:0>.

[0098] An embodiment of the present application also provides an electronic device, comprising the above-mentioned two-dimensional signal conversion in-memory computing architecture.

[0099] Next, the two-dimensional signal transformation in-memory calculation method proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0100] Fig.13 It is a flow chart of the in-memory calculation method for two-dimensional signal transformation according to an embodiment of the present application.

[0101] like Fig.13 As shown, the two-dimensional signal transformation in-memory calculation method includes the following steps:

[0102] In step S101 , an externally input two-dimensional signal is divided into column data by column.

[0103] The two-dimensional signal may be image data.

[0104] It is understandable that the embodiment of the present application can divide the externally input two-dimensional signal into column data by column to facilitate subsequent data calculation.

[0105] In step S102, a cross array is used to store a parameter matrix of a one-dimensional signal transformation, and a matrix-vector multiplication operation is performed between the input vector and the parameter matrix to obtain a one-dimensional signal transformation result.

[0106] It is understandable that the embodiments of the present application can utilize a cross array to store a parameter matrix of a one-dimensional signal transformation, and perform a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result.

[0107] In step S103, the row lines of the cross array are driven to convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result based on the column data, and the cross array calculates the second one-dimensional signal transformation result based on the first one-dimensional signal transformation result.

[0108] Among them, the cross array calculates the first one-dimensional signal transformation result based on the column data, and the cross array calculates the second one-dimensional signal transformation result based on the first one-dimensional signal transformation result

[0109] It can be understood that the embodiments of the present application can directly drive the row lines of the cross array, reduce the intermediate data processing steps, improve efficiency, and convert the column data and the first one-dimensional signal transformation results of the cross array into the input vector of the cross array.

[0110] In step S104, the column lines of the crossbar array are driven and the crossbar array calculation results are output.

[0111] In step S105, all the first one-dimensional signal transformation results of the cross array calculation are stored, and a transposition operation is performed on all the first one-dimensional signal transformation results of the cross array calculation to achieve in-memory calculation of two-dimensional signal transformation.

[0112] It can be understood that the embodiments of the present application can store all the first one-dimensional signal transformation results of the cross array calculation, and perform a transposition operation on all the first one-dimensional signal transformation results of the cross array calculation to realize in-memory calculation of two-dimensional signal transformation, and directly complete the transposition operation to avoid repeated movement of data in and out of the core.

[0113] In step S106, the result of the in-memory calculation of the two-dimensional signal transformation is determined according to the second one-dimensional signal transformation result of the cross array, and the result of the in-memory calculation of the two-dimensional signal transformation is output.

[0114] It can be understood that the embodiments of the present application can determine the result of the two-dimensional signal transformation in-memory calculation based on the second one-dimensional signal transformation result of the cross array, and output the result of the two-dimensional signal transformation in-memory calculation to implement the two-dimensional signal transformation in-memory calculation in a two-dimensional signal transformation in-memory calculation architecture.

[0115] It should be noted that the aforementioned explanation of the embodiment of the two-dimensional signal transformation in-memory computing architecture is also applicable to the two-dimensional signal transformation in-memory computing architecture method of this embodiment, and will not be repeated here.

[0116] According to the two-dimensional signal transformation in-memory computing architecture method proposed in the embodiment of the present application, direct caching, reorganization and transposition operations of intermediate results can be implemented directly in the computing core, avoiding repeated transfer of data in and out of the core, significantly improving computing efficiency, and reducing hardware overhead.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0118] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0119] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0120] It should be understood that the various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0121] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0122] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A two-dimensional signal transformation in-memory computing architecture, characterized in that: include: A data input module, used for dividing an externally input two-dimensional signal into column data by columns; A cross array, used to store a parameter matrix of a one-dimensional signal transformation, and to perform a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; A row driving module, used to drive the row lines of the cross array, and convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; A column driving module, used to drive the column lines of the cross array and output the cross array calculation results; A cache module, used for storing all the first one-dimensional signal transformation results of the cross array calculation, and performing a transposition operation on all the first one-dimensional signal transformation results of the cross array calculation to realize the in-memory calculation of the two-dimensional signal transformation; The data output module is used to determine the result of the two-dimensional signal transformation in-memory calculation according to the second one-dimensional signal transformation result of the cross array, and output the result of the two-dimensional signal transformation in-memory calculation.

2. The two-dimensional signal transformation in-memory computing architecture according to claim 1, characterized in that: Also includes: A data conversion module is used to obtain the output data of the column driving module, convert the first one-dimensional signal transformation result into a first digital signal vector, store the first one-dimensional signal transformation result into the cache module based on the first digital signal vector, and convert all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and output the second digital signal vector to the data output module.

3. The two-dimensional signal transformation in-memory computing architecture according to claim 2, characterized in that: Also includes: The first encoding module is used to perform encoding operation on the first digital signal vector, and write the output data of the first encoding module into the cache module in parallel via row lines.

4. The two-dimensional signal transformation in-memory computing architecture according to claim 3, characterized in that: If the data input module performs a multi-bit data batch input operation, the first encoding module performs a shift-and-add operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces an inverse code or a complement code, the first encoding module re-encodes the first digital signal vector into the original code form; after the first encoding module performs a shift-and-add operation or re-encoding, the first digital signal vector is sorted according to the cache sequence format of the cache module.

5. The two-dimensional signal transformation in-memory computing architecture according to claim 1, characterized in that: Also includes: The second encoding module is used to perform at least one encoding operation of an accumulation operation, a shift operation and an original code and a complement code exchange operation on the data cached by the cache module, and input the data after the encoding operation into the row driving module.

6. The two-dimensional signal transformation in-memory computing architecture according to claim 1, characterized in that: The data input module allows buffering of part of the data of the two-dimensional signal each time.

7. The two-dimensional signal transformation in-memory computing architecture according to claim 1, characterized in that: The parameter matrix includes at least one of a discrete Fourier transform matrix, a discrete cosine transform matrix, and a discrete wavelet transform matrix.

8. The two-dimensional signal transformation in-memory computing architecture according to claim 1, characterized in that: The computing core of the crossbar array includes: different types of memristors, storage-computing integrated architectures of different structures, or storage-computing integrated units of different structures.

9. An electronic device, characterized in that: It includes a two-dimensional signal transformation in-memory computing architecture as described in any one of claims 1-8.

10. A two-dimensional signal transformation in-memory calculation method, characterized in that: The following steps are involved: Divide the external input two-dimensional signal into column data by column; Using a cross array to store a parameter matrix of a one-dimensional signal transformation, performing a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; driving the row lines of the cross array, converting the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; driving the column lines of the cross array and outputting the calculation results of the cross array; Storing all the first one-dimensional signal transformation results of the cross array calculation, and performing a transposition operation on all the first one-dimensional signal transformation results of the cross array calculation to realize the in-memory calculation of the two-dimensional signal transformation; The result of the in-memory calculation of the two-dimensional signal transformation is determined according to the second one-dimensional signal transformation result of the cross array, and the result of the in-memory calculation of the two-dimensional signal transformation is output.

Images