Two-dimensional signal transform in-memory computing architecture, electronic device and method

Abstract

The application relates to the technical field of signals, in particular to a two-dimensional signal transformation in-memory computing architecture, an electronic device and a method. The architecture comprises a data input module, a cross array, a row driving module, a column driving module, a cache module and a data output module. The cross array performs a matrix vector multiplication operation between an input vector and a parameter matrix to obtain a one-dimensional signal transformation result. The row driving module converts column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array. The column driving module is used for outputting the calculation result of the cross array. The cache module is used for storing all the first one-dimensional signal transformation results of the cross array calculation, and performing a transposition operation on all the first one-dimensional signal transformation results of the cross array calculation to realize two-dimensional signal transformation in-memory computing. The data output module is used for outputting the result of the two-dimensional signal transformation in-memory computing. Thus, the problems that complete two-dimensional signal transformation cannot be performed in the related art, data is repeatedly transported in and out of a core, the computing efficiency is reduced, and the hardware cost is high are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of signal processing, in particular to a two-dimensional signal transformation in-memory computing architecture, an electronic device and a method. BACKGROUND

[0002] Two-dimensional signal transformation is a widely used image processing method, including two-dimensional discrete Fourier transform (2D-DFT), two-dimensional discrete cosine transform (2D-DCT), two-dimensional discrete wavelet transform (2D-DWT), etc. The calculation process of the two-dimensional signal transformation algorithm can be roughly divided into three steps: first one-dimensional signal transformation, intermediate result caching, reorganization and transposition, and second signal transformation. In recent years, with the continuous increase in image size and the continuous improvement in video frame rate, the traditional computing hardware will bring serious energy consumption and operation delay when executing the two-dimensional signal transformation algorithm due to the bottleneck of the Von Neumann architecture.

[0003] One-dimensional signal transformation algorithm can be efficiently implemented on a cross-array (including but not limited to resistive random access memory RRAM, phase change memory PCM, and magnetic random access memory MRAM) based computing core, with an energy efficiency improvement of about 2 orders of magnitude [1-3]. This is because the one-dimensional signal transformation algorithm can be equivalent to the matrix-vector multiplication operation between the input vector and the transformation matrix, and the cross-array computing core can efficiently complete the matrix-vector multiplication operation based on physical laws.

[0004] However, the current cross-array computing core can only complete one-dimensional signal transformation, and cannot perform complete two-dimensional signal transformation. The intermediate result data can only be moved out of the computing core for caching and reorganization, and the transposition operation can be completed by a digital processing unit (CPU, FPGA, MCU, etc.) before being input to the cross-array computing core to complete the second one-dimensional signal transformation. The repeated movement of data will cause additional hardware overhead. SUMMARY

[0005] The present application provides a two-dimensional signal transformation in-memory computing architecture, an electronic device and a method to solve the problems in the related art that the complete two-dimensional signal transformation cannot be performed, the data is repeatedly moved in and out of the core, the computing efficiency is reduced, and the hardware overhead is large.

[0006] The first aspect of the present application provides an in-memory computing architecture for two-dimensional signal transformation, including: a data input module for dividing an externally input two-dimensional signal into column data by columns; a cross array for storing a parameter matrix for one-dimensional signal transformation, and performing a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; a row driving module for driving the row lines of the cross array, and converting the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result based on the column data, and the cross array calculates the second one-dimensional signal transformation result based on the first one-dimensional signal transformation result; a column driving module for driving the column lines of the cross array and outputting the calculation results of the cross array; a cache module for storing all the first one-dimensional signal transformation results calculated by the cross array, and performing a transposition operation on all the first one-dimensional signal transformation results calculated by the cross array to realize in-memory computing of two-dimensional signal transformation; a data output module for determining the result of the in-memory computing of the two-dimensional signal transformation based on the second one-dimensional signal transformation result of the cross array, and outputting the result of the in-memory computing of the two-dimensional signal transformation.

[0007] Optionally, it also includes: a data conversion module, used to obtain the output data of the column driving module, convert the first one-dimensional signal transformation result into a first digital signal vector, store the first one-dimensional signal transformation result into a cache module based on the first digital signal vector, and convert all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and output the second digital signal vector to the data output module.

[0008] Optionally, it further includes: a first encoding module, used to perform an encoding operation on the first digital signal vector, and write the output data of the first encoding module into the cache module in parallel by row lines.

[0009] Optionally, if the data input module performs a multi-bit data batch input operation, the first encoding module performs a shift-and-add operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces an inverse code or a complement code, the first encoding module re-encodes the first digital signal vector into the original code form; after the first encoding module performs the shift-and-add operation or re-encoding, the first digital signal vector is sorted according to the cache sequence format of the cache module.

[0010] Optionally, it further includes: a second encoding module, which is used to perform at least one encoding operation of accumulation operation, shift operation and original code complement exchange operation on the data cached by the cache module, and input the data after the encoding operation into the row driving module.

[0011] Optionally, the data input module allows buffering of partial data of the two-dimensional signal each time.

[0012] Optionally, the parameter matrix comprises at least one of a discrete Fourier transform matrix, a discrete cosine transform matrix, and a discrete wavelet transform matrix.

[0013] Optionally, the computing core of the cross array comprises different types of memristors, different structures of memory-computing integrated architectures, or different structures of memory-computing integrated units.

[0014] The second aspect embodiment of the present application provides an electronic device comprising the two-dimensional signal transformation in-memory computing architecture of the above embodiments.

[0015] The second aspect embodiment of the present application provides a two-dimensional signal transformation in-memory computing method, comprising the following steps: dividing an externally input two-dimensional signal into column data according to columns; storing a parameter matrix of one-dimensional signal transformation in a cross array, performing a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; driving a row line of the cross array to convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates a second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; driving a column line of the cross array to output a calculation result of the cross array; storing all the first one-dimensional signal transformation results calculated by the cross array, and performing a transposition operation on all the first one-dimensional signal transformation results calculated by the cross array to realize two-dimensional signal transformation in-memory computing; determining a result of the two-dimensional signal transformation in-memory computing according to the second one-dimensional signal transformation result of the cross array, and outputting the result of the two-dimensional signal transformation in-memory computing.

[0016] Thus, the present application includes the following beneficial effects:

[0017] Since two-dimensional signal transformation needs to pass through two transformations, the first one-dimensional transformation and the second one-dimensional transformation, the output final two-dimensional signal transformation in-memory calculation result, but the existing cross array calculation core can only complete one one-dimensional signal transformation, and cannot perform complete two-dimensional signal transformation, therefore, the two-dimensional signal transformation in-memory calculation architecture provided by the embodiment of the application, the two-dimensional signal first passes through the data input module, the cross array, the row driving module, the column driving module to perform the first one-dimensional signal transformation, and the cache module is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can directly complete the cache, transposition and other operations in the cache module to realize the reorganization of the intermediate result, and then the second one-dimensional signal transformation is performed in place, and finally the data output module determines the two-dimensional signal transformation in-memory calculation result according to the second one-dimensional signal transformation result to output the two-dimensional signal transformation in-memory calculation result, so that the cache, transposition and other operations of the intermediate result can be directly completed in the calculation core, the repeated carrying of data in and out of the core is avoided, the calculation efficiency is significantly improved, and the hardware overhead is reduced. Therefore, the technical problems that complete two-dimensional signal transformation cannot be performed in the related art, the calculation efficiency is reduced due to repeated carrying of data in and out of the core, and the hardware overhead is large are solved.

[0018] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and / or additional aspects and advantages of the application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0020] Figure 1 The calculation flowchart of the two-dimensional signal transformation algorithm is shown in the figure;

[0021] Figure 2 The matrix vector multiplication operation schematic diagram based on the cross array calculation core is shown in the figure;

[0022] Figure 3 The schematic diagram of the cross array calculation core architecture in the related art is shown in the figure;

[0023] Figure 4 The schematic diagram of the two-dimensional signal transformation in-memory calculation architecture provided by the embodiment of the application is shown in the figure;

[0024] Figure 5 The specific structure diagram of the two-dimensional signal transformation in-memory calculation architecture provided by the embodiment of the application is shown in the figure;

[0025] Figure 6 The data input schematic diagram provided by the embodiment of the application is shown in the figure;

[0026] Figure 7A multi-bit data splitting input diagram provided according to an embodiment of the present application;

[0027] Figure 8 A first one-dimensional signal transformation diagram provided according to an embodiment of the present application;

[0028] Figure 9 A data caching and reorganizing diagram provided according to an embodiment of the present application;

[0029] Figure 10 A second one-dimensional signal transformation diagram provided according to an embodiment of the present application;

[0030] Figure 11 A data output diagram provided according to an embodiment of the present application;

[0031] Figure 12 A two-dimensional signal transformation in-memory computing architecture diagram based on a memristor and an SRAM provided according to an embodiment of the present application;

[0032] Figure 13 A two-dimensional signal transformation in-memory computing method flowchart provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar elements or elements having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0034] Before describing the technical solutions of the present application, the following related art is described in the form of words and pictures to assist in understanding the technical solutions of the present application.

[0035] As shown in Figure 1 , the calculation flow of the two-dimensional signal transformation algorithm mainly includes a first one-dimensional signal transformation, intermediate result caching, reorganization and transposition, and a second signal transformation. The one-dimensional signal transformation algorithm can be efficiently implemented on a cross-array-based computing core. The one-dimensional signal transformation algorithm can be equivalent to matrix-vector multiplication between an input vector and a transformation matrix, and the cross-array computing core can efficiently complete the matrix-vector multiplication operation based on physical laws, as shown in Figure 2 .

[0036] The current cross-array computing core, as shown in Figure 3As shown, only one-dimensional signal transformation can be completed, and complete two-dimensional signal transformation cannot be performed. This is due to two reasons. First, there is a lack of cache space in the core, and the intermediate result matrix cannot be cached. Second, there is a lack of a transpose operation unit in the core, and the transpose operation cannot be performed on the intermediate result matrix. Both of these result in intermediate result data being moved out of the computing core for caching and reorganization, and the transpose operation being completed by the digital processing unit before being input to the cross array computing core to complete the second one-dimensional signal transformation. This repeated movement of data causes additional hardware overhead.

[0037] To this end, the present application provides a two-dimensional signal transformation in-memory computing architecture. In the architecture, the two-dimensional signal is first subjected to a first one-dimensional signal transformation by a data input module, a cross array, a row drive module, and a column drive module. A cache module is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can be directly cached, transposed, and the like in the cache module to reorganize the intermediate result, and then a second one-dimensional signal transformation is performed. Finally, a data output module determines the result of the two-dimensional signal transformation in-memory computing according to the second one-dimensional signal transformation result to output the result of the two-dimensional signal transformation in-memory computing. Thus, the intermediate result can be directly cached, transposed, and the like in the computing core, avoiding repeated movement of data in and out of the core, thereby significantly improving computing efficiency and reducing hardware overhead. Thus, the problems of being unable to perform complete two-dimensional signal transformation, and the repeated movement of data in and out of the core resulting in reduced computing efficiency and large hardware overhead are solved.

[0038] Specifically, Figure 4 A schematic diagram of a two-dimensional signal transformation in-memory computing architecture provided by an embodiment of the present application.

[0039] As Figure 4 shown, the two-dimensional signal transformation in-memory computing architecture 10 includes a data input module 11, a cross array 12, a row drive module 13, a column drive module 14, a cache module 15, and a data output module 16.

[0040] The data input module 11 is configured to split the externally input two-dimensional signal into column data according to columns; the cross array 12 is configured to store a parameter matrix of one-dimensional signal transformation, and perform a matrix vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; the row driving module 13 is configured to drive a row line of the cross array, and convert the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates a second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; the column driving module 14 is configured to drive a column line of the cross array, and output a calculation result of the cross array; the cache module 15 is configured to store all the first one-dimensional signal transformation results calculated by the cross array, and perform a transposition operation on all the first one-dimensional signal transformation results calculated by the cross array, so as to realize in-memory calculation of two-dimensional signal transformation; and the data output module 16 is configured to determine a result of the in-memory calculation of two-dimensional signal transformation according to the second one-dimensional signal transformation result of the cross array, and output the result of the in-memory calculation of two-dimensional signal transformation.

[0041] The two-dimensional signal transformation needs to pass through two transformations, i.e., a first one-dimensional transformation and a second one-dimensional transformation, and finally outputs a result of in-memory calculation of two-dimensional signal transformation. However, the existing cross array calculation core can only complete one one-dimensional signal transformation, and cannot perform complete two-dimensional signal transformation. Therefore, in the two-dimensional signal transformation in-memory calculation architecture 10, the two-dimensional signal first passes through the data input module 11, the cross array 12, the row driving module 13 and the column driving module 14 to perform the first one-dimensional signal transformation, and the cache module 15 is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can directly complete cache, transposition and other operations in the cache module 15, to recombine an intermediate result, and then perform the second one-dimensional signal transformation in place. Finally, the data output module 16 determines a result of in-memory calculation of two-dimensional signal transformation according to the second one-dimensional signal transformation result, to output the result of in-memory calculation of two-dimensional signal transformation. Therefore, the cache, transposition and other operations of the intermediate result can be directly completed in the calculation core, the repeated carrying of data between the inside and outside of the calculation core is avoided, the calculation efficiency is significantly improved, and the hardware cost is reduced.

[0042] Specifically, the two-dimensional signal transformation in-memory calculation architecture 10 includes:

[0043] The data input module 11 splits the externally input two-dimensional signal into column data according to columns, and buffers and encodes the data into a data form supported by the row driving module 13, and is usually composed of a storage unit and a decoder, wherein the two-dimensional signal is, for example, picture data;

[0044] The cross array 12 is used for storing a parameter matrix of one-dimensional signal transformation and performing a matrix connection multiplication operation between an input connection and the parameter matrix to obtain a one-dimensional signal transformation result. The cross array can be a memristor 1T1R array, a 2T2R array, etc.

[0045] The row driving module 13 is used for driving a row line of the cross array, converting column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array. The cross array calculates the first one-dimensional signal transformation result according to the column data. The cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result.

[0046] The column driving module 14 is used for driving a column line of the cross array.

[0047] The cache module 15 is used for storing all the first one-dimensional signal transformation results calculated by the cross array, recombining and transposing to realize in-memory calculation of two-dimensional signal transformation. It needs to be noted that the input of the cache module 15 is completed on the row line, and the output is completed on the column line. Thus, the transposition operation of the intermediate result matrix can be directly completed in the memory to realize in-memory calculation.

[0048] The data output module 16 is used for storing the second one-dimensional signal transformation result, arranging and outputting to outside the core.

[0049] In the embodiment of the application, the parameter matrix includes at least one of a discrete Fourier transform matrix, a discrete cosine transform matrix and a discrete wavelet transform matrix.

[0050] In the embodiment of the application, the architecture 10 of the embodiment of the application further includes a data conversion module.

[0051] The data conversion module is used for obtaining output data of the column driving module 14, converting the first one-dimensional signal transformation result into a first digital signal vector, storing the first one-dimensional signal transformation result into the cache module 15 based on the first digital signal vector, converting all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and outputting the second digital signal vector to the data output module 16.

[0052] The first digital signal vector is used for representing the first one-dimensional signal transformation result, and the second digital signal vector is used for representing the second one-dimensional signal transformation result.

[0053] It can be understood that the architecture 10 of the embodiment of the present application further comprises a data conversion module, which is configured to convert the first one-dimensional signal transformation result into a first digital signal vector, store the first one-dimensional signal transformation result in the cache module 15, convert the second one-dimensional signal transformation result into a second digital signal vector, and output the second digital signal vector to the data output module 16 for subsequent output of the calculation result.

[0054] In the embodiment of the present application, the architecture 10 of the embodiment of the present application further comprises a first encoding module.

[0055] The first encoding module is configured to perform an encoding operation on the first digital signal vector, and write the output data of the first encoding module to the cache module in parallel through the row line.

[0056] It can be understood that the architecture 10 of the embodiment of the present application further comprises a first encoding module, which is configured to perform an encoding operation on the first digital signal vector, and write the encoded data to the cache module 15 in parallel through the row line.

[0057] In the embodiment of the present application, if the data input module 11 performs a multi-bit data batch input operation, the first encoding module performs a shift accumulation operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces a reverse code or a complement code, the first encoding module re-encodes the first digital signal vector into a normal code form; and after the first encoding module performs the shift accumulation operation or the re-encoding, the first digital signal vector is sorted according to the cache order format of the cache module 15.

[0058] It can be understood that the architecture 10 of the embodiment of the present application further supports a multi-bit data batch input operation, so as to flexibly process data of different formats, enhance the flexibility of the architecture, perform a shift accumulation operation on the first digital signal vector, ensure that the data input in batches still maintains a corresponding weight after operation, re-encode the first digital signal vector into a normal code form if the first one-dimensional signal transformation result introduces a reverse code or a complement code, and sort the first digital signal vector according to the cache order format of the cache module 15 after the first encoding module performs the shift accumulation operation or the re-encoding.

[0059] In the embodiment of the present application, the architecture 10 of the embodiment of the present application further comprises a second encoding module.

[0060] The second encoding module is configured to perform at least one encoding operation of an accumulation operation, a shift operation and a normal code complement code interconversion operation on the data cached by the cache module 15, and input the data after the encoding operation to the row driving module 13.

[0061] It can be understood that the architecture 10 of the embodiment of the present application further comprises a second encoding module for performing at least one encoding operation of accumulation operation, shift operation and original code and complement interchanging operation on the data cached by the cache module, and inputting the data after the encoding operation into the row driving module 13, so as to realize the subsequent in-memory calculation of two-dimensional signal transformation by performing the encoding operation on the data of the cache module 15.

[0062] In the embodiment of the present application, the data input module 11 allows to cache part of the data of the two-dimensional signal each time.

[0063] It can be understood that the data input module 11 of the embodiment of the present application allows to cache part of the data of the two-dimensional signal each time, thereby reducing the requirement for the capacity of the cache module.

[0064] In the embodiment of the present application, the computing core of the cross array 12 comprises different types of memristors, different structures of memory-computing integrated architecture or different structures of memory-computing integrated units.

[0065] It can be understood that the computing core of the cross array 12 of the embodiment of the present application comprises different types of memory (such as resistive random access memory, phase change memory, magnetic random access memory, etc.), different structures of memory-computing integrated architecture or different structures of memory-computing integrated units (such as 1T1R, 2T2R, 2T1R, etc.), so as to adapt to different application requirements and technical conditions.

[0066] In summary, the specific structure of the two-dimensional signal transformation in-memory computing architecture of the embodiment of the present application is as shown in Figure 5 As shown in the figure, it comprises nine modules, specifically:

[0067] 1. Data input module: receiving the input data transmitted from outside the core, caching and encoding the data into the data form supported by the subsequent row driving module; usually composed of a storage unit and a decoder.

[0068] 2. Row driving module: receiving the data of the data input module or the second encoding module, and driving the row line of the cross array based on the data. For example, the BL driver of the memristor memory-computing integrated chip.

[0069] 3. Cross array: storing the parameter matrix of one-dimensional signal transformation (for example, discrete Fourier transform matrix, discrete cosine transform matrix, discrete wavelet transform matrix, etc.), and performing the matrix vector multiplication operation between the input vector and the one-dimensional signal transformation matrix. For example, the cross array can be a memristor 1T1R array or a 2T2R array.

[0070] 4. Column driving module: driving the column line of the cross array. For example, the SL driver of the memristor memory-computing integrated chip.

[0071] 5. Data conversion module: This module converts the computational results of the crossbar array into a data format that is convenient for storage, transmission, or subsequent computation. For example, in a memristor memory-computing integrated chip, the "data conversion module" is typically a transimpedance amplifier (TIA) and an analog-to-digital converter (ADC).

[0072] 6. First encoding module: encodes the cross-array calculation results so that the data can be directly stored in the cache module later; usually this module includes operations such as accumulation, shift, and original code and complement exchange.

[0073] 7. Cache module: Caches, reorganizes, and transposes the calculation results of the first one-dimensional signal transformation. It should be noted that the input of the cache module is completed on the row line, and the output is completed on the column line. In this way, the transposition operation of the intermediate result matrix can be completed directly in the memory, realizing in-memory calculation.

[0074] 8. Second encoding module: encodes the output data of the cache module into a data format supported by the row driver module.

[0075] 9. Data output module: caches, organizes and outputs the calculation results of the second one-dimensional signal transformation to the outside of the core.

[0076] based on Figure 5 The in-memory computing architecture for two-dimensional signal transformation shown in FIG. specifically describes the data flow during two-dimensional signal transformation operations, including the following steps:

[0077] 1. Data input, such as Figure 6 shown.

[0078] Data input: The input two-dimensional signal (such as image data) is input from outside the core to the data input module. The data input module completes data caching and preprocessing, and then divides the two-dimensional signal into columns. After encoding, each column of data will be input into the row driver module in units of column vectors. It should be noted that if each pixel is represented by multi-bit data, it needs to be input in batches according to the accuracy supported by the row driver module; for example, if each pixel is represented by 8-bit data, and the DAC (digital-to-analog converter) set in the row driver module has a precision of 2 bits, then the data of a column vector needs to be input into the row driver module in 4 batches, such as Figure 7 As shown. It should also be noted that in some implementations, the data input module does not need to cache the entire two-dimensional signal (e.g., the entire image). Instead, it can receive only one column of input data from the computing core at a time, encode this column of input data, and send it to the row driver module, which then receives and processes a second column of input data from the computing core.

[0079] Second, the first one-dimensional signal transformation, such as Figure 8 shown.

[0080] The row driving module receives the data processed by the data input module, converts the data into a signal form suitable for the cross array to perform operation (for example, if the cross array is a memristor array, the signal form is "voltage"), and then applies the data to the cross array to complete the matrix vector multiplication operation. The calculation result is converted by the data conversion module into a column of digital signals, which represent the result of the one-dimensional signal transformation corresponding to the column of input data.

[0081] III. Data buffering and reorganization, as shown in Figure 9

[0082] The column of digital signals output by the data conversion module is encoded in the first encoding module. Specifically, three steps are needed: first, if the aforementioned data input step adopts the method of batch input of multi-bit data, then the shift accumulation operation needs to be completed in the first encoding module to ensure that the batch input data still maintains the corresponding weight after operation; second, if the inverse code or the complement code is introduced in the first one-dimensional signal transformation step, it needs to be re-encoded into the original code form; third, after obtaining the calculation result of the one-dimensional signal transformation, the order of the calculation result needs to be rearranged according to the data input requirements of the buffer module. After completing the above three steps of data encoding operation, the column of intermediate result data buffered in the data conversion module is written into the buffer module in parallel through the row line. At this time, the column of storage units of the buffer module corresponds to a column of intermediate result data. Then, the first one-dimensional signal transformation and the first half of the data buffering and reorganization steps are repeated until all column vectors buffered in the data input module are calculated by the cross array to complete the one-dimensional signal transformation and stored in the buffer module.

[0083] IV. Second one-dimensional signal transformation, as shown in Figure 10

[0084] The buffer module is full of intermediate result data after the first one-dimensional signal transformation. Next, the intermediate result data is read out in parallel from the column line of the buffer module in turn and sent to the second encoding module. In the second encoding module, a row of intermediate result data is encoded into a data form supported by the subsequent row driving module. Then, the row driving module receives the data processed by the second encoding module, converts the data into a signal form suitable for the cross array to perform operation (for example, if the cross array is a memristor array, the signal form is "voltage"), and then applies the data to the cross array to complete the matrix vector multiplication operation. The calculation result is converted by the data conversion module into a digital signal vector, which represents the result after the second one-dimensional signal transformation after transposition of a row of intermediate result data, and is then buffered to the data output module.

[0085] V. Data output, as shown in Figure 11

[0086] ​​​The data output module performs the final encoding and post-processing on the data stored therein to achieve interface matching with the external circuit and output the final two-dimensional signal transformation calculation result.

[0087] According to the two-dimensional signal transformation in-memory computing architecture proposed in the embodiment of the present application, since the two-dimensional signal transformation needs to undergo two transformations, the first one-dimensional transformation and the second one-dimensional transformation, to output the final two-dimensional signal transformation in-memory computing result, but the existing cross array computing core can only complete one-dimensional signal transformation once, and cannot perform a complete two-dimensional signal transformation, therefore, in the two-dimensional signal transformation in-memory computing architecture of the embodiment of the present application, the two-dimensional signal first passes through the data input module, the cross array, the row driver module, and the column driver module to perform the first one-dimensional signal transformation, and a cache module is added to store the first one-dimensional signal transformation result, so that the first one-dimensional signal transformation result can be directly cached, transposed, and other operations in the cache module to achieve the reorganization of the intermediate result, and then the second one-dimensional signal transformation is performed in situ, and the final data output module determines the result of the two-dimensional signal transformation in-memory computing based on the second one-dimensional signal transformation result to output the result of the two-dimensional signal transformation in-memory computing, so that the intermediate result caching, transposition, and other operations can be completed directly in the computing core, avoiding repeated data transfer in and out of the core, thereby significantly improving computing efficiency and reducing hardware overhead.

[0088] The following describes the two-dimensional signal conversion memory computing architecture of the present application through a specific embodiment, taking the two-dimensional signal conversion memory computing architecture based on memristor and SRAM as an example. Figure 12 Shown, including:

[0089] 1. BL_R<1023:0>: 1024-bit register, corresponding to Figure 5 The data input module shown is used to organize and cache input data. The input data is input via the bus BL_DATAIN<63:0>, with 64 bits of data input per cycle. The 64 bits of data need to be cached to a specific location in BL_R<1023:0> based on the address signal BLROW_REGA<4:0>. Here, at least 16 cycles are required to complete the input of 1024 bits of data.

[0090] 2. BL_IN_MUX: used to determine whether the data input to the subsequent driver module BL_DRV is BL_IN<1023:0> from BL_R<1023:0> or RTN_DATA<1023:0> from the DECODER module (corresponding to Figure 5 The second transform input data in ).

[0091] 3. BL_DRV: corresponding Figure 5The “row driver module” shown in FIG receives 1024 bits of data from BL_IN_MUX, encodes it into BL_POS<575:0> and BL_NEG<575:0>, and applies it to the memristor array (ARRAY) in the form of a voltage signal.

[0092] 4. ARRAY: memristor array, corresponding to Figure 5 The cross array in .

[0093] 5. ADC<63:0>: 64-channel analog-to-digital converter, corresponding to Figure 5 The data conversion module in the memory array receives the current of the source line (SL) SL<511:0> and converts it into a digital signal <*64>ADC_OUT<7:0>. Note that for simplicity, Figure 5 The column driver module shown in FIG needs to be retained during actual chip design.

[0094] 6. ADD_ENCODER and <*2>ADD_12BIT<255:0>: corresponding Figure 5 The first encoding module shown can implement the functions of shift accumulation, encoding, and data caching; it receives the signal <*64>ADC_OUT<7:0> output by ADC<63:0> and finally organizes the data into <*256>ADD_OUT<7:0>; this module is controlled by the signals WLGRP_SEL<2:0>, ADD_EN, ADD_MD, and ADD_CLR.

[0095] 7. SRAM: corresponding Figure 5 In the cache module shown, the SRAM array data is written in parallel by row lines and read out in parallel by column lines, so the in-situ transposition in-memory calculation function can be realized; the module is controlled by the signals STATE, REG_SCLK, and SRAM_ADDR<10:0>.

[0096] 8. SIGN_BUF and DECODER: corresponding Figure 5 The second encoding module shown is used to encode the buffered intermediate result data to generate the second transformation input data RTN_DATA<1023:0>; this module is controlled by the signal SIGN_EN.

[0097] 9. OUTPUT: Corresponding Figure 5 The data output module shown is used to output the calculation results of the two-dimensional signal transformation to the outside of the computing core. The data bus is DO<7:0> and is controlled by the signal DO_SEL<7:0>.

[0098] An embodiment of the present application also provides an electronic device, comprising the above-mentioned two-dimensional signal conversion in-memory computing architecture.

[0099] Next, a two-dimensional signal transformation in-memory computing method according to an embodiment of the present application is described with reference to the accompanying drawings.

[0100] Figure 13 is a flowchart of the two-dimensional signal transformation in-memory computing method according to an embodiment of the present application.

[0101] As shown in Figure 13 , the two-dimensional signal transformation in-memory computing method includes the following steps:

[0102] In step S101, the externally input two-dimensional signal is divided into column data by column.

[0103] The two-dimensional signal can be picture data.

[0104] It can be understood that the externally input two-dimensional signal can be divided into column data by column, so as to perform subsequent data calculation.

[0105] In step S102, the parameter matrix of one-dimensional signal transformation is stored in the crossbar array, and matrix vector multiplication operation between the input vector and the parameter matrix is performed to obtain one-dimensional signal transformation result.

[0106] It can be understood that the parameter matrix of one-dimensional signal transformation can be stored in the crossbar array, and matrix vector multiplication operation between the input vector and the parameter matrix is performed to obtain one-dimensional signal transformation result.

[0107] In step S103, the row line of the crossbar array is driven, and the column data and the first one-dimensional signal transformation result of the crossbar array are converted into the input vector of the crossbar array, wherein the crossbar array calculates the first one-dimensional signal transformation result according to the column data, and the crossbar array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result.

[0108] The crossbar array calculates the first one-dimensional signal transformation result according to the column data, and the crossbar array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result

[0109] It can be understood that the row line of the crossbar array can be directly driven, the step of intermediate data processing is reduced, the efficiency is improved, and the column data and the first one-dimensional signal transformation result of the crossbar array are converted into the input vector of the crossbar array.

[0110] In step S104, the column line of the crossbar array is driven, and the crossbar array calculation result is output.

[0111] In step S105, all first one-dimensional signal transformation results of the cross array calculation are stored, and a transposition operation is performed on all first one-dimensional signal transformation results of the cross array calculation to realize the in-memory calculation of the two-dimensional signal transformation.

[0112] It can be understood that the embodiment of the present application can store all first one-dimensional signal transformation results of the cross array calculation, perform a transposition operation on all first one-dimensional signal transformation results of the cross array calculation to realize the in-memory calculation of the two-dimensional signal transformation, and directly complete the transposition operation, thereby avoiding repeated data transfer between the core and outside.

[0113] In step S106, the result of the in-memory calculation of the two-dimensional signal transformation is determined according to the second one-dimensional signal transformation result of the cross array, and the result of the in-memory calculation of the two-dimensional signal transformation is output.

[0114] It can be understood that the embodiment of the present application can determine the result of the in-memory calculation of the two-dimensional signal transformation according to the second one-dimensional signal transformation result of the cross array, and output the result of the in-memory calculation of the two-dimensional signal transformation, so as to realize the in-memory calculation of the two-dimensional signal transformation in one two-dimensional signal transformation in-memory calculation architecture.

[0115] It should be noted that the foregoing explanation and description of the embodiment of the two-dimensional signal transformation in-memory calculation architecture are also applicable to the method of the two-dimensional signal transformation in-memory calculation architecture of the embodiment, which will not be described herein.

[0116] According to the method of the two-dimensional signal transformation in-memory calculation architecture provided by the embodiment of the present application, the direct caching, recombination and transposition operation of the intermediate result can be directly realized in the calculation core, repeated data transfer between the core and outside is avoided, the calculation efficiency is significantly improved, and the hardware cost is reduced.

[0117] In the description of the specification, the description of the terms “one embodiment”, “some embodiments”, “example”, “specific example”, or “some examples” means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms is not necessarily for the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0118] Furthermore, the terms "first", "second", etc. are used herein only to describe different steps or features and do not imply a relative importance or a specific order of steps or features. Thus, a feature defined with "first", "second", etc. can explicitly or implicitly include at least one of the feature. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0119] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments of modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps, and alternate implementations are possible. The steps or functions described in the process or method descriptions or blocks can be implemented as program instructions. In some embodiments, the steps or functions described can be implemented as software executed by a processing unit. In some embodiments, the steps or functions described can be implemented as hardware or as a combination of software and hardware.

[0120] It should be understood that portions of the present application can be realized with hardware, software, firmware or a combination thereof. In the above embodiments, the steps or methods can be realized with software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if realized with hardware and in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays, field programmable gate arrays, etc.

[0121] Those of ordinary skill in the art can understand that all or part of the steps carried out by the method of the above embodiments can be instructed by a program to complete the relevant hardware, and the above program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiments or a combination thereof.

[0122] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those of ordinary skill in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A two-dimensional signal transformation in-memory computing architecture, comprising: The application comprises: a data input module for dividing the externally input two-dimensional signal into column data by column; a cross array for storing a parameter matrix of one-dimensional signal transformation and performing matrix-vector multiplication operation between an input vector and the parameter matrix to obtain one-dimensional signal transformation result; a row drive module for driving the row line of the cross array and converting the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates the second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; a column drive module for driving the column line of the cross array and outputting the calculation result of the cross array; a cache module for storing all the first one-dimensional signal transformation results calculated by the cross array and performing transpose operation on all the first one-dimensional signal transformation results calculated by the cross array to realize in-memory calculation of two-dimensional signal transformation; a data output module for determining the result of in-memory calculation of two-dimensional signal transformation according to the second one-dimensional signal transformation result of the cross array and outputting the result of in-memory calculation of two-dimensional signal transformation; a data conversion module for obtaining the output data of the column drive module, converting the first one-dimensional signal transformation result into a first digital signal vector, storing the first one-dimensional signal transformation result into the cache module based on the first digital signal vector, converting all the second one-dimensional signal transformation results of the cross array into a second digital signal vector, and outputting the second digital signal vector to the data output module; a first encoding module for performing encoding operation on the first digital signal vector and writing the output data of the first encoding module into the cache module in parallel through the row line; a second encoding module for performing at least one encoding operation of accumulation operation, shift operation and original code and complement code interchanging operation on the data cached by the cache module and inputting the data after the encoding operation into the row drive module.

2. The 2D signal transform in-memory computing architecture of claim 1, wherein, If the data input module performs batch input operation of multi-bit data, the first encoding module performs shift accumulation operation on the first digital signal vector; if the first one-dimensional signal transformation result introduces inverse code or complement code, the first encoding module re-encodes the first digital signal vector into original code form; after the first encoding module performs shift accumulation operation or re-encoding, the first digital signal vector is sorted according to the cache order format of the cache module.

3. The two-dimensional signal transform in-memory computing architecture of claim 1, wherein, The data input module allows to cache part of the two-dimensional signal data each time.

4. The 2D signal transform in-memory computing architecture of claim 1, wherein, The parameter matrix comprises at least one of discrete Fourier transform matrix, discrete cosine transform matrix and discrete wavelet transform matrix.

5. The 2D signal transform in-memory computing architecture of claim 1, wherein, The calculation core of the cross array comprises different kinds of memristors, different structures of memory-computing integrated architecture or different structures of memory-computing integrated units.

6. An electronic device, comprising: The application comprises the two-dimensional signal transformation in-memory calculation architecture according to any one of claims 1-5.

7. A two-dimensional signal transformation in-memory computing method, characterized by, The method utilizes the two-dimensional signal transformation in-memory computing architecture of any one of claims 1-5 to perform computation, and the method comprises the following steps: segmenting an externally input two-dimensional signal into column data according to columns by using a data input module; storing a parameter matrix of one-dimensional signal transformation in a cross array, performing a matrix-vector multiplication operation between an input vector and the parameter matrix to obtain a one-dimensional signal transformation result; driving row lines of the cross array by using a row driving module, and converting the column data and the first one-dimensional signal transformation result of the cross array into an input vector of the cross array, wherein the cross array calculates the first one-dimensional signal transformation result according to the column data, and the cross array calculates a second one-dimensional signal transformation result according to the first one-dimensional signal transformation result; outputting a calculation result of the cross array by driving column lines of the cross array by using a column driving module; storing all first one-dimensional signal transformation results calculated by the cross array in a cache module, and performing a transposition operation on all first one-dimensional signal transformation results calculated by the cross array to realize the two-dimensional signal transformation in-memory computation; determining a result of the two-dimensional signal transformation in-memory computation according to the second one-dimensional signal transformation result of the cross array by using a data output module, and outputting the result of the two-dimensional signal transformation in-memory computation; obtaining output data of the column driving module by using a data conversion module, converting the first one-dimensional signal transformation result into a first digital signal vector, storing the first one-dimensional signal transformation result in the cache module based on the first digital signal vector, converting all second one-dimensional signal transformation results of the cross array into a second digital signal vector, and outputting the second digital signal vector to the data output module; performing an encoding operation on the first digital signal vector by using a first encoding module, and writing output data of the first encoding module into the cache module in parallel from row lines; performing at least one encoding operation of an accumulation operation, a shift operation and a ones' complement and two's complement interconversion operation on data cached in the cache module by using a second encoding module, and inputting data after the encoding operation into the row driving module.

Images