WHT-based extensible in-memory image processing system and method

Abstract

The invention belongs to the technical field of signal analysis and signal processing, and particularly discloses a WHT-based extensible in-memory image processing system and method. According to the application, the image is split into the plurality of first data groups according to the first direction through the first input processing module, and the corresponding first voltage pulse signals are output. And mapping a matrix corresponding to the WHT operator by using the memristor array through an operation unit, and processing the first voltage pulse signal corresponding to each first data group to obtain a plurality of corresponding first current pulse signals. And the output processing module is used for converting the first current pulse signal output by the arithmetic unit into a corresponding second data group, and recombining and combining the plurality of second data groups according to a second direction to obtain final output data. According to the application, the WHT operation of any scale and size can be realized by using the memristor array of a fixed size, and the operation amount of the system is reduced while low power consumption is maintained.

Description

Technical Field

[0001] The present application belongs to the technical field of signal analysis and signal processing, and more specifically, to an expandable in-memory image processing system and method based on WHT. Background Art

[0002] Walsh-Hadamard Transform (WHT) is a signal orthogonal transformation method, which is widely used in image processing, speech processing, filtering and power spectrum analysis. The two-dimensional WHT has the characteristic of energy concentration, especially when there are evenly distributed numbers in the original data, the data after the transformation will be more concentrated on the corners of the matrix, so it can be used to compress image information.

[0003] In the process of WHT calculation, traditional computer architecture involves a large number of multiplication and addition operations, which requires continuous extraction of data from the storage module and transmission to the computing module for calculation, and then the calculated data is transmitted back to the storage module for storage. As images are currently developing towards larger sizes, the amount of data that needs to be processed is also increasing. The continuous transmission of data between the computing unit and the storage unit will consume a lot of power and cause increased system latency.

[0004] Current in-memory computing technology uses non-volatile memory to store information while performing in-situ computing, thereby realizing the functions of storage and computing at the same time, saving energy consumption and delay problems caused by data transmission in traditional architectures. However, since the size of the memristor array is usually fixed in practical applications, and as the image size continues to expand, the size of the WHT operator will also change in response to different requirements. Therefore, how to use a fixed-size memristor array to implement WHT operations of any size is an urgent problem that needs to be solved. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of the present application is to provide a scalable in-memory image processing system and method based on WHT, aiming to solve the problem that the existing in-memory computing system using a fixed-size memristor array is difficult to implement WHT operations of different scales.

[0006] To achieve the above objectives, in a first aspect, the present application provides an extensible in-memory image processing system based on WHT, comprising: A first input processing module, used for splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction, and outputting first voltage pulse signals respectively corresponding to the plurality of first data groups; An operation unit connected to the first input processing module, used to process the plurality of first voltage pulse signals by mapping the matrix corresponding to the Walsh-Hadamard transform operator using the memristor array to obtain a plurality of first current pulse signals; The output processing module is connected to the operation unit and is used to convert the multiple first current pulse signals into corresponding second data groups respectively, and reorganize and merge the multiple second data groups in the second direction to obtain output data.

[0007] In some embodiments, the computing unit includes: An operation module, connected to the first input processing module, for processing the plurality of first voltage pulse signals using a matrix corresponding to a Walsh-Hadamard transform operator to obtain a plurality of second current pulse signals; an intermediate processing module, connected to the operation module, and configured to convert the plurality of second current pulse signals into corresponding third data groups, and to reorganize and merge the plurality of third data groups in a first direction to obtain a second image block; A second input processing module, connected to the intermediate processing module and the operation module respectively, for splitting the second image block into a plurality of fourth data groups according to the second direction, and outputting second voltage pulse signals respectively corresponding to the plurality of fourth data groups to the operation module; Correspondingly, the operation module is also used to process the multiple second voltage pulse signals using a matrix to obtain multiple first current pulse signals.

[0008] In some embodiments, the computing module includes: Memristor arrays, registers, adders, and subtractors.

[0009] In some embodiments, the length of the memristor array is twice its width, and the width is 2 to the power of N.

[0010] In some embodiments, the matrix is ​​split into a positive matrix corresponding to the Walsh-Hadamard transform operator and a negative matrix corresponding to the Walsh-Hadamard transform operator, and the matrix is ​​equal to the positive matrix minus the negative matrix.

[0011] In some embodiments, the memristors in the memristor array are binary devices, divided into a high resistance state and a low resistance state. The low resistance state memristors are used to map a logic value of 1, and the high resistance state memristors are used to map a logic value of 0.

[0012] In some embodiments, the length and width of the first image block are the same and are both 2 to the power of N.

[0013] In a second aspect, the present application provides a scalable in-memory image processing method based on WHT, comprising: Splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction through a first input processing module, and outputting voltage pulse signals respectively corresponding to the plurality of first data groups; A matrix corresponding to the Walsh-Hadamard transform operator is mapped by a memristor array in a computing unit connected to the first input processing module, and a plurality of first voltage pulse signals are processed to obtain a plurality of first current pulse signals; The output processing module connected to the operation unit converts the multiple first current pulse signals into corresponding second data groups respectively, and the multiple second data groups are reorganized and merged in the second direction to obtain output data.

[0014] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory, wherein when the programs stored in the memory are executed, the processor is used to execute the method described in the second aspect or any embodiments of the second aspect.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the second aspect or any embodiments of the second aspect.

[0016] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the second aspect or any embodiments of the second aspect.

[0017] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: The expandable in-memory image processing system and method based on WHT provided in the present application splits the image into multiple first data groups in a first direction through a first input processing module, and outputs a corresponding first voltage pulse signal. Through the operation unit, the matrix corresponding to the WHT operator is mapped using a memristor array, and the first voltage pulse signal corresponding to each first data group is processed to obtain a corresponding multiple first current pulse signals. Through the output processing module, it is used to convert the first current pulse signal output by the operation unit into a corresponding second data group, and reorganize and merge the multiple second data groups in a second direction to obtain the final output data. The present application can realize WHT operations of any size by using a fixed-size memristor array, while maintaining low power consumption and reducing the amount of system operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is one of the flow charts of the expandable in-memory image processing system based on WHT provided in the embodiment of the present application; Figure 2 It is one of the processing flow diagrams of the first input processing module in the expandable in-memory image processing system based on WHT provided in an embodiment of the present application; Figure 3 This is the second flow chart of the expandable in-memory image processing system based on WHT provided in the embodiment of the present application; Figure 4 It is a structural schematic diagram of a computing module in an expandable in-memory image processing system based on WHT provided in an embodiment of the present application; Figure 5 It is one of the processing flow diagrams of the second input processing module in the expandable in-memory image processing system based on WHT provided in an embodiment of the present application; Figure 6 This is the second processing flow diagram of the first input processing module in the expandable in-memory image processing system based on WHT provided in an embodiment of the present application; Figure 7 This is the second structural schematic diagram of the computing module in the expandable in-memory image processing system based on WHT provided in an embodiment of the present application; Figure 8 is a schematic diagram of the structure of a memristor array in a computing module provided in an embodiment of the present application; Fig. 9 It is a schematic diagram of the processing flow of the intermediate processing module provided in an embodiment of the present application; Fig.10 is a processing flow diagram of an output processing module provided in an embodiment of the present application; Fig.11 It is a flowchart of an expandable in-memory image processing method based on WHT provided in an embodiment of the present application; Fig.12 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0021] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first voltage pulse signal and a second voltage pulse signal are used to distinguish different voltage pulse signals rather than to describe a specific order of the voltage pulse signals.

[0022] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0023] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple first data groups refer to two or more data groups, etc.; multiple first current pulse signals refer to two or more first current pulse signals, etc.

[0024] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0025] See also Figure 1 An embodiment of the present application provides an expandable in-memory image processing system based on WHT, including: a first input processing module, a computing unit and an output processing module.

[0026] A first input processing module, used for splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction, and outputting first voltage pulse signals respectively corresponding to the plurality of first data groups; An operation unit connected to the first input processing module, used to process the plurality of first voltage pulse signals by mapping the matrix corresponding to the Walsh-Hadamard transform operator using the memristor array to obtain a plurality of first current pulse signals; The output processing module is connected to the operation unit and is used to convert the multiple first current pulse signals into corresponding second data groups respectively, and reorganize and merge the multiple second data groups in the second direction to obtain output data.

[0027] In an embodiment of the present application, the first input processing module can be specifically used to split the image block corresponding to the image to be processed (i.e., the first image block) into multiple data groups (i.e., first data groups) along a first direction in the current operation, and output a voltage pulse signal corresponding to each first data group, i.e., a first voltage pulse signal.

[0028] Wherein, the first data group is a one-dimensional data group, and the first direction can be a row or a column. The image is a two-dimensional image, and the first image block refers to the two-dimensional data to be processed by WHT. The first image block can be obtained by dividing the above image according to a fixed number of rows and columns. For example, a 256x256 image is divided into 16 64x64 image blocks.

[0029] Further, in some embodiments, the length and width of the first image block are the same and are both 2 to the power of N.

[0030] In the embodiment of the present application, the length and width of the first image block are the same, and are both 2 to the power of N, where N is a positive integer.

[0031] For example, see further Figure 2 When the input is a 4x4 first image block, it can be regarded as a 4x4 two-dimensional data. The first input processing module splits the two-dimensional data according to the first direction (such as row) to obtain 4 one-dimensional data groups of size 1x4, namely the first data group.

[0032] Specifically, the first input processing module can map each one-dimensional data group (i.e., the first data group) into a set of voltage pulse signals (i.e., first voltage pulse signals) of corresponding sizes according to the values ​​in each one-dimensional data group (i.e., the first data group) through linear conversion, and output the first voltage pulse signal corresponding to each one-dimensional data group. The pulse width of these first voltage pulse signals remains consistent, but the voltage size of the pulse maintains a linear transformation relationship with the value of the input data, and then these first voltage pulse signals will be divided into 4 time sequences for input. It can be understood that each voltage pulse signal value in each first voltage pulse signal corresponds one-to-one to the value in each first data group.

[0033] It should be noted that, in the embodiment of the present application, the length of each first data group after the first image block is expanded is twice the length of each row of the WHT operator used. By analogy, the system can be applied to process any input data of a size of 2 to the power of N, and it is also helpful to reduce the amount of calculation.

[0034] The operation unit is connected to the first input processing module, and includes a memristor array, which can be used to map the matrix corresponding to the WHT operator.

[0035] The operation unit processes the multiple first voltage pulse signals output by the first input processing module by using a matrix corresponding to the WHT operator to obtain multiple current pulse signals, namely, first current pulse signals.

[0036] The output processing module is connected to the above-mentioned operation unit, and can be specifically used to convert the multiple first current pulse signals output by the above-mentioned operation unit into corresponding one-dimensional data groups (i.e., second data groups), and reorganize and merge the second data groups in the second direction to obtain final output data.

[0037] In the embodiment of the present application, the second direction may be a row or a column. It should be noted that when the first direction is a row, the second direction is a column; when the first direction is a column, the second direction is a row.

[0038] An expandable in-memory image processing system based on WHT is provided in an embodiment of the present application. The image is split into multiple first data groups in a first direction through a first input processing module, and a corresponding first voltage pulse signal is output. Through an operation unit, the matrix corresponding to the WHT operator is mapped using a memristor array, and the first voltage pulse signal corresponding to each first data group is processed to obtain a corresponding multiple first current pulse signals. Through an output processing module, the first current pulse signal output by the operation unit is converted into a corresponding second data group, and the multiple second data groups are reorganized and merged in a second direction to obtain final output data. In the present application, WHT operations of any size can be realized by using a fixed-size memristor array, thereby reducing the amount of computation of the system while maintaining low power consumption.

[0039] Furthermore, in some embodiments, the computing unit includes: An operation module, connected to the first input processing module, for processing the plurality of first voltage pulse signals using a matrix corresponding to a Walsh-Hadamard transform operator to obtain a plurality of second current pulse signals; an intermediate processing module, connected to the operation module, and configured to convert the plurality of second current pulse signals into corresponding third data groups, and to reorganize and merge the plurality of third data groups in a first direction to obtain a second image block; A second input processing module, connected to the intermediate processing module and the operation module respectively, for splitting the second image block into a plurality of fourth data groups according to the second direction, and outputting second voltage pulse signals respectively corresponding to the plurality of fourth data groups to the operation module; Correspondingly, the operation module is also used to process the multiple second voltage pulse signals using a matrix to obtain multiple first current pulse signals.

[0040] Please see further Figure 3 The operation unit may include an operation module, an intermediate processing module and a second input processing module. The operation module is connected to the first input processing module, the intermediate processing module is connected to the operation module, and the second input processing module is connected to the operation module and the intermediate processing module respectively.

[0041] The expandable in-memory image processing system based on WHT comprises a first input processing module, a calculation module, an intermediate processing module, a second input processing module and an output processing module.

[0042] The operation module can be used to use the Walsh-Hadamard transform operator to output the voltage pulse signal corresponding to each one-dimensional data group in two times, and output the current pulse signal corresponding to each one-dimensional data group. The specific implementation is as follows: The operation module processes the multiple first voltage pulse signals output by the first input processing module by using a matrix corresponding to the WHT operator to obtain current pulse signals corresponding to the multiple first voltage pulse signals, namely, second current pulse signals.

[0043] The operation module outputs the above-mentioned multiple second current pulse signals to the intermediate processing module. The intermediate processing module converts the multiple second current pulse signals output by the operation module for the first time into corresponding one-dimensional data groups (i.e., the third data groups), and reorganizes and merges the multiple third data groups along the first direction to obtain a two-dimensional intermediate data block, i.e., the second image block.

[0044] The intermediate processing module inputs the above-mentioned two-dimensional intermediate data block to the second input processing module, and the second input processing module splits it into multiple one-dimensional data groups (i.e., fourth data groups) along the second direction, and uses the same processing method as the above-mentioned first input processing module to convert the multiple fourth data groups into corresponding voltage pulse signals (i.e., second voltage pulse signals).

[0045] The second input processing module inputs the above-mentioned multiple second voltage pulse signals to the operation module, and the operation module processes the multiple second voltage pulse signals through the matrix corresponding to the above-mentioned WHT operator to obtain the corresponding multiple first current pulse signals.

[0046] It should be noted that the input first image block is processed once by the first input processing module, the operation unit and the output processing module in sequence, which is an iterative operation process.

[0047] Please see further Figure 4 The operation module is pre-constructed by a non-volatile memory and mapped with a WHT operator. Therefore, the operation module can perform WHT operation on the voltage pulse signal corresponding to each one-dimensional data group output by the first input processing module in two time sequences. Combined with the conductivity characteristics of the non-volatile memory, the voltage pulse signal corresponding to each one-dimensional data group can be converted into a corresponding current pulse signal, and the current pulse signal corresponding to each one-dimensional data group is output to the intermediate processing module.

[0048] Furthermore, in an embodiment of the present application, the intermediate processing module converts each group of current pulse signals into a corresponding one-dimensional data group according to the mapping relationship between the aforementioned first image block and the first voltage pulse signal, and reorganizes each one-dimensional data group according to the aforementioned row splitting to obtain a two-dimensional intermediate data block for the first operation.

[0049] Next, the second operation is performed, and the second input processing module receives and processes the two-dimensional intermediate data group. The second input processing module splits the two-dimensional intermediate data block by column and splits it into multiple one-dimensional data groups (i.e., fourth data groups). Figure 5 When the two-dimensional intermediate data block is 4x4 two-dimensional data, the second input processing module splits the two-dimensional intermediate data block by columns, thereby also obtaining corresponding 4 one-dimensional data groups of size 4x1 (i.e., the fourth data group), and outputs the voltage pulse signal corresponding to the fourth data group (i.e., the second voltage pulse signal).

[0050] In an embodiment of the present application, the output processing module converts each group of current pulse signals into a corresponding one-dimensional data group according to the mapping relationship between the aforementioned two-dimensional intermediate data block and the second voltage pulse signal, and reorganizes the data of each one-dimensional data group according to the aforementioned column splitting to obtain the output data of the second operation.

[0051] The expandable in-memory image processing system based on WHT provided by the embodiment of the present application, through the input processing module, in the current operation, the two-dimensional image block is split into multiple one-dimensional data groups according to the first direction, and the corresponding voltage pulse signal is output. Through the operation module, the matrix corresponding to the Walsh-Hadamard transform operator is used to output the voltage pulse signal corresponding to each one-dimensional data group in two times, and the current pulse signal corresponding to each one-dimensional data group is output. Each group of current pulse signals output by the operation module for the first time is converted into a corresponding one-dimensional data group by the intermediate processing module, and the one-dimensional conversion data group is reorganized and merged according to the first direction to obtain a two-dimensional intermediate data block. The two-dimensional intermediate data block is split into multiple one-dimensional data groups according to the second direction by the second input processing module, and the corresponding voltage pulse signal is output. The current pulse signal output by the operation module is converted into a corresponding one-dimensional data group by the output processing module, and the one-dimensional data group is reorganized and merged according to the second direction to obtain the final output data. Finally, WHT operations of any size can be realized by using a fixed-size memristor array, while maintaining low power consumption, reducing the amount of operation of the system.

[0052] Furthermore, in some embodiments, the operation module includes: Memristor arrays, registers, adders, and subtractors.

[0053] In an embodiment of the present application, the above-mentioned operation module includes a memristor array, a register, an adder and a subtractor.

[0054] Among them, the memristor array is used to map the matrix corresponding to the WHT operator, wherein the upper half of the memristor array with the same length and width is used to map the positive matrix corresponding to the WHT operator, and the lower half of the memristor array with the same length and width is used to map the negative matrix corresponding to the WHT operator.

[0055] For example, see further Figure 6 , the first input processing module divides the first image block of size 8x8 into eight one-dimensional data groups of size 1x8 by row, and then maps them into eight voltage pulse signals of size 1x8, namely, first voltage pulse signals. Please refer to Figure 7 , the operation module divides each 1x8 voltage pulse signal into two 1x4 voltage pulse signals, and then divides each 1x4 voltage pulse signal into two time steps (i.e., two time series) inputs, and performs addition and subtraction operations on the current pulse signal output in the first time step and the current pulse signal output in the second time step to obtain a 1x4 current pulse signal. Then, the first 1x4 current pulse signal and the second 1x4 current pulse signal are added and subtracted to obtain a 1x8 current pulse signal corresponding to each 1x8 first voltage pulse signal, i.e., the second current pulse signal. That is, a 1x8 input vector is multiplied by a vector matrix with an 8x8 WHT operator to obtain a 1x8 output vector.

[0056] The register can be used to store the current pulse signal output by the memristor array in the first time step. The adder can be used to implement the addition operation of the current pulse signal output by the memristor array in the first time step and the current pulse signal output in the second time step. The subtractor can be used to implement the subtraction operation of the current pulse signal output by the memristor array in the first time step and the current pulse signal output in the second time step. The current pulse signal finally output by the operation module can be determined according to the current pulse signal output by the adder and the current pulse signal output by the subtractor.

[0057] Further, in some embodiments, the matrix is ​​split into a positive matrix corresponding to the Walsh-Hadamard transform operator and a negative matrix corresponding to the Walsh-Hadamard transform operator, and the matrix is ​​equal to the positive matrix minus the negative matrix.

[0058] In an embodiment of the present application, the matrix corresponding to the WHT operator is split into a positive matrix corresponding to the WHT operator and a negative matrix corresponding to the WHT operator, and the matrix corresponding to the WHT operator is equal to the positive matrix corresponding to the WHT operator minus the negative matrix corresponding to the WHT operator.

[0059] Furthermore, in some embodiments, the length of the memristor array is twice its width, and the width is 2 to the Nth power.

[0060] In the embodiment of the present application, the length of the memristor array in the computing module is twice the width, and the width can be any size of 2 to the power of N, where N is a positive integer.

[0061] Furthermore, in some embodiments, the memristors in the memristor array are binary devices, divided into a high resistance state and a low resistance state, the low resistance state memristors are used to map logic values ​​1, and the high resistance state memristors are used to map logic values ​​0.

[0062] In the embodiment of the present application, the memristors in the memristor array are binary devices, which are divided into a high-resistance state and a low-resistance state, wherein the memristor in the low-resistance state can be used to map "1", and the memristor in the high-resistance state is used to map "0".

[0063] It should be noted that, in the embodiment of the present application, the sizes of the first image block, the two-dimensional intermediate data block and the output data are all the same.

[0064] In the embodiment of the present application, a memristor is introduced in the process of WHT operation. Memristor has the advantages of small area, low power consumption, high speed, and compatibility with complementary metal oxide semiconductor (CMOS) process, etc., and is a main candidate device for improving system computing performance. At the same time, memristor has a relatively stable second-order conductivity characteristic, which can be used to store the values ​​of some basic operators that need to be frequently used for calculation, so that Ohm's law and Kirchhoff's law can be directly used to realize matrix multiplication calculation, and convert the input voltage pulse signal into the output current pulse signal.

[0065] It should also be noted that in the embodiments of the present application, the memristor array may adopt a 1T1R memristor array structure or a 1S1R memristor array structure, wherein the 1T1R memristor array structure is composed of a transistor and a memristor, and the 1S1R memristor array structure is composed of a selector and a memristor.

[0066] In a specific embodiment of the present application, the first image block is two-dimensional data of a size of 4x4, and the size of the memristor array is 4x2, that is, the size of the positive matrix and the negative matrix of the WHT operator are both 2x2.

[0067] It should be noted that in the embodiment of the present application, the size of the memristor array is equal to twice the size of the adopted WHT operator, and the matrix corresponding to the WHT operator inputs positive voltage pulse signals and negative voltage pulse signals respectively. The conductance value of the memristor array is set according to the size of the WHT operator, and there is a linear mapping relationship between the two, where the maximum conductance value is used to map "1" and the minimum conductance value is used to map "0".

[0068] In an embodiment of the present application, after obtaining the voltage pulse signal corresponding to each one-dimensional data group, the data of the voltage pulse signal corresponding to each one-dimensional data group and the negative number of the data of each voltage pulse signal are input into the memristor array, and the conductivity value of each memristor in the memristor array based on the WHT operator mapping is obtained. The partial data of each voltage pulse signal and the negative number of the partial data of each voltage pulse signal are weighted and summed with the conductance value of the corresponding memristor in the memristor array to obtain the data of the current pulse signal corresponding to each one-dimensional data group.

[0069] Please see further Figure 8 In an embodiment of the present application, the size of the memristor array is 4x2, and two positive voltage pulses and two corresponding negative voltage pulses are input simultaneously each time, corresponding to four rows of the memristor array respectively. After the voltage passes through the memristor array, two columns of current are generated, corresponding to the output result of the matrix multiplication of the one-dimensional data group vector.

[0070] Specifically, assuming that the conductance values ​​of each memristor are , , , , , , and , the data of the voltage pulse signal corresponding to the one-dimensional data group (in, ) and the corresponding negative signals The two columns are input into the memristor array respectively, and the conductance values ​​of the corresponding memristors in the internal memristor array are weighted and summed to obtain the partial data of the current pulse signal corresponding to the two columns. ,Right now:

[0071] in, .

[0072] The computing system of the embodiment of the present application realizes WHT computing by introducing a memristor array structure, using a memristor device with a smaller area, and designs half of the memristor array to process positive voltage pulse signals and half of the memristor array to process negative voltage pulse signals, thereby realizing the processing of negative values ​​of the WHT operator and ensuring the effectiveness and reliability of the WHT in-memory computing.

[0073] Please see further Fig. 9 The intermediate processing module processes the 4 sets of current pulse signals output by the operation module, wherein the 4 sets of current pulse signals are ; Through linear conversion, the four groups of current pulse signals are converted into corresponding one-dimensional data groups. Assume that the one-dimensional data group is specifically , and then reorganize and merge them according to the rows to output a two-dimensional intermediate data block of size 4x4.

[0074] Please continue to see Figure 5 The second input processing module processes the two-dimensional intermediate data block output by the intermediate processing module and divides it into columns to obtain the corresponding one-dimensional data group. ; The four one-dimensional data sets are converted into corresponding voltage pulse signals through linear mapping .

[0075] Please see further Fig.10 The output processing module processes the 4 sets of current pulse signals output by the operation module, wherein the 4 sets of current pulse signals are ; Through linear transformation, the four groups of current pulse signals are converted into corresponding one-dimensional data groups, assuming that , and then merge them by column to output a two-dimensional output data block of size 4x4.

[0076] The expandable in-memory image processing system based on WHT provided in the embodiment of the present application uses memristors to construct operation modules, multiplexes a memristor array for processing, and uses adders and subtractors to implement current addition and subtraction, thereby reducing the multiple data transmission processes in traditional WHT operations, reducing the circuit area of ​​WHT operations implemented using hardware circuits, and implementing low-power, low-latency WHT operations.

[0077] The following describes the scalable in-memory image processing method based on WHT provided in the present application. The scalable in-memory image processing method based on WHT described below can be implemented based on the scalable in-memory image processing system based on WHT described above.

[0078] See also Fig.11 The scalable in-memory image processing method based on WHT provided in an embodiment of the present application may include: step 1110, step 1120 and step 1130.

[0079] Step 1110: splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction through a first input processing module, and outputting voltage pulse signals corresponding to the plurality of first data groups respectively; Step 1120 maps a matrix corresponding to a Walsh-Hadamard transform operator through a memristor array in a computing unit connected to the first input processing module, and processes a plurality of first voltage pulse signals to obtain a plurality of first current pulse signals; In step 1130, the output processing module connected to the operation unit converts the plurality of first current pulse signals into corresponding second data groups respectively, and reorganizes and merges the plurality of second data groups in a second direction to obtain output data.

[0080] The expandable in-memory image processing method based on WHT provided in an embodiment of the present application splits the image into multiple first data groups in a first direction through a first input processing module, and outputs a corresponding first voltage pulse signal. Through the operation unit, the matrix corresponding to the WHT operator is mapped using a memristor array, and the first voltage pulse signal corresponding to each first data group is processed to obtain a corresponding multiple first current pulse signals. Through the output processing module, it is used to convert the first current pulse signal output by the operation unit into a corresponding second data group, and reorganize and merge the multiple second data groups in a second direction to obtain the final output data. The present application can realize WHT operations of any size by using a fixed-size memristor array, while maintaining low power consumption and reducing the amount of system operations.

[0081] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0082] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0083] Based on the method in the above embodiment, the present application embodiment provides an electronic device, see Fig.12 The electronic device may include: a processor 1210, a communication interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communication interface 1220 and the memory 1230 communicate with each other via the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the method in the above embodiment.

[0084] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0085] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0086] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0087] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0088] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0089] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0090] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0091] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A scalable in-memory image processing system based on WHT, characterized in that: include: A first input processing module, used for splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction, and outputting first voltage pulse signals respectively corresponding to the plurality of first data groups; an operation unit connected to the first input processing module, and configured to use a memristor array to map a matrix corresponding to a Walsh-Hadamard transform operator, process a plurality of first voltage pulse signals, and obtain a plurality of first current pulse signals; The output processing module is connected to the operation unit and is used to convert the multiple first current pulse signals into corresponding second data groups respectively, and reorganize and merge the multiple second data groups in the second direction to obtain output data.

2. The scalable in-memory image processing system based on WHT as claimed in claim 1, characterized in that: The computing unit comprises: an operation module, connected to the first input processing module, and configured to process the plurality of first voltage pulse signals using a matrix corresponding to a Walsh-Hadamard transform operator to obtain a plurality of second current pulse signals; an intermediate processing module, connected to the operation module, for converting the plurality of second current pulse signals into corresponding third data groups, and recombining and merging the plurality of third data groups in a first direction to obtain a second image block; a second input processing module, connected to the intermediate processing module and the operation module respectively, and configured to split the second image block into a plurality of fourth data groups according to a second direction, and output second voltage pulse signals respectively corresponding to the plurality of fourth data groups to the operation module; Correspondingly, the operation module is further used to process multiple second voltage pulse signals using the matrix to obtain multiple first current pulse signals.

3. The scalable in-memory image processing system based on WHT as claimed in claim 2, characterized in that: The operation module comprises: Memristor arrays, registers, adders, and subtractors.

4. The scalable in-memory image processing system based on WHT as claimed in claim 3, characterized in that: The length of the memristor array is twice its width, and the width is 2 to the Nth power.

5. The scalable in-memory image processing system based on WHT as claimed in claim 3, characterized in that: The matrix is ​​split into a positive matrix corresponding to a Walsh-Hadamard transform operator and a negative matrix corresponding to a Walsh-Hadamard transform operator, and the matrix is ​​equal to the positive matrix minus the negative matrix.

6. The scalable in-memory image processing system based on WHT as claimed in claim 3, characterized in that: The memristors in the memristor array are binary devices, which are divided into a high resistance state and a low resistance state. The memristors in the low resistance state are used to map a logic value of 1, and the memristors in the high resistance state are used to map a logic value of 0.

7. The scalable in-memory image processing system based on WHT according to any one of claims 1 to 6, characterized in that: The length and width of the first image block are the same and are both 2 to the power of N.

8. A scalable in-memory image processing method based on WHT, characterized in that: include: Splitting a first image block corresponding to the image into a plurality of first data groups according to a first direction through a first input processing module, and outputting voltage pulse signals respectively corresponding to the plurality of first data groups; Mapping a matrix corresponding to a Walsh-Hadamard transform operator through a memristor array in a computing unit connected to the first input processing module, and processing a plurality of first voltage pulse signals to obtain a plurality of first current pulse signals; The output processing module connected to the operation unit converts the multiple first current pulse signals into corresponding second data groups respectively, and reorganizes and merges the multiple second data groups in the second direction to obtain output data.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is used to execute the method according to claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is run on a processor, the processor is caused to execute the method as claimed in claim 8.

Images