Image feature extraction method and apparatus based on magnetic tunnel junction array

Abstract

Provided in the embodiments of the present application are an image feature extraction method and apparatus based on a magnetic tunnel junction array. The method comprises: a training stage, involving: on the basis of a preset voltage-pixel mapping relationship, a voltage-current mapping relationship and a preset convolution kernel, obtaining a pixel-current mapping relationship, wherein the convolution kernel is a magnetic tunnel junction array; and a feature extraction stage, involving: mapping, into a voltage, pixels of an image to be processed; using the voltage as an input, and on the basis of the convolution kernel, outputting current coordinates corresponding to said image; and on the basis of the current coordinates and the pixel-current mapping relationship, determining pixels, which have been subjected to convolution, of said image. The technical solution provided in the present application is used for solving the problems in the prior art of the writing rate being low, the energy consumption being high, the reliability being poor, etc.

Description

A method and device for extracting image features based on magnetic tunnel junction array Technical Field

[0001] This document relates to the field of hardware neural network technology, and in particular to a method and device for extracting image features based on a magnetic tunnel junction array. Background Art

[0002] Image feature extraction is a key step in image processing. The efficiency and accuracy of image feature extraction have a decisive impact on the overall processing results. Therefore, researching and developing efficient feature extraction methods is a hot topic in the field of image processing.

[0003] Since convolutional neural networks require a lot of computing and storage resources, existing technologies develop hardware neural networks based on new non-volatile memories to reduce computing and storage resources.

[0004] However, the above solutions have problems such as slow writing speed, high energy consumption, and poor reliability.

[0005] Summary of the Invention

[0006] In view of the above analysis, the present application aims to propose a method and device for extracting image features based on a magnetic tunnel junction array to solve at least one of the above technical problems.

[0007] In a first aspect, one or more embodiments of the present application provide a method for extracting image features based on a magnetic tunnel junction array, comprising:

[0008] In the training phase, a pixel-current mapping relationship is obtained based on a preset voltage-pixel mapping relationship, a voltage-current mapping relationship, and a preset convolution kernel; the convolution kernel is a magnetic tunnel junction array;

[0009] In the feature extraction stage, the pixels of the image to be processed are mapped to voltages; the voltages are used as input, and based on the convolution kernel, the current coordinates corresponding to the image to be processed are output; and the pixels of the image to be processed after convolution are determined based on the current coordinates and the pixel-current mapping relationship.

[0010] Furthermore, the pixel-current mapping relationship is obtained based on the preset voltage-pixel mapping relationship, the voltage-current mapping relationship and the preset convolution kernel, including:

[0011] Mapping the pixel matrix into a voltage matrix according to the voltage-pixel mapping relationship;

[0012] According to the voltage-current mapping relationship and the convolution kernel, the voltage matrix is ​​mapped into a current matrix; wherein the voltage-current mapping relationship is a quantitative relationship between voltage, conductance, and current; each column vector in the voltage matrix corresponds to a column of magnetic tunnel junctions in the magnetic tunnel junction array; and each column of the magnetic tunnel junctions corresponds to a conductance matrix;

[0013] The pixel-current mapping relationship is obtained according to the voltage matrix and the current matrix.

[0014] Further, the magnetic tunnel junction array includes: a plurality of magnetic tunnel junctions;

[0015] Two adjacent magnetic tunnel junctions map a weight in the convolution kernel.

[0016] Furthermore, the two adjacent magnetic tunnel junctions map a weight value in the convolution kernel, specifically:

[0017] Each of the magnetic tunnel junctions has two resistance states;

[0018] The corresponding weight value is obtained by changing the resistance state of two adjacent magnetic tunnel junctions.

[0019] Furthermore, mapping the voltage matrix into a current matrix according to the voltage-current mapping relationship and the convolution kernel includes:

[0020] Split the convolution kernel into vertical matrix and parallel matrix;

[0021] Writing preset resistance states into the magnetic tunnel junction array corresponding to the vertical matrix and the magnetic tunnel junction array corresponding to the parallel matrix respectively to obtain a vertical conductivity matrix and a parallel conductivity matrix;

[0022] According to Ohm's law and Kirchhoff's law, a current matrix is ​​obtained by using the voltage-current mapping relationship, the vertical conductivity matrix and the parallel conductivity matrix.

[0023] Furthermore, the operator of the convolution kernel is specifically: Roberts operator or Prewitt operator.

[0024] In a second aspect, an embodiment of the present application provides an image feature extraction device based on a magnetic tunnel junction array, comprising:

[0025] The word line conductive portion is arranged along the word line;

[0026] The bit line conductive portion is arranged along the bit line;

[0027] The magnetic tunnel junction is arranged at a position where the word line conductive portion and the bit line conductive portion intersect;

[0028] One end of the magnetic tunnel junction is connected to the word line conductive portion, and the other end is connected to the bit line conductive portion.

[0029] Furthermore, the word line conductive portion and the bit line conductive portion form a grid structure;

[0030] Each node in the grid structure corresponds to a magnetic tunnel junction;

[0031] Two adjacent magnetic tunnel junctions correspond to a weight value of the convolution kernel.

[0032] Furthermore, the word line conductive portion includes: a metal line;

[0033] or

[0034] The word line conductive portion includes a metal line and a transistor gate.

[0035] Furthermore, the bit line conductive portion includes: a metal line;

[0036] or,

[0037] The bit line conductive portion includes a metal line and a transistor gate.

[0038] Compared with the existing technology, this application can at least achieve the following technical effects:

[0039] By replacing the convolution kernel with a magnetic tunnel junction array and establishing a pixel-current mapping relationship based on preset voltage-pixel and voltage-current mapping relationships, the corresponding relationship between input and output in the feature extraction process is obtained. This method transforms the feature extraction process into a query process based on the corresponding relationship, thereby reducing the computational intensity and resource consumption of the image feature extraction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] FIG1 is a schematic diagram of an input mapping rule provided by one or more embodiments of the present application;

[0042] FIG2 is a schematic diagram of a convolution kernel weight mapping rule provided by one or more embodiments of the present application;

[0043] FIG3 is a flow chart of determining a current matrix according to one or more embodiments of the present application;

[0044] FIG4 is a current coordinate diagram provided by one or more embodiments of the present application;

[0045] FIG5 is a flow chart of a mapping relationship between standing pixels and currents provided by one or more embodiments of the present application;

[0046] FIG6 is a schematic structural diagram of an image feature extraction device based on a magnetic tunnel junction array provided by one or more embodiments of the present application;

[0047] FIG7 is a schematic structural diagram of another image feature extraction device based on a magnetic tunnel junction array provided by one or more embodiments of the present application;

[0048] FIG8 is a schematic structural diagram of another magnetic tunnel junction array-based image feature extraction device provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present application, the technical solutions in one or more embodiments of the present application will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on one or more embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this document.

[0050] In order to solve the problem of low bit density of memory in the prior art, an embodiment of the present application provides an image feature extraction method based on a magnetic tunnel junction array, which includes two stages: a training stage and a feature extraction stage.

[0051] During the training phase, a pixel-current mapping relationship is obtained based on a preset voltage-pixel mapping relationship, a voltage-current mapping relationship, and a preset convolution kernel; the convolution kernel is a magnetic tunnel junction array.

[0052] In the embodiment of the present application, the image grayscale values ​​0-255 are mapped to read voltage amplitudes, as shown in Figure 1. The step size and amplitude of the read voltage are determined according to actual needs to determine the voltage-pixel mapping relationship. For example, the grayscale value is mapped to 0-25.5mV with a step size of 0.05mV, that is, one pixel value corresponds to a 0.05mV read voltage amplitude. Then, according to the voltage-pixel mapping relationship, the pixel matrix is ​​mapped to a voltage matrix.

[0053] The default convolution kernel is: Roberts operator or Prewitt operator.

[0054] In an embodiment of the present application, a magnetic tunnel junction array includes: a plurality of magnetic tunnel junctions; two adjacent magnetic tunnel junctions map a weight in the convolution kernel. Since the convolution kernel can be regarded as a matrix of weights, the magnetic tunnel junction array can obtain the weight matrix of the convolution kernel by mapping the weights in the convolution kernel, thereby realizing the use of a magnetic tunnel junction array to replace the convolution kernel. Specifically, each of the magnetic tunnel junctions has two resistance states. By changing the resistance states of the two adjacent magnetic tunnel junctions, the corresponding weights are obtained. As shown in Figure 2, there are two adjacent magnetic tunnel junctions: a first magnetic tunnel junction and a second magnetic tunnel junction. The first magnetic tunnel junction is in a parallel state (P, low resistance state), and the second magnetic tunnel junction is in an antiparallel state (AP, high resistance state), mapped to weights.

[0055] "1"; the first magnetic tunnel junction is in a parallel state (AP, high resistance state), and the second magnetic tunnel junction is in an antiparallel state (AP, high resistance state), which is mapped to a weight of "0"; the first magnetic tunnel junction is in a parallel state (AP, high resistance state), and the second magnetic tunnel junction is in an antiparallel state (P, low resistance state), which is mapped to a weight of "-1".

[0056] It can also be stipulated that the first magnetic tunnel junction is in a parallel state (P, low resistance state), the second magnetic tunnel junction is in an antiparallel state (AP, high resistance state), and is mapped to a weight of "-1"; the first magnetic tunnel junction is in a parallel state (AP, high resistance state), the second magnetic tunnel junction is in an antiparallel state (AP, high resistance state), and is mapped to a weight of "0"; the first magnetic tunnel junction is in a parallel state (AP, high resistance state), the second magnetic tunnel junction is in an antiparallel state (P, low resistance state), and is mapped to a weight of "1".

[0057] In the embodiment of the present application, the voltage matrix is ​​mapped into a current matrix based on the voltage-current mapping relationship and the convolution kernel. Based on the voltage matrix and the current matrix, a pixel-current mapping relationship is obtained. The voltage-current mapping relationship is the quantitative relationship between voltage, conductance, and current; each column vector in the voltage matrix corresponds to a column of magnetic tunnel junctions in a magnetic tunnel junction array; and each column of magnetic tunnel junctions corresponds to a conductance matrix.

[0058] The above process is explained using the Roberts operator as an example, as shown in Figure 1.

[0059] In the magnetic tunnel junction array, the pixel matrix is ​​mapped according to the preset voltage-pixel mapping relationship. Mapped into a voltage matrix After that, the convolution kernel is split into a vertical matrix and a parallel matrix. The Roberts operator is a 2×2 operator, which is divided into two parts: the horizontal and vertical directions, namely the vertical matrix and the parallel matrix:

[0060] Preset resistance states are written into the magnetic tunnel junction array corresponding to the vertical matrix and the magnetic tunnel junction array corresponding to the parallel matrix respectively, to obtain a vertical conductivity matrix and a parallel conductivity matrix:

[0061] According to Ohm's law and Kirchhoff's law, the convolution operation of the pixel matrix P and the Roberts operator can be directly equivalent to the reading operation of the magnetic tunnel junction array, that is:

[0062] Then according to V ij and I out The current coordinate diagram shown in FIG2 is obtained, that is, the pixel-current mapping relationship.

[0063] In Figure 2, the first column of output current corresponds to the first column of the voltage matrix V, and the second column of output current corresponds to the second column of the voltage matrix V. The horizontal and vertical axes represent the two elements in the first and second columns of the matrix, respectively, and the current I is distinguished by color. out .

[0064] Specifically, as shown in Figure 3, the pixel matrix in the image is split into the first and second columns. Since there is a corresponding relationship between the pixel matrix and the voltage matrix, the first and second columns of the voltage matrix V can be obtained based on the first and second columns of the pixel matrix. The relationship between current and voltage is found using Figure 2. Once found, the first and second columns of the voltage matrix V are replaced with the corresponding first and second columns of the pixel matrix, thus establishing a pixel-current mapping relationship.

[0065] Based on the above method, an embodiment of the present application provides an image feature extraction device based on a magnetic tunnel junction array, as shown in FIG4 , comprising a magnetic tunnel junction, a word line conductive portion, and a bit line conductive portion;

[0066] The word line conductive portion is arranged along the word line; the bit line conductive portion is arranged along the bit line; a magnetic tunnel junction is set at the intersection of the word line conductive portion and the bit line conductive portion; one end of the magnetic tunnel junction is connected to the word line conductive portion, and the other end is connected to the bit line conductive portion.

[0067] The word line conductive portion and the bit line conductive portion form a grid structure; each node in the grid structure corresponds to a magnetic tunnel junction; and two adjacent magnetic tunnel junctions correspond to a weight value of a convolution kernel.

[0068] In an embodiment of the present application, the word line conductive portion includes: a metal line, or the word line conductive portion includes: a metal line and a transistor gate.

[0069] In an embodiment of the present application, the bit line conductive portion includes: a metal line, or the bit line conductive portion includes: a metal line and a transistor gate.

[0070] Specifically, as shown in FIG4 , the image feature extraction device based on the magnetic tunnel junction array uses a passive (no transistors, diodes, or other gates) STT-MRAM array. In this embodiment, when the weight of the magnetic tunnel junction unit M(i, j) needs to be written, the write voltage Vwrite is applied to the bottom electrode metal wire of the i-th row, and the top electrode metal wire of the j-th column is grounded, and the weight writing of M(i, j) is achieved through spin transfer torque. When the j-th column needs to be read, the top electrode metal wire of the j-th column is grounded, and the read voltage vector (V1, V2, …, Vn)T is added to the bottom electrode metal wires of each row, and the read current on the top electrode metal wire of the j-th column is read. Compared with active arrays, passive arrays can reduce area overhead and achieve higher density integration.

[0071] As shown in Figure 5, the magnetic tunnel junction array-based image feature extraction device uses an active (transistor-based) STT-MRAM array. In this embodiment, when a weight is written to a magnetic tunnel junction cell M(i, j), the gate WL[i] of the control transistor in the i-th row is connected to a high level, the write voltage Vwrite is applied to the bottom electrode metal line in the i-th row, and the top electrode metal line in the j-th column is grounded. The weight writing of M(i, j) is achieved through spin transfer torque. When a read operation is performed on the j-th column, the gate WL of the control transistor in each row is connected to a high level, the top electrode metal line in the j-th column is grounded, and the read voltage vector (V1, V2, …, Vn)T is applied to the bottom electrode metal lines in each row. The read current on the top electrode metal line in the j-th column is read. Compared to passive arrays, active arrays avoid ideal factors such as leakage current and facilitate the implementation of larger arrays.

[0072] As shown in Figure 6, an image feature extraction device based on a magnetic tunnel junction array uses an active (transistor-based) SOT-MRAM array. In this embodiment, when a weight is written to a magnetic tunnel junction cell M(i, j), the gate WL[i] of the write control transistor in the i-th row is connected high, and the drain is applied with a write voltage Vwrite. The gate BL[j] of the read control transistor in the j-th column is connected low, and the weight is written to M(i, j) via spin-orbit moment. When a read operation is performed on the j-th column, the gate BL[j] of the read control transistor in the j-th column is connected high, and the drain is grounded. The gate WL of each row's write control transistor is connected low, and the gate is turned off. A read voltage vector (V1, V2, ..., Vn)T is applied to the other end of each row's bottom electrode metal line, away from the write control transistor, to read the read current on the top electrode metal line in the j-th column.

[0073] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0074] One or more embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0075] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0076] The foregoing description is merely an example of the present invention and is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims herein.

Claims

1. A method for extracting features from a magnetic tunnel junction array image, characterized in that: include: In the training phase, a pixel-current mapping relationship is obtained based on a preset voltage-pixel mapping relationship, a voltage-current mapping relationship, and a preset convolution kernel; The convolution kernel is a magnetic tunnel junction array; In the feature extraction stage, the pixels of the image to be processed are mapped to voltages; the voltages are used as inputs, and the current coordinates corresponding to the image to be processed are output based on the convolution kernel; and the pixels of the image to be processed after convolution are determined according to the current coordinates and the pixel-current mapping relationship.

2. The method according to claim 1, characterized in that The pixel-current mapping relationship is obtained based on the preset voltage-pixel mapping relationship, the voltage-current mapping relationship and the preset convolution kernel, including: According to the voltage-pixel mapping relationship, mapping the pixel matrix into a voltage matrix; According to the voltage-current mapping relationship and the convolution kernel, the voltage matrix is ​​mapped into a current matrix; wherein the voltage-current mapping relationship is a quantitative relationship between voltage, conductance and current; each column vector in the voltage matrix corresponds to a column of magnetic tunnel junctions in the magnetic tunnel junction array; each column of the magnetic tunnel junctions corresponds to a conductance matrix; The pixel-current mapping relationship is obtained according to the voltage matrix and the current matrix.

3. The method according to claim 2, characterized in that The magnetic tunnel junction array comprises: a plurality of magnetic tunnel junctions; Two adjacent magnetic tunnel junctions map a weight in the convolution kernel.

4. The method according to claim 3, characterized in that The two adjacent magnetic tunnel junctions map a weight in the convolution kernel, specifically: Each of the magnetic tunnel junctions has two resistance states; By changing the resistance states of two adjacent magnetic tunnel junctions, corresponding weights are obtained.

5. The method according to claim 2, characterized in that: Mapping the voltage matrix into a current matrix according to the voltage-current mapping relationship and the convolution kernel includes: Split the convolution kernel into vertical matrix and parallel matrix; Writing preset resistance states into the magnetic tunnel junction array corresponding to the vertical matrix and the magnetic tunnel junction array corresponding to the parallel matrix respectively, to obtain a vertical conductivity matrix and a parallel conductivity matrix; According to Ohm's law and Kirchhoff's law, a current matrix is ​​obtained by using the voltage-current mapping relationship, the vertical conductivity matrix and the parallel conductivity matrix.

6. The method according to any one of claims 1 to 5, characterized in that: The operator of the convolution kernel is specifically: Roberts operator or Prewitt operator.

7. A device for extracting image features based on a magnetic tunnel junction array, characterized in that: It includes a magnetic tunnel junction, a word line conductive part and a bit line conductive part; The word line conductive part is arranged along the word line; The bit line conductive part is arranged along the bit line; The magnetic tunnel junction is arranged at a position where the word line conductive portion and the bit line conductive portion intersect; One end of the magnetic tunnel junction is connected to the word line conductive portion, and the other end is connected to the bit line conductive portion.

8. The device according to claim 7, characterized in that The word line conductive portion and the bit line conductive portion form a grid structure; Each node in the grid structure corresponds to one magnetic tunnel junction; Two adjacent magnetic tunnel junctions correspond to a weight of the convolution kernel.

9. The device according to claim, characterized in that The word line conductive part includes: a metal line; or The word line conductive part includes: a metal line and a transistor gate.

10. The method according to claim 7, characterized in that The bit line conductive part includes: a metal line; or, The bit line conductive part includes: a metal line and a transistor gate.

Images