A neural network compression method based on memristor and neural network accelerator

Abstract

The present invention relates to the technical field of neural network compression, and proposes a neural network compression method and a neural network accelerator based on a memristor, comprising the following steps: converting the weights of a pre-trained neural network stored in a memristor into a plurality of column vector subsets; performing a pruning operation on each column vector subset to obtain a first pruned column vector subset; searching for an optimal pruning rate layer by layer; obtaining a second pruned column vector subset using the optimal pruning rate, and encoding the second pruned column vector subset to obtain a plurality of mask subsets; obtaining each third pruned column vector subset using each mask subset; solving the optimal third pruned column vector subset, and clustering them to obtain a clustering sub-matrix set; mapping the clustering sub-matrix set to a memristor, and after the mapping is completed, retraining the pre-trained neural network based on the memristor to complete the compression of the memristor-based neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network compression, and more specifically, to a memristor-based neural network compression method and a neural network accelerator. Background Art

[0002] With the advent of the information age and the rapid development of big data and artificial intelligence technologies, the amount of data generated daily is growing exponentially, placing higher demands on computer computing and storage capabilities. In the traditional von Neumann architecture, memory and processors are separated. When processing large amounts of data, memory and processors need to interact frequently. This frequent data communication consumes a significant portion of information processing time and power consumption, leading to bottlenecks in computing speed and energy efficiency. To address this bottleneck, researchers have begun researching the use of new hardware devices to accelerate neural network computations.

[0003] A memristor is a hardware device capable of fast multiplication. It possesses both computational and storage capabilities, eliminating the need for data communication between two devices. This effectively improves the computing speed and energy efficiency of computing devices, and the stored data is protected from loss after a power outage, offering strong security. Due to the memristor's advantages of integrated storage and computation, as well as its non-volatility, memristor-based neural network accelerators have become an important research direction. However, neural networks, such as deep neural networks (DNNs), exhibit weight sparsity, with a significant portion of their weights being close to or equal to zero. While these weights have little or no impact on neural network performance, they do increase the computational overhead of memristor-based neural network accelerators. Therefore, a neural network compression method suitable for memristor-based neural network accelerators is urgently needed. This method could remove redundant weights close to or equal to zero in the neural network, thereby reducing the computational overhead of memristor-based neural network accelerators.

[0004] The prior art proposes a method and system for structured pruning, which includes the following steps: S1, setting a specified network layer in a deep neural network as a to-be-pruned layer to obtain a to-be-processed deep neural network; S2, using an image dataset to train the to-be-processed deep neural network multiple times and performing multiple pre-pruning processes during the training process, each pre-pruning process including: according to the importance of all to-be-pruned layers on the image dataset, setting multiple to-be-pruned layers that are ranked low in importance and can meet the pruning amount of each pre-pruning after pre-pruning as to-be-pruned, and the importance is determined based on the weight parameters, gradients, and activation frequencies of the to-be-pruned layers; S3, when the number of pre-pruning processes reaches a predetermined number of pre-pruning processes, using the image dataset to fine-tune the to-be-processed deep neural network after pre-pruning, and setting the network layers corresponding to the pre-pruned to-be-pruned layers to be pre-pruned. The network layer is pruned; the weight matrix obtained by this method can match the memristor cross array composed of memristors, so this method is suitable for memristor-based neural network accelerators and can reduce the computational overhead of memristor-based neural network accelerators. However, the structured pruning method adopted by this method is a coarse-grained pruning method that prunes the entire convolution kernel of the neural network as a unit. Because the entire filter of the neural network contains both redundant weights and important weights, pruning the entire filter of the neural network is likely to cause some important weights to be pruned. If the neural network is compressed by this method and the compression method is used in the process of accelerating the neural network by the memristor-based neural network accelerator, the computational accuracy of the accelerated calculation of the memristor-based neural network accelerator will be low.

[0005] Prior art also proposes a convolutional neural network pruning method for industrial production applications. This method first performs unstructured pruning on the convolutional neural network to be pruned, pruning some neurons based on their importance, and then recovering the accuracy from training. Then, structured pruning is performed to prune some convolution kernels based on their perceived size, and then recovering the accuracy from training. This method first uses fine-grained unstructured pruning to prune some unimportant neurons, and then combines it with structured pruning to prune some convolution kernels. This method, combining unstructured and structured pruning, can be applied to neural network accelerators, reducing the computational overhead of the network accelerator while maintaining the computational accuracy of the accelerated calculations. However, because unstructured pruning causes the neural network to generate a sparser weight matrix, which cannot be matched with a memristor crossbar array composed of memristors, this method is not suitable for memristor-based neural network accelerators and cannot use memristors to improve the computing speed of neural network accelerators.

[0006] In summary, the pruning methods proposed in the prior art cannot simultaneously improve the computational accuracy and computational speed of the neural network accelerator when compressing the neural network. Summary of the Invention

[0007] In order to overcome the defect of the above-mentioned prior art that the calculation accuracy and calculation speed of the neural network accelerator cannot be improved simultaneously, the present invention provides a neural network compression method and a neural network accelerator based on a memristor that can simultaneously improve the calculation accuracy and calculation speed of the neural network accelerator.

[0008] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0009] A neural network compression method based on memristor comprises the following steps:

[0010] S1: Convert the weights of the pre-trained neural network stored in the memristor into a set of column vectors;

[0011] S2: Perform a pruning operation on each column vector subset to obtain a first pruned column vector subset corresponding to each column vector subset;

[0012] S3: Search for the optimal pruning rate of the first pruning column vector subset along the layers of the pre-trained neural network layer by layer;

[0013] S4: performing a pruning operation on each column vector subset using the optimal pruning rate to obtain a second pruned column vector subset corresponding to each column vector subset;

[0014] S5: Encode each second pruned column vector in each second pruned column vector subset to obtain a mask corresponding to each second pruned column vector, and form a mask subset corresponding to each second pruned column vector subset;

[0015] S6: using each mask subset, perform a pruning operation on each column vector subset to obtain each third pruned column vector subset;

[0016] S7: solving the optimal third pruned column vector subset corresponding to each third pruned column vector subset, and clustering each optimal third pruned column vector in each optimal third pruned column vector subset to obtain a plurality of clustering submatrices corresponding to each optimal third pruned column vector subset to form a clustering submatrix set;

[0017] S8: Mapping the cluster submatrix set to the memristor, and after the mapping is completed, retraining the pre-trained neural network based on the memristor to complete the compression of the memristor-based neural network.

[0018] The present invention also proposes a neural network accelerator, wherein the computing unit of the neural network accelerator is a memristor cross array composed of memristors. The neural network accelerator is used to accelerate the calculation of a memristor-based neural network, and in the process of accelerating the calculation of a memristor-based neural network, the neural network accelerator executes a memristor-based neural network compression method.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0020] The present invention converts the weights of a pre-trained neural network stored in a memristor into several column vector subsets, and prunes the weights of the pre-trained neural network by pruning the column vectors in the column vector subsets, thereby achieving the purpose of compressing the pre-trained neural network. The column vectors in the column vector subsets are pruned using an optimal pruning rate obtained by layer-by-layer search and a mask subset obtained by encoding, ensuring that the pruned weights do not affect the performance of the pre-trained neural network, thereby improving the computational accuracy of the neural network accelerator performing accelerated calculations on the pre-trained neural network. At the same time, the optimal third pruned column vectors are clustered to obtain a clustering submatrix that can match a memristor cross array composed of memristors, so that the clustering submatrix can be mapped to the memristor. Therefore, the memristor-based neural network compression method of the present invention is applicable to the memristor-based neural network accelerator, thereby achieving the purpose of using the memristor to improve the computing speed of the neural network accelerator, thereby achieving the effect of simultaneously improving the computing accuracy and computing speed of the neural network accelerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the flow of the memristor-based neural network compression method of Example 1;

[0022] Figure 2 This is a schematic diagram of the process of obtaining a column vector set in Example 1;

[0023] Figure 3 This is a schematic diagram of the process of obtaining the third pruned column vector set in Example 2;

[0024] Figure 4 Schematic diagram of the process of obtaining a clustering submatrix set in Example 2;

[0025] Figure 5 Schematic diagram of a memristor crossbar array of a neural network accelerator of Example 3. DETAILED DESCRIPTION

[0026] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0027] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0028] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0029] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0030] Example 1

[0031] This embodiment proposes a neural network compression method based on memristor. Figure 1 Schematic diagram of the flow of the memristor-based neural network compression method of this embodiment.

[0032] The memristor-based neural network compression method proposed in this embodiment includes the following steps:

[0033] S1: Convert the weights of the pre-trained neural network stored in the memristor into a set of column vectors;

[0034] S2: Perform a pruning operation on each column vector subset to obtain a first pruned column vector subset corresponding to each column vector subset;

[0035] S3: Search for the optimal pruning rate of the first pruning column vector subset along the layers of the pre-trained neural network layer by layer;

[0036] S4: performing a pruning operation on each column vector subset using the optimal pruning rate to obtain a second pruned column vector subset corresponding to each column vector subset;

[0037] S5: Encode each second pruned column vector in each second pruned column vector subset to obtain a mask corresponding to each second pruned column vector, and form a mask subset corresponding to each second pruned column vector subset;

[0038] S6: using each mask subset, perform a pruning operation on each column vector subset to obtain each third pruned column vector subset;

[0039] S7: solving the optimal third pruned column vector subset corresponding to each third pruned column vector subset, and clustering each optimal third pruned column vector in each optimal third pruned column vector subset to obtain a plurality of clustering submatrices corresponding to each optimal third pruned column vector subset to form a clustering submatrix set;

[0040] S8: Mapping the cluster submatrix set to the memristor, and after the mapping is completed, retraining the pre-trained neural network based on the memristor to complete the compression of the memristor-based neural network.

[0041] During the specific implementation process, the weights of the pre-trained neural network stored in the memristor are converted into several column vector subsets, and the optimal pruning rate obtained by layer-by-layer search and the mask subset obtained by encoding are used to solve the optimal third pruned column vector to ensure that the pruned weights of the optimal third pruned column vector do not affect the performance of the pre-trained neural network, thereby improving the computational accuracy of the neural network accelerator for accelerating the pre-trained neural network. At the same time, the optimal third pruned column vector is clustered to obtain a clustering sub-matrix that can match the memristor cross array composed of memristors, so that the clustering sub-matrix can be mapped to the memristor, so that the memristor-based neural network compression method of the present invention can be applied to the memristor-based neural network accelerator, thereby achieving the purpose of using the memristor to improve the computing speed of the neural network accelerator, thereby achieving the effect of simultaneously improving the computing accuracy and computing speed of the neural network accelerator.

[0042] In an alternative embodiment, if Figure 2 As shown, it is a flow chart of obtaining a column vector subset of the present embodiment; the process of converting the weights of the pre-trained neural network stored in the memristor into a number of column vector subsets is as follows: removing the weights of the input layer and the output layer of the pre-trained neural network stored in the memristor, and converting the remaining weights of each layer of the pre-trained neural network into a two-dimensional matrix to obtain a weight matrix corresponding to the remaining weights of each layer; dividing each weight matrix into a number of weight sub-matrices, and representing each column of each weight sub-matrix in the form of a column vector to obtain a column vector subset corresponding to each weight matrix.

[0043] In an optional embodiment, the pruning operation in step S2 is:

[0044] According to the absolute values ​​of the elements in the column vectors, the elements in the column vectors with the smallest absolute values ​​1 to the smallest d are pruned to obtain the pruned column vectors corresponding to each column vector, and the first pruned column vector subset corresponding to each column vector subset is obtained; d is a positive integer value.

[0045] As an exemplary description, in this optional embodiment, the specific value of the number of elements pruned from each column vector is a random value.

[0046] Example 2

[0047] This embodiment further explains steps S3 to S8 in embodiment 1.

[0048] In an optional embodiment, step S3 is:

[0049] S3.1: Construct a pruning rate optimization model, specifically:

[0050]

[0051]

[0052]

[0053] Where, F i Represents the fitness function corresponding to the i-th layer of the pre-trained neural network, represents the current pruning rate of the i-th layer of the pre-trained neural network, β i express The weighted value, acc i Indicates based on The accuracy of the pre-trained neural network after pruning, α i Indicates acc i The weighted value of represents the number of elements in each first pruned column vector in the first pruned column vector corresponding to the i-th layer of the pre-trained neural network, and row represents the number of elements in each column vector in the subset of column vectors corresponding to the i-th layer of the pre-trained neural network;

[0054] S3.2: Using the pruning rate optimization model, after completing the penultimate layer of the pre-trained neural network, iteratively update the pruning rate optimization model, and N-1 When the minimum is reached, the iteration is ended, and the current pruning rate of the penultimate layer of the pre-trained neural network is obtained when the iteration is ended, that is, the optimal pruning rate of the penultimate layer of the pre-trained neural network is obtained;

[0055] Among them, the fitness function F N-1 The expression is:

[0056]

[0057] Where, represents the current pruning rate of the penultimate layer of the pre-trained neural network, β N-1 express The weighted value, acc N-1 Indicates based on The accuracy of the pre-trained neural network after pruning, α N-1 Indicates acc N-1 The weighted value of

[0058] S3.3: According to the method of step S3.2, the optimal pruning rate from the third to last layer to the second layer of the pre-trained neural network is solved layer by layer.

[0059] In this optional embodiment, α i and β i The ratio is dynamically adjusted to complete the α from the penultimate layer to the second layer of the pre-trained neural network. i and β i The ratio of decreases as the number of layers increases. As an example, when the α of the second layer of the pre-trained neural network is completed, i and β i The ratio is set to 9:1, completing the α of the third layer of the pre-trained neural network i and β i The ratio is set to 6:4.

[0060] In an optional embodiment, in step S4, the step of performing a pruning operation on the i-th column vector subset using the optimal pruning rate is:

[0061] According to the absolute value of the element value in the column vector, each column vector in the i-th column vector subset with the absolute value from the smallest 1 to the smallest b i ′The elements in the smaller column vector are pruned to obtain the second pruned column vector corresponding to each column vector, and the second pruned column vector subset corresponding to the i-th column vector subset is obtained;

[0062] Among them, b i ′ represents the optimal pruning number corresponding to the i-th column vector subset, and the optimal pruning number b i The calculation expression of ′ is:

[0063]

[0064]

[0065] Where, rate p B i Indicates the optimal pruning rate corresponding to the i-th layer of the pre-trained neural network.

[0066] In this optional embodiment, by pruning the same number of elements from each column vector in any column vector set, it is ensured that the number of elements in each second pruned column vector in the second pruned column vector subset is the same, thereby ensuring that the pruning result is applicable to the memristor-based neural network accelerator, thereby improving the computing speed of the neural network accelerator; wherein, the number of pruned elements from each column vector in any column vector set adopts the optimal pruning number, ensuring that the number of deleted elements is optimal, which helps to subsequently determine the positions of elements corresponding to unimportant weights, and prune the elements corresponding to unimportant weights, ensuring that the pruned elements are all elements corresponding to unimportant weights, thereby improving the computing accuracy of the neural network accelerator.

[0067] In an optional embodiment, in step S5, the step of encoding each second pruned column vector in the i-th subset of second pruned column vectors is:

[0068] The pruned elements of each second pruned column vector in the i-th second pruned column vector subset are encoded as 0, and the retained elements of each second pruned column vector in the i-th second pruned column vector subset are encoded as 1, to obtain a mask corresponding to each second pruned column vector.

[0069] In an alternative embodiment, if Figure 3 , which is a schematic diagram of the process of obtaining the third pruned column vector set according to this embodiment;

[0070] In step S6, the steps of obtaining the i-th third pruned column vector subset using the i-th mask subset are:

[0071] Count the types of masks and the number of each mask in the i-th mask subset, sort each mask in descending order according to the number of each mask, retain the first several masks to form a retained mask subset; match the masks in the retained mask subset with the i-th second pruned column vector subset; if the second pruned column vector in the i-th second pruned column vector subset cannot match the corresponding mask, randomly select a mask from the retained mask subset as its corresponding mask;

[0072] According to the mask corresponding to each second pruned column vector in the i-th second pruned column vector subset, a pruning operation is performed on the i-th column vector subset, and the elements at the positions encoded as 0 in the mask corresponding to the i-th column vector of the i-th column vector subset are pruned to obtain the i-th third pruned column vector subset.

[0073] As an exemplary illustration, the number of mask types retained in the i-th mask subset is determined according to the hyperparameters in the pre-trained neural network.

[0074] In an optional embodiment, the step of solving the optimal third pruned column vector subset corresponding to each third pruned column vector subset is:

[0075] Construct an optimal mask search model, specifically:

[0076]

[0077] Z i =W i

[0078]

[0079] Where W i represents the set of the third pruned column vectors of the i-th column; Si Indicates the constraint condition that the number of pruning branches is the optimal number of pruning branches, U q represents the objective function, and f(·) represents the loss function of the pre-trained neural network;

[0080] The optimal mask search model is iteratively updated using the alternating direction multiplier method. q When the minimum is reached, the iteration ends and the optimal third pruned column vector subset corresponding to each column vector subset is obtained.

[0081] In this optional embodiment, by fixing the number of pruned column vector elements as the optimal pruning number, a search is performed for the pruned column vector elements that, after pruning, minimize the loss value corresponding to the loss function of the pre-trained neural network, thereby ensuring that the weights corresponding to the pruned column vector elements are all non-important weights, thereby improving the computational accuracy of the neural network accelerator.

[0082] In an alternative embodiment, if Figure 4 FIG. 1 is a flow chart of obtaining a clustering submatrix set according to this embodiment; the steps of obtaining the clustering submatrix set are:

[0083] The optimal third pruned column vectors with the same mask in each optimal third pruned column vector subset are clustered into a clustering submatrix, and several clustering submatrices corresponding to each optimal third pruned column vector subset are obtained to form a clustering submatrix set.

[0084] In this embodiment, the optimal third pruned column vectors with the same mask are clustered into a cluster sub-matrix, so that the weights in the cluster sub-matrix can be mapped in the memristor cross array formed by the memristors, thereby enabling the weight calculation in the cluster sub-matrix to be completed in a single clock cycle of the memristor, which is beneficial to speeding up the computing speed of the neural network accelerator.

[0085] Example 3

[0086] This embodiment proposes a neural network accelerator, such as Figure 5 As shown, this is a schematic diagram of the memristor cross array of the neural network accelerator of this embodiment; the computing unit of the neural network accelerator is a memristor cross array composed of memristors, and the neural network accelerator is used to accelerate the calculation of a memristor-based neural network, and in the process of accelerating the calculation of a memristor-based neural network, the neural network accelerator executes the memristor-based neural network compression method described in Example 1 or Example 2.

[0087] It can be understood that the computer device of this embodiment is applied to the method of the above-mentioned embodiment 1, and the optional items in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described again here.

[0088] The same or similar reference numerals correspond to the same or similar components;

[0089] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0090] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A neural network compression method based on memristor, characterized in that: The following steps are involved: S1: Convert the weights of the pre-trained neural network stored in the memristor into a set of column vectors; S2: Perform a pruning operation on each column vector subset to obtain a first pruned column vector subset corresponding to each column vector subset; S3: Search for the optimal pruning rate of the first pruning column vector subset along the layers of the pre-trained neural network layer by layer; S4: performing a pruning operation on each column vector subset using the optimal pruning rate to obtain a second pruned column vector subset corresponding to each column vector subset; S5: Encode each second pruned column vector in each second pruned column vector subset to obtain a mask corresponding to each second pruned column vector, and form a mask subset corresponding to each second pruned column vector subset; S6: using each mask subset, perform a pruning operation on each column vector subset to obtain each third pruned column vector subset; S7: solving the optimal third pruned column vector subset corresponding to each third pruned column vector subset, and clustering each optimal third pruned column vector in each optimal third pruned column vector subset to obtain a plurality of clustering submatrices corresponding to each optimal third pruned column vector subset to form a clustering submatrix set, so that the weights in the clustering submatrices can be mapped in the memristor crossbar array formed by the memristors, so that the weight calculation in the clustering submatrix can be completed in a single clock cycle of the memristor; S8: Mapping the cluster submatrix set to the memristor, and after the mapping is completed, retraining the pre-trained neural network based on the memristor to complete the compression of the memristor-based neural network.

2. The memristor-based neural network compression method according to claim 1, characterized in that: In step S1, the weights of the input layer and output layer of the pre-trained neural network stored in the memristor are removed, and the remaining weights of each layer of the pre-trained neural network are converted into a two-dimensional matrix to obtain a weight matrix corresponding to the remaining weights of each layer; each weight matrix is ​​divided into several weight sub-matrices, and each column of each weight sub-matrix is ​​represented in the form of a column vector to obtain a column vector subset corresponding to each weight matrix.

3. The memristor-based neural network compression method according to claim 2, characterized in that: The pruning operation described in step S2 is: According to the absolute values ​​of the elements in the column vectors, the elements in the column vectors with the smallest absolute values ​​1 to the smallest d are pruned to obtain the pruned column vectors corresponding to each column vector, and the first pruned column vector subset corresponding to each column vector subset is obtained; d is a positive integer value.

4. The memristor-based neural network compression method according to any one of claims 1 to 3, characterized in that: The S3 steps are: S3.1: Construct a pruning rate optimization model, specifically: Where, Indicates the first step of the pre-trained neural network The fitness function corresponding to the layer, Indicates the first step of the pre-trained neural network The current pruning rate of the layer, express The weighted value of Indicates based on The accuracy of the pre-trained neural network after pruning, express The weighted value of Indicates the first step of the pre-trained neural network The number of elements in each first pruned column vector in the first pruned column vector corresponding to the layer, Indicates the first step of the pre-trained neural network The number of elements in each column vector in the column vector subset corresponding to the layer; S3.2: Using the pruning rate optimization model, after completing the penultimate layer of the pre-trained neural network, iteratively update the pruning rate optimization model, in the fitness function When the minimum is reached, the iteration is ended, and the current pruning rate of the penultimate layer of the pre-trained neural network is obtained when the iteration is ended, that is, the optimal pruning rate of the penultimate layer of the pre-trained neural network is obtained; Among them, the fitness function The expression is: Where, represents the current pruning rate of the penultimate layer of the pre-trained neural network, express The weighted value of Indicates based on The accuracy of the pre-trained neural network after pruning, express The weighted value of S3.3: According to the method of step S3.2, the optimal pruning rate from the third to last layer to the second layer of the pre-trained neural network is solved layer by layer.

5. The memristor-based neural network compression method according to claim 4, characterized in that: In step S4, the optimal pruning rate is used to prune the first The steps for pruning a set of column vectors are: According to the absolute value of the element value in the column vector, the In each column vector of the subset of column vectors, the absolute value of The elements in the small column vector are pruned to obtain the second pruned column vector corresponding to each column vector, and the first The second pruned column vector subset corresponding to the column vector subset; in, Indicates the The optimal number of pruning branches corresponding to the set of column vectors, the optimal number of pruning branches The calculation expression is: Where, Indicates the first step of the pre-trained neural network The optimal pruning rate corresponding to the layer.

6. The memristor-based neural network compression method according to claim 5, characterized in that: In step S5, The steps of encoding each second pruned column vector in the subset of second pruned column vectors are: The first In the subset of the second pruned column vectors, the pruned elements of each second pruned column vector are encoded as 0, and the first In the subset of second pruned column vectors, the retained elements of each second pruned column vector are encoded as 1, and a mask corresponding to each second pruned column vector is obtained.

7. The memristor-based neural network compression method according to claim 6, characterized in that: In step S6, the mask subsets, and obtain the The steps for the third pruning column vector set are: Statistics In the mask subsets, the types of masks and the number of each mask are sorted in descending order according to the number of each mask, and the first several masks are retained to form the retained mask subsets; the masks in the retained mask subsets are compared with the first The second pruned column vector matches the subset, The second pruned column vector that cannot match the corresponding mask in the second pruned column vector subset will randomly select a mask from the retained mask subset as its corresponding mask; According to In the subset of the second pruned column vectors, the mask corresponding to each second pruned column vector is Column vector subsets are pruned, and the The first column vector set In the column vector, the elements at the positions where the corresponding codes are 0 in the mask are cut off to obtain the first The third pruned column vector set.

8. The memristor-based neural network compression method according to claim 7, characterized in that: The steps for solving the optimal third pruned column vector subset corresponding to each third pruned column vector subset are: Construct an optimal mask search model, specifically: Where, Indicates the A third set of pruned column vectors; Indicates the constraint that the number of pruning branches is the optimal number of pruning branches, represents the objective function, Represents the loss function of the pre-trained neural network; The optimal mask search model is iteratively updated using the alternating direction multiplier method. When the minimum is reached, the iteration ends and the optimal third pruned column vector subset corresponding to each column vector subset is obtained.

9. The memristor-based neural network compression method according to claim 8, characterized in that: The steps to obtain the set of clustering submatrices are: The optimal third pruned column vectors with the same mask in each optimal third pruned column vector subset are clustered into a clustering submatrix, and several clustering submatrices corresponding to each optimal third pruned column vector subset are obtained to form a clustering submatrix set.

10. A neural network accelerator, characterized in that: The computing unit of the neural network accelerator is a memristor cross array composed of memristors. The neural network accelerator is used to accelerate the calculation of a neural network based on a memristor, and in the process of accelerating the calculation of a neural network based on a memristor, the neural network accelerator executes the memristor-based neural network compression method described in any one of claims 1 to 9.

Images