Hybrid clustering optimization method and system for memristor array weight mapping

Abstract

The invention discloses a hybrid clustering optimization method and system for memristor array weight mapping, belongs to the technical field of storage and calculation integrated chips, and solves the problem that the reasoning precision is reduced due to the fact that edge weights are forcibly classified into unmatched clustering centers due to the fact that an existing clustering method is poor in adaptability to weight distribution characteristics. According to the method, the posterior probability is obtained based on the weight of the target neural network and the discrete level number of the conductance state of the initialized target memristor array; performing hierarchical clustering based on a result obtained in the step 1 to obtain a corresponding clustering center; performing resistance drift pre-compensation based on all the clustering centers to obtain a corrected clustering center; and carrying out weight-conductance mapping deployment based on the corrected clustering center. The memristor array weight mapping method is used for memristor array weight mapping.

Description

Technical Field

[0001] A hybrid clustering optimization method and system for memristor array weight mapping are used for memristor array weight mapping and belong to the technical field of storage and computing integrated chip. Background Art

[0002] In recent years, memristors have shown great potential in neuromorphic computing and integrated computing architectures due to their natural ability to simulate synaptic weights. In particular, in the field of deep learning acceleration, by mapping neural network weights to the discrete conductance states of the memristor array, highly energy-efficient matrix multiplication and addition operations can be achieved. However, due to the non-idealities of the memristor manufacturing process (such as conductance discretization and nonlinear drift) and the complexity of the neural network weight distribution, how to achieve high-precision and low-loss weight-conductance mapping remains a core problem that restricts its practical application. In the existing technology, weight discretization methods based on clustering algorithms are widely used, but the following problems still exist:

[0003] 1. Traditional clustering methods have poor adaptability to weight distribution characteristics, which results in edge weights being forced to be assigned to incompatible cluster centers, resulting in reduced inference accuracy.

[0004] 2. The lack of a parameter sharing mechanism across network layers will cause different layers to adopt the same conductance state allocation strategy, resulting in a waste of resources;

[0005] 3. The resistance drift compensation mechanism is not perfect, which causes the drift of the conductance value after writing, making the original cluster center invalid and increasing the mapping error. Summary of the Invention

[0006] In response to the above research problems, the purpose of the present invention is to provide a hybrid clustering optimization method and system for memristor array weight mapping, so as to solve the problem that the existing clustering methods have poor adaptability to the weight distribution characteristics, resulting in edge weights being forced to be classified into mismatched cluster centers, resulting in a decrease in reasoning accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A hybrid clustering optimization method for memristor array weight mapping includes the following steps:

[0009] Step 1: Obtain the posterior probability based on the weight of the target neural network and the number of discrete levels of the conductivity state of the initialized target memristor array;

[0010] Step 2: Perform hierarchical clustering based on the results obtained in step 1 to obtain the corresponding cluster centers;

[0011] Step 3: Perform resistance drift pre-compensation based on all cluster centers to obtain corrected cluster centers;

[0012] Step 4: Deploy weight-conductance mapping based on the corrected cluster centers.

[0013] Furthermore, the specific steps of step 1 are:

[0014] Step 1.1: Data preprocessing and parameter initialization, that is, artificially extracting the weight matrix W∈R of each layer of the target neural network model m×n , and mark the layer type. At the same time, initialize the number of discrete levels of the conductance state of the target memristor array. The marked layer type includes the marked convolution layer Conv and the fully connected layer FC, R represents the real matrix space, m represents the output dimension, and n is the input dimension;

[0015] Step 1.2: Normalization, that is, linearly normalizing the weight values ​​in the weight matrix to the dynamic range of the memristor conductance [g min , g max ], the normalized formula is:

[0016]

[0017] Among them, w norm is the normalized weight matrix, w max and w min are the maximum and minimum values ​​of the weight values ​​in the weight matrix respectively;

[0018] Step 1.3, initialize the parameters of the Gaussian mixture model GMM, that is, set the number of Gaussian components of the Gaussian mixture model based on the number of discrete levels of the conductivity state, and the initial mean value of each Gaussian component μ k To distribute the conductivity intervals equally, the covariance ∑ k Set to 1 / 3 of the distance between adjacent conductances, the mixing coefficient π k =1 / K, K represents the number of Gaussian components, that is, the number of cluster centers;

[0019] Step 1.4: EM algorithm iteratively optimizes the initialized Gaussian mixture model GMM, that is, calculates the i-th weight sample w i The posterior probability of belonging to the kth Gaussian component is:

[0020]

[0021] Among them, N(w|μ,∑) is the Gaussian distribution function, γ ik is the weight sample w i The posterior probability of belonging to the kth Gaussian component, π k and π j are the mixing coefficients of the kth and jth Gaussian components, w i is the i-th weight sample, that is, w normThe normalized weight value of the ith element in μ k and ∑ k is the mean and covariance of the kth Gaussian component;

[0022] Step 1.5: Update parameters based on the posterior probability. The formula is:

[0023]

[0024] in, and represents the updated mean, covariance and mixing coefficient of the k-th Gaussian component, and M represents the total number of weight samples in the current layer;

[0025] Step 1.6: When the likelihood function change rate is less than the predetermined value, stop the iteration and obtain the final posterior probability. Otherwise, the updated probability obtained in step 1.5 is and As μ k ,∑ k and π k Repeat step 1.4, where the likelihood function is M represents the total number of weight samples in the current layer, K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian component, represents the variance of the kth Gaussian component, Represents the Gaussian probability density function, and the rate of change of the likelihood function is the difference between the results of the two likelihood functions divided by the previous one.

[0026] Furthermore, the specific steps of step 2 are:

[0027] Convolutional layer clustering:

[0028] Step 2.1: For the sensitivity weighted GMM clustering method of the convolutional layer, first obtain the weight gradient of the cth weight sample of the convolutional layer through back propagation Calculating sensitivity And perform normalization processing to obtain normalized sensitivity. The formula for normalization processing is:

[0029]

[0030] Among them, s norm represents the normalized sensitivity to the cth weight sample, s max and s min Respectively represent the minimum and maximum values ​​of the current layer weight sensitivity:

[0031] Step 2.2: Correct the posterior probability. This is to obtain the weighted posterior probability of the convolutional layer based on the posterior probability of the Gaussian mixture model GMM and the normalized sensitivity of the convolutional layer, and correct the original posterior probability. The formula is:

[0032]

[0033] in, is the weighted posterior probability of the convolutional layer. When i is within the numerical range of the convolutional layer, c is equal to i.

[0034] Step 2.3: Calculate the divergence between the two Gaussian components based on the corrected posterior probability. If the divergence is less than 0.1, merge the Gaussian components and update K = K-1 as the cluster center of the convolutional layer. Otherwise, do not merge.

[0035] Density-adaptive clustering of fully connected layers:

[0036] Step 2.4: Calculate the density weight based on the weight samples of the fully connected layer. The formula is:

[0037]

[0038] Among them, d j is the weight sample w j The density weight of reflects its importance in clustering. M is the total number of weight samples in the fully connected layer. When i is within the numerical range of the fully connected layer, j is equal to i.

[0039] Step 2.5: Select the weight samples corresponding to the first S largest density weights in the fully connected layer as the initial cluster centers;

[0040] Step 2.6: Based on the results obtained in step 2.5, calculate the distance from each remaining weight sample in the fully connected layer to each initial cluster center, and assign each weight sample to the nearest cluster center c. m , and finally multiple clusters are obtained, and the distance is calculated as:

[0041]

[0042] in, ∈ represents the noise term;

[0043] Step 2.7, if the intra-class variance of each cluster Then create a new cluster center for the cluster, split the cluster into two sub-clusters, and delete the original cluster center. At this time, S=S+1, and execute step 2.6 again based on the new cluster center. Otherwise, get the final cluster center of the fully connected layer, where is the variance of the mth cluster, and α is the variance scaling factor.

[0044] Furthermore, the specific steps of step 3 are:

[0045] Step 3.1: Model the drift statistics, i.e. test the memristor array and record the ath conductance state G a The actual drift △g a , fitting the quadratic model, the formula is:

[0046] △g a =η·(G a -G mid ) 2 +∈

[0047] Among them, △g a The conductivity state G a The actual drift, η is the drift coefficient, through experimental calibration, G mid is the midpoint of the conductance interval, ∈ is the noise term, which obeys the Gaussian distribution with mean zero;

[0048] Step 3.2: All cluster centers c output in step 2 b Mapping to the nearest conductance state G a , then calculate the reverse compensation, the formula is:

[0049]

[0050] in, is the target conductivity value after compensation;

[0051] Step 3.3: Then, based on the compensated target conductance value, the cluster center is corrected, that is, the cluster center is adjusted to approach the compensated conductance value. The formula is:

[0052]

[0053] Among them, c b is the b-th cluster center output in step 2, is the target conductivity value after compensation, β is the compensation intensity coefficient, which is used to control the correction amplitude. is the modified cluster center, G b is the bth effective conductance state in the effective conductance state set.

[0054] Furthermore, the specific steps of step 4 are:

[0055] Step 4.1: The corrected cluster center Map to the target conductivity state according to the minimum distance, specifically:

[0056]

[0057] Among them, Gb is the optimized conductivity state set finally assigned to cluster center a, T(w i ) is the weight sample w i The target conductance state to be mapped, G a is the ath effective conductance state of the memristor, argmin represents the input when it returns to the minimum value;

[0058] Step 4.2: If multiple weight samples w i Mapped to the same target conductance state, according to T(w i ) prioritizes retaining the original distribution of highly sensitive weights, and finally generates the final discretization mapping table T;

[0059] Step 4.3: Generate pulse parameters according to the discretization mapping table T and write them into the memristor array, where the pulse parameters include amplitude and width;

[0060] Step 4.4, forward reasoning verification, that is, after the deployment of step 4.3, the accuracy loss of the test set memristor array is tested. If the loss is greater than the given threshold, the parameters λ or α in steps 1.6 and 2.7 are adjusted and the process returns to step 1. Otherwise, the final mapping result is obtained.

[0061] A hybrid clustering optimization system for memristor array weight mapping, comprising:

[0062] Posterior probability acquisition module: obtains the posterior probability based on the weights of the target neural network and the number of discrete levels of the conductivity state of the initialized target memristor array;

[0063] Hierarchical clustering module: Perform hierarchical clustering based on the results obtained in step 1 to obtain the corresponding cluster centers;

[0064] Correction module: pre-compensate resistance drift based on all cluster centers to obtain the corrected cluster centers;

[0065] Deployment module: Deploy weight-conductance mapping based on the corrected cluster centers.

[0066] Furthermore, the specific implementation steps of the posterior probability acquisition module are:

[0067] Step 1.1: Data preprocessing and parameter initialization, that is, artificially extracting the weight matrix W∈R of each layer of the target neural network model m×n , and mark the layer type. At the same time, initialize the number of discrete levels of the conductance state of the target memristor array. The marked layer type includes the marked convolution layer Conv and the fully connected layer FC, R represents the real matrix space, m represents the output dimension, and n is the input dimension;

[0068] Step 1.2: Normalization, that is, linearly normalizing the weight values ​​in the weight matrix to the dynamic range of the memristor conductance [g min , g max ], the normalized formula is:

[0069]

[0070] Among them, w norm is the normalized weight matrix, w max and w min are the maximum and minimum values ​​of the weight values ​​in the weight matrix respectively;

[0071] Step 1.3, initialize the parameters of the Gaussian mixture model GMM, that is, set the number of Gaussian components of the Gaussian mixture model based on the number of discrete levels of the conductivity state, and the initial mean value of each Gaussian component μ k To distribute the conductivity intervals equally, the covariance ∑ k Set to 1 / 3 of the distance between adjacent conductances, the mixing coefficient π k =1 / K, K represents the number of Gaussian components, that is, the number of cluster centers;

[0072] Step 1.4: EM algorithm iteratively optimizes the initialized Gaussian mixture model GMM, that is, calculates the i-th weight sample w i The posterior probability of belonging to the kth Gaussian component is:

[0073]

[0074] Among them, N(w|μ,∑) is the Gaussian distribution function, γ ik is the weight sample w i The posterior probability of belonging to the kth Gaussian component, π k and π j are the mixing coefficients of the kth and jth Gaussian components, w i is the i-th weight sample, that is, w norm The normalized weight value of the ith element in μ k and ∑ k is the mean and covariance of the kth Gaussian component;

[0075] Step 1.5: Update parameters based on the posterior probability. The formula is:

[0076]

[0077] in, and represents the updated mean, covariance and mixing coefficient of the k-th Gaussian component, and M represents the total number of weight samples in the current layer;

[0078] Step 1.6: When the likelihood function change rate is less than the predetermined value, stop the iteration and obtain the final posterior probability. Otherwise, the updated probability obtained in step 1.5 is and As μ k ,∑ k and π k Repeat step 1.4, where the likelihood function is M represents the total number of weight samples in the current layer, K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian component, represents the variance of the kth Gaussian component, Represents the Gaussian probability density function, and the rate of change of the likelihood function is the difference between the results of the two likelihood functions divided by the previous one.

[0079] Furthermore, the specific implementation steps of the hierarchical clustering module are:

[0080] Convolutional layer clustering:

[0081] Step 2.1: For the sensitivity weighted GMM clustering method of the convolutional layer, first obtain the weight gradient of the cth weight sample of the convolutional layer through back propagation Calculating sensitivity And perform normalization processing to obtain normalized sensitivity. The formula for normalization processing is:

[0082]

[0083] Among them, s norm represents the normalized sensitivity to the cth weight sample, s max and s min Represent the minimum and maximum values ​​of the weight sensitivity of the current layer respectively;

[0084] Step 2.2: Correct the posterior probability. This is to obtain the weighted posterior probability of the convolutional layer based on the posterior probability of the Gaussian mixture model GMM and the normalized sensitivity of the convolutional layer, and correct the original posterior probability. The formula is:

[0085]

[0086] in, is the weighted posterior probability of the convolutional layer. When i is within the numerical range of the convolutional layer, c is equal to i.

[0087] Step 2.3: Calculate the divergence between the two Gaussian components based on the corrected posterior probability. If the divergence is less than 0.1, merge the Gaussian components and update K = K-1 as the cluster center of the convolutional layer. Otherwise, do not merge.

[0088] Density-adaptive clustering of fully connected layers:

[0089] Step 2.4: Calculate the density weight based on the weight samples of the fully connected layer. The formula is:

[0090]

[0091] Among them, d j is the weight sample w j The density weight of reflects its importance in clustering. M is the total number of weight samples in the fully connected layer. When i is within the numerical range of the fully connected layer, j is equal to i.

[0092] Step 2.5: Select the weight samples corresponding to the first S largest density weights in the fully connected layer as the initial cluster centers;

[0093] Step 2.6: Based on the results obtained in step 2.5, calculate the distance from each remaining weight sample in the fully connected layer to each initial cluster center, and assign each weight sample to the nearest cluster center c. m , and finally multiple clusters are obtained, and the distance is calculated as:

[0094]

[0095] in, ∈ represents the noise term;

[0096] Step 2.7, if the intra-class variance of each cluster Then create a new cluster center for the cluster, split the cluster into two sub-clusters, and delete the original cluster center. At this time, S=S+1, and execute step 2.6 again based on the new cluster center. Otherwise, get the final cluster center of the fully connected layer, where is the variance of the mth cluster, and α is the variance scaling factor.

[0097] 9. The hybrid clustering optimization system for memristor array weight mapping according to claim 8, wherein the specific implementation steps of the correction module are:

[0098] Step 3.1: Model the drift statistics, i.e. test the memristor array and record the ath conductance state G a The actual drift △g a , fitting the quadratic model, the formula is:

[0099] △g a =η·(G a -G mid ) 2 +∈

[0100] Among them, △g a The conductivity state G aThe actual drift, η is the drift coefficient, through experimental calibration, G mid is the midpoint of the conductance interval, ∈ is the noise term, which obeys a Gaussian distribution with a mean of zero;

[0101] Step 3.2: All cluster centers c output in step 2 b Mapping to the nearest conductance state G a , then calculate the reverse compensation, the formula is:

[0102]

[0103] in, is the target conductivity value after compensation;

[0104] Step 3.3: Then, based on the compensated target conductance value, the cluster center is corrected, that is, the cluster center is adjusted to approach the compensated conductance value. The formula is:

[0105]

[0106] Among them, c b is the b-th cluster center output in step 2, is the target conductivity value after compensation, β is the compensation intensity coefficient, which is used to control the correction amplitude. is the modified cluster center, G b is the bth effective conductance state in the effective conductance state set.

[0107] Furthermore, the specific implementation steps of the deployment module are:

[0108] Step 4.1: The corrected cluster center Map to the target conductivity state according to the minimum distance, specifically:

[0109]

[0110] Among them, G b is the optimized conductivity state set finally assigned to cluster center a, T(w i ) is the weight sample w i The target conductance state to be mapped, G a is the ath effective conductance state of the memristor, argmin represents the input when it returns to the minimum value;

[0111] Step 4.2: If multiple weight samples w i Mapped to the same target conductance state, according to T(w i ) prioritizes retaining the original distribution of highly sensitive weights, and finally generates the final discretization mapping table T;

[0112] Step 4.3: Generate pulse parameters according to the discretization mapping table T and write them into the memristor array, where the pulse parameters include amplitude and width;

[0113] Step 4.4, forward reasoning verification, that is, after the deployment of step 4.3, the accuracy loss of the test set memristor array is tested. If the loss is greater than the given threshold, the parameters λ or α in steps 1.6 and 2.7 are adjusted and the process returns to step 1. Otherwise, the final mapping result is obtained.

[0114] Compared with the prior art, the present invention has the following beneficial effects:

[0115] The present invention achieves high-precision discretization mapping of neural network weights to memristor conductance states by integrating probabilistic modeling with hardware characteristic compensation. That is, through algorithm-hardware co-design and cross-layer dynamic optimization, it overcomes core problems such as poor distribution adaptability and hardware drift error accumulation in the discretization mapping of neural network weights. The core of this method lies in the construction of a full-chain optimization framework of probabilistic modeling-hierarchical clustering-drift compensation. While ensuring low precision loss (<2%), this method significantly improves the hardware adaptability of weight-conductance mapping. This is reflected in the deep coupling of statistical models and physical constraints, the differentiated design of cross-layer clustering strategies, and the feedforward suppression mechanism of hardware non-idealities. It not only promotes the leap of neuromorphic computing from theoretical models to industrial implementation, but also opens up a new path for low-power deployment of edge intelligent devices, which is specifically reflected in:

[0116] 1. Conductance-weight probability joint modeling: The EM algorithm is used to establish a probabilistic mapping relationship between the weight distribution and the conductance state of the memristor. The matching degree between the cluster center and the conductance value range is dynamically adjusted by maximizing the likelihood function.

[0117] 2. Cross-layer hybrid clustering mechanism: To address the spatial sensitivity of convolutional layers, we use gradient-weighted GMM clustering to preserve the distribution details of highly sensitive weights. For fully connected layers, we design a density-adaptive weighted K-means algorithm that dynamically adjusts the cluster radius based on weight amplitude to address center shifts caused by amplitude differences.

[0118] 3. Resistance drift pre-compensation: A resistance offset compensation factor is introduced during the cluster center calculation stage. The cluster center target value is reversely corrected based on the statistical model of memristor conductance drift to achieve error feedforward compensation under hardware non-ideality. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 This is an overall flow chart of a memristor weight mapping method based on hybrid clustering optimization for a memristor array in an embodiment of the present invention;

[0120] Figure 2 Schematic diagram of a memristor crossbar array deployed with weight mapping in an embodiment of the present invention, wherein WL0…WLM The input data is converted into current through the DAC digital-to-analog converter, and the weight is mapped into the memristor conductance g 00 …g MN , and finally the output voltage is converted into data output BL0…BL through the analog-to-digital converter ADC N , SL0…SL N The source level is grounded to prevent leakage. DETAILED DESCRIPTION

[0121] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0122] In order to solve the problems of parameter sharing and error accumulation in the discretization mapping of memristor array weights.

[0123] The present invention provides a hybrid clustering optimization method for memristor array weight mapping. In order to solve the parameter sharing problem and the non-uniform quantization error accumulation problem caused by the discretization of resistance in memristor weight mapping, a three-level collaborative optimization mechanism is proposed: first, a conductance-weight joint probability model is established, and the mapping relationship between weight distribution and memristor conductance state is fitted by the EM algorithm; secondly, a hierarchical clustering framework is designed, and gradient sensitivity weighted GMM clustering is adopted for the Gaussian mixture characteristics of convolutional layer weights, and an improved weighted K-means algorithm is adopted for the sparse distribution characteristics of the fully connected layer; finally, a resistance offset compensation factor is introduced to dynamically adjust the cluster center to offset the interference of hardware non-ideal characteristics. The present invention provides a high-precision and strong robust weight encoding method for memristor array weight mapping, significantly improves the deployment efficiency of neural networks on edge devices, and promotes artificial intelligence hardware acceleration technology to industrial-grade reliability and practicality. The specific steps are:

[0124] A hybrid clustering optimization method for memristor array weight mapping includes the following steps:

[0125] Step 1: Conductance-weight joint probability modeling is performed based on the weight of the target neural network and the number of discrete levels of conductance states of the initialized target memristor array;

[0126] The specific steps are:

[0127] Step 1.1: Data preprocessing and parameter initialization, that is, artificially extracting the weight matrix W∈R of each layer of the target neural network model m×n , and mark the layer type. At the same time, initialize the target, the number of discrete levels of the conductance state of the memristor array. Among them, the marked layer type includes the marked convolution layer Conv and the fully connected layer FC, R represents the real matrix space, m represents the output dimension, and n is the input dimension;

[0128] Step 1.2: Normalization, that is, linearly normalizing the weight values ​​in the weight matrix to the dynamic range of the memristor conductance [gmin , g max ], the normalized formula is:

[0129]

[0130] Among them, w norm is the normalized weight matrix, w max and w min are the maximum and minimum values ​​of the weight values ​​in the weight matrix respectively;

[0131] Step 1.3, initialize the parameters of the Gaussian mixture model GMM, that is, set the number of Gaussian components of the Gaussian mixture model K = [N·0.8] based on the number of discrete levels of the conductivity state, where N is the number of discrete levels of the conductivity state, and the initial mean value of each Gaussian component μ k To distribute the conductivity intervals equally, the covariance ∑ k Set to 1 / 3 of the distance between adjacent conductances, the mixing coefficient π k =1 / K, K represents the number of Gaussian components, that is, the number of cluster centers;

[0132] Step 1.4: EM algorithm iteratively optimizes the initialized Gaussian mixture model GMM, that is, calculates the i-th weight sample w i The posterior probability of belonging to the kth Gaussian component is:

[0133]

[0134] Among them, N(w|μ,∑) is the Gaussian distribution function, γ ik is the weight sample w i The posterior probability of belonging to the kth Gaussian component, π k and π j are the mixing coefficients of the kth and jth Gaussian components, w i is the i-th weight sample, that is, w norm The normalized weight value of the ith element in μ k and ∑ k is the mean and covariance of the kth Gaussian component;

[0135] Step 1.5: Update parameters based on the posterior probability. The formula is:

[0136]

[0137] in, and represents the updated mean, covariance and mixing coefficient of the k-th Gaussian component, and M represents the total number of weight samples in the current layer;

[0138] Step 1.6: When the likelihood function change rate is less than the predetermined value, stop the iteration and obtain the final posterior probability. Otherwise, the updated probability obtained in step 1.5 is and As μ k ,∑ k and π k Repeat step 1.4, where the likelihood function is M represents the total number of weight samples in the current layer, K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian component, represents the variance of the kth Gaussian component, Represents the Gaussian probability density function, and the rate of change of the likelihood function is the difference between the results of the two likelihood functions divided by the previous one.

[0139] Step 2: Perform hierarchical clustering based on the results obtained in step 1 to obtain the corresponding cluster centers;

[0140] The specific steps are:

[0141] Convolutional layer clustering:

[0142] Step 2.1: For the sensitivity weighted GMM clustering method of the convolutional layer, first obtain the weight gradient of the cth weight sample of the convolutional layer through back propagation Calculating sensitivity And perform normalization processing to obtain normalized sensitivity. The formula for normalization processing is:

[0143]

[0144] Among them, s norm represents the normalized sensitivity to the cth weight sample, s max and s min Represent the minimum and maximum values ​​of the weight sensitivity of the current layer respectively;

[0145] Step 2.2: Correct the posterior probability. This is to obtain the weighted posterior probability of the convolutional layer based on the posterior probability of the Gaussian mixture model GMM and the normalized sensitivity of the convolutional layer, and correct the original posterior probability. The formula is:

[0146]

[0147] in, is the weighted posterior probability of the convolutional layer. When i is within the numerical range of the convolutional layer, c is equivalent to i:

[0148] Step 2.3: Calculate the divergence between the two Gaussian components based on the corrected posterior probability. If the divergence is less than 0.1, merge the Gaussian components and update K = K-1 as the cluster center of the convolutional layer. Otherwise, do not merge.

[0149] Density-adaptive clustering of fully connected layers:

[0150] Step 2.4: Calculate the density weight based on the weight samples of the fully connected layer. The formula is:

[0151]

[0152] Among them, d j is the weight sample w j The density weight of reflects its importance in clustering. M is the total number of weight samples in the fully connected layer. When i is within the numerical range of the fully connected layer, j is equal to i.

[0153] Step 2.5: Select the weight samples corresponding to the first S largest density weights in the fully connected layer as the initial cluster centers;

[0154] Step 2.6: Based on the results obtained in step 2.5, calculate the distance from each remaining weight sample in the fully connected layer to each initial cluster center, and assign each weight sample to the nearest cluster center c. m , and finally multiple clusters are obtained, and the distance is calculated as:

[0155]

[0156] in, ∈ represents the noise term;

[0157] Step 2.7, if the intra-class variance of each cluster Then create a new cluster center for the cluster, split the cluster into two sub-clusters, and delete the original cluster center. At this time, S=S+1, and execute step 2.6 again based on the new cluster center. Otherwise, get the final cluster center of the fully connected layer, where is the variance of the mth cluster, and α is the variance scaling factor (default value 0.3).

[0158] Step 3: Perform resistance drift pre-compensation based on all cluster centers to obtain corrected cluster centers;

[0159] The specific steps are:

[0160] Step 3.1: Model the drift statistics, i.e. test the memristor array and record the ath conductance state G a The actual drift △g a , fitting the quadratic model, the formula is:

[0161] △g a =η·(G a -G mid ) 2 +∈

[0162] Among them, △g a The conductivity state G a The actual drift, η is the drift coefficient, through experimental calibration, G mid is the midpoint of the conductance interval, ∈ is the noise term, which obeys a Gaussian distribution with a mean of zero;

[0163] Step 3.2: All cluster centers c output in step 2 b Mapping to the nearest conductance state G a , then calculate the reverse compensation, the formula is:

[0164]

[0165] in, is the target conductivity value after compensation;

[0166] Step 3.3: Then, based on the compensated target conductance value, the cluster center is corrected, that is, the cluster center is adjusted to approach the compensated conductance value. The formula is:

[0167]

[0168] Among them, c b is the b-th cluster center output in step 2, is the target conductivity value after compensation, β is the compensation intensity coefficient (default β=0.8), which is used to control the correction amplitude. is the modified cluster center, G b is the bth effective conductance state in the effective conductance state set.

[0169] Step 4: Deploy weight-conductance mapping based on the corrected cluster centers.

[0170] The specific steps are:

[0171] Step 4.1: The corrected cluster center Map to the target conductivity state according to the minimum distance, specifically:

[0172]

[0173] Among them, G b is the optimized conductivity state set finally assigned to cluster center a, T(w i ) is the weight sample w i The target conductance state to be mapped, G a is the ath effective conductance state of the memristor, argmin represents the input when it returns to the minimum value;

[0174] Step 4.2: If multiple weight samples w iMapped to the same target conductance state, according to T(w i ) prioritizes retaining the original distribution of highly sensitive weights, and finally generates the final discretization mapping table T;

[0175] Step 4.3: Generate pulse parameters according to the discretization mapping table T and write them into the memristor array, where the pulse parameters include amplitude and width;

[0176] Step 4.4, forward reasoning verification, that is, after the deployment of step 4.3, the accuracy loss of the test set memristor array (2%) is tested. If the loss is greater than the given threshold, adjust the parameters λ or α in steps 1.6 and 2.7 and return to step 1. Otherwise, the final mapping result is obtained.

[0177] like Figure 2 As shown, this is a schematic diagram of the memristor cross array deployed for weight mapping in this embodiment; the present invention systematically achieves high precision and hardware robustness of weight mapping through a hierarchical and progressive optimization process (i.e., a three-level collaborative optimization mechanism): First, based on the probabilistic joint modeling of conductivity and weight, the statistical distribution at the algorithm level and the conductivity constraint at the physical level are deeply integrated, solving the mapping mismatch problem caused by the decoupling of software and hardware parameters in traditional methods; Second, in view of the characteristic differences between the convolutional layer and the fully connected layer, a hybrid clustering strategy of gradient sensitivity weighted GMM and density adaptive K-means is designed, which significantly improves the retention rate of key weights and resource allocation efficiency; Finally, through the drift model-driven clustering center pre-compensation mechanism, the hardware non-ideal error is suppressed in the algorithm optimization stage instead of relying on a posteriori calibration, providing high-precision and robust weight encoding for the memristor array weight mapping, significantly improving the deployment efficiency of neural networks on edge devices, thereby realizing feedforward blocking of the error chain and laying a key technical foundation for low-power AI acceleration of edge computing and smart terminals.

[0178] The above are only representative embodiments of the present invention in many specific application scopes and do not constitute any limitation on the protection scope of the present invention. Any technical solutions formed by transformation or equivalent replacement fall within the scope of protection of the present invention.

Claims

1. A hybrid clustering optimization method for memristor array weight mapping, characterized in that: The steps include: Step 1: Obtain the posterior probability based on the weight of the target neural network and the number of discrete levels of the conductivity state of the initialized target memristor array; Step 2: Perform hierarchical clustering based on the results obtained in step 1 to obtain the corresponding cluster centers; Step 3: Perform resistance drift pre-compensation based on all cluster centers to obtain corrected cluster centers; Step 4: Deploy weight-conductance mapping based on the corrected cluster centers.

2. The hybrid clustering optimization method for memristor array weight mapping according to claim 1, characterized in that: The specific steps of step 1 are: Step 1.1: Data preprocessing and parameter initialization, that is, artificially extracting the weight matrix W∈R of each layer of the target neural network model m×n , and mark the layer type. At the same time, initialize the number of discrete levels of the conductance state of the target memristor array. The marked layer type includes the marked convolution layer Conv and the fully connected layer FC, R represents the real matrix space, m represents the output dimension, and n is the input dimension; Step 1.2: Normalization, that is, linearly normalizing the weight values ​​in the weight matrix to the dynamic range of the memristor conductance [g min , g max ], the normalized formula is: Among them, w norm is the normalized weight matrix, w max and w min are the maximum and minimum values ​​of the weight values ​​in the weight matrix respectively; Step 1.3, initialize the parameters of the Gaussian mixture model GMM, that is, set the number of Gaussian components of the Gaussian mixture model based on the number of discrete levels of the conductivity state, and the initial mean value of each Gaussian component μ k To distribute the conductivity intervals equally, the covariance ∑ k Set to 1 / 3 of the distance between adjacent conductances, the mixing coefficient π k =1 / K, K represents the number of Gaussian components, that is, the number of cluster centers; Step 1.4: EM algorithm iteratively optimizes the initialized Gaussian mixture model GMM, that is, calculates the i-th weight sample w i The posterior probability of belonging to the kth Gaussian component is: Among them, N(w|μ,∑) is the Gaussian distribution function, γ ik is the weight sample w i The posterior probability of belonging to the kth Gaussian component, π k and π j are the mixing coefficients of the kth and jth Gaussian components, w i is the i-th weight sample, that is, w norm The normalized weight value of the ith element in μ k and ∑ k is the mean and covariance of the kth Gaussian component; Step 1.5: Update parameters based on the posterior probability. The formula is: in, and represents the updated mean, covariance and mixing coefficient of the k-th Gaussian component, and M represents the total number of weight samples in the current layer; Step 1.6: When the likelihood function change rate is less than the predetermined value, stop the iteration and obtain the final posterior probability. Otherwise, the updated probability obtained in step 1.5 is and As μ k ,∑ k and π k Repeat step 1.4, where the likelihood function is M represents the total number of weight samples in the current layer, K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian component, represents the variance of the kth Gaussian component, Represents the Gaussian probability density function, and the rate of change of the likelihood function is the difference between the results of the two likelihood functions divided by the previous one.

3. The hybrid clustering optimization method for memristor array weight mapping according to claim 2, characterized in that: The specific steps of step 2 are: Convolutional layer clustering: Step 2.1: For the sensitivity weighted GMM clustering method of the convolutional layer, first obtain the weight gradient of the cth weight sample of the convolutional layer through back propagation Calculating sensitivity And perform normalization processing to obtain normalized sensitivity. The formula for normalization processing is: Among them, s norm represents the normalized sensitivity to the cth weight sample, s max and s min Represent the minimum and maximum values ​​of the weight sensitivity of the current layer respectively; Step 2.2: Correct the posterior probability. This is to obtain the weighted posterior probability of the convolutional layer based on the posterior probability of the Gaussian mixture model GMM and the normalized sensitivity of the convolutional layer, and correct the original posterior probability. The formula is: in, is the weighted posterior probability of the convolutional layer. When i is within the numerical range of the convolutional layer, c is equal to i. Step 2.3: Calculate the divergence between the two Gaussian components based on the corrected posterior probability. If the divergence is less than 0.1, merge the Gaussian components and update K = K-1 as the cluster center of the convolutional layer. Otherwise, do not merge. Density-adaptive clustering of fully connected layers: Step 2.4: Calculate the density weight based on the weight samples of the fully connected layer. The formula is: Among them, d j is the weight sample w j The density weight of reflects its importance in clustering. M is the total number of weight samples in the fully connected layer. When i is within the numerical range of the fully connected layer, j is equal to i. Step 2.5: Select the weight samples corresponding to the first S largest density weights in the fully connected layer as the initial cluster centers; Step 2.6: Based on the results obtained in step 2.5, calculate the distance from each remaining weight sample in the fully connected layer to each initial cluster center, and assign each weight sample to the nearest cluster center c. m , and finally multiple clusters are obtained, and the distance is calculated as: in, ∈ represents the noise term; Step 2.7, if the intra-class variance of each cluster Then create a new cluster center for the cluster, split the cluster into two sub-clusters, and delete the original cluster center. At this time, S=S+1, and execute step 2.6 again based on the new cluster center. Otherwise, get the final cluster center of the fully connected layer, where is the variance of the mth cluster, and α is the variance scaling factor.

4. The hybrid clustering optimization method for memristor array weight mapping according to claim 3, characterized in that: The specific steps of step 3 are: Step 3.1: Model the drift statistics, i.e. test the memristor array and record the ath conductance state G a The actual drift Δg a , fitting the quadratic model, the formula is: Δg a =η·(G a -G mid ) 2 +∈ Among them, △g a The conductivity state G a The actual drift, η is the drift coefficient, through experimental calibration, G mid is the midpoint of the conductance interval, ∈ is the noise term, which obeys a Gaussian distribution with a mean of zero; Step 3.2: All cluster centers c output in step 2 b Mapping to the nearest conductance state G a , then calculate the reverse compensation, the formula is: in, is the target conductivity value after compensation; Step 3.3: Then, based on the compensated target conductance value, the cluster center is corrected, that is, the cluster center is adjusted to approach the compensated conductance value. The formula is: Among them, c b is the b-th cluster center output in step 2, is the target conductivity value after compensation, β is the compensation intensity coefficient, which is used to control the correction amplitude. is the modified cluster center, G b is the bth effective conductance state in the effective conductance state set.

5. The hybrid clustering optimization method for memristor array weight mapping according to claim 4, characterized in that: The specific steps of step 4 are: Step 4.1: The corrected cluster center Map to the target conductivity state according to the minimum distance, specifically: Among them, G b is the optimized conductivity state set finally assigned to cluster center a, T(w i ) is the weight sample w i The target conductance state to be mapped, G a is the ath effective conductance state of the memristor, argmin represents the input when it returns to the minimum value; Step 4.2: If multiple weight samples w i Mapped to the same target conductance state, according to T(w i ) prioritizes retaining the original distribution of highly sensitive weights, and finally generates the final discretization mapping table T; Step 4.3: Generate pulse parameters according to the discretization mapping table T and write them into the memristor array, where the pulse parameters include amplitude and width; Step 4.4, forward reasoning verification, that is, after the deployment of step 4.3, the accuracy loss of the test set memristor array is tested. If the loss is greater than the given threshold, the parameters λ or α in steps 1.6 and 2.7 are adjusted and the process returns to step 1. Otherwise, the final mapping result is obtained.

6. A hybrid clustering optimization system for memristor array weight mapping, characterized in that: include: Posterior probability acquisition module: obtains the posterior probability based on the weights of the target neural network and the number of discrete levels of the conductivity state of the initialized target memristor array; Hierarchical clustering module: Perform hierarchical clustering based on the results obtained in step 1 to obtain the corresponding cluster centers; Correction module: pre-compensate resistance drift based on all cluster centers to obtain the corrected cluster centers; Deployment module: Deploy weight-conductance mapping based on the corrected cluster centers.

7. The hybrid clustering optimization system for memristor array weight mapping according to claim 6, characterized in that: The specific implementation steps of the posterior probability acquisition module are: Step 1.1: Data preprocessing and parameter initialization, that is, artificially extracting the weight matrix W∈R of each layer of the target neural network model m×n , and mark the layer type. At the same time, initialize the number of discrete levels of the conductance state of the target memristor array. The marked layer type includes the marked convolution layer Conv and the fully connected layer FC, R represents the real matrix space, m represents the output dimension, and n is the input dimension; Step 1.2: Normalization, that is, linearly normalizing the weight values ​​in the weight matrix to the dynamic range of the memristor conductance [g min , g max ], the normalized formula is: Among them, w norm is the normalized weight matrix, w max and w min are the maximum and minimum values ​​of the weight values ​​in the weight matrix respectively; Step 1.3, initialize the parameters of the Gaussian mixture model GMM, that is, set the number of Gaussian components of the Gaussian mixture model based on the number of discrete levels of the conductivity state, and the initial mean value of each Gaussian component μ k To distribute the conductivity intervals equally, the covariance ∑ k Set to 1 / 3 of the distance between adjacent conductances, the mixing coefficient π k =1 / K, K represents the number of Gaussian components, that is, the number of cluster centers; Step 1.4: EM algorithm iteratively optimizes the initialized Gaussian mixture model GMM, that is, calculates the i-th weight sample w i The posterior probability of belonging to the kth Gaussian component is: Among them, N(w|μ,∑) is the Gaussian distribution function, γ ik is the weight sample w i The posterior probability of belonging to the kth Gaussian component, π k and π j are the mixing coefficients of the kth and jth Gaussian components, w i is the i-th weight sample, that is, w norm The normalized weight value of the ith element in μ k and ∑ k is the mean and covariance of the kth Gaussian component; Step 1.5: Update parameters based on the posterior probability. The formula is: in, and represents the updated mean, covariance and mixing coefficient of the k-th Gaussian component, and M represents the total number of weight samples in the current layer; Step 1.6: When the likelihood function change rate is less than the predetermined value, stop the iteration and obtain the final posterior probability. Otherwise, the updated probability obtained in step 1.5 is and As μ k ,∑ k and π k Repeat step 1.4, where the likelihood function is M represents the total number of weight samples in the current layer, K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian component, represents the variance of the kth Gaussian component, Represents the Gaussian probability density function, and the rate of change of the likelihood function is the difference between the results of the two likelihood functions divided by the previous one.

8. The hybrid clustering optimization system for memristor array weight mapping according to claim 7, characterized in that: The specific implementation steps of the hierarchical clustering module are: Convolutional layer clustering: Step 2.1: For the sensitivity weighted GMM clustering method of the convolutional layer, first obtain the weight gradient of the cth weight sample of the convolutional layer through back propagation Calculating sensitivity And perform normalization processing to obtain normalized sensitivity. The formula for normalization processing is: Among them, s norm represents the normalized sensitivity to the cth weight sample, s max and s min Represent the minimum and maximum values ​​of the weight sensitivity of the current layer respectively; Step 2.2: Correct the posterior probability. This is to obtain the weighted posterior probability of the convolutional layer based on the posterior probability of the Gaussian mixture model GMM and the normalized sensitivity of the convolutional layer, and correct the original posterior probability. The formula is: in, is the weighted posterior probability of the convolutional layer. When i is within the numerical range of the convolutional layer, c is equal to i. Step 2.3: Calculate the divergence between the two Gaussian components based on the corrected posterior probability. If the divergence is less than 0.1, merge the Gaussian components and update K = K-1 as the cluster center of the convolutional layer. Otherwise, do not merge. Density-adaptive clustering of fully connected layers: Step 2.4: Calculate the density weight based on the weight samples of the fully connected layer. The formula is: Among them, d j is the density weight of the weighted sample wj, reflecting its importance in the clustering. M is the total number of weighted samples in the fully connected layer. When i is within the numerical range of the fully connected layer, j is equal to i. Step 2.5: Select the weight samples corresponding to the first S largest density weights in the fully connected layer as the initial cluster centers; Step 2.6: Based on the results obtained in step 2.5, calculate the distance from each remaining weight sample in the fully connected layer to each initial cluster center, and assign each weight sample to the nearest cluster center c. m , and finally multiple clusters are obtained, and the distance is calculated as: in, ∈ represents the noise term; Step 2.7, if the intra-class variance of each cluster Then create a new cluster center for the cluster, split the cluster into two sub-clusters, and delete the original cluster center. At this time, S=S+1, and execute step 2.6 again based on the new cluster center. Otherwise, get the final cluster center of the fully connected layer, where is the variance of the mth cluster, and α is the variance scaling factor.

9. The hybrid clustering optimization system for memristor array weight mapping according to claim 8, characterized in that: The specific implementation steps of the correction module are: Step 3.1: Model the drift statistics, i.e. test the memristor array and record the ath conductance state G a The actual drift △g a , fitting the quadratic model, the formula is: △g a =η·(G a -G mid ) 2 +∈ Among them, △g a The conductivity state G a The actual drift, η is the drift coefficient, through experimental calibration, G mid is the midpoint of the conductance interval, ∈ is the noise term, which obeys a Gaussian distribution with a mean of zero; Step 3.2: All cluster centers c output in step 2 b Mapping to the nearest conductance state G a , then calculate the reverse compensation, the formula is: in, is the target conductivity value after compensation; Step 3.3: Then, based on the compensated target conductance value, the cluster center is corrected, that is, the cluster center is adjusted to approach the compensated conductance value. The formula is: Among them, c b is the b-th cluster center output in step 2, is the target conductivity value after compensation, β is the compensation intensity coefficient, which is used to control the correction amplitude. is the modified cluster center, G b is the bth effective conductance state in the effective conductance state set.

10. The hybrid clustering optimization system for memristor array weight mapping according to claim 9, characterized in that: The specific implementation steps of the deployment module are: Step 4.1: The corrected cluster center Map to the target conductivity state according to the minimum distance, specifically: Among them, G b is the optimized conductivity state set finally assigned to cluster center a, T(w i ) is the weight sample w i The target conductance state to be mapped, G a is the ath effective conductance state of the memristor, argmin represents the input when it returns to the minimum value; Step 4.2: If multiple weight samples w i Mapped to the same target conductance state, according to T(w i ) prioritizes retaining the original distribution of highly sensitive weights, and finally generates the final discretization mapping table T; Step 4.3: Generate pulse parameters according to the discretization mapping table T and write them into the memristor array, where the pulse parameters include amplitude and width; Step 4.4, forward reasoning verification, that is, after the deployment of step 4.3, the accuracy loss of the test set memristor array is tested. If the loss is greater than the given threshold, the parameters λ or α in steps 1.6 and 2.7 are adjusted and the process returns to step 1. Otherwise, the final mapping result is obtained.

Images