Voice noise reduction method and device based on storage and calculation integrated chip, medium and equipment

Abstract

The invention discloses a voice noise reduction method and device based on a storage and calculation integrated chip, a medium and equipment, and the method comprises the steps: extracting weight data corresponding to a preset level of a voice noise reduction model, and obtaining a first weight matrix; performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix; performing weight quantization on the second weight matrix to obtain a third weight matrix; adding an additional weight in the third weight matrix to obtain a target weight matrix; and according to the elements of the target weight matrix, adjusting the conductance value of a memristor in a pre-constructed storage and calculation integrated chip so as to embed the voice noise reduction model into the storage and calculation integrated chip to obtain a voice noise reduction chip. According to the invention, the voice noise reduction model is integrated on the storage and calculation integrated chip for noise reduction, the difficulty of end-side deployment of the voice noise reduction model is reduced, and the efficiency of voice noise reduction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech noise reduction processing, and in particular to a speech noise reduction method, device, medium and equipment based on a storage and computing integrated chip. Background Art

[0002] On some end-side devices (such as smart watches), it is difficult to integrate speech noise reduction models due to physical conditions such as size, available computing resources, and power consumption. Usually, a cloud model needs to be deployed online for noise reduction, resulting in low efficiency of speech noise reduction and difficulty in end-side deployment. Summary of the invention

[0003] In view of the above problems, the present invention is proposed to provide a speech noise reduction method, device, medium and equipment based on a storage and computing integrated chip that overcomes the above problems or at least partially solves the above problems.

[0004] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0005] According to a first aspect of an embodiment of the present invention, a method for voice noise reduction based on a storage-computing integrated chip is provided, and the method for voice noise reduction based on a storage-computing integrated chip includes:

[0006] Extracting weight data corresponding to a preset level of the speech noise reduction model to obtain a first weight matrix;

[0007] Performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix;

[0008] Based on a preset chip data bit width, weight quantizing the second weight matrix to obtain a third weight matrix;

[0009] Adding additional weights to the third weight matrix to obtain a target weight matrix, wherein the sum of the weights of the elements in each column of the target weight matrix is ​​the same;

[0010] According to the elements of the target weight matrix, the conductance value of the memristor in the pre-built storage and computing integrated chip is adjusted to embed the speech noise reduction model into the storage and computing integrated chip to obtain the speech noise reduction chip;

[0011] When the speech to be de-noised is received, the speech de-noising chip is used to de-noise the speech to be de-noised to obtain de-noised speech.

[0012] According to a second aspect of an embodiment of the present invention, a speech noise reduction device based on a storage and computing integrated chip is provided, the device comprising:

[0013] A parsing module, used to extract weight data corresponding to a preset level of a speech noise reduction model to obtain a first weight matrix;

[0014] A first processing module, configured to perform positive and negative weight division processing on the first weight matrix to obtain a second weight matrix;

[0015] A second processing module, configured to perform weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix;

[0016] A third processing module is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same;

[0017] A model embedding module, used for adjusting the conductance value of the memristor in the pre-built storage and computing integrated chip according to the elements of the target weight matrix, so as to embed the speech noise reduction model into the storage and computing integrated chip to obtain the speech noise reduction chip;

[0018] The noise reduction processing module is used to reduce the noise of the speech to be reduced by using the speech noise reduction chip when receiving the speech to be reduced, so as to obtain the reduced noise speech.

[0019] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which computer program instructions are stored. The computer program instructions are loaded and executed by a processor to implement the operations performed by any of the methods described above.

[0020] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements instructions of any of the methods described above.

[0021] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0022] The embodiment of the present invention provides a method, device, medium and equipment for voice noise reduction based on a storage and computing integrated chip. Compared with the prior art, the method for voice noise reduction based on a storage and computing integrated chip described in the embodiment of the present invention is

[0023] The first weight matrix is ​​divided into positive and negative weights to obtain a second weight matrix. Based on the preset chip data bit width, the second weight matrix is ​​weight quantized to obtain a third weight matrix. Additional weights are added to the third weight matrix to obtain a target weight matrix, and the weights of each column element in the target weight matrix are the same. According to the elements of the target weight matrix, the conductance value of the memristor in the pre-built storage and computing integrated chip is adjusted to embed the speech noise reduction model into the storage and computing integrated chip to obtain a speech noise reduction chip. When the speech to be noise-reduced is received, the speech noise reduction chip is used to reduce the noise of the speech to be noise-reduced to obtain the noise-reduced speech. The problem that the conductance value of the memristor cannot represent negative weights and the calculation result of the memristor array matrix realized by the voltage mode is inconsistent with the actual matrix multiplication calculation result is solved, so that the storage and computing integrated chip can realize speech noise reduction, and there is no need to deploy a cloud language noise reduction model for noise reduction, which improves the speed of speech noise reduction. At the same time, the speech noise reduction model integrated in the storage and computing integrated chip can be more conveniently deployed on the end side, which reduces the threshold for the end-side device to deploy the speech noise reduction model, and has a high promotion prospect.

[0024] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flow chart of a method for voice noise reduction based on a storage-computing integrated chip provided in an embodiment of the present invention;

[0027] Figure 2 A structural reference diagram of a memristor array;

[0028] Figure 3 This is a schematic diagram for reference of the structure of the memristor array;

[0029] Figure 4 A schematic diagram of the flow principle structure of a speech noise reduction device based on a storage and computing integrated chip provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0031] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0032] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intervening layer / element therebetween. Additionally, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0033] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with specific implementation methods. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical scheme of the present invention, rather than limitations on the technical scheme of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0034] Figure 1 is a flow chart of a method for voice noise reduction based on a storage and computing integrated chip provided by an embodiment of the present invention, such as Figure 1 As shown, the voice noise reduction method based on the storage and computing integrated chip includes the following steps:

[0035] S1. Extract weight data corresponding to a preset level of a speech noise reduction model to obtain a first weight matrix;

[0036] The embodiment of the present invention analyzes the trained speech denoising model, extracts weight data corresponding to a preset level of the speech denoising model, and obtains a first weight matrix.

[0037] In an embodiment of the present invention, the speech noise reduction model is, for example, an end-to-end speech noise reduction network using deep learning, which uses convolutional neural networks to construct complex acoustic features. At the same time, traditional statistical noise reduction methods, such as spectral subtraction, can also be integrated to give full play to the respective advantages of neural networks and classical algorithms. It needs to be explained that in order to accelerate the speech noise reduction model using the storage and computing integrated chip in the embodiment of the present invention, the speech noise reduction model needs to be embedded in the storage and computing integrated chip. Therefore, it is necessary to extract the weight data of the level (i.e., the preset level) that the storage and computing integrated chip can accelerate. The preset level includes but is not limited to: convolution layer, fully connected layer.

[0038] The training process of the speech denoising model can adopt a conventional model training method, for example, using the feature of the audio signal extracted after the audio data preprocessing - the Mel spectrum of the audio as the input feature of the speech denoising model. Correspondingly, the output of the speech denoising model is the Mel spectrum of the clean human voice audio. The goal of the speech denoising model is to output a result that is as close as possible to the clean human voice Mel spectrum from the input noisy audio Mel spectrum. In the training process of the speech denoising model, MSE (mean square error) or L1 loss function can be used as the optimization target to calculate the difference between the Mel spectrum output by the model and the corresponding clean human voice Mel spectrum. And the Adam algorithm optimization algorithm is used to iteratively update the model parameters. At the same time, the model performance can be monitored on the validation set, and strategies such as Early Stopping can be used to prevent overfitting. Finally, the speech denoising effect of the speech denoising model is evaluated on the validation set; and according to the validation results, the hyperparameters are adjusted to further optimize the denoising performance of the speech denoising model.

[0039] The embodiment of the present invention uses a training model framework such as Pytorch or Tensorflow to load the trained speech denoising model, extracts weight data corresponding to a preset level of the speech denoising model, and obtains a first weight matrix.

[0040] Specifically, in an embodiment of the present invention, when the preset level is a convolutional layer, the weight data corresponding to the convolutional layer is the convolution kernel of the convolutional layer, and the convolution kernel data of the convolutional layer is extracted as the first weight matrix; when the preset level is a fully connected layer, the weight data corresponding to the fully connected layer is the weight matrix of the fully connected layer.

[0041] Since different layers in the speech noise reduction model, such as the fully connected layer, may contain a bias vector, or may not contain a bias vector, in order to maintain the accuracy of the weight matrix data, when the fully connected layer corresponding to the first weight matrix contains a bias vector, the first weight matrix needs to be updated, including: obtaining the bias vector of the fully connected layer corresponding to the first weight matrix, the first weight matrix is ​​a matrix of dimension h×k, where h and k are both positive integers; placing the bias vector in the last column of the first weight matrix for splicing to update the first weight matrix to obtain an updated first weight matrix, the updated first weight matrix dimension is h×(k+1), and at this time, the updated first weight matrix is ​​the final first weight matrix.

[0042] For example, if the first weight matrix of a fully connected layer is defined as H and the bias vector is b, then H and b can be concatenated to form a new weight matrix H', and the last column of the new first weight matrix H' is the original bias vector b. Specifically, if the input dimension of a fully connected layer is defined as 4 and the output dimension is 3, the original first weight matrix H is (3×4) in size and the bias vector b is (3×1) in size. After merging, the new first weight matrix H' is (3×5) in size and the last column is the bias vector b.

[0043] S2, performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix;

[0044] In an embodiment of the present invention, in order to map the weights in the first weight matrix to the conductance value of the memristor (the inverse of the resistance of the memristor), and the conductance value of the memristor cannot represent a negative value, the first weight matrix is ​​divided into positive and negative weights to obtain a second weight matrix.

[0045] The embodiment of the present invention performs positive and negative weight division processing on the first weight matrix to obtain a second weight matrix including: determining each column in the first weight matrix as a target weight column; replacing all negative numbers in the target weight column with zero to obtain a positive weight column corresponding to the target weight column; replacing all positive numbers in the target weight column with zero and all negative numbers with their absolute values ​​to obtain a negative weight column corresponding to the target weight column; combining the positive weight column and the negative weight column corresponding to the target weight column to obtain a positive and negative weight column pair corresponding to the target weight column; replacing each column in the first weight matrix with the positive and negative weight column pair corresponding to the column to obtain a second weight matrix.

[0046] In the first weight matrix, all the elements contained have different numerical attributes, and each element is a weight value, which can be positive or negative, that is, the element may be positive or negative. Since the memristor in the storage and computing integrated chip cannot directly represent negative elements, based on this, negative numbers are represented by the difference between zero and positive numbers, and all negative numbers in the target weight column are replaced by zero to obtain the positive weight column corresponding to the target weight column, and all positive numbers in the target weight column are replaced by zero and all negative numbers are replaced by their absolute values ​​to obtain the negative weight column corresponding to the target weight column. The positive weight column and negative weight column corresponding to the same target weight column are taken as a group to obtain the positive and negative weight column pairs corresponding to the target weight column. The target weight column corresponding to the positive and negative weight column pair can be represented by subtracting the negative weight column from the positive weight column in each positive and negative weight column pair, which solves the problem that the negative weight memristor in the target weight column cannot be represented.

[0047] For example: The first weight matrix is Then the first column Replace all negative numbers in with 0 to get the first column of positive weights The first column All positive numbers are replaced with 0, and all negative numbers are replaced with their absolute values ​​to obtain the negative weight column of the first column. The first column can be obtained by the difference between the positive weight column and the negative weight column in the first column. The positive and negative weight column pair of the first column is Similarly, the positive weight column in the second column Negative weight column for the second column The positive and negative weight pairs in the second column are Replace each column in the first weight matrix with the positive and negative weight pairs of the column to obtain the second weight matrix

[0048] Through the above-mentioned positive and negative weight division, the embodiment of the present invention can make each memristor correspond to an element in the second weight matrix in the memristor array of the subsequent voltage-implemented storage and computing integrated chip, so that the difference in the conductance values ​​of two adjacent columns of memristors can be used to characterize the elements in any target weight column of the first weight matrix in the storage and computing integrated chip, avoiding the problem that the memristor cannot represent negative numbers.

[0049] Optionally, other embodiments of the present invention may also use the following method to perform positive and negative weight division processing on the first weight matrix, and the specific process is as follows:

[0050] If the first weight matrix is ​​an m*n matrix weight, it is necessary to expand the number of columns n of the first weight matrix to 2n, create a new matrix new_weight with a shape of m*2n, initialize the elements in the new matrix new_weight to 0, and simulate the voltage to realize the memristor array in the storage and computing integrated chip;

[0051] Set the even columns of the new matrix new_weight to negative weight columns for storing negative numbers, and the odd columns to positive weight columns for storing positive numbers. Traverse each element of the matrix weight. If the element is greater than or equal to 0, assign the element to the element in the same row of the odd column corresponding to the element in new_weight (for example: the value of the element is 5, and it is in the third column and fifth row of the matrix weight. Then the odd column corresponding to the element in new_weight is the third odd column, that is, the fifth row element of the third odd column in new_weight is assigned to 5); if the element is less than 0, assign the absolute value of the element to the even column corresponding to the element in new_weight (for example: the value of the element is -3, and it is in the third column and fourth row of the matrix weight. Then the even column corresponding to the element in new_weight is the third even column, that is, the fourth row element of the third odd column in new_weight is assigned to 3). Therefore, the adjacent "positive weight column value - negative weight column value" in new_weight can represent the corresponding element in weight.

[0052] For example, if the element in the first row and first column of weight is -4, which is less than 0, its absolute value 4 is assigned to the element in the first row of the first even column of new_weight, that is, the element in the first row of the first even column of new_weight becomes 4, and the element in the first row of the first odd column of new_weight is 0, then the element -4 in the first row and first column of weight can be represented by the element in the first row of the first odd column of new_weight (that is, the value in the first row of the positive weight column) - the element in the first row of the first even column (that is, the value in the first row of the negative weight column) = 0-4 = -4. That is, a negative number can be represented by the difference between two non-negative elements, thereby realizing that the differential conductance of a pair of memristors represents a negative weight, and the differential conductance of two columns of memristors represents a column of elements of the second weight matrix.

[0053] S3. Based on a preset chip data bit width, weight quantize the second weight matrix to obtain a third weight matrix;

[0054] In order to reduce the complexity of subsequent chip calculations, the embodiment of the present invention quantizes the second weight matrix, for example, using linear quantization technology to quantize the weight values ​​of floating-point numbers and convert them from floating-point numbers to integers. The embodiment of the present invention quantizes the third weight matrix, converts the floating-point numbers in the second weight matrix into integers to obtain an integer matrix, namely the third weight matrix.

[0055] The embodiment of the present invention performs weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix, which includes: respectively determining the maximum and minimum values ​​of the elements in the second weight matrix; determining a quantization factor according to the maximum and minimum values ​​of the second weight matrix and the chip data bit width; and using the quantization factor to perform quantization rounding and update on each element in the second weight matrix to obtain a third weight matrix.

[0056] Specifically, the embodiment of the present invention first determines the maximum value data of the elements in the second weight matrix max and minimum data min , and then calculate and determine the quantization factor according to the preset chip data bit width, that is, the pre-built data bit width bits of the main MCU on the storage and computing integrated chip to be embedded in the speech noise reduction model, for example, bits = 8. Specifically, the embodiment of the present invention can be determined by formula Each element in the second weight matrix is ​​multiplied by the quantization factor, and the elements multiplied by the quantization factor are rounded using the round() function, thereby obtaining a quantized integer matrix, namely the third weight matrix.

[0057] S4, adding additional weights to the third weight matrix to obtain a target weight matrix, wherein the sum of the weights of the elements in each column of the target weight matrix is ​​the same;

[0058] Specifically, the embodiment of the present invention adds additional weights to the third weight matrix to obtain a target weight matrix, including: calculating the weight sum of each column element in the third weight matrix (the weight sum of each column element is the sum of all elements in each column), determining the maximum value of the weight sum of each column in the third weight matrix, and obtaining the target weight value; adding a preset number of additional weights to each column of the third weight matrix to obtain the target weight matrix, wherein the weight sum of each column element of the target weight matrix is ​​equal to the target weight value, and a preset number of rows in the target weight matrix are all composed of additional weights, for example: the preset number is 2, then there are two rows in the target weight matrix that are all composed of additional weights.

[0059] In the embodiment of the present invention, the goal is to map the weight matrix to the memristor array to embed the speech noise reduction model into the storage and computing integrated chip. A column in the weight matrix can be equivalent to a column of memristors in the memristor array, and the value of the element in the weight matrix can be equivalent to the conductance value of the memristor. The embodiment of the present invention takes a column of memristor array as an example. The structure of the memristor array is as follows: Figure 2 As shown in the reference, assuming the input is The weight data stored in the first column of the memristor array is (that is, the weight (conductance value) of the first row of memristors is w1, the weight (conductance value) of the second row of memristors is w2, the weight (conductance value) of the third row of memristors is w3, and the weight (conductance value) of the second row of memristors is w4), the voltage of the input line SL is v (the input voltages of each input line from top to bottom are x1, x2, x3, x4 respectively), and the voltage of the negative line BL is Y.

[0060] Then it is expanded to n input voltages, and the input voltage of each input line is x i , the weight (conductance value) of each memristor is w i According to Ohm’s law, the current of each memristor can be calculated as I i =(x i -Y)·w i Since the voltage on the negative line BL may be greater than or less than the voltage on the input line SL, the direction of the current includes two cases: when the voltage on the input line SL is greater than the voltage on the negative line BL, the current flows from the input line to the negative line through the memristor; when the voltage on the input line SL is less than the voltage on the negative line BL, the current flows from the negative line to the input line through the memristor.

[0061] Since the current will only flow in this memristor array network and the current is conserved, I1+I2+I3+I4=0. Expanding it, we get (x1-Y)·w1+…+(x4-Y)·w4=0. Further deduction yields

[0062] For the case with n inputs, the final output is derived as shown in the following formula:

[0063]

[0064] Therefore, the embodiment of the present invention can obtain the output result shown in the above formula by measuring the voltage on the negative line BL, where is the result of the matrix multiplication that actually needs to be calculated.

[0065] By using the weight quantization method, the voltage output on each negative line can be changed to Wherein c is a constant. Therefore, the embodiment of the present invention can obtain a value proportional to the actual calculation result by measuring the voltage on the negative line BL. By multiplying the constant c after each weight matrix multiplication, the actual required calculation result can be restored.

[0066] In the memristor array of the integrated memory and computing chip, the output obtained by measuring the voltage on the negative line BL in the embodiment of the present invention is actually In fact, the result of matrix multiplication is If the result of each column is divided by That is, the output of each column is divided by the sum of the weights of this column, and the sum of the weights in different columns of the memristor array is different. In order to solve the problem that the result obtained by measuring the negative line voltage output is inconsistent with the result of the actual matrix multiplication calculation, the present invention adds an additional weight to make the sum of the weights on each column of memristors the same, which is the maximum value of the sum of the weight values ​​of each column of the weight matrix, that is, the target weight value, even if the sum of the weights of each column is a constant, so that the actual result to be calculated can be obtained by multiplying the output of each column by this constant.

[0067]

[0068] For example, we can introduce two additional inputs with a value of 0 and two additional weights p1 and p2 (which can be understood as ), so that the sum of the weights of each column after adding the extra weights is the target weight value, that is, The output of each column is becomes Even though the output of each column of memristors is the result of dividing the actual matrix multiplication calculation result by the constant summax. In order to restore the actual calculation result, the voltage output can be multiplied by summax; or when the restoration is not performed, an output result proportional to the actual result is obtained. The embodiment of the present invention can eliminate the influence of the proportion through the Softmax layer.

[0069] Specifically, the third weight matrix W is calculated, the maximum value of the sum of weights in each column is summax; a preset number of additional weights p are added to each column i , so that the sum of the weights of each column is summax. For example, assuming that two additional weights p1 and p2 are added, then p1 of each column forms a row vector P1, and p2 of each column forms a row vector P2, then a new weight matrix can be constructed, and the original weights and additional weights in the second weight data are spliced ​​together, that is, the third weight matrix is ​​spliced ​​together with the row vector P1 and the row vector P2 (after splicing, the row vector P1 and the row vector P2 are used as the rows of the new matrix), and the target weight matrix is ​​obtained.

[0070] It should be noted that the additional weight added in the embodiment of the present invention is a non-negative integer.

[0071] S5. According to the elements of the target weight matrix, the conductance value of the memristor in the pre-built storage and computing integrated chip is adjusted to embed the speech noise reduction model into the storage and computing integrated chip to obtain a speech noise reduction chip;

[0072] Based on the foregoing embodiments, the present invention forms a mapping relationship between each element of the target weight matrix and a memristor in the storage-computing integrated chip; specifically, in the embodiments of the present invention, each target weight matrix in the storage-computing integrated chip corresponds to a memristor array, and each element in the target weight matrix corresponds to the memristor in the same row and column position in the memristor array corresponding to the target weight matrix. The output voltage of each odd-numbered memristor column in the memristor array and the even-numbered memristor column adjacent to its right side are differentially output through a differential structure connected to the end of the negative line (BL). The memristor array is as follows: Figure 3 As shown (the first and third columns are odd-numbered memristor columns, the second and fourth columns are even-numbered memristor columns, the first column and the second column are connected via a differential structure at the end of the negative line BL for differential voltage output, and the third column and the fourth column are connected via a differential structure at the end of the negative line BL for differential voltage output).

[0073] Since memristors cannot represent negative numbers, in order to implement multiplication and addition operations involving negative weights through a memristor array, the above-mentioned embodiment represents a column of original weights as two columns of non-negative weights, and then maps these two columns of non-negative weights to the conductance values ​​of two adjacent columns of memristors. The voltage outputs of the two columns of memristors are then subtracted through a voltage differential structure connected at the end of the negative line, and the true result of the original weight multiplication and addition can be obtained, solving the problem that the memristor array cannot perform multiplication and addition operations involving non-negative weights. For example: a memristor array has n input lines, which are x, i , the weight of each column of the memristor array is the same as (denoted as C), and the weight (conductance value) of the memristor in the odd-numbered memristor column is w pi , the weight (conductance value) of the memristor in the even-numbered memristor column is The calculation result (output voltage) of the odd-numbered memristor column is The calculation result (output voltage) for an even-numbered memristor column is

[0074]

[0075] Subtracting the output results of the two columns of memristors gives a differential output voltage of again The final derivation result of the output voltage differential output of each odd-numbered memristor column in the memristor array and its even-numbered memristor column adjacent to the right is:

[0076] Among them, the embodiment of the present invention according to the elements of the target weight matrix, adjusting the conductance value of the memristor in the storage and computing integrated chip includes: traversing each element in the target weight matrix, and performing a non-zero judgment on each traversed element; when the traversed element is a non-zero element, adjusting the conductance value of the memristor corresponding to the element in the storage and computing integrated chip to the value of the element; when the traversed element is a zero element, adjusting the conductance value of the memristor corresponding to the element in the storage and computing integrated chip to a preset high resistance value.

[0077] It needs to be explained that, in the embodiment of the present invention, since the memristor cannot represent 0, when the traversed element is a zero element, the conductance value of the memristor corresponding to the element in the storage and computing integrated chip is adjusted to a preset high resistance value, and the preset high resistance value is a preset inverse of the high resistance. When the resistance of the memristor is set to a preset high resistance, the current contributed by the memristor is very small and can be approximated to zero. Therefore, the preset high resistance value is used to represent the zero element.

[0078] By adjusting the conductance values ​​of the memristors corresponding to the elements of the target weight matrix in the storage-computing integrated chip one by one, each element of the target weight matrix can be mapped to a memristor in the storage-computing integrated chip, so as to embed the speech noise reduction model into the storage-computing integrated chip to obtain the speech noise reduction chip. The pre-built storage-computing integrated chip is a voltage-implemented storage-computing integrated chip (that is, by measuring the negative line voltage of the memristor array instead of the result of the multiplication and addition calculation of each column of memristors of the current). The embedding of the speech noise reduction model into other parts of the storage-computing integrated chip adopts the commonly used technical means in the field, such as using the STM32 burning project to burn the model code data to achieve model embedding, which will not be repeated here.

[0079] S6. When receiving the speech to be denoised, the speech denoising chip is used to denoise the speech to be denoised to obtain denoised speech.

[0080] After obtaining the speech noise reduction chip, when receiving speech to be noise-reduced, the speech noise reduction chip can be used to reduce the noise of the received speech to be noise-reduced to obtain noise-reduced speech, thereby achieving the purpose of performing speech noise reduction on the storage and computing integrated chip.

[0081] In the embodiment of the present invention, the speech to be denoised is pre-processed and converted into a data form recognizable by a speech denoising chip. When the speech to be denoised is denoised using the speech denoising chip, when the memristor array where the additional weights are located in the speech denoising chip has data input, the memristor input voltage corresponding to the additional weights is set to 0V (i.e., the influence of the additional weight multiplication and addition is eliminated. For the detailed process, refer to the example of introducing additional weights in S4). Finally, the output of the speech denoising chip is restored to speech to obtain the denoised speech.

[0082] It should also be noted in the embodiment of the present invention that when the first weight matrix includes a bias vector, the element corresponding to the bias vector in the target weight matrix is ​​determined as the bias element (for example, the fifth column in the first weight matrix is ​​the bias vector, then the elements in the ninth and tenth columns of the target weight matrix that are not additional weights are bias elements), then when the speech noise reduction chip is used to reduce the noise of the speech to be reduced, when the memristor array where the bias element in the speech noise reduction chip is located has data input, the memristor input voltage corresponding to the bias element is set to 1V to ensure the stability of the memristor in calculating the bias vector (when the input of the speech noise reduction chip is the original input data multiplied by a preset integer multiple for data input, when the speech noise reduction chip is used to reduce the noise of the speech to be reduced, when the memristor array where the bias element in the speech noise reduction chip is located has data input, the memristor input voltage corresponding to the bias element is set to a preset integer multiple V).

[0083] Compared with the prior art, in the embodiment of the present invention, the first weight matrix is ​​processed by dividing the positive and negative weights to obtain the second weight matrix, and the second weight matrix is ​​weight quantized based on the preset chip data bit width to obtain the third weight matrix; additional weights are added to the third weight matrix to obtain a target weight matrix, and the weights of each column element in the target weight matrix are the same; according to the elements of the target weight matrix, the conductance value of the memristor in the pre-constructed storage and computing integrated chip is adjusted to embed the speech noise reduction model into the storage and computing integrated chip to obtain a speech noise reduction chip; when the speech to be denoised is received, the speech to be denoised is denoised using the speech noise reduction chip to obtain noise-reduced speech. The problem that the conductivity value of the memristor cannot represent negative weights and the calculation results of the memristor array matrix implemented in voltage mode are inconsistent with the actual matrix multiplication calculation results is solved. Therefore, the storage and computing integrated chip can be used to achieve voice noise reduction. There is no need to deploy a cloud-based language noise reduction model for noise reduction, which improves the speed of voice noise reduction. At the same time, the voice noise reduction model integrated in the storage and computing integrated chip can be more conveniently deployed on the terminal side, which lowers the threshold for deploying the voice noise reduction model on the terminal side device and has a high promotion prospect.

[0084] Based on the above embodiment, as the above Figure 1The present invention provides an embodiment of a speech noise reduction device based on a storage and computing integrated chip. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices, see Figure 4 As shown, the speech noise reduction device based on the storage and computing integrated chip includes:

[0085] The parsing module 100 is used to extract weight data corresponding to a preset level of the speech noise reduction model to obtain a first weight matrix;

[0086] A first processing module 200 is used to perform positive and negative weight division processing on the first weight matrix to obtain a second weight matrix;

[0087] A second processing module 300 is used to perform weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix;

[0088] A third processing module 400 is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same;

[0089] A model embedding module 500 is used to adjust the conductance value of the memristor in the pre-built storage and computing integrated chip according to the elements of the target weight matrix, so as to embed the speech noise reduction model into the storage and computing integrated chip to obtain the speech noise reduction chip;

[0090] The noise reduction processing module 600 is used to, when receiving the speech to be noise reduced, use the speech noise reduction chip to perform noise reduction on the speech to be noise reduced to obtain noise reduced speech.

[0091] The speech noise reduction device based on the storage and computing integrated chip described in the embodiment of the present invention can execute the speech noise reduction method based on the storage and computing integrated chip provided in the above embodiment. The speech noise reduction device based on the storage and computing integrated chip has the corresponding functional steps and beneficial effects of the speech noise reduction method based on the storage and computing integrated chip described in the above embodiment. Please refer to the embodiment of the speech noise reduction method based on the storage and computing integrated chip for details. The embodiment of the present invention will not be repeated here.

[0092] The embodiment of the present invention also provides an electronic device, which may include a processor and a memory, wherein the processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various types of chips. The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the voice noise reduction method based on the storage-computing integrated chip in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the voice noise reduction method based on the storage-computing integrated chip in the above method embodiment is realized.

[0093] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. The one or more modules are stored in the memory, and when executed by the processor, the voice noise reduction method based on the integrated storage and computing chip in the above-mentioned method embodiment is executed. The specific details of the above-mentioned electronic device can be understood by corresponding to the corresponding descriptions and effects in the above-mentioned method embodiment, and will not be repeated here. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0094] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0095] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0096] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A method for reducing speech noise based on a storage and computing integrated chip, characterized in that: The method comprises: Extracting weight data corresponding to a preset level of the speech noise reduction model to obtain a first weight matrix; Performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix; Based on a preset chip data bit width, weight quantizing the second weight matrix to obtain a third weight matrix; Adding additional weights to the third weight matrix to obtain a target weight matrix, wherein the sum of the weights of the elements in each column of the target weight matrix is ​​the same; According to the elements of the target weight matrix, the conductance value of the memristor in the pre-built storage and computing integrated chip is adjusted to embed the speech noise reduction model into the storage and computing integrated chip to obtain the speech noise reduction chip; When the speech to be de-noised is received, the speech de-noising chip is used to de-noise the speech to be de-noised to obtain de-noised speech.

2. The method for voice noise reduction based on a storage and computing integrated chip according to claim 1, characterized in that: The performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix comprises: Determine each column in the first weight matrix as a target weight column; Replace all negative numbers in the target weight column with zero to obtain a positive weight column corresponding to the target weight column; Replace all positive numbers in the target weight column with zero and all negative numbers with their absolute values ​​to obtain a negative weight column corresponding to the target weight column; Combine the positive weight column and the negative weight column corresponding to the target weight column to obtain a positive and negative weight column pair corresponding to the target weight column; Each column in the first weight matrix is ​​replaced by a positive and negative weight column pair corresponding to the column to obtain a second weight matrix.

3. The method for voice noise reduction based on a storage and computing integrated chip according to claim 1, characterized in that: Adding additional weights to the third weight matrix to obtain a target weight matrix includes: Calculate the weight sum of each column of the third weight matrix respectively, determine the maximum value of the weight sum of each column in the third weight matrix, and obtain the target weight value; Adding a preset number of additional weights to each column of the third weight matrix to obtain the target weight matrix, The sum of the weights of the elements in each column of the target weight matrix is ​​equal to the target weight value and a preset number of rows in the target weight matrix are all composed of additional weights.

4. The method for voice noise reduction based on a storage and computing integrated chip according to claim 1, characterized in that: The step of performing weight quantization on the second weight matrix based on the preset chip data bit width to obtain a third weight matrix includes: Respectively determining the maximum value and the minimum value of the elements in the second weight matrix; Determine a quantization factor according to the maximum value and the minimum value of the second weight matrix and the chip data bit width; Each element in the second weight matrix is ​​quantized and updated by using the quantization factor to obtain a third weight matrix.

5. The method for voice noise reduction based on a storage-computing integrated chip according to any one of claims 1 to 4, characterized in that: In the storage-computing integrated chip, each of the target weight matrices corresponds to a memristor array, each element in the target weight matrix corresponds to the memristor in the same row and column position in the memristor array corresponding to the target weight matrix, and the output voltage of each odd-numbered memristor column in the memristor array and the even-numbered memristor column adjacent to its right side are differentially output through a differential structure connected at the end of the negative line.

6. The method for voice noise reduction based on a storage and computing integrated chip according to claim 5 is characterized in that: The step of adjusting the conductance value of the memristor in the pre-built storage-computation integrated chip according to the elements of the target weight matrix includes: Traversing each element in the target weight matrix, and performing a non-zero judgment on each traversed element; When the traversed element is a non-zero element, the conductance value of the memristor corresponding to the element in the storage-computation integrated chip is adjusted to the value of the element; When the traversed element is a zero element, the conductance value of the memristor corresponding to the element in the storage-computing integrated chip is adjusted to a preset high resistance value.

7. The method for voice noise reduction based on a storage and computing integrated chip according to claim 5, characterized in that: When the speech to be de-noised is de-noised using the speech de-noising chip, when data is input to the memristor array where the additional weights are located in the speech de-noising chip, the memristor input voltage corresponding to the additional weights is set to 0V.

8. A speech noise reduction device based on a storage and computing integrated chip, applied to any method in claims 1-7, characterized in that: The device comprises: A parsing module, used to extract weight data corresponding to a preset level of a speech noise reduction model to obtain a first weight matrix; A first processing module, configured to perform positive and negative weight division processing on the first weight matrix to obtain a second weight matrix; A second processing module, configured to perform weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix; A third processing module is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same; A model embedding module, used for adjusting the conductance value of the memristor in the pre-built storage and computing integrated chip according to the elements of the target weight matrix, so as to embed the speech noise reduction model into the storage and computing integrated chip to obtain the speech noise reduction chip; The noise reduction processing module is used to, when receiving the speech to be denoised, use the speech noise reduction chip to perform noise reduction on the speech to be denoised to obtain the noise-reduced speech.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which are loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 7.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the processor implements instructions of any one of the methods described in claims 1-7.

Images