Voice noise reduction method and device based on memory-computing integrated chip, medium and equipment

Abstract

The application discloses a voice noise reduction method and device based on a memory-computing integrated chip, a medium and equipment. The method comprises the following steps: extracting weight data corresponding to a preset level of a voice noise reduction model to obtain a first weight matrix; performing positive and negative weight division processing on the first weight matrix to obtain a second weight matrix; performing weight quantization on the second weight matrix to obtain a third weight matrix; adding an additional weight to the third weight matrix to obtain a target weight matrix; and adjusting the conductance value of a memristor in a pre-constructed memory-computing integrated chip according to the elements of the target weight matrix to embed the voice noise reduction model in the memory-computing integrated chip to obtain a voice noise reduction chip. The voice noise reduction model is integrated on the memory-computing integrated chip to reduce noise, the difficulty of deploying the voice noise reduction model on the terminal side is reduced, and the efficiency of voice noise reduction is improved.

Description

Technical Field

[0001] This invention relates to the field of speech noise reduction technology, and in particular to a speech noise reduction method, apparatus, medium and device based on a memory computing chip. Background Technology

[0002] On some edge devices (such as smartwatches), due to limitations in size, available computing resources, power consumption, and other physical conditions, it is difficult to integrate a voice noise reduction model. Usually, a cloud-based model needs to be deployed online for noise reduction, resulting in low efficiency of voice noise reduction and greater difficulty in edge deployment. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide a speech noise reduction method, apparatus, medium and device based on a memory computing chip to overcome or at least partially solve the above problems.

[0004] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0005] According to a first aspect of the present invention, a speech noise reduction method based on a memory computing chip is provided, the speech noise reduction method based on a memory computing chip includes:

[0006] Extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix;

[0007] The first weight matrix is ​​divided into positive and negative weights to obtain the second weight matrix;

[0008] Based on the preset chip data bit width, the second weight matrix is ​​weighted and quantized to obtain the third weight matrix;

[0009] Add additional weights to the third weight matrix to obtain the target weight matrix, wherein the sum of the weights of each column element in the target weight matrix is ​​the same;

[0010] Based on the elements of the target weight matrix, the conductance value of the memristor in the pre-constructed in-memory computing chip is adjusted to embed the speech denoising model into the in-memory computing chip, thereby obtaining the speech denoising chip.

[0011] When a voice message to be denoised is received, the voice denoising chip is used to denoise the voice message to obtain the denoised voice message.

[0012] According to a second aspect of the present invention, a speech noise reduction device based on a memory computing chip is provided, the device comprising:

[0013] The parsing module is used to extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix;

[0014] The first processing module is used to perform positive and negative weight partitioning on the first weight matrix to obtain a second weight matrix.

[0015] The second processing module is used to perform weight quantization on the second weight matrix based on the preset chip data bit width to obtain a third weight matrix;

[0016] The third processing module is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same.

[0017] The model embedding module is used to adjust the conductance value of the memristor in the pre-built in-memory computing chip according to the elements of the target weight matrix, so as to embed the speech denoising model into the in-memory computing chip to obtain the speech denoising chip.

[0018] The noise reduction processing module is used to reduce the noise of the voice to be denoised by the voice noise reduction chip when the voice to be denoised is received, so as to obtain the denoised voice.

[0019] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein computer program instructions are stored therein, the computer program instructions being loaded and executed by a processor to perform the operations performed by the method described in any of the preceding claims.

[0020] According to a fourth aspect of the present invention, an electronic device is provided, including a processor and a memory, the memory storing computer program instructions executable by the processor, wherein when the processor executes the computer program instructions, it implements the instructions of any of the methods described above.

[0021] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0022] This invention provides a speech noise reduction method, apparatus, medium, and device based on an in-memory computing chip. Compared to existing technologies, the speech noise reduction method based on an in-memory computing chip described in this invention achieves [advantages / advantages].

[0023] By dividing the first weight matrix into positive and negative weights, a second weight matrix is ​​obtained. Based on a preset chip data bit width, the second weight matrix is ​​weighted and quantized to obtain a third weight matrix. Additional weights are added to the third weight matrix to obtain a target weight matrix, where the sum of the weights of each column element is the same. Based on the elements of the target weight matrix, the conductance value of the memristor in the pre-built in-memory computing chip is adjusted to embed the speech denoising model into the in-memory computing chip, resulting in a speech denoising chip. When speech to be denoised is received, the speech denoising chip is used to denoise the speech, resulting in denoised speech. This solves the problems that the conductance value of the memristor cannot represent negative weights and that the calculation results of the memristor array matrix implemented by voltage method are inconsistent with the actual matrix multiplication calculation results. Therefore, speech denoising can be implemented in the in-memory computing chip without deploying a cloud-based speech denoising model, improving the speed of speech denoising. Furthermore, integrating the speech denoising model into the in-memory computing chip makes it easier to deploy on the edge, lowering the barrier to entry for deploying speech denoising models on edge devices, and has high potential for widespread application.

[0024] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic flowchart of a speech noise reduction method based on a memory computing chip provided in an embodiment of the present invention;

[0027] Figure 2 This is a structural reference diagram of a memristor array;

[0028] Figure 3 This is a schematic diagram of the structure of a memristor array;

[0029] Figure 4 This is a schematic diagram of the flow principle structure of a speech noise reduction device based on a memory computing chip provided in an embodiment of the present invention. Detailed Implementation

[0030] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings.

[0031] The accompanying drawings illustrate various structural schematics according to embodiments of the present disclosure. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0032] In the context of this disclosure, when a layer / component is referred to as being "above" another layer / component, that layer / component may be directly above the other layer / component, or there may be an intermediate layer / component between them. Additionally, if a layer / component is "above" another layer / component in one orientation, then when the orientation is reversed, that layer / component may be "below" the other layer / component. In the context of this disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0033] To better understand the above technical solutions, the following will describe the above technical solutions in detail with reference to specific implementation methods. It should be understood that the embodiments of this disclosure and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0034] Figure 1 This is a flowchart illustrating a speech noise reduction method based on a memory computing chip provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the speech noise reduction method based on a memory computing chip includes the following steps:

[0035] S1. Extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix;

[0036] In this embodiment of the invention, the trained speech denoising model is analyzed to extract the weight data corresponding to the preset level of the speech denoising model, and a first weight matrix is ​​obtained.

[0037] In this embodiment of the invention, the speech denoising model is, for example, an end-to-end speech denoising network using deep learning, which utilizes convolutional neural networks to construct complex acoustic features. It can also incorporate traditional statistical denoising methods, such as spectral subtraction, to fully leverage the advantages of both neural networks and classical algorithms. It should be noted that, in order to accelerate the speech denoising model using an in-memory computing chip, the speech denoising model needs to be embedded in the in-memory computing chip. Therefore, it is necessary to extract the weight data of the layers that the in-memory computing chip can accelerate (i.e., preset layers). These preset layers include, but are not limited to, convolutional layers and fully connected layers.

[0038] The training process of the speech denoising model can employ conventional model training methods. For example, the Mel spectrogram of the audio signal extracted after audio data preprocessing can be used as the input feature of the speech denoising model. Correspondingly, the output of the speech denoising model is the Mel spectrogram of the clean human voice audio. The goal of the speech denoising model is to output a result that is as close as possible to the clean human voice Mel spectrogram from the input noisy audio Mel spectrogram. During the training process of the speech denoising model, MSE (mean squared error) or L1 loss function can be used as optimization objectives to calculate the difference between the model's output Mel spectrogram and the corresponding clean human voice Mel spectrogram. The Adam algorithm can be used to iteratively update the model parameters. Simultaneously, model performance can be monitored on a validation set, and strategies such as Early Stopping can be used to prevent overfitting. Finally, the speech denoising effect of the speech denoising model is evaluated on a validation set; and based on the validation results, the hyperparameters are adjusted to further optimize the denoising performance of the speech denoising model.

[0039] In this embodiment of the invention, the pre-trained speech denoising model is loaded using a training framework such as PyTorch or Tensorflow, and the weight data corresponding to the preset level of the speech denoising model is extracted to obtain the first weight matrix.

[0040] Specifically, in this embodiment of the invention, when the preset layer is a convolutional layer, the weight data corresponding to the convolutional layer is the convolutional kernel of the convolutional layer, and the convolutional kernel data of the convolutional layer is extracted as the first weight matrix; when the preset layer is a fully connected layer, the weight data corresponding to the fully connected layer is the weight matrix of the fully connected layer.

[0041] Because different layers in the speech denoising model may contain bias vectors or not, in order to maintain the accuracy of the weight matrix data, when the fully connected layer corresponding to the first weight matrix contains a bias vector, it is necessary to update the first weight matrix. This includes: obtaining the bias vector of the fully connected layer corresponding to the first weight matrix, where the first weight matrix is ​​a matrix of dimension h×k, and h and k are both positive integers; concatenating the bias vector in the last column of the first weight matrix to update the first weight matrix, resulting in an updated first weight matrix with dimension h×(k+1). At this point, the updated first weight matrix is ​​the final first weight matrix.

[0042] For example, if we define the first weight matrix of a fully connected layer as H and the bias vector as b, then we can concatenate H and b to form a new weight matrix H'. The last column of the new first weight matrix H' is the original bias vector b. Specifically, if we define a fully connected layer with an input dimension of 4 and an output dimension of 3, the original first weight matrix H is (3×4) in size, and the bias vector b is (3×1) in size. After merging, the new first weight matrix H' is (3×5) in size, and the last column is the bias vector b.

[0043] S2. Perform positive and negative weight division on the first weight matrix to obtain the second weight matrix;

[0044] In this embodiment of the invention, in order to map the weights in the first weight matrix to the conductance value of the memristor (the reciprocal of the memristor's resistance), and since the conductance value of the memristor cannot represent a negative value, the first weight matrix is ​​divided into positive and negative weights to obtain the second weight matrix.

[0045] In this embodiment of the invention, the process of dividing the first weight matrix into positive and negative weights to obtain the second weight matrix includes: determining each column of the first weight matrix as a target weight column; replacing all negative numbers in the target weight column with zero to obtain a positive weight column corresponding to the target weight column; replacing all positive numbers in the target weight column with zero and all negative numbers with their absolute values ​​to obtain a negative weight column corresponding to the target weight column; combining the positive and negative weight columns corresponding to the target weight column to obtain a pair of positive and negative weight columns corresponding to the target weight column; and replacing each column of the first weight matrix with the pair of positive and negative weight columns corresponding to that column to obtain the second weight matrix.

[0046] In the first weight matrix, all elements have different numerical attributes. Each element is a weight value, which can be positive or negative. Since the memristor in the in-memory computing chip cannot directly represent negative elements, negative numbers are represented by the difference between zero and positive numbers. All negative numbers in the target weight column are replaced with zero to obtain the positive weight column corresponding to the target weight column. Similarly, all positive numbers in the target weight column are replaced with zero, and all negative numbers are replaced with their absolute values ​​to obtain the negative weight column corresponding to the target weight column. The positive and negative weight columns corresponding to the same target weight column are grouped together to obtain the positive and negative weight column pairs corresponding to the target weight column. Subtracting the negative weight column from the positive weight column in each positive and negative weight column pair represents the target weight column corresponding to that pair, thus solving the problem that the negative weights in the target weight column cannot be represented by memristors.

[0047] For example: the first weight matrix is Then the first column Replace all negative numbers with 0 to obtain the first column of positive weights. First column Replace all positive numbers with 0 and all negative numbers with their absolute values ​​to obtain the negative weight column in the first column. The first column can be obtained by taking the difference between the positive and negative weight columns. The positive and negative weight pairs in the first column are then... Similarly, the second column has positive weights. The second column is a negative weight column. The positive and negative weight pairs in the second column are Replace each column of the first weight matrix with a pair of positive and negative weights for that column to obtain the second weight matrix.

[0048] Through the above-described positive and negative weighting, the memristor array of the in-memory computing chip implemented in the subsequent voltage implementation can make each memristor correspond to an element in the second weighting matrix. This allows the difference in conductance between two adjacent columns of memristors to characterize any element in the target weight column of the first weighting matrix in the in-memory computing chip, thus avoiding the problem that memristors cannot characterize negative numbers.

[0049] Optionally, other embodiments of the present invention may further perform positive and negative weight division on the first weight matrix using the following method, the specific process of which is as follows:

[0050] If the first weight matrix is ​​an m*n matrix weight, then the number of columns n of the first weight matrix needs to be expanded to 2n, and a new matrix new_weight with shape m*2n needs to be created. The elements in the new matrix new_weight are initialized to 0, which is analog to the memristor array in the in-memory computing chip implemented by voltage simulation.

[0051] The new matrix `new_weight` is structured so that even-numbered columns store negative weights and odd-numbered columns store positive weights. Each element in the matrix `weight` is iterated over. If an element is greater than or equal to 0, it is assigned to the corresponding element in the same row of the odd-numbered column in `new_weight` (e.g., if the element's value is 5 and it's in the third column and fifth row of `weight`, then the element in the fifth row of the third odd-numbered column in `new_weight` is assigned the value 5). If an element is less than 0, its absolute value is assigned to the corresponding element in the even-numbered column in `new_weight` (e.g., if the element's value is -3 and it's in the third column and fourth row of `weight`, then the element in the fourth row of the third odd-numbered column in `new_weight` is assigned the value 3). Thus, the adjacent values ​​in the positive and negative weight columns of `new_weight` represent the corresponding elements in `weight`.

[0052] For example, if the element in the first row and first column of `weight` is -4, which is less than 0, its absolute value of 4 is assigned to the element in the first even-numbered column and first row of `new_weight`, making the element in the first even-numbered column and first row of `new_weight` 4. Since the element in the first odd-numbered column and first row of `new_weight` is 0, the element in the first row and first column of `weight` can be represented as -4 by the difference between the element in the first odd-numbered column and first row of `new_weight` (i.e., the value of the first row of the positive weight column) and the element in the first even-numbered column and first row of `new_weight` (i.e., the value of the first row of the negative weight column) = 0 - 4 = -4. Thus, the difference between two non-negative elements represents a negative number, thereby enabling the differential conductance of a pair of memristors to represent a negative weight, and the differential conductance of two columns of memristors to represent a column of elements in the second weight matrix.

[0053] S3. Based on the preset chip data bit width, the second weight matrix is ​​weighted and quantized to obtain the third weight matrix;

[0054] To reduce the computational complexity of subsequent chip calculations, this embodiment of the invention performs quantization processing on the second weight matrix, for example, using linear quantization technology to quantize the floating-point weight values, converting them from floating-point numbers to integers. This embodiment of the invention also performs quantization processing on the third weight matrix, converting the floating-point numbers in the second weight matrix into integers to obtain an integer matrix, i.e., the third weight matrix.

[0055] The embodiments of the present invention perform weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix, including: determining the maximum and minimum values ​​of the elements in the second weight matrix respectively; determining a quantization factor based on the maximum and minimum values ​​of the second weight matrix and the chip data bit width; and using the quantization factor to quantize and round down each element in the second weight matrix to obtain the third weight matrix.

[0056] Specifically, in this embodiment of the invention, the maximum value data of the elements in the second weight matrix is ​​first determined. max and minimum value data min Then, based on the preset chip data bit width, i.e., the data bit width (bits) of the main MCU on the in-memory computing chip to which the speech denoising model will be embedded (e.g., bits = 8), the quantization factor is calculated and determined. Specifically, embodiments of the present invention can be achieved through the formula... The quantization factor is obtained by multiplying each element in the second weight matrix by the quantization factor, and then using the round() function to round the elements multiplied by the quantization factor to obtain the quantized integer matrix, which is the third weight matrix.

[0057] S4. Add additional weights to the third weight matrix to obtain the target weight matrix, wherein the sum of the weights of each column element in the target weight matrix is ​​the same;

[0058] Specifically, in this embodiment of the invention, adding additional weights to the third weight matrix to obtain a target weight matrix includes: calculating the sum of the weights of each column element in the third weight matrix (the sum of the weights of each column element is the sum of all elements in that column), determining the maximum value among the sums of the weights of each column in the third weight matrix, and obtaining a target weight value; adding a preset number of additional weights to each column of the third weight matrix to obtain the target weight matrix, wherein the sum of the weights of each column element in the target weight matrix is ​​equal to the target weight value, and a preset number of rows in the target weight matrix are all composed of additional weights. For example, if the preset number is 2, then there will be two rows in the target weight matrix that are all composed of additional weights.

[0059] In this embodiment of the invention, the goal is to map a weight matrix onto a memristor array to embed the speech denoising model into a memory computing chip. A column in the weight matrix can be equivalently understood as a column of memristors in the memristor array, and the values ​​of the elements in the weight matrix can be equivalently understood as the conductance values ​​of the memristors. This embodiment of the invention uses a column of memristors as an example, and the structure of the memristor array is as follows... Figure 2 As shown in the reference, assume the input is The first column of the memristor array stores the weight data as follows: (That is, the weight (conductance value) of the first row of memristors is w1, the weight (conductance value) of the second row of memristors is w2, the weight (conductance value) of the third row of memristors is w3, and the weight (conductance value) of the second row of memristors is w4), the voltage of the input line SL is v (the input voltage of each input line from top to bottom is x1, x2, x3, x4 respectively), and the voltage of the negative line BL is Y.

[0060] This then extends to n input voltages, with each input line having an input voltage of x. i The weight (conductance value) of each memristor is w. i According to Ohm's law, the current of each memristor can be calculated as I. i =(x i -Y)·w i Since the voltage on the negative line BL may be greater than or less than the voltage on the input line SL, the direction of the current includes two cases: when the voltage on the input line SL is greater than the voltage on the negative line BL, the current flows from the input line through the memristor to the negative line; when the voltage on the input line SL is less than the voltage on the negative line BL, the current flows from the negative line through the memristor to the input line.

[0061] Since the current flows only on this memristor array network, and current is conserved, we have I1 + I2 + I3 + I4 = 0. Expanding this, we get (x1 - Y)·w1 + ... + (x4 - Y)·w4 = 0. Further derivation yields...

[0062] For the case with n inputs, the final output is derived as shown in the following formula:

[0063]

[0064] Therefore, by measuring the voltage on the negative line BL, the embodiment of the present invention can obtain the output result shown in the above formula, wherein... It is the result of the matrix multiplication that actually needs to be calculated.

[0065] By using weighted quantization, the voltage output on each negative line can be transformed into... Here, c is a constant. Therefore, in this embodiment of the invention, a value proportional to the actual calculation result can be obtained by measuring the voltage on the negative line BL. By multiplying by the constant c after each weight matrix multiplication, the actual required calculation result can be restored.

[0066] In the memristor array of the in-memory computing chip, the output obtained by measuring the voltage on the negative line BL in this embodiment of the invention is actually... In reality, the result of matrix multiplication is... If the calculation result of each column is divided by That is, the output of each column is divided by the sum of the weights of that column, but the sum of the weights is different in different columns of the memristor array. To solve the problem of inconsistency between the result obtained from measuring the negative line voltage output and the actual matrix multiplication calculation result, this invention adds extra weights to make the sum of the weights on each column of memristors the same, equal to the maximum sum of the weight values ​​of each column in the weight matrix, i.e., the target weight value. This means that the sum of the weights in each column is a constant, and thus the actual calculation result can be obtained by multiplying the output of each column by this constant.

[0067]

[0068] For example, we can introduce two additional inputs with a value of 0 and two additional weights p1 and p2 (which can be understood as...) This ensures that the sum of the weights of each column after adding the external weights equals the target weight value. Then the output of each column starts from... Become Even though the output of each memristor column is the result of the actual matrix multiplication calculation divided by the constant summax, to restore the actual calculation result, the voltage output can be multiplied by summax; or, without restoration, an output result proportional to the actual result is obtained. In this embodiment of the invention, the influence of the ratio can be eliminated through the Softmax layer.

[0069] Specifically, calculate the maximum sum of weights in each column of the third weight matrix W (summax); and add a predetermined number of additional weights p to each column. i This ensures that the sum of the weights in each column is summax. For example, assuming two additional weights p1 and p2 are added, each column of p1 forms a row vector P1, and each column of p2 forms a row vector P2. A new weight matrix can then be constructed by concatenating the original weights and additional weights from the second weight data, i.e., concatenating the third weight matrix with the row vectors P1 and P2 (after concatenation, the row vectors P1 and P2 become the rows of the new matrix), thus obtaining the target weight matrix.

[0070] It should be noted that the additional weights added in this embodiment of the invention are non-negative integers.

[0071] S5. Adjust the conductance value of the memristor in the pre-constructed in-memory computing chip according to the elements of the target weight matrix, so as to embed the speech noise reduction model into the in-memory computing chip and obtain the speech noise reduction chip.

[0072] Based on the foregoing embodiments, this invention maps each element of the target weight matrix to a memristor in the in-memory computing chip. Specifically, in this embodiment, each target weight matrix in the in-memory computing chip corresponds to a memristor array. Each element of the target weight matrix corresponds to a memristor in the same row and column of the memristor array corresponding to that target weight matrix. The output voltage of each odd-numbered memristor column and its adjacent even-numbered memristor column on the right is output differentially through a differential structure connected at the end of the negative line (BL). The memristor array is as follows: Figure 3 As shown (the first and third columns are odd-numbered memristor columns, the second and fourth columns are even-numbered memristor columns, the first and second columns are connected by a differential structure through the negative line BL to output differential voltage, and the third and fourth columns are connected by a differential structure through the negative line BL to output differential voltage).

[0073] Since memristors cannot represent negative numbers, in order to achieve multiplication and addition operations with negative weights using a memristor array, the aforementioned embodiment represents a column of original weights as two columns of non-negative weights. These two columns of non-negative weights are then mapped to the conductance values ​​of two adjacent columns of memristors. The voltage outputs of the two columns of memristors are then subtracted through a voltage differential structure connected at the end of the negative line, thus obtaining the true result of the original weight multiplication and addition. This solves the problem that memristor arrays cannot perform multiplication and addition calculations with non-negative weights. For example: a memristor array has n input lines, with x... i The weights (denoted as C) of each column in the memristor array are the same, and the weights (conductance values) of the memristors in odd-numbered memristor columns are w. pi The weight (conductance value) of the memristors in the even-numbered memristor column is: The calculated result (output voltage) for an odd-numbered memristor array is: The calculated output voltage for an even-numbered memristor array is:

[0074]

[0075] The differential output voltage is obtained by subtracting the outputs of the two memristors. again The final derivation yields the calculated output voltage differential between each odd-numbered memristor column and its adjacent even-numbered memristor column to the right of the memristor array.

[0076] In this embodiment of the invention, adjusting the conductance value of the memristor in the in-memory computing chip according to the elements of the target weight matrix includes: traversing each element in the target weight matrix and performing a non-zero check on each traversed element; when the traversed element is a non-zero element, adjusting the conductance value of the memristor corresponding to that element in the in-memory computing chip to the value of that element; when the traversed element is a zero element, adjusting the conductance value of the memristor corresponding to that element in the in-memory computing chip to a preset high resistance value.

[0077] It should be explained that, in this embodiment of the invention, since the memristor cannot represent 0, when the traversed element is a zero element, the conductance value of the memristor corresponding to that element in the in-memory computing chip is adjusted to a preset high resistance value. The preset high resistance value is the reciprocal of a preset high resistance. When the resistance of the memristor is set to the preset high resistance, the current contributed by the memristor is very small and can be approximated as zero. Therefore, the preset high resistance value is used to characterize the zero element.

[0078] By adjusting the conductance values ​​of the memristors corresponding to the elements of the target weight matrix in the in-memory computing chip one by one, each element of the target weight matrix can be mapped to a memristor in the in-memory computing chip, thereby embedding the speech denoising model into the in-memory computing chip to obtain the speech denoising chip. The pre-built in-memory computing chip is a voltage-implemented in-memory computing chip (i.e., the result of multiplying and adding each column of memristors by measuring the negative line voltage of the memristor array instead of the current). The other parts of embedding the speech denoising model into the in-memory computing chip adopt commonly used techniques in the field, such as using STM32 programming to program model code data to achieve model embedding, which will not be elaborated here.

[0079] S6. When the voice to be denoised is received, the voice denoising chip is used to denoise the voice to be denoised to obtain the denoised voice.

[0080] After obtaining the speech noise reduction chip, when receiving speech to be denoised, the speech noise reduction chip can be used to denoise the received speech to be denoised, thereby obtaining denoised speech, thus achieving the purpose of speech noise reduction on the in-memory computing chip.

[0081] In this embodiment of the invention, the speech to be denoised is preprocessed and converted into a data format that can be recognized by the speech denoising chip. When the speech denoising chip is used to denoise the speech to be denoised, when there is data input to the memristor array where the extra weights are located in the speech denoising chip, the input voltage of the memristor corresponding to the extra weights is set to 0V (that is, to eliminate the influence of the multiplication and addition of extra weights; for details, please refer to the example of the introduction of extra weights in S4). Finally, the output of the speech denoising chip is restored to speech to obtain the denoised speech.

[0082] In this embodiment of the invention, it should also be noted that when the first weight matrix contains a bias vector, the element corresponding to the bias vector in the target weight matrix is ​​determined as the bias element (for example, if the fifth column of the first weight matrix is ​​the bias vector, then the elements in the ninth and tenth columns of the target weight matrix that are not additional weights are the bias elements). When the speech denoising chip is used to denoise the speech to be denoised, when there is data input to the memristor array where the bias element is located in the speech denoising chip, the memristor input voltage corresponding to the bias element is set to 1V to ensure the stability of the memristor in calculating the bias vector. (When the input of the speech denoising chip is the original input data multiplied by a preset integer multiple, when the speech denoising chip is used to denoise the speech to be denoised, when there is data input to the memristor array where the bias element is located in the speech denoising chip, the memristor input voltage corresponding to the bias element is set to a preset integer multiple V.)

[0083] In this embodiment of the invention, compared with the prior art, a second weight matrix is ​​obtained by dividing the first weight matrix into positive and negative weights. Based on a preset chip data bit width, the second weight matrix is ​​weighted to obtain a third weight matrix. Additional weights are added to the third weight matrix to obtain a target weight matrix, in which the weights of each column element are equal. The conductance of the memristor in the pre-constructed in-memory computing chip is adjusted according to the elements of the target weight matrix to embed the speech denoising model into the in-memory computing chip, thus obtaining a speech denoising chip. When speech to be denoised is received, the speech denoising chip is used to denoise the speech to obtain denoised speech. This invention solves the problems that the conductance value of memristors cannot represent negative weights and that the calculation results of memristor array matrices implemented by voltage method are inconsistent with the actual matrix multiplication calculation results. As a result, speech denoising can be implemented in in-memory computing chips without the need to deploy cloud-based speech denoising models, which improves the speed of speech denoising. At the same time, the speech denoising model is integrated into the in-memory computing chip, which makes it easier to deploy on the edge, lowers the threshold for deploying speech denoising models on edge devices, and has high prospects for promotion.

[0084] Based on the above embodiments, as a supplement to the above... Figure 1The present invention provides an embodiment of a speech noise reduction device based on a memory computing chip, which is similar to the implementation of the method shown. Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices, see reference. Figure 4 As shown, the speech noise reduction device based on a memory computing chip includes:

[0085] The parsing module 100 is used to extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix;

[0086] The first processing module 200 is used to perform positive and negative weight division processing on the first weight matrix to obtain a second weight matrix;

[0087] The second processing module 300 is used to perform weight quantization on the second weight matrix based on a preset chip data bit width to obtain a third weight matrix;

[0088] The third processing module 400 is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same.

[0089] The model embedding module 500 is used to adjust the conductance value of the memristor in the pre-built in-memory computing chip according to the elements of the target weight matrix, so as to embed the speech denoising model into the in-memory computing chip to obtain the speech denoising chip.

[0090] The noise reduction processing module 600 is used to reduce the noise of the voice to be denoised by the voice noise reduction chip when the voice to be denoised is received, so as to obtain the denoised voice.

[0091] The speech noise reduction device based on a memory computing chip described in this embodiment of the invention can execute the speech noise reduction method based on a memory computing chip provided in the above embodiments. The speech noise reduction device based on a memory computing chip has the corresponding functional steps and beneficial effects of the speech noise reduction method based on a memory computing chip described in the above embodiments. For details, please refer to the embodiments of the speech noise reduction method based on a memory computing chip described above. The embodiments of the present invention will not be repeated here.

[0092] This invention also provides an electronic device, which may include a processor and a memory, wherein the processor and memory can be connected via a bus or other means. The processor may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the speech denoising method based on an in-memory computing chip in this invention embodiment. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the speech denoising method based on an in-memory computing chip in the above method embodiment.

[0093] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. The one or more modules are stored in the memory and, when executed by the processor, perform the speech noise reduction method based on an in-memory computing chip as described in the above method embodiments. Specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it may include the processes of the embodiments of the above methods. The storage medium may be a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above types of memory.

[0094] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0095] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of a single foregoingly disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0096] It should be noted that the above embodiments are illustrative of the invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims.

Claims

1. A speech noise reduction method based on a memory computing chip, characterized in that, The method includes: Extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix; The first weight matrix is ​​divided into positive and negative weights to obtain the second weight matrix; Based on the preset chip data bit width, the second weight matrix is ​​weighted and quantized to obtain the third weight matrix; Add additional weights to the third weight matrix to obtain the target weight matrix, wherein the sum of the weights of each column element in the target weight matrix is ​​the same; Based on the elements of the target weight matrix, the conductance value of the memristor in the pre-constructed in-memory computing chip is adjusted to embed the speech denoising model into the in-memory computing chip, thereby obtaining the speech denoising chip. When a voice message to be denoised is received, the voice denoising chip is used to denoise the voice message to obtain the denoised voice message.

2. The speech noise reduction method based on a memory computing chip according to claim 1, characterized in that, The step of dividing the first weight matrix into positive and negative weights to obtain the second weight matrix includes: Each column in the first weight matrix is ​​determined as the target weight column; Replace all negative numbers in the target weight column with zero to obtain the positive weight column corresponding to the target weight column; Replace all positive numbers in the target weight column with zero and all negative numbers with their absolute values ​​to obtain the negative weight column corresponding to the target weight column; The positive and negative weight columns corresponding to the target weight column are combined to obtain the positive and negative weight column pairs corresponding to the target weight column; Replace each column in the first weight matrix with the corresponding positive and negative weight pair to obtain the second weight matrix.

3. The speech noise reduction method based on a memory computing chip according to claim 1, characterized in that, The step of adding additional weights to the third weight matrix to obtain the target weight matrix includes: Calculate the sum of the weights of each column element in the third weight matrix, determine the maximum value of the sum of the weights in each column of the third weight matrix, and obtain the target weight value; By adding a predetermined number of additional weights to each column of the third weight matrix, the target weight matrix is ​​obtained. Wherein, the sum of the weights of each column element in the target weight matrix is ​​equal to the target weight value, and a predetermined number of rows in the target weight matrix are all composed of additional weights.

4. The speech noise reduction method based on a memory computing chip according to claim 1, characterized in that, The third weight matrix is ​​obtained by weight quantization of the second weight matrix based on the preset chip data bit width, including: Determine the maximum and minimum values ​​of the elements in the second weight matrix, respectively; The quantization factor is determined based on the maximum and minimum values ​​of the second weight matrix and the chip data bit width; Using the quantization factor, each element in the second weight matrix is ​​quantized and rounded to update it, resulting in the third weight matrix.

5. The speech noise reduction method based on a memory computing chip according to any one of claims 1 to 4, characterized in that, In the in-memory computing chip, each target weight matrix corresponds to a memristor array. Each element in the target weight matrix corresponds to the memristor in the same row and column position in the memristor array corresponding to the target weight matrix. The output voltage of each odd-numbered memristor column in the memristor array and its adjacent even-numbered memristor column on the right is output differentially through a differential structure connected at the end of the negative line.

6. The speech noise reduction method based on a memory computing chip according to claim 5, characterized in that, The step of adjusting the conductance value of the memristor in the pre-constructed in-memory computing chip according to the elements of the target weight matrix includes: Iterate through each element in the target weight matrix and check if each element is non-zero. When the traversed element is a non-zero element, the conductance value of the memristor corresponding to that element in the in-memory computing chip is adjusted to the value of that element. When the traversed element is zero, the conductance value of the memristor corresponding to that element in the in-memory computing chip is adjusted to a preset high resistance value.

7. The speech noise reduction method based on a memory computing chip according to claim 5, characterized in that, When the speech noise reduction chip is used to reduce the noise of the speech to be denoised, when there is data input in the memristor array where the extra weights are located in the speech noise reduction chip, the memristor input voltage corresponding to the extra weights is set to 0V.

8. A speech noise reduction device based on a memory computing chip, applied to the method described in any one of claims 1-7, characterized in that, The device includes: The parsing module is used to extract the weight data corresponding to the preset level of the speech denoising model to obtain the first weight matrix; The first processing module is used to perform positive and negative weight partitioning on the first weight matrix to obtain a second weight matrix. The second processing module is used to perform weight quantization on the second weight matrix based on the preset chip data bit width to obtain a third weight matrix; The third processing module is used to add additional weights to the third weight matrix to obtain a target weight matrix, wherein the weights of each column element in the target weight matrix are the same. The model embedding module is used to adjust the conductance value of the memristor in the pre-built in-memory computing chip according to the elements of the target weight matrix, so as to embed the speech denoising model into the in-memory computing chip to obtain the speech denoising chip. The noise reduction processing module is used to reduce the noise of the voice to be denoised by the voice noise reduction chip when the voice to be denoised is received, so as to obtain the denoised voice.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that are loaded and executed by a processor to perform the operations described in any one of claims 1-7.

10. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements the instructions of the method as described in any one of claims 1-7.

Images