Data processing methods, audio noise reduction methods and neural network models

Abstract

The present application discloses a data processing method, including: obtaining the first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence; obtaining short-term memory cache data corresponding to the first feature data of the subsequence; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order; inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with some noise removed. The above method is used to solve the problem of slow operation and large delay of large models when the neural network model is used to perform noise reduction processing on input data in the prior art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a data processing method, a data processing device, an electronic device and a storage device; the present application also relates to a method for data processing based on a neural network model and a device for data processing based on a neural network model; the present application also relates to an audio noise reduction method, an audio noise reduction device and a neural network model. Background Art

[0002] In live broadcast scenarios, audio is often mixed with noise, which affects the listening experience. The increasingly common sudden noise, complex noise, and human voices pose a severe challenge to the capabilities of traditional audio noise reduction algorithms.

[0003] The use of neural networks, especially data-driven algorithms of convolutional neural networks (CNNs), has achieved good results in complex noise scenarios. However, the reliable noise reduction effect of neural networks depends on the depth and computational complexity of the neural networks (NNs), as well as the number of input data and input features. Generally speaking, the deeper the NN, the more parameters, the higher the complexity, and the longer the input features, the better the algorithm effect. However, NN models with deeper depth, more parameters, and more input features run slower, the delay generated by the algorithm will also increase, and the impact on the user experience will be more significant. In actual research and development, the trade-off between the "size, speed" and "effect" of the NN model has always been a difficult problem.

[0004] In summary, the existing technology uses neural networks to perform noise reduction on input data, but there are problems such as slow operation of large models and large delays. Summary of the invention

[0005] The present application provides a data processing method, apparatus, electronic device and storage device to solve the problems of slow operation and large delay of large models when the prior art uses neural networks to perform noise reduction on input data.

[0006] The present application provides a data processing method, comprising:

[0007] Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0008] Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order;

[0009] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0010] Optionally, the first convolution layer is the first convolution layer of a neural network model, and the obtaining of first feature data of the subsequence for inputting into the neural network model includes:

[0011] Obtain target sequence data;

[0012] Performing frame processing on the target sequence data according to a specified number of frames to obtain original data of the subsequence;

[0013] The original data of the subsequence is subjected to feature conversion processing to obtain first feature data of the subsequence for input into a neural network model.

[0014] Optionally, the first convolution layer is not the first convolution layer of the neural network model, and the obtaining of the first feature data of the subsequence for inputting into the neural network model includes:

[0015] Obtain feature data output by a convolutional layer above the first convolutional layer for a subsequence;

[0016] The feature data output for the subsequence is used as the first feature data of the subsequence for input into the neural network model.

[0017] Optionally, the obtaining of short-term memory cache data corresponding to the first feature data of the subsequence includes:

[0018] Determine whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache;

[0019] If it does not exist, data filling processing is performed on the first characteristic data of the subsequence to obtain short-term memory cache data corresponding to the first characteristic data of the subsequence;

[0020] When it exists, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained from the cache.

[0021] Optionally, the determining whether there is short-term memory cache data corresponding to the first feature data of the subsequence in the cache includes:

[0022] Determine whether the subsequence corresponding to the first characteristic data of the subsequence is the first subsequence; the first subsequence is the subsequence that is located first in time in the target sequence;

[0023] If so, it is determined that the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache;

[0024] If not, it is determined that there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache.

[0025] Optionally, the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache is obtained in the following manner:

[0026] Obtain the first feature data of the previous subsequence;

[0027] From the end of the first characteristic data of the previous subsequence, extract characteristic data of a preset length;

[0028] The feature data of the preset length is used as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

[0029] Optionally, the preset length is the number of short-term memory frames required for the first convolutional layer; the number of short-term memory frames required for the first convolutional layer is determined according to the size of the convolution kernel of the first convolutional layer and the void ratio of the convolution of the first convolutional layer.

[0030] Optionally, the step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for noise reduction to obtain the second feature data includes:

[0031] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain second feature data.

[0032] Optionally, the step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into a first convolutional layer of a neural network model for convolution operation and regularization processing to obtain the second feature data includes:

[0033] Obtaining a mean and a standard deviation of the feature data of each frame in the subsequence according to the first feature data of the subsequence and the corresponding short-term memory buffer data;

[0034] Normalizing the feature data of the same frame according to the mean and standard deviation of the feature data of each frame to obtain normalized feature data corresponding to each frame;

[0035] Obtaining a regularization transformation parameter value for a regularization operation;

[0036] According to the regularization transformation parameters and the normalized feature data corresponding to each frame, a convolution operation and a regularization process are performed to obtain second feature data.

[0037] Optionally, the subsequence includes at least one frame; and obtaining regularized transformation parameters for regularization operation includes: obtaining the same regularized transformation parameter value for regularization operation for each frame included in the subsequence.

[0038] Optionally, the subsequence includes at least one of the following:

[0039] A subsequence of an audio sequence;

[0040] a subsequence of a video sequence;

[0041] A subsequence of a text sequence.

[0042] The present application also provides a data processing device, comprising:

[0043] A first feature data obtaining unit, used to obtain first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0044] A short-term memory cache data acquisition unit, used to acquire short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is part of the first characteristic data of the previous subsequence of the subsequence in time order;

[0045] The second feature data obtaining unit is used to input the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolution layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0046] The present application also provides an electronic device, including:

[0047] Processor; and

[0048] The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0049] Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0050] Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order;

[0051] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0052] The present application also provides a storage device storing a program of a data processing method, wherein the program is executed by a processor to perform the following steps:

[0053] Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0054] Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order;

[0055] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0056] The present application also provides a method for data processing based on a neural network model, comprising:

[0057] Perform frame processing on the target sequence data to obtain the original data of the subsequence;

[0058] According to the original data of the subsequence, first feature data of the subsequence for inputting into the neural network model is obtained;

[0059] The first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data are input into the neural network model, and the denoised target sequence data is output; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order.

[0060] The present application also provides a device for data processing based on a neural network model, comprising:

[0061] The subsequence data obtaining unit performs frame processing on the target sequence data to obtain the original data of the subsequence;

[0062] A first feature data obtaining unit, used for obtaining first feature data of the subsequence for inputting into the neural network model according to the original data of the subsequence;

[0063] The denoised target sequence data output unit is used to input the first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data into the neural network model, and output the denoised target sequence data; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order.

[0064] The present application also provides an audio noise reduction method, comprising:

[0065] Acquire first feature data of an audio subsequence for inputting into a neural network; the audio subsequence is a data subset obtained by performing frame processing on an audio target sequence;

[0066] Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order;

[0067] The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0068] The present application also provides an audio noise reduction device, comprising:

[0069] Acquire first feature data of an audio subsequence for inputting into a neural network; the audio subsequence is a data subset obtained by performing frame processing on an audio target sequence;

[0070] Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order;

[0071] The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0072] The present application also provides a neural network model, comprising: at least one convolutional layer;

[0073] The convolution layer is used to obtain the first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data; and to perform noise reduction processing on the first feature data and the short-term memory cache data corresponding to the first feature data to generate second feature data; wherein the subsequence is a data subset obtained by performing frame processing on the target sequence; and the second feature data is the first feature data with part of the noise removed.

[0074] Compared with the prior art, this application has the following advantages:

[0075] The present application provides a data processing method, including: obtaining first feature data of a subsequence for input into a neural network model; the subsequence is a data subset obtained by frame processing of a target sequence; obtaining short-term memory cache data corresponding to the first feature data of the subsequence; the short-term memory cache data is partial data of the first feature data of a previous subsequence of the subsequence in time order; inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed. The method of the present application inputs the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for processing. Since the use of short-term memory cache data enhances the receptive field (receptive field, which refers to the amount of data that the algorithm can use. For image algorithms, the amount of data refers to the size of the pixel space range; for audio algorithms, the amount of data refers to the length of the time span), the input of the neural network model is enriched, and the fitting ability is improved, thereby reducing the burden of a single operation. In addition, the present application can perform frame processing on the target sequence (for example, an audio target sequence) to obtain data subsets of arbitrary length, take short-term memory for the first feature data of each data subset, and output the denoised second feature data, which will not cause delay problems and can adapt to various delay requirements, thereby solving the problems of slow operation and large delay of large models when using a neural network model to perform denoising on input data (for example, audio sequence data). BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1a It is a scene graph provided in the first embodiment of the present application.

[0077] Figure 1 It is a flowchart of a data processing method provided in the first embodiment of the present application.

[0078] Figure 2 This is a deduction schematic diagram of a specified frame number of 5 frames provided in the first embodiment of the present application.

[0079] Figure 3 This is a deduction schematic diagram of a specified frame number of 1 frame provided in the first embodiment of the present application.

[0080] Figure 4 This is another deduction schematic diagram with a specified frame number of 5 frames provided in the first embodiment of the present application.

[0081] Figure 5 It is a flow chart of an overall solution provided in the first embodiment of the present application.

[0082] Figure 6It is a schematic diagram of a data processing device provided in the second embodiment of the present application. DETAILED DESCRIPTION

[0083] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0084] The first embodiment of the present application provides a data processing method. Figures 1 to 5 Provide explanation.

[0085] In order to enable those skilled in the art to better understand the present application, a specific application scenario embodiment of the present application is first described in detail.

[0086] The image category recognition method provided in the first embodiment of the present application can be applied to the scene where the client and the server interact, such as Figure 1a When the user needs to perform noise reduction processing on the audio target sequence data, first, the client establishes a connection with the server, and then sends the audio target sequence data to the server through the client device; after obtaining the audio target sequence data, the server first performs frame processing to obtain a subsequence of the audio target sequence data, and then obtains the first feature data of the subsequence for inputting into the neural network model in the first feature data obtaining unit 101; the subsequence is a data subset obtained by performing frame processing on the target sequence; in the short-term memory cache data obtaining unit 102, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order; in the second feature data obtaining unit 103, the first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for noise reduction processing to obtain the second feature data; the second feature data is the first feature data with part of the noise removed, and after the server processes each subsequence of the audio target sequence using the above three units through multiple first convolution layers of the neural network model, the audio target sequence data with noise removed is obtained and provided to the client, and the client receives the audio target sequence data with noise removed.

[0087] like Figure 1 As shown, in step S101, the first feature data of the subsequence for inputting into the neural network model is obtained; the subsequence is a data subset obtained by performing frame processing on the target sequence.

[0088] The target sequence refers to a data format that appears and is arranged in sequence. A frame is the smallest unit of a sequence. For example, the target sequence includes the first frame, the second frame, the third frame, and the nth frame. The target sequence may include an audio sequence, a text sequence, an image sequence, and the like.

[0089] The subsequence refers to a data subset obtained after the target sequence is framed, that is, a shorter series of frame sequences generated after the target sequence is framed. Generally speaking, in actual use, several, dozens, or even hundreds of frames are often used as a subsequence and processed together. This method of treating N frames as a subsequence is called framing. In this application, the number of frames contained in the subsequence can be any number of frames.

[0090] The subsequence may include: a subsequence of an audio sequence, a subsequence of a video sequence, a subsequence of a text sequence, etc.

[0091] by Figure 2 For example, 2-1 is a subsequence containing 5 frames. 2-2 is also a subsequence containing 5 frames. Figure 3 As shown, 3-1 is a subsequence containing 1 frame.

[0092] The first feature data refers to the feature data corresponding to the subsequence to be input into the neural network. The first feature data is feature data containing noise to be input into the first convolutional layer. After the first feature data is input into the first convolutional layer, it is processed by the first convolutional layer to obtain second feature data with some noise removed.

[0093] like Figure 1 As shown, in step S102, short-term memory cache data corresponding to the first feature data of the subsequence is obtained; the short-term memory cache data is partial data of the first feature data of the previous subsequence of the subsequence in time order.

[0094] The short-term memory cache data refers to cached short-term memory data. The short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order.

[0095] For example, if the subsequence is the third subsequence in time order, the short-term memory cache data corresponding to the first characteristic data of the subsequence is part of the first characteristic data of the second subsequence in time order.

[0096] like Figure 1 As shown, in step S103, the first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0097] The second feature data refers to feature data obtained by removing part of the noise after the first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for denoising. The second feature data may be feature data obtained by performing convolution operation and regularization on the first feature data of the subsequence and the corresponding short-term memory cache data through the first convolution layer of the neural network model.

[0098] The first convolutional layer may refer to any convolutional layer of a neural network model.

[0099] When the first convolutional layer is the first convolutional layer of the neural network, the obtaining the first feature data of the subsequence includes:

[0100] Obtain target sequence data;

[0101] Performing frame processing on the target sequence data according to a specified number of frames to obtain original data of the subsequence;

[0102] The original data of the subsequence is subjected to feature conversion processing to obtain first feature data of the subsequence for input into a neural network model.

[0103] The first convolution layer refers to a convolution layer that processes the first feature data directly obtained from the original data of the subsequence. That is, the original data of the subsequence is feature processed to obtain the first feature data, and the convolution layer that processes the first feature data is the first convolution layer.

[0104] The target sequence data may refer to the sequence data that needs to be de-noised. If a sequence data containing noise needs to be de-noised, the sequence data may be determined as the target sequence data. For example, 3 seconds or 3 minutes of audio sequence data to be de-noised may be a target sequence data.

[0105] The specified number of frames can be any number of frames. For example, the specified number of frames is 1 frame or 5 frames. Figure 2 , specify the number of frames as 5 frames, divide the target sequence data into frames according to 5 frames, obtain the original data of the subsequence, and then perform feature conversion on the original data of the subsequence to obtain the first feature data of the subsequence (including the feature data of 5 frames).

[0106] The obtaining of target sequence data refers to obtaining sequence data from which noise is to be removed. Specifically, the data processing method of the present application is run on the server side, and the user can send the target sequence data to the server side through the user side, and the server side obtains the target sequence data.

[0107] The frame processing of the target sequence data according to the specified frame number to obtain the original data of the subsequence may refer to dividing the target sequence data into multiple subsets according to the specified frame number to obtain the original data of the subsequence. For example, the target sequence data includes 30 frames of data, the specified frame number is 5 frames, and the frame processing according to the specified frame number can obtain the original data of 6 subsequences.

[0108] The feature conversion processing is performed on the original data of the subsequence to obtain the first feature data of the subsequence for inputting into the neural network model, which refers to converting the original data of the subsequence into the input data type of the neural network model and inputting into the neural network model.

[0109] When the first convolutional layer is not the first convolutional layer of the neural network, obtaining the first feature data of the subsequence includes:

[0110] Obtain feature data output by a convolutional layer above the first convolutional layer for a subsequence;

[0111] The feature data output for the subsequence is used as the first feature data of the subsequence for input into the neural network model.

[0112] For example, if the first convolutional layer is the second convolutional layer (the second convolutional layer refers to a convolutional layer that takes the output of the first convolutional layer as input), then the first feature data of the subsequence refers to the feature data output by the previous convolutional layer of the first convolutional layer, i.e., the first convolutional layer, for the subsequence.

[0113] The obtaining of short-term memory cache data corresponding to the first feature data of the subsequence includes:

[0114] Determine whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache;

[0115] If it does not exist, data filling processing is performed on the first characteristic data of the subsequence to obtain short-term memory cache data corresponding to the first characteristic data of the subsequence;

[0116] When it exists, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained from the cache.

[0117] The step of determining whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache includes:

[0118] Determine whether the subsequence corresponding to the first characteristic data of the subsequence is the first subsequence;

[0119] If so, it is determined that the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache;

[0120] If not, it is determined that there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache.

[0121] The first subsequence refers to the subsequence that is located earliest in time in the target sequence.

[0122] Since when the subsequence corresponding to the first feature data of a subsequence is the first subsequence, the first subsequence does not have a previous subsequence in time order, and therefore the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache, data filling processing is performed on the first feature data of the subsequence; when the subsequence is not the first subsequence in time order, the short-term memory cache data corresponding to the first feature data of the subsequence is stored in the cache, and the short-term memory cache data corresponding to the first feature data of the subsequence can be directly obtained from the cache.

[0123] by Figure 2 For example, 2-1 is the first subsequence, and there is no previous subsequence. Therefore, there is no short-term memory cache data corresponding to the first feature data of the subsequence in the cache. Therefore, the first feature data of the subsequence is filled with data (the first two frames are filled with 0). The data containing 0 in the two frames can be considered as the short-term memory cache data of the first subsequence. 2-2 is the second subsequence, not the first subsequence. The first subsequence is its previous subsequence. Therefore, there is short-term memory cache data corresponding to the first feature data of the subsequence in the cache.

[0124] The short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache can be obtained in the following manner:

[0125] Obtain the first feature data of the previous subsequence;

[0126] From the end of the first characteristic data of the previous subsequence, extract characteristic data of a preset length;

[0127] The feature data of the preset length is used as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

[0128] like Figure 2 , the feature data in the dotted box is the feature data of the preset length intercepted from the previous subsequence, and the feature data of the preset length intercepted is used as the short-term memory cache data corresponding to the first feature data of the subsequence. For example, the feature data of the last two frames of 2-1 is the short-term memory cache data corresponding to the feature data of 2-2 (the first feature data of the subsequence).

[0129] The preset length may be the number of short-term memory frames required for the first convolutional layer; the number of short-term memory frames required for the first convolutional layer is determined according to the size of the convolution kernel of the first convolutional layer and the void ratio of the convolution of the first convolutional layer.

[0130] Specifically, the number of short-term memory frames required for the l-th convolutional layer can be determined using the following formula:

[0131] f l =(k l -1)*d l ;

[0132] Among them, k l : kernel size of the lth convolution layer (kernel size, for sequence data, it refers specifically to the time domain width of 1D convolution);

[0133] d l : dilation rate of the lth convolution layer.

[0134] dilation: The distance between valid elements in the CNN convolution kernel matrix. When dilation = 1, the convolution kernel is a matrix / tensor in the common sense. When dilation = 2, a space is inserted between two convolution kernel parameters.

[0135] The number of floating points required for the lth convolution layer is N l :

[0136] N l =(k l -1)*d l* c l ;

[0137] c l is the number of input channels of the l-th convolution layer.

[0138] The number of short-term memory frames F required for the entire model:

[0139]

[0140] The number of short-term memory floating-point numbers N required for the entire model:

[0141]

[0142] For example, if the size of the convolution kernel of the first convolution layer is 3 and the dilution rate of the convolution of the first convolution layer is 1, the number of short-term memory frames required for the first convolution layer is: (3-1)*1=2 frames. That is, the feature data of the last two frames of the first feature data of the subsequence is intercepted (for a time series, the feature data of the last two frames refers to the two frames of data with the latest arrival time) as the short-term memory cache data of the next subsequence.

[0143] like Figure 3As shown, the size of the convolution kernel of the first convolution layer is 3, and the hole rate of the convolution of the first convolution layer is 1, then the number of short-term memory frames required by the first convolution layer is: (3-1)*1=2 frames. The subsequence contains 1 frame of data. When the subsequence is the first subsequence, two frames of 0 are added as short-term memory cache data. When the subsequence is the second subsequence, since the first subsequence only contains one frame of feature data, in addition to using this frame of feature data as short-term memory cache data, it is also necessary to add a frame of data containing 0 in front to combine into two frames of short-term memory cache data. Two frames of feature data can be taken from the end of the second subsequence containing short-term memory frames as the short-term memory cache data of the third subsequence.

[0144] The step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for processing to obtain the second feature data comprises:

[0145] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain second feature data.

[0146] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain the second feature data, including:

[0147] Obtaining a mean and a standard deviation of the feature data of each frame in the subsequence according to the first feature data of the subsequence and the corresponding short-term memory buffer data;

[0148] Normalizing the feature data of the same frame according to the mean and standard deviation of the feature data of each frame to obtain normalized feature data corresponding to each frame;

[0149] Obtaining a regularization transformation parameter value for a regularization operation;

[0150] According to the regularization transformation parameters and the normalized feature data corresponding to each frame, a convolution operation and a regularization process are performed to obtain second feature data.

[0151] The mean value of the feature data of each frame refers to a result obtained by summing the feature data contained in each frame and dividing it by the number of elements contained in the frame.

[0152] The standard deviation of the feature data of each frame refers to the square root of the variance of the feature data of each frame.

[0153] The variance of the feature data of each frame refers to the average of the sum of the squares of the differences between each feature data of each frame and the mean value of the feature data of each frame.

[0154] Specifically, according to the mean and standard deviation of the feature data of each frame, the feature data of the same frame is normalized to obtain the normalized feature data corresponding to each frame, as shown below: for each frame of input data (feature data of each frame) x, use its mean mu and standard deviation stddev to normalize x to x'=(x-mu) / stddev.

[0155] The regularization transformation parameter value used for the regularization operation refers to two variable parameters alpha and beta, where alpha or beta can be a scalar or a vector.

[0156] Specifically, according to the regularization transformation parameters and the normalized feature data corresponding to each frame, convolution operation and regularization processing are performed to obtain the second feature data, which can be obtained using the following formula: Second feature data: y=x'*alpha+beta.

[0157] The subsequence includes at least one frame; the obtaining of the regularization transformation parameter for the regularization operation includes: obtaining the same regularization transformation parameter value for the regularization operation for each frame included in the subsequence. Figure 2 , the same layer uses the same regularization transformation parameter values ​​α and β for regularization operation for each frame of input data, and different layers use different regularization transformation parameter values ​​α and β.

[0158] This application can perform frame self-regularization, that is, calculate the mean and standard deviation for each frame, and then perform regularization operation on each frame. The segmented deduction has the same effect as the whole segment deduction. This application can process a frame of data through frame self-regularization. When a frame of data is obtained, it can be processed in time without waiting for multiple frames of data to be processed together, which improves the real-time processing.

[0159] The present application may also use regularization with subsequence splitting invariance. Figure 4 As shown, it is another deduction schematic diagram of specifying the number of frames as 5 frames provided in the first embodiment of the present application.

[0160]

[0161]

[0162] S is an artificially specified subsequence length.

[0163] When changing the length of the subsequence deduced by the neural network, follow the following rules:

[0164] Constructing a circular queue generation function Generate a new infinite array:

[0165]

[0166] The corresponding infinite array alpha becomes:

[0167]

[0168] Beta and so on.

[0169] In the regularization of each layer, when alpha beta is taken out and used, the subscript is pressed to match the data frame number of this layer.

[0170] Only the part of each layer of data that does not have short-term memory added to it requires alpha beta.

[0171] Short-term memory frames do not consume alpha beta and are not counted in the data sequence number.

[0172] The following are the effects that this application can achieve compared with the prior art solutions:

[0173] 1. Short frame input solution:

[0174] Disadvantages: limited receptive field resulting in poor results, long delays, waste of computing resources due to discarding some outputs, and high energy consumption.

[0175] The present application inputs the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for processing. Since the use of short-term memory cache data enhances the receptive field (receptive field refers to the amount of data that the algorithm can use. For image algorithms, the amount of data refers to the size of the pixel space range; for audio algorithms, the amount of data refers to the length of the time span), it enriches the input of the neural network model and improves the fitting ability, thereby reducing the burden of a single operation.

[0176] 2. Single frame input solution:

[0177] Disadvantages: The receptive field is highly limited, the fitting ability is poor, and there is a waste of computing resources due to zero padding.

[0178] The present application uses the first feature data of the subsequence and the corresponding short-term memory cache data to be input into the first convolutional layer of the neural network model for processing. The use of short-term memory cache data enhances the receptive field, enriches the input of the neural network model, avoids padding with zeros, and does not waste computing due to padding with zeros.

[0179] 3. Long-segment input solution

[0180] Disadvantages: The disadvantages of the long-segment input solution are high latency and limited application scenarios.

[0181] The present application can perform frame processing on a target sequence (for example, an audio target sequence) to obtain data subsets of arbitrary length, take short-term memory of the first feature data of each data subset, and output the second feature data after noise reduction, without causing delay problems, and can adapt to various delay requirements.

[0182] 4. Offline solution for the entire input

[0183] Disadvantages: The disadvantages of the entire input are: high latency and extremely limited application scenarios.

[0184] The present application can perform frame processing on a target sequence (for example, an audio target sequence) to obtain data subsets of arbitrary length, take short-term memory of the first feature data of each data subset, and output the second feature data after noise reduction, without causing delay problems, and can adapt to various delay requirements.

[0185] This application can make the algorithm effect the same as the whole / long segment solution, provide the same good fitting ability, and at the same time have the low latency of single frame input / short frame input and lower computing power consumption than short frame input.

[0186] Figure 5 A flowchart of an overall solution provided for the first embodiment of the present application.

[0187] like Figure 5 As shown, in step S501, any length sequence data is input, for example, 3 seconds of audio sequence data is input.

[0188] like Figure 5 As shown, in step S502, the frame is divided according to the specified frame number. For example, the specified frame number is 5 frames, and sub-sequence data is obtained after the frame division.

[0189] like Figure 5 As shown, in step S503, the subsequence data is preprocessed and feature converted.

[0190] like Figure 5 As shown, in step S504, it is determined whether there is a short-term memory cache, if so, step S505 is executed, if not, step S506 is executed.

[0191] like Figure 5 As shown, in step S505, the input short-term memory cache is used.

[0192] like Figure 5 As shown, in step S506, the short-term memory cache is initialized using 0 or other padding means.

[0193] like Figure 5 As shown, in step S507, a convolutional neural network with single-frame regularization is used for computational deduction.

[0194] like Figure 5 As shown, in step S508, the result is output.

[0195] like Figure 5 As shown, in step S509, it is determined whether the whole sequence has been processed. If not, step S510 is executed to integrate the features into a memory cache; if so, the process ends.

[0196] like Figure 5 As shown, in step S510, intermediate features of each layer are extracted.

[0197] It should be noted that the data processing method provided in the first embodiment of the present application can be provided to users in the form of a plug-in. After the user downloads the plug-in, the plug-in is used to process the target sequence data with noise, and the sequence data with noise removed is generated after processing.

[0198] Corresponding to the data processing method provided in the first embodiment of the present application, the second embodiment of the present application also provides a data processing device.

[0199] Please refer to Figure 6 , the data processing device comprises:

[0200] A first feature data obtaining unit 601 is used to obtain first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0201] The short-term memory cache data obtaining unit 602 is used to obtain the short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is part of the first characteristic data of the previous subsequence of the subsequence in time order;

[0202] The second feature data obtaining unit 603 is used to input the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolution layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0203] Optionally, the first convolution layer is the first convolution layer of a neural network model, and the first feature data obtaining unit is specifically used for:

[0204] Obtain target sequence data;

[0205] Performing frame processing on the target sequence data according to a specified number of frames to obtain original data of the subsequence;

[0206] The original data of the subsequence is subjected to feature conversion processing to obtain first feature data of the subsequence for input into a neural network model.

[0207] Optionally, the first convolution layer is not the first convolution layer of the neural network model, and the first feature data obtaining unit is specifically used for:

[0208] Obtain feature data output by a convolutional layer above the first convolutional layer for a subsequence;

[0209] The feature data output for the subsequence is used as the first feature data of the subsequence for input into the neural network model.

[0210] Optionally, the short-term memory cache data obtaining unit is specifically used for:

[0211] Determine whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache;

[0212] If it does not exist, data filling processing is performed on the first characteristic data of the subsequence to obtain short-term memory cache data corresponding to the first characteristic data of the subsequence;

[0213] When it exists, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained from the cache.

[0214] Optionally, the short-term memory cache data obtaining unit is specifically used for:

[0215] Determine whether the subsequence corresponding to the first characteristic data of the subsequence is the first subsequence; the first subsequence is the subsequence that is located first in time in the target sequence;

[0216] If so, it is determined that the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache;

[0217] If not, it is determined that there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache.

[0218] Optionally, the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache is obtained in the following manner:

[0219] Obtain the first feature data of the previous subsequence;

[0220] From the end of the first characteristic data of the previous subsequence, extract characteristic data of a preset length;

[0221] The feature data of the preset length is used as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

[0222] Optionally, the preset length is the number of short-term memory frames required for the first convolutional layer; the number of short-term memory frames required for the first convolutional layer is determined according to the size of the convolution kernel of the first convolutional layer and the void ratio of the convolution of the first convolutional layer.

[0223] Optionally, the second feature data obtaining unit is specifically used for:

[0224] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain second feature data.

[0225] Optionally, the second feature data obtaining unit is specifically used for:

[0226] Obtaining a mean and a standard deviation of the feature data of each frame in the subsequence according to the first feature data of the subsequence and the corresponding short-term memory buffer data;

[0227] Normalizing the feature data of the same frame according to the mean and standard deviation of the feature data of each frame to obtain normalized feature data corresponding to each frame;

[0228] Obtaining a regularization transformation parameter value for a regularization operation;

[0229] According to the regularization transformation parameters and the normalized feature data corresponding to each frame, a convolution operation and a regularization process are performed to obtain second feature data.

[0230] Optionally, the subsequence includes at least one frame; and the second feature data obtaining unit is specifically used to obtain the same regularization transformation parameter value for regularization operation for each frame included in the subsequence.

[0231] Optionally, the subsequence includes at least one of the following:

[0232] A subsequence of an audio sequence;

[0233] a subsequence of a video sequence;

[0234] A subsequence of a text sequence.

[0235] It should be noted that for the detailed description of the device provided in the second embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0236] Corresponding to the data processing method provided in the first embodiment of the present application, the third embodiment of the present application also provides an electronic device.

[0237] The electronic device comprises:

[0238] Processor; and

[0239] The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed:

[0240] Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0241] Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order;

[0242] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0243] Optionally, the first convolution layer is the first convolution layer of a neural network model, and the obtaining of first feature data of the subsequence for inputting into the neural network model includes:

[0244] Obtain target sequence data;

[0245] Performing frame processing on the target sequence data according to a specified number of frames to obtain original data of the subsequence;

[0246] The original data of the subsequence is subjected to feature conversion processing to obtain first feature data of the subsequence for input into a neural network model.

[0247] Optionally, the first convolution layer is not the first convolution layer of the neural network model, and the obtaining of the first feature data of the subsequence for inputting into the neural network model includes:

[0248] Obtain feature data output by a convolutional layer above the first convolutional layer for a subsequence;

[0249] The feature data output for the subsequence is used as the first feature data of the subsequence for input into the neural network model.

[0250] Optionally, the obtaining of short-term memory cache data corresponding to the first feature data of the subsequence includes:

[0251] Determine whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache;

[0252] If it does not exist, data filling processing is performed on the first characteristic data of the subsequence to obtain short-term memory cache data corresponding to the first characteristic data of the subsequence;

[0253] When it exists, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained from the cache.

[0254] Optionally, the determining whether there is short-term memory cache data corresponding to the first feature data of the subsequence in the cache includes:

[0255] Determine whether the subsequence corresponding to the first characteristic data of the subsequence is the first subsequence; the first subsequence is the subsequence that is located first in time in the target sequence;

[0256] If so, it is determined that the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache;

[0257] If not, it is determined that there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache.

[0258] Optionally, the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache is obtained in the following manner:

[0259] Obtain the first feature data of the previous subsequence;

[0260] From the end of the first characteristic data of the previous subsequence, extract characteristic data of a preset length;

[0261] The feature data of the preset length is used as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

[0262] Optionally, the preset length is the number of short-term memory frames required for the first convolutional layer; the number of short-term memory frames required for the first convolutional layer is determined according to the size of the convolution kernel of the first convolutional layer and the void ratio of the convolution of the first convolutional layer.

[0263] Optionally, the step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for noise reduction to obtain the second feature data includes:

[0264] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain second feature data.

[0265] Optionally, the step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into a first convolutional layer of a neural network model for convolution operation and regularization processing to obtain the second feature data includes:

[0266] Obtaining a mean and a standard deviation of the feature data of each frame in the subsequence according to the first feature data of the subsequence and the corresponding short-term memory buffer data;

[0267] Normalizing the feature data of the same frame according to the mean and standard deviation of the feature data of each frame to obtain normalized feature data corresponding to each frame;

[0268] Obtaining a regularization transformation parameter value for a regularization operation;

[0269] According to the regularization transformation parameters and the normalized feature data corresponding to each frame, a convolution operation and a regularization process are performed to obtain second feature data.

[0270] Optionally, the subsequence includes at least one frame; and obtaining regularized transformation parameters for regularization operation includes: obtaining the same regularized transformation parameter value for regularization operation for each frame included in the subsequence.

[0271] Optionally, the subsequence includes at least one of the following:

[0272] A subsequence of an audio sequence;

[0273] a subsequence of a video sequence;

[0274] A subsequence of a text sequence.

[0275] It should be noted that for the detailed description of the electronic device provided in the third embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0276] Corresponding to the data processing method provided in the first embodiment of the present application, the fourth embodiment of the present application further provides a storage device. A program of the data processing method is stored, and the program is executed by a processor to perform the following steps:

[0277] Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence;

[0278] Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order;

[0279] The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0280] It should be noted that for the detailed description of the storage device provided in the fourth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0281] The fifth embodiment of the present application provides a method for data processing based on a neural network model, including:

[0282] Perform frame processing on the target sequence data to obtain the original data of the subsequence;

[0283] According to the original data of the subsequence, first feature data of the subsequence for inputting into the neural network model is obtained;

[0284] The first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data are input into the neural network model, and the denoised target sequence data is output; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order.

[0285] It should be noted that the detailed description of the fifth embodiment of the present application can refer to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0286] A sixth embodiment of the present application provides a device for performing data processing based on a neural network model, including:

[0287] The subsequence data obtaining unit performs frame processing on the target sequence data to obtain the original data of the subsequence;

[0288] A first feature data obtaining unit, used for obtaining first feature data of the subsequence for inputting into the neural network model according to the original data of the subsequence;

[0289] The denoised target sequence data output unit is used to input the first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data into the neural network model, and output the denoised target sequence data; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order.

[0290] It should be noted that for the detailed description of the sixth embodiment of the present application, reference may be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0291] A seventh embodiment of the present application provides an audio noise reduction method, including:

[0292] Acquire first feature data of an audio subsequence for inputting into a neural network; the audio subsequence is a data subset obtained by performing frame processing on an audio target sequence;

[0293] Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order;

[0294] The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0295] It should be noted that for the detailed description of the seventh embodiment of the present application, reference may be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0296] An eighth embodiment of the present application provides an audio noise reduction device, comprising:

[0297] Acquire first feature data of an audio subsequence for inputting into a neural network; the audio subsequence is a data subset obtained by performing frame processing on an audio target sequence;

[0298] Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order;

[0299] The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

[0300] It should be noted that for the detailed description of the eighth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0301] A ninth embodiment of the present application provides a neural network model, including: at least one convolutional layer;

[0302] The convolution layer is used to obtain the first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data; and to perform noise reduction processing on the first feature data and the short-term memory cache data corresponding to the first feature data to generate second feature data; wherein the subsequence is a data subset obtained by performing frame processing on the target sequence; and the second feature data is the first feature data with part of the noise removed.

[0303] Based on user feedback, this solution can optimize and intervene in the algorithm in two ways to form a closed-loop improvement iterative process:

[0304] 1. Feedback on the algorithm effect can be achieved by changing the training objective function to update the output effect. With the help of the features and cache utilization methods described in this solution, the algorithm improvement can use continuous sequence data to efficiently retrain and update the algorithm quality.

[0305] 2. For feedback on the algorithm delay speed, the algorithm delay can be adjusted arbitrarily to match user needs by changing the framing parameters of this method.

[0306] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0307] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0308] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0309] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0310] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. A data processing method, characterized in that: include: Obtaining first feature data of the subsequence for inputting into the neural network model; The subsequence is a data subset obtained by performing frame processing on the target sequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the subsequence includes: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache; The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

2. The method according to claim 1, characterized in that The first convolution layer is the first convolution layer of the neural network model, and the obtaining of the first feature data of the subsequence for inputting into the neural network model includes: Obtain target sequence data; Performing frame processing on the target sequence data according to a specified number of frames to obtain original data of the subsequence; The original data of the subsequence is subjected to feature conversion processing to obtain first feature data of the subsequence for input into a neural network model.

3. The method according to claim 1, characterized in that The first convolutional layer is not the first convolutional layer of the neural network model, and the obtaining of the first feature data of the subsequence for inputting into the neural network model includes: Obtain feature data output by a convolutional layer above the first convolutional layer for a subsequence; The feature data output for the subsequence is used as the first feature data of the subsequence for input into the neural network model.

4. The method according to claim 1, characterized in that: The step of obtaining the short-term memory cache data corresponding to the first feature data of the subsequence further includes: Determine whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache; If it does not exist, data filling processing is performed on the first characteristic data of the subsequence to obtain short-term memory cache data corresponding to the first characteristic data of the subsequence; When it exists, the short-term memory cache data corresponding to the first feature data of the subsequence is obtained from the cache.

5. The method according to claim 4, characterized in that The step of determining whether there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache includes: Determine whether the subsequence corresponding to the first characteristic data of the subsequence is the first subsequence; the first subsequence is the subsequence that is located first in time in the target sequence; If so, it is determined that the short-term memory cache data corresponding to the first feature data of the subsequence does not exist in the cache; If not, it is determined that there is short-term memory cache data corresponding to the first characteristic data of the subsequence in the cache.

6. The method according to claim 1, characterized in that The preset length is the number of short-term memory frames required by the first convolutional layer; the number of short-term memory frames required by the first convolutional layer is determined according to the size of the convolution kernel of the first convolutional layer and the void ratio of the convolution of the first convolutional layer.

7. The method according to claim 1, characterized in that The step of inputting the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolutional layer of the neural network model for noise reduction to obtain the second feature data comprises: The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain second feature data.

8. The method according to claim 7, characterized in that The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolution layer of the neural network model for convolution operation and regularization processing to obtain the second feature data, including: Obtaining a mean and a standard deviation of the feature data of each frame in the subsequence according to the first feature data of the subsequence and the corresponding short-term memory buffer data; Normalizing the feature data of the same frame according to the mean and standard deviation of the feature data of each frame to obtain normalized feature data corresponding to each frame; Obtaining a regularization transformation parameter value for a regularization operation; According to the regularization transformation parameters and the normalized feature data corresponding to each frame, a convolution operation and a regularization process are performed to obtain second feature data.

9. The method according to claim 8, characterized in that The subsequence includes at least one frame; and obtaining the regularization transformation parameter for the regularization operation includes: obtaining the same regularization transformation parameter value for the regularization operation for each frame included in the subsequence.

10. A data processing device, characterized in that: include: A first feature data obtaining unit, used to obtain first feature data of a subsequence for input into a neural network model; The subsequence is a data subset obtained by performing frame processing on the target sequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; A short-term memory cache data acquisition unit, used to acquire short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is part of the first characteristic data of the previous subsequence of the subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the subsequence includes: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache; The second feature data obtaining unit is used to input the first feature data of the subsequence and the corresponding short-term memory cache data into the first convolution layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

11. An electronic device, comprising: processor; as well as The memory is used to store a program of the data processing method. After the device is powered on and the program of the data processing method is run by the processor, the following steps are performed: Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the subsequence includes: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache; The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

12. A storage device storing a program of a data processing method, wherein the program is executed by a processor to perform the following steps: Obtaining first feature data of a subsequence for inputting into a neural network model; the subsequence is a data subset obtained by performing frame processing on a target sequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; Obtaining short-term memory cache data corresponding to the first characteristic data of the subsequence; the short-term memory cache data is partial data of the first characteristic data of the previous subsequence of the subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the subsequence includes: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache; The first feature data of the subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network model for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

13. A method for data processing based on a neural network model, characterized in that: include: Performing frame processing on the target sequence data to obtain original data of a subsequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; Subsequence of a video sequence; Subsequence of a text sequence; According to the original data of the subsequence, first feature data of the subsequence for inputting into the neural network model is obtained; The first feature data of a subsequence and the short-term memory cache data corresponding to the first feature data are input into a neural network model, and the denoised target sequence data is output; the short-term memory cache data is part of the first feature data of the previous subsequence of the subsequence in time order; the short-term memory cache data corresponding to the first feature data of the subsequence is obtained in the following manner: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

14. A device for data processing based on a neural network model, characterized in that: include: The subsequence data obtaining unit performs frame processing on the target sequence data to obtain the original data of the subsequence; the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; A first feature data obtaining unit, used for obtaining first feature data of the subsequence for inputting into the neural network model according to the original data of the subsequence; The denoised target sequence data output unit is used to input the first feature data of a subsequence and the short-term memory cache data corresponding to the first feature data into a neural network model, and output the denoised target sequence data; the short-term memory cache data corresponding to the first feature data of the subsequence is obtained in the following manner: obtaining the first feature data of the previous subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous subsequence; using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache; the short-term memory cache data is partial data of the first feature data of the previous subsequence of the subsequence in time order.

15. An audio noise reduction method, characterized in that: include: Obtaining first feature data of the audio subsequence for input into the neural network; The audio subsequence is a data subset obtained by performing frame processing on the audio target sequence; Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the audio subsequence includes: obtaining the first feature data of the previous audio subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous audio subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the audio subsequence existing in the cache; The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

16. An audio noise reduction device, characterized in that: include: Obtaining first feature data of the audio subsequence for input into the neural network; The audio subsequence is a data subset obtained by performing frame processing on the audio target sequence; Obtaining short-term memory cache data corresponding to the first feature data of the audio subsequence; the short-term memory cache data is partial data of the first feature data of the previous audio subsequence of the audio subsequence in time order; The obtaining of the short-term memory cache data corresponding to the first feature data of the audio subsequence includes: obtaining the first feature data of the previous audio subsequence; intercepting feature data of a preset length from the end of the first feature data of the previous audio subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the audio subsequence existing in the cache; The first feature data of the audio subsequence and the corresponding short-term memory cache data are input into the first convolutional layer of the neural network for noise reduction processing to obtain second feature data; the second feature data is the first feature data with part of the noise removed.

17. A neural network model, characterized in that: include: At least one convolutional layer; The convolution layer is used to obtain the first feature data of the subsequence and the short-term memory cache data corresponding to the first feature data; and to perform noise reduction processing on the first feature data and the short-term memory cache data corresponding to the first feature data to generate second feature data; wherein the subsequence is a data subset obtained by frame processing of the target sequence; the second feature data is the first feature data with part of the noise removed; wherein the subsequence includes at least one of the following: a subsequence of an audio sequence; a subsequence of a video sequence; a subsequence of a text sequence; the short-term memory cache data corresponding to the first feature data of the subsequence is obtained in the following manner: obtaining the first feature data of the previous subsequence; cutting feature data of a preset length from the end of the first feature data of the previous subsequence; and using the feature data of the preset length as the short-term memory cache data corresponding to the first feature data of the subsequence existing in the cache.

Images