Method for processing signal and related apparatus

Abstract

The present application provides a method for processing a signal and a related apparatus. The method comprises: acquiring N quantized audio features of an original non-speech audio, wherein N is a positive integer greater than or equal to 1; determining a first predicted silence audio feature on the basis of the N quantized audio features and a first neural network; and on the basis of the first predicted silence audio feature and a second neural network, determining a first non-speech audio corresponding to the original non-speech audio. According to the technical solution, more natural noise can be generated.

Description

Method of processing a signal and related apparatus

[0001] This application claims priority to the Russian patent application No. 2024129151, filed on October 01, 2024, with the Russian Federal Service for Intellectual Property, and having the title “Method of processing a signal and related apparatus”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of signal processing, and more particularly, to a method of processing a signal and related apparatus. BACKGROUND

[0003] In voice communication, although the speech content of the speaker is the main information, if the environmental background noise is not transmitted, a completely silent state will occur in the communication process, which may make the user mistakenly think that the communication is interrupted or has a fault. Transmitting certain environmental background noise can make the user still feel the continuity of the communication during the non-speech period. In addition, the environmental background noise can also provide some information about the environment of the sender to the receiver. For example, if there is the sound of a vehicle driving in the background, it can be roughly judged that the other party is probably on the street outdoors. Therefore, transmitting background noise in voice communication helps to improve the user experience.

[0004] However, if the environmental background noise is normally encoded and transmitted to the receiving end through the link, unnecessary communication bandwidth will be occupied. If the environmental background noise can be transmitted discontinuously, and the non-speech period audio is approximately recovered at the receiving end using discontinuous frames, the communication link pressure will be greatly reduced. Therefore, how to transmit environmental background noise is a problem that the industry is concerned about. SUMMARY

[0005] The present application provides a method of processing a signal and related apparatus, which can generate more natural noise.

[0006] The first aspect of the present application provides a method of processing a signal. The method comprises: obtaining N quantized audio features of an original non-speech audio, N being a positive integer greater than or equal to 1; determining a first predicted silence audio feature according to the N quantized audio features and a first neural network; determining a first non-speech audio corresponding to the original non-speech audio according to the first predicted silence audio feature and a second neural network.

[0007] The technical solution of the present application proposes a noise generation scheme based on a neural network (i.e., the first non-speech audio). Compared with the traditional scheme, the noise generated by the neural network is more natural. In addition, the above technical solution can predict the subsequent silence audio feature based on the received quantized audio feature, and the subsequent silence audio feature is predicted using a neural network, which makes the noise generated by the neural network more natural.

[0008] With reference to the first aspect, in a possible implementation form of the first aspect, the first neural network is an autoregressive-based neural network.

[0009] With reference to the first aspect, in a possible implementation form of the first aspect, N is a positive integer less than or equal to 3.

[0010] The greater the value of N is, the more silence information needs to be transmitted through the link, and the greater the communication overhead is. Therefore, N less than or equal to 3 can save the communication overhead.

[0011] With reference to the first aspect, in a possible implementation form of the first aspect, the method further includes: determining a second predicted silence audio feature according to the N reference audio features and the first neural network, wherein, in the case of N equaling 1, the N reference audio features are the first predicted silence audio feature, and in the case of N being a positive integer greater than 1, the N reference audio features include the first predicted silence audio feature and the last N-1 quantized audio features in the N quantized audio features; determining a second non-speech audio corresponding to the original non-speech audio according to the second predicted silence audio feature and the second neural network.

[0012] With reference to the first aspect, in a possible implementation form of the first aspect, the obtaining of the N quantized audio features includes: receiving a code stream through a low-code-rate audio communication channel, the low-code-rate audio communication channel at least including a satellite communication channel, the code stream containing N audio features extracted from the original non-speech audio and quantized and encoded; and decoding and dequantizing the code stream to obtain the N quantized audio features, the N quantized audio features corresponding to the N audio features one by one.

[0013] The low-code-rate audio communication channel has limited communication capability, and therefore it is desirable to further compress the non-speech audio (i.e., noise) with a relatively small amount of information contained. The above technical solution can reduce the silence audio features that need to be transmitted through the way of predicting silence audio features. In this way, the occupation of the low-code-rate audio communication channel can be reduced.

[0014] With reference to the first aspect, in a possible implementation form of the first aspect, the N audio features are a mel-frequency spectrogram of the original non-speech audio or a mel-frequency cepstral coefficient of the original non-speech audio, or the N audio features are features extracted by a third neural network from the original non-speech audio.

[0015] Optionally, any two or all of the first neural network, the second neural network and the third neural network can be jointly trained.

[0016] The audio features in the technical solution are extracted based on a mel spectrum or a neural network. The mel spectrum or the neural network can extract audio features that are better perceived by humans. Therefore, the predicted audio features predicted based on the quantized audio features are also better perceived by humans. In this way, the effect and naturalness of the noise generated by the neural network can be more in line with the hearing habits of human ears.

[0017] With reference to the first aspect, in a possible implementation form of the first aspect, the method further includes: determining N non-speech audios according to the N quantized audio features and the second neural network, wherein the reconstructed audio of the original non-speech audio includes the N non-speech audios and the first non-speech audio.

[0018] In a second aspect, an electronic device is provided, which includes units for implementing the first aspect or any possible implementation form of the first aspect.

[0019] In a third aspect, an electronic device is provided, which includes a processor configured to couple with a memory, read and execute instructions and / or program codes in the memory to perform the first aspect or any possible implementation form of the first aspect.

[0020] In a fourth aspect, a chip system is provided, which includes a logic circuit configured to couple with an input / output interface, transmit data through the input / output interface to perform the first aspect or any possible implementation form of the first aspect.

[0021] In a fifth aspect, a computer readable storage medium is provided, which stores program codes, and when the program codes stored in the computer readable storage medium are run on an electronic device, the electronic device performs the first aspect or any possible implementation form of the first aspect.

[0022] In a sixth aspect, a computer program product is provided, which includes computer program codes, and when the computer program codes are run on an electronic device, the electronic device performs the first aspect or any possible implementation form of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0023] FIG. 1 is a schematic diagram of a speech communication encoding process;

[0024] FIG. 2 is a schematic diagram of a satellite call system;

[0025] FIG. 3 is a process schematic diagram of a method for processing a signal according to an embodiment of the present application;

[0026] FIG. 4 is a schematic flowchart of a method for processing a signal according to an embodiment of the present application;

[0027] FIG. 5 is a schematic diagram of a process of extracting audio features;

[0028] FIG. 6 is a schematic diagram of a system architecture according to an embodiment of the present application;

[0029] FIG. 7 is a schematic diagram of a convolutional neural network;

[0030] FIG. 8 is a schematic diagram of a chip hardware structure according to an embodiment of the present application;

[0031] FIG. 9 is a schematic flowchart of another method of processing a signal according to an embodiment of the present application;

[0032] FIG. 10 is a schematic diagram of another system architecture according to an embodiment of the present application;

[0033] FIG. 11 is a schematic structural block diagram of an electronic device according to an embodiment of the present application;

[0034] FIG. 12 is a schematic structural block diagram of another electronic device according to an embodiment of the present application;

[0035] FIG. 13 is a schematic structural block diagram of still another electronic device according to an embodiment of the present application;

[0036] FIG. 14 is a schematic diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0038] FIG. 1 is a schematic diagram of a speech communication encoding process.

[0039] As shown in FIG. 1, a voice activity detection (VAD) module can divide an audio signal into non-silence segments and silence segments. For example, the non-silence segments and the silence segments can be determined based on the energy, the zero-crossing rate, the spectral features, etc. of the audio signal. For the non-silence segments, a continuous encoding manner can be used to encode the non-silence segments, obtaining non-silence segment signals (which can also be referred to as non-silence bitstreams). For the silence segments, a discontinuous encoding manner can be used to encode the silence segments, obtaining silence segment signals (which can also be referred to as silence bitstreams).

[0040] The non-silence segment signals and the silence segment signals are transmitted to a receiving end through a communication link. The receiving end continuously decodes the received non-silence segment signals, obtaining the audio signals of the non-silence segments. A comfort noise generation module obtains the received silence segment signals, and restores the audio signals of the silence segments by using a comfort noise generation technology. Then, the audio signals of the non-silence segments and the audio signals of the silence segments are combined, obtaining a final decoding result.

[0041] The technical scheme provided by the embodiments of the present application can be applied to a low code rate audio communication system.

[0042] The satellite communication system shown in FIG. 2 is a typical low code rate audio communication system.

[0043] The terminal device 1 (for example, a mobile phone) transmits the encoded code stream to a satellite. The satellite sends the received code stream to a satellite ground station. The satellite ground station transcodes the code stream from the satellite and sends the transcoded code stream to the terminal device 2 through a network device.

[0044] FIG. 3 is a process schematic diagram of processing a signal provided by the embodiments of the present application.

[0045] As shown in FIG. 3, the feature extraction module of the sending end can extract the audio features of the non-speech audio. For the convenience of description, the non-speech audio whose audio features are extracted by the feature extraction module can be referred to as original non-speech audio. In other words, the feature extraction module can extract the audio features of the original non-speech audio. After the audio features of the original non-speech audio are extracted, the sending end can obtain a code stream by quantizing and encoding the audio features of the non-speech audio, and then send the code stream to the receiving end through a transmission link.

[0046] Correspondingly, the receiving end receives the code stream, decodes and dequantizes the code stream to obtain quantized audio features. The non-speech audio generation module performs frame prediction based on the quantized audio features to obtain predicted mute audio features, and then generates target non-speech audio according to the predicted mute audio features. The target non-speech audio is the speech audio that can be heard by the user of the receiving end.

[0047] FIG. 4 is a schematic flowchart of a method of processing a signal provided by the embodiments of the present application. The method shown in FIG. 4 can be executed by a sending end device, or can be executed by a component (for example, a chip, a system on chip (SoC), etc.) in the sending end device. The sending end device can be a mobile phone, a tablet computer, a smart watch, etc. that can communicate. For the convenience of description, it is assumed that the method shown in FIG. 4 is executed by the sending end device, and the method can include steps 401-403.

[0048] 401, the sending end device extracts audio features of original non-speech audio to obtain N audio features.

[0049] 402, the sending end device quantizes and encodes the N audio features to obtain a code stream containing the N audio features.

[0050] 403, the sending end device sends the code stream.

[0051] The original non-speech audio is non-speech audio for which audio features need to be extracted. The original non-speech audio can be a part of the non-speech audio in the silence segment output by the VAD module. Taking Figure 5 as an example, Figure 5 is a schematic diagram of the process of extracting audio features. Assume that the duration T total = t5 - t1 of the silence segment output by the VAD module, then the original non-speech audio can be this segment of non-speech audio from time t1 to time t4. As shown in Figure 5, the original non-speech audio can be divided into multiple original non-speech frames. The lengths of different original non-speech frames can be the same or different. However, for the sake of description, it is assumed below that the lengths of different original non-speech frames are the same. Figure 5 shows two original non-speech frames, original non-speech frame 1 and original non-speech frame 2. The start time of original non-speech frame 1 is t1, and the end time is t3; the start time of original non-speech frame 2 is t2, and the end time is t4, where t3 - t1 = t4 - t2. As shown in Figure 5, t 1 < t 2 < t 3 < t 4 . In other words, there is an overlap between original non-speech frame 1 and original non-speech frame 2, that is, there is an overlap between adjacent original non-speech frames as shown in Figure 5.

[0052] It can be understood that Figure 5 is only an example of two original non-speech frames, rather than a limitation on the original non-speech frames. The embodiments of the present application do not limit whether there is an overlap between two adjacent original non-speech frames. In other words, in some embodiments, there is an overlap between adjacent original non-speech frames. In other embodiments, there may also be no overlap between adjacent original non-speech frames. In addition, the embodiments of the present application do not limit the number of silence segments output by the VAD module. For example, in some embodiments, the VAD module may only output one silence segment, while in other embodiments, the VAD module may output two or more silence segments.

[0053] The sending device can extract the audio features of N original non-speech frames respectively to obtain N audio features. The N audio features correspond to the N original non-speech frames one by one, and each audio feature in the N audio features is the audio feature extracted from the corresponding original non-speech frame.

[0054] Optionally, in some embodiments, the sending device can extract the mel spectrogram of the original non-speech frame as the audio feature of the original non-speech frame.

[0055] Optionally, in other embodiments, the sending device can extract the mel-frequency cepstral coefficients (MFCC) of the original non-speech frame as the audio feature of the original non-speech frame.

[0056] Optionally, in some embodiments, the sender device can extract the audio features of the original non-speech frame by using an artificial intelligence (AI) model. Embodiments of the present application involve multiple neural networks. For the sake of distinction, the neural network used for extracting audio features can be referred to as a feature extraction model.

[0057] The feature extraction model can be a neural network, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a Transformer model, etc.

[0058] In some embodiments, the input signal of the feature extraction model can be the original non-speech frame. The output of the feature extraction model is the extracted T audio features. The dimension of the audio features extracted by the feature extraction model can be denoted by the letter D, and the number of extracted features can be denoted by the letter T, where T is a positive integer greater than or equal to 1. The number of features T can satisfy the following relationship:

[0059] T = signal_length / hop_size, (Equation 1.1)

[0060] where signal_length represents the length of the original non-speech audio, and hop_size represents the length of the starting time of adjacent two frames. For example, as shown in FIG. 5, hop_size is the length between t1 and t2.

[0061] In some other embodiments, the framing operation can also be implemented by the feature extraction model. In this case, the input signal of the feature extraction model can be the original non-speech audio. The feature extraction model can frame the input original non-speech audio to obtain T original non-speech frames, and then determine the audio features of each original non-speech frame.

[0062] For the sake of description, it is assumed in the following embodiments that the input of the feature extraction model is the original non-speech frame.

[0063] The following describes how to train the feature extraction model by taking a convolutional neural network as an example.

[0064] The convolutional neural network is a deep convolutional neural network with a convolutional structure. The convolutional neural network has better feature learning ability and can automatically extract useful information from non-speech frames. The core feature of the convolutional neural network is the convolution operation, which uses filters (also known as convolution kernels) and pooling layers to extract features of non-speech frames. The convolution operation can effectively reduce the number of weights and reduce the amount of calculation. The pooling layer can reduce the amount of calculation and improve the robustness of the model without changing the dimension of the feature map.

[0065] The convolution kernel can be initialized in the form of a random size vector or matrix, and the convolution kernel can obtain reasonable weights through learning in the training process of the convolutional neural network. In addition, the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0066] As an unsupervised learning algorithm, the autoencoder (AE) is widely used in data dimensionality reduction, feature extraction and other aspects. The combination of convolutional neural network and autoencoder can obtain the feature extraction model.

[0067] The CNN autoencoder is composed of an encoder and a decoder. The encoder is responsible for feature extraction and compression of the input data through convolutional layers, pooling layers and other structures to generate low-dimensional encoding representations. The decoder is responsible for reconstructing the original input data through convolutional layers and up-sampling layers.

[0068] In the encoding stage, the input data is gradually extracted through the alternating action of convolutional layers and pooling layers, and the high-level feature representation is compressed into a low-dimensional encoding representation. In this process, the redundant information in the data is removed, and the important features are retained.

[0069] In the decoding stage, the encoding representation is gradually restored to the spatial dimension and detail information of the data through the alternating action of convolutional layers and up-sampling layers, and finally reconstructs the data close to the original input. In this process, the decoder needs to learn how to reconstruct the original data from the encoding representation.

[0070] In the process of training the CNN auto-encoder, because it is desired that the output of the convolutional neural network is as close as possible to the value that is actually desired to be predicted, the weight vector of each layer of the convolutional neural network can be updated according to the difference between the predicted value of the current network and the value that is actually desired to be predicted (of course, before the first update, there is usually an initialization process, that is, the parameters of each layer of the convolutional neural network are pre-configured), for example, if the predicted value of the network is too high, the weight vector is adjusted to make it predict a lower value, and the adjustment is continuously made until the convolutional neural network can predict the value that is actually desired to be predicted or a value very close to the value that is actually desired to be predicted. For the CNN auto-encoder, the target value is the input training sample of the encoder, and the predicted value is the output value of the decoder, which is determined based on the input training sample. For example, the input training sample of the encoder is x, and the output of the encoder is the feature of the training sample x, denoted as z. The input of the decoder is the feature z, and the output is the predicted value x'. By comparing the difference between the training sample x and the predicted value x', the weight vector of the convolutional neural network can be adjusted.

[0071] In order to achieve the above purpose, it is necessary to define in advance "how to compare the difference between the predicted value and the target value", which is the loss function or the objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the convolutional neural network becomes a process of trying to minimize the loss.

[0072] The convolutional neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the convolutional neural network during the training process, so that the error loss between the predicted value output by the convolutional neural network and the value that is actually desired to be predicted becomes smaller and smaller. Specifically, the forward propagation of the input signal until the output produces an error loss, and the initial parameters in the convolutional neural network are updated by back propagating the error loss information, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the convolutional neural network, such as the weight vector or matrix, that is, the convolution kernel of the convolution layer.

[0073] After the training of the CNN auto-encoder is completed, the encoder in the trained CNN auto-encoder can be used as a feature extraction model.

[0074] The system architecture provided by the embodiments of the present application is introduced below.

[0075] Referring to FIG. 6, a system architecture 600 is provided in an embodiment of the present application. As shown in the system architecture 600, a data collection device 640 is configured to collect training data, and the training data can be stored in a database 630. In an embodiment of the present application, the training data includes one or more non-speech frames. A training device 620 can train a target model / rule 601 based on the training data maintained in the database 630. The target model / rule 601 is a feature extraction model for extracting audio features of original non-speech audio. That is, the original non-speech frames or original non-speech audio can be input into the target model / rule 601, and audio features can be obtained. In an embodiment of the present application, the target model / rule 601 can be a trained convolutional neural network. In an embodiment provided in the present application, the convolutional neural network is obtained by training an initialized convolutional neural network. It should be noted that, in actual applications, the training data maintained in the database 630 can not all be collected by the data collection device 640, but can also be received from other devices. In addition, it should be noted that the training device 620 can not train the target model / rule 601 based on the training data maintained in the database 630, but can also obtain training data from the cloud or other places to train the model. The above description should not be regarded as a limitation on the embodiments of the present application. The training data can be noise collected in public scenes, traffic scenes, work scenes, and the like.

[0076] The target model / rule 601 trained by the training device 620 can be applied to an execution device 610 shown in FIG. 6. The execution device 610 can be the sender device mentioned above. In FIG. 6, the execution device 610 is configured with an I / O interface 612 for data interaction with external devices. A user can input data to the I / O interface 612, and the input data can include an audio signal collected by a microphone in an embodiment of the present application. The microphone can be a microphone built in the execution device 610, or a microphone of a wired earphone connected to the execution device 610 through a wired interface (for example, a headphone interface, a universal serial bus (USB) interface, and the like), or a microphone of a wireless earphone connected to the execution device 610 through a wireless communication technology (for example, Wi-Fi).

[0077] The preprocessing module 613 is configured to perform preprocessing on the input data (e.g., the audio signal) received by the I / O interface 612. In an embodiment of the present application, the preprocessing module 613 can be configured to implement voice activity detection to determine a silent segment signal (i.e., the original non-speech audio), and can also be configured to segment the original non-speech audio to obtain a plurality of original non-speech frames. The preprocessing module 613 can also be configured to implement other preprocessing operations, which are not limited in the present application.

[0078] During the preprocessing of the input data performed by the execution device 610, or during the processing performed by the computing module 611 of the execution device 610, the modules (e.g., the computing module 611 and the preprocessing module 613) in the execution device 610 can call the data and codes stored in the storage module 615 to implement corresponding processing, and can also store the data and instructions obtained through the corresponding processing in the storage module 615.

[0079] After obtaining the audio features of the original non-speech frames based on the target model / rule 601, the computing module 611 can quantize and encode the audio features of the original non-speech frames to obtain a code stream containing the audio features. The communication module 614 is responsible for sending the code stream.

[0080] It is worth noting that FIG. 6 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in FIG. 6 does not constitute any limitation. For example, in FIG. 6, the storage module 615 is disposed in the execution device 610. In other cases, the storage module 615 can also be an external memory of the execution device 610.

[0081] As described above, the convolutional neural network is a deep learning architecture with a convolutional structure, which means that the algorithm of machine learning is used to learn at different levels of abstraction. As a deep learning architecture, the CNN is a feed-forward artificial convolutional neural network, and each neuron in the feed-forward artificial convolutional neural network can respond to the non-speech frames input therein.

[0082] As shown in FIG. 7, the convolutional neural network (CNN) 700 can include an input layer 710, a convolutional layer / down-sampling layer 720, a convolutional layer / up-sampling layer 730, and an output layer 740. The convolutional layer / down-sampling layer 720 is configured to implement the function of an encoder, and the convolutional layer / up-sampling layer 730 is configured to implement the function of a decoder.

[0083] As shown in FIG. 7, the convolutional layer / down-sampling layer 720 can include layers such as example 721-726, for example: in one implementation, the 721 layer is a convolutional layer, the 722 layer is a down-sampling layer, the 723 layer is a convolutional layer, the 724 layer is a down-sampling layer, the 725 layer is a convolutional layer, and the 726 layer is a down-sampling layer; in another implementation, the 721 and 722 layers are convolutional layers, the 723 layer is a down-sampling layer, the 724 and 725 layers are convolutional layers, and the 726 layer is a down-sampling layer. That is, the output of a convolutional layer can be used as the input of a subsequent down-sampling layer, or as the input of another convolutional layer to continue the convolution operation.

[0084] The core idea of a convolutional layer is to map the input features to the output feature space through a convolution operation. In a convolutional neural network, the convolution operator (which can be referred to as a convolution kernel) is a learnable parameter, and the parameters of the convolution kernel are adjusted through training to optimize the performance of the model. The convolution kernel can be represented as one or more dimensions, corresponding to one or more dimensions of the input signal. The convolution operator can essentially be a weight vector or matrix, and the weight values in the weight vector or matrix need to be obtained through a large amount of training in practical applications. The weight vectors or matrices formed by the weight values obtained through training can be used to extract information from the input, so that the convolutional neural network 700 can make correct predictions.

[0085] When the convolutional neural network 700 has multiple convolutional layers, the initial convolutional layer (e.g., 721) often extracts more general features, which can also be referred to as low-level features, corresponding to high-resolution feature maps. As the depth of the convolutional neural network 700 increases, the convolutional layers further back (e.g., 726) extract more complex features, such as high-level semantic features, corresponding to low-resolution feature maps. The higher the semantic features, the more suitable they are for the problem to be solved.

[0086] Because it is often necessary to reduce the number of training parameters, a down-sampling layer is often periodically introduced after a convolutional layer. In the example 721-726 layers shown in FIG. 7, one convolutional layer can be followed by one down-sampling layer, or multiple convolutional layers can be followed by one or more down-sampling layers. The core idea of a down-sampling layer is to map the input features to a lower resolution space through a down-sampling operation. Common pooling operations include max pooling and average pooling. Max pooling selects the maximum value in the input window as the output, and average pooling selects the average value in the input window as the output.

[0087] As shown in FIG. 7, the convolution layer / upsampling layer 730 can include layers such as examples 731-736, for example: in one implementation, the 731 layer is an upsampling layer, the 732 layer is a convolution layer, the 723 layer is an upsampling layer, the 724 layer is a convolution layer, the 725 is an upsampling layer, and the 726 is a convolution layer; in another implementation, the 731 is an upsampling layer, the 732 and 733 are convolution layers, the 734 is an upsampling layer, and the 735 and 736 are convolution layers.

[0088] The convolution layer in the convolution layer / upsampling layer 730 can also be referred to as a transposed convolution layer, which is used to upsample the encoded representation and gradually recover the spatial dimension of the data.

[0089] The upsampling layer:

[0090] The upsampling layer corresponds to the downsampling layer and is used to increase the spatial resolution of the data.

[0091] Similarly, as shown in FIG. 7, the 731-732 layers in the 730 can be one upsampling layer followed by one convolution layer, or one or more upsampling layers followed by multiple convolution layers.

[0092] The output layer 740 has a loss function similar to the classification cross-entropy, which is specifically used to calculate the prediction error. Once the forward propagation (as shown in FIG. 7, the propagation from 710 to 740 is forward propagation) of the entire convolutional neural network 700 is completed, the back propagation (as shown in FIG. 7, the propagation from 740 to 710 is back propagation) will start to update the weight values and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 700, and the error between the result output by the convolutional neural network 700 through the output layer and the ideal result.

[0093] The input layer 710 and the convolution layer / downsampling layer 720 in the trained convolutional neural network 700 can be used as a feature extraction model.

[0094] It should be noted that the convolutional neural network 700 shown in FIG. 7 is only an example of a convolutional neural network, and in specific applications, the convolutional neural network can also exist in the form of other network models.

[0095] Next, a chip hardware structure provided by an embodiment of the present application is introduced.

[0096] Figure 8 is a chip hardware structure provided by an embodiment of the present application, which includes a convolutional neural network processor 80. The chip can be arranged in the execution device 610 shown in Figure 6 to complete the computing work of the computing module 611. The chip can also be arranged in the training device 620 shown in Figure 6 to complete the training work of the training device 620 and output the target model / rule 601. The algorithms of each layer in the convolutional neural network shown in Figure 7 can be implemented in the chip shown in Figure 8.

[0097] The convolutional neural network processor 80 can be a neural network processing unit (NPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), or any other processor suitable for large-scale XOR operation processing. Taking the NPU as an example: the NPU can be mounted as a coprocessor to the central processing unit (CPU), i.e., the Host CPU, and tasks are allocated to it by the Host CPU. The core part of the NPU is the operation circuit 803, which extracts matrix data in the memory (801 and 802) and performs multiplication and addition operations under the control of the controller 804.

[0098] In some implementations, the operation circuit 803 includes a plurality of processing engines (PEs) inside. In some implementations, the operation circuit 803 is a two-dimensional systolic array. The operation circuit 803 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 803 is a general matrix processor.

[0099] For example, assuming there are input data A, weight data B, and output data C. The operation circuit 803 takes the weight values of data B from the weight memory 802 and caches them on each PE in the operation circuit 803. The operation circuit 803 takes the input data of data A from the input memory 801, performs operations according to the input data of data A and the weight values of data B, and saves the partial results or final results in the accumulator 808. The input data can be original non-speech frames, and the weight data can be a weight vector or matrix, i.e., a convolution kernel.

[0100] The unified memory 806 is used to store input data and output data. The weight vector or matrix is carried into the weight memory 802 by the direct memory access controller (DMAC) 805. The input data is also carried into the unified memory 806 by the direct memory access controller 805.

[0101] The bus interface unit (BIU) 810 is used for the interaction between the direct memory access controller 805 and the instruction fetch buffer 809. The bus interface unit 810 is also used for the instruction fetch buffer 809 to fetch instructions from an external memory. The bus interface unit 810 is also used for the direct memory access controller 805 to fetch the original data of the input data A or the weight data B from an external memory.

[0102] The vector calculation unit 807 can include a plurality of operation processing units, which further process the output of the operation circuit 803 as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and the like. The vector calculation unit 807 is mainly used for the calculation of the non-convolution layer or the fully connected layer (FC) in the convolutional neural network, and can specifically process the calculation of pooling, normalization, and the like. For example, the vector calculation unit 807 can apply a nonlinear function to the output of the operation circuit 803, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 807 generates a normalized value, a merged value, or both.

[0103] In some implementations, the vector calculation unit 807 stores the processed vector to the unified memory 806. In some implementations, the vector processed by the vector calculation unit 807 can be used as an activation input of the operation circuit 803, for example, for use in a subsequent layer in the convolutional neural network, such as the current processing layer 731 in FIG. 7 is the hidden layer 1, and the vector processed by the vector calculation unit 807 can also be used for the calculation in the hidden layer 2 (such as 732).

[0104] The instruction fetch buffer 809 connected to the controller 804 is used to store instructions used by the controller 804.

[0105] The unified memory 806, the input memory 801, the weight memory 802, and the instruction fetch buffer 809 are all on-chip memories.

[0106] In the convolutional neural network shown in FIG. 7, the operations of each layer can be performed by the operation circuit 803 or the vector calculation unit 807.

[0107] Optionally, in some embodiments, the sending device can further extract a linear prediction coding (LPC) of the original non-speech frame as the audio feature of the original non-speech frame.

[0108] FIG. 9 is a schematic flowchart of another method for processing a signal according to an embodiment of the present application. The method shown in FIG. 9 can be performed by a receiving device, or can be performed by a component (e.g., a chip, a system on chip (SoC), etc.) in the receiving device. The receiving device can be a mobile phone, a tablet computer, a smart watch, etc. For ease of description, it is assumed that the method shown in FIG. 9 is performed by a receiving device, and the method includes steps 901 to 903.

[0109] 901, the receiving device obtains N quantized audio features of an original non-speech audio.

[0110] The receiving device can receive a bitstream from a sending device. The received bitstream is decoded and dequantized to obtain the N quantized audio features. The N quantized audio features correspond to the N audio features one-to-one. Each of the N quantized audio features is obtained by quantizing, encoding, decoding, and dequantizing the corresponding audio feature.

[0111] The audio features extracted from the non-speech audio can also be referred to as silence audio features. Correspondingly, the corresponding quantized audio features can also be referred to as quantized silence audio features.

[0112] 902, the receiving device determines a first predicted silence audio feature according to the N quantized audio features and a neural network.

[0113] The neural network can be referred to as a prediction model or a first neural network.

[0114] The input of the prediction model is the N quantized audio features. The output of the prediction model is the first predicted silence audio feature. The prediction model can be an autoregressive-based neural network. The autoregressive-based neural network can be implemented by a neural network, a DNN, a CNN, an RNN, a Transformer model, etc.

[0115] The prediction model is used to predict the most restored silence audio features. The training samples for training the prediction model can include a plurality of silence audio features. In the process of training the prediction model, the parameters of the current model can be updated based on the continuous N+1 silence audio features. Specifically, in the process of training the prediction model, the difference between the current model based on the first N silence audio features in the continuous N+1 silence audio features and the last silence audio feature in the continuous N+1 silence audio features is compared. Then the parameters in the current model are updated according to the difference between the two. For example, if the non-speech audio determination model is an RNN, the weight vector of each layer of the RNN can be updated according to the difference between the silence audio feature output by the current model and the last silence audio feature in the N+1 silence audio features in the training sample, until the RNN can predict the N+1 silence audio feature or basically predict the N+1 silence audio feature. The RNN can use the back propagation (BP) algorithm to correct the size of the parameters in the RNN during the training process, so that the error loss between the predicted value output by the RNN and the target value that is actually wanted becomes smaller and smaller. Specifically, the forward propagation of the input signal until the output produces an error loss, and the parameters in the initial RNN are updated by back propagating the error loss information, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the RNN, such as weight vectors or matrices, or convolution kernels of convolution layers.

[0116] In some embodiments, the training samples for training the prediction model can be determined according to the feature extraction model. As described above, the training samples for training the prediction model can include a plurality of silence audio features, which can be determined by the feature extraction model according to a plurality of non-speech audio frames. Assuming that there is a data set X, the data set X includes a plurality of non-speech audio frames. The non-speech audio frames in the data set X are respectively input into the trained feature extraction model to obtain a feature set Z, the feature set Z includes a plurality of silence audio features, and the plurality of silence audio features correspond one-to-one to the plurality of non-speech audio frames. The obtained silence audio features are quantization encoded to obtain a code stream containing silence audio features, and the code stream is obtained through a transmission link or an analog transmission link. The dequantization silence audio features obtained after decoding and dequantization of the code stream can be used as training samples for training the prediction model.

[0117] 903, the receiving end device determines the first non-speech audio corresponding to the original non-speech audio according to the first predicted silence audio feature and another neural network.

[0118] The neural network used to determine the first non-speech audio can be referred to as a non-speech audio determination model or a second neural network.

[0119] Optionally, in some embodiments, after the first non-speech audio is determined, the receiving end device can further predict a silence audio feature and determine a second non-speech audio corresponding to the original non-speech audio according to the predicted silence audio feature.

[0120] For example, in some embodiments, when N is equal to 1, the receiving end device can determine a second predicted silence audio feature according to the first predicted silence audio feature and the prediction model.

[0121] For another example, in some embodiments, when N is a positive integer greater than 1, the receiving end device can determine a second predicted silence audio feature according to the first predicted silence audio feature and the last N-1 quantized audio features of the N quantized audio features and the prediction model.

[0122] After the second silence audio feature is determined, a second non-speech audio corresponding to the original non-speech audio can be determined according to the second silence audio feature and the non-speech audio determination model. The determination process of the second non-speech audio is similar to that of the first non-speech audio, and thus will not be described here for brevity.

[0123] Optionally, in some embodiments, the receiving end device can further determine N non-speech audios according to the N quantized audio features and a non-speech audio determination model.

[0124] For example, in some embodiments, the quantized audio features can be directly input into the non-speech audio determination model. The non-speech audio determination model can determine a non-speech audio corresponding to the quantized audio features based on the input quantized audio features.

[0125] The reconstructed audio of the original non-speech audio includes the N non-speech audios and a non-speech audio predicted according to the N non-speech audios, for example, the first non-speech audio.

[0126] For another example, in some embodiments, the VAD module outputs a silence segment with a time length T total where the original non-speech audio is a segment of non-speech audio from time t1 to time t4. The encoding end device extracts N audio features from the segment of non-speech audio, quantizes and encodes the N extracted audio features, and transmits the code stream to the receiving end device. However, the segment of non-speech audio from time t4 to time t5 can not need to go through the above process. The receiving end device can obtain a reconstructed audio of the original non-speech audio based on the N quantized audio features obtained from the code stream. The time length of the reconstructed audio of the original non-speech audio is Ttotal wherein the receiving end device can obtain N non-speech audios based on the N quantized audio features and the neural network, and the time length of the N non-speech audios is t4-t1. Then, the receiving end device can predict the silence audio features by using the N quantized audio features, and obtain the predicted non-speech audio based on the predicted silence audio features and the neural network, and the time length of the predicted non-speech audio is t5-t4.

[0127] The non-speech audio determined by the receiving end device corresponds to the original non-speech audio, which is the target non-speech audio mentioned above.

[0128] The input of the non-speech audio determination model is the first predicted silence audio features, and the output of the non-speech audio determination model is the first non-speech audio. The non-speech audio determination model can be a neural network, a DNN, a CNN, an RNN, a Transformer model, etc.

[0129] The non-speech audio determination model is used to output the non-speech audio as much as possible. The training samples used to train the non-speech audio determination model can include a plurality of non-speech audios and silence audio features determined according to the plurality of non-speech audios. In the process of training the non-speech audio determination model, the difference between the non-speech audio output by the current model and the non-speech audio in the training sample can be compared. Then, the parameters in the current model are updated according to the difference between the two. For example, if the non-speech audio determination model is a CNN, the weight vector of each layer of the convolutional neural network can be updated according to the difference between the non-speech audio output by the current model and the non-speech audio in the training sample (of course, before the first update, there is usually an initialization process, that is, the parameters of each layer of the convolutional neural network are pre-configured), until the convolutional neural network can restore the non-speech audio or basically restore the non-speech audio. The convolutional neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the convolutional neural network during the training process, so that the error loss between the predicted value output by the convolutional neural network and the target value that is actually wanted becomes smaller and smaller. Specifically, the forward propagation of the input signal until the output produces an error loss, and the error loss information is propagated backward to update the parameters in the initial convolutional neural network, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the convolutional neural network, such as weight vectors or matrices, that is, the convolution kernel of the convolutional layer.

[0130] In some embodiments, the training samples for training the non-speech audio determination model can be determined according to the feature extraction model. As described above, the training samples for training the non-speech audio determination model can include a plurality of non-speech audios and the silence audio features determined according to the plurality of non-speech audios, and the silence audio features in the training samples can be determined by the feature extraction model according to the plurality of non-speech audios. Assuming that there is a data set X, the data set X includes a plurality of non-speech frames. The non-speech frames in the data set X are respectively input into the trained feature extraction model to obtain a feature set Z, and the feature set Z includes a plurality of silence audio features, which correspond to the plurality of non-speech frames in a one-to-one manner. The obtained audio features are quantized and encoded to obtain a code stream containing the plurality of silence audio features, and the code stream is obtained through a transmission link or an analog transmission link. The code stream is decoded and dequantized to obtain a plurality of quantized silence audio features. The plurality of quantized silence audio features and the plurality of non-speech frames in the data set X can be used as training samples of the non-speech audio determination model. The non-speech frames are target values, and the output of the non-speech audio determination model based on the quantized silence audio features is a predicted value. By comparing the difference between the predicted value and the target value, the weight vector of the convolutional neural network can be adjusted.

[0131] FIG. 10 is a system architecture 1000 provided by an embodiment of the present application.

[0132] The system architecture 1000 shown in FIG. 10 can be used for the prediction model and the non-speech audio determination model.

[0133] As shown in the system architecture 1000 shown in FIG. 10, the data acquisition device 1040 is used to acquire training data. If the prediction model is trained, the training data can include a plurality of silence audio features. If the non-speech audio determination model is trained, the training data includes one or more non-speech audios and the silence audio features corresponding to the one or more non-speech audios.

[0134] The training data can be stored in the database 1030, and the training device 1020 can train the target model / rule 1001 and the target model / rule 1002 based on the training data maintained in the database 1030. The target model / rule 1001 (i.e., the prediction model) can predict the next silence audio feature based on the consecutive N silence audio features. The target model / rule 1002 (i.e., the non-speech audio determination model) can determine the non-speech audio based on the silence audio feature, i.e., inputting the silence audio feature into the target model / rule 1002 to obtain the non-speech audio.

[0135] It should be noted that in actual application, the training data maintained in the database 1030 does not necessarily come from the collection of the data collection device 1040, but can also be received from other devices. In addition, it should be noted that the training device 1020 does not necessarily train the target model / rule 1001 based on the training data maintained in the database 1030, but can also obtain training data from the cloud or other places for model training. The above description should not be regarded as a limitation of the embodiments of the present application.

[0136] The target model / rule 1001 and the target model / rule 1002 trained by the training device 1020 can be applied to the execution device 1010 shown in FIG. 10, which can be the receiving end device mentioned above. In FIG. 10, the execution device 1010 is configured with a communication module 1012. The communication module 1012 is configured to receive a code stream. A calculation module 1011 is configured to decode and dequantize the code stream to obtain N quantized audio features. Then, the calculation module 1011 determines non-speech audio based on the target model / rule 1001, the target model / rule 1002 and the N quantized audio features. Then, the non-speech audio can be played through a loudspeaker or a headset connected to the execution device 1010. For example, the calculation module 1011 can predict first predicted silence audio features based on the target model / rule 1001 and the N quantized audio features, and then determine the non-speech audio based on the first predicted silence audio features and the target model / rule 1002.

[0137] In the pre-processing of the input data by the execution device 1010, or in the related processing process of the calculation performed by the calculation module 1011 of the execution device 1010, the modules (such as the calculation module 1011) in the execution device 1010 can call the data, code, etc. in the storage module 1013 to implement the corresponding processing, or store the data, instructions, etc. obtained by the corresponding processing in the storage module 1013.

[0138] It should be noted that FIG. 10 is only a schematic diagram of a system architecture provided by the embodiments of the present application, and the positional relationship between the devices, components, modules, etc. shown in FIG. 10 does not constitute any limitation. For example, in FIG. 10, the storage module 1013 is placed in the execution device 1010. In other cases, the storage module 1013 can also be an external memory of the execution device 1010.

[0139] If the target model / rule 1001 / 1002 is a convolutional neural network, the convolutional neural network processor 80 shown in FIG. 8 can also be arranged in the execution device 1010 shown in FIG. 10 to complete the calculation work of the calculation module 1011, or can also be arranged in the training device 1020 shown in FIG. 10 to complete the training work of the training device 1020 and output the target model / rule 1001 / 1002.

[0140] FIG. 11 is a schematic structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 1100 shown in FIG. 11 can be used as the sending-end device in the above embodiments. As shown in FIG. 11, the electronic device 1100 includes a processing unit 1101 and a sending unit 1102.

[0141] The processing unit 1101 is configured to extract audio features of the original non-speech audio to obtain N audio features.

[0142] The processing unit 1101 is further configured to quantize and encode the N audio features to obtain a code stream containing the N audio features.

[0143] The sending unit 1102 is configured to send the code stream obtained by the processing unit 1101.

[0144] The specific functions and advantages of the processing unit 1101 and the sending unit 1102 can refer to the above embodiments, and for the sake of brevity, will not be repeated here.

[0145] The processing unit 1101 can be implemented by a processor, and the sending unit 1102 can be implemented by a transmitter.

[0146] FIG. 12 is a schematic structural block diagram of another electronic device provided by an embodiment of the present application. The electronic device 1200 shown in FIG. 12 can be used as the receiving-end device in the above embodiments. As shown in FIG. 12, the electronic device 1200 includes an obtaining unit 1201 and a processing unit 1202.

[0147] The obtaining unit 1201 is configured to obtain N quantized audio features of the original non-speech audio, where N is a positive integer greater than or equal to 1.

[0148] The processing unit 1202 is configured to determine a first predicted mute audio feature according to the N quantized audio features and a first neural network.

[0149] The processing unit 1202 is further configured to determine a first non-speech audio corresponding to the original non-speech audio according to the first predicted mute audio feature and a second neural network.

[0150] The specific functions and advantages of the acquisition unit 1201 and the processing unit 1202 can refer to the above embodiments, and for the sake of brevity, will not be repeated here.

[0151] The acquisition unit 1201 can be implemented by a processor and a receiver. For example, the receiver receives a code stream, and the processor decodes and inverse quantizes the code stream to obtain the N quantized audio features.

[0152] The processing unit 1202 can be implemented by a processor.

[0153] FIG. 13 is a schematic diagram of another electronic device according to an embodiment of the present application. As shown in FIG. 13, the electronic device 1300 includes a processor 1301 configured to execute computer programs or instructions stored in a memory 1302, or read data / signaling stored in the memory 1302, to perform the methods in the above method embodiments. Optionally, the processor 1301 is one or more.

[0154] The memory 1302 is configured to store computer programs or instructions and / or data. The memory 1302 can be integrated with the processor 1301, or can be separately arranged. Optionally, the memory 1302 is one or more.

[0155] Optionally, as shown in FIG. 13, the electronic device 1300 further includes a transceiver 1303 configured to receive and / or send signals. For example, the processor 1301 is configured to control the transceiver 1303 to receive and / or send signals.

[0156] As an option, the electronic device 1300 is configured to implement the operations performed by the sending-end device in the above method embodiments.

[0157] As another option, the electronic device 1300 is configured to implement the operations performed by the receiving-end device in the above method embodiments.

[0158] It should be understood that the processor mentioned in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0159] It should also be appreciated that the memory referred to in the embodiments of the present application can be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM). For example, the RAM can be used as an external cache. By way of example, and not limitation, RAM can include the following varieties: static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0160] It should be noted that when the processor is a general processor, a DSP, an ASIC, a FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, the memory (storage module) can be integrated in the processor.

[0161] It should also be noted that the memory described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0162] FIG. 14 is a schematic diagram of a chip system 1400 provided by an embodiment of the present application. The chip system 1400 (or also can be referred to as a processing system) includes a logic circuit 1401 and an input / output interface 1402.

[0163] The logic circuit 1401 can be a processing circuit in the chip system 1400. The logic circuit 1401 can be coupled to a storage unit, and invoke instructions in the storage unit, so that the chip system 1400 can implement the methods and functions of the embodiments of the present application. The input / output interface 1402 can be an input / output circuit in the chip system 1400, and output information processed by the chip system 1400, or input data or signaling information to be processed by the chip system 1400.

[0164] As an option, the chip system 1400 is configured to implement operations performed by the sending device or the receiving device in the above various method embodiments.

[0165] The embodiments of the present application also provide a computer readable storage medium, which stores computer instructions for implementing the method performed by the sending device or the receiving device in the above various method embodiments.

[0166] For example, the computer program is executed by an electronic device, so that the electronic device can implement the method performed by the sending device or the receiving device in the above various method embodiments.

[0167] The embodiments of the present application also provide a computer program product, which contains instructions, and the instructions are executed by a computer to implement the method performed by the sending device or the receiving device in the above various method embodiments.

[0168] The embodiments of the present application also provide a communication system, which includes the sending device and the receiving device as described above.

[0169] The above provides any one of the devices related to the explanation and beneficial effects of the above-mentioned corresponding method embodiments, and will not be repeated here.

[0170] The above, only for the specific implementation of the present application, but the protection scope of the present application is not limited to this, any skilled in the art of the technical personnel in the technical range of the present application, can easily think of changes or replacement, should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of processing a signal, characterized by, The method comprises: obtaining N quantized audio features of the original non-speech audio, N being a positive integer greater than or equal to 1; determining a first predicted mute audio feature according to the N quantized audio features and a first neural network; determining a first non-speech audio corresponding to the original non-speech audio according to the first predicted mute audio feature and a second neural network.

2. The method of claim 1, wherein, The first neural network is an autoregressive-based neural network.

3. The method according to claim 1 or 2, characterized in that, N is a positive integer less than or equal to 3.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: determining a second predicted mute audio feature according to N reference audio features and the first neural network, wherein, in the case of N equal to 1, the N reference audio features are the first predicted mute audio feature, and in the case of N being a positive integer greater than 1, the N reference audio features comprise the first predicted mute audio feature and the last N-1 quantized audio features among the N quantized audio features; determining a second non-speech audio corresponding to the original non-speech audio according to the second predicted mute audio feature and the second neural network.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: receiving a code stream through a low-code-rate audio communication channel, the low-code-rate audio communication channel at least comprising a satellite communication channel, the code stream containing N audio features that are quantized and encoded, the N audio features being extracted from the original non-speech audio; decoding and dequantizing the code stream to obtain the N quantized audio features, the N quantized audio features corresponding one-to-one to the N audio features.

6. The method of claim 5, wherein, The N audio features are mel-spectrograms of the original non-speech audio or mel-frequency cepstral coefficients of the original non-speech audio, or the N audio features are features extracted from the original non-speech audio by a third neural network.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: determining N non-speech audios according to the N quantized audio features and the second neural network, wherein a reconstructed audio of the original non-speech audio comprises the N non-speech audios and the first non-speech audio.

8. An electronic device, comprising: The method comprises: an obtaining unit configured to obtain N quantized audio features of an original non-speech audio, N being a positive integer greater than or equal to 1; a processing unit configured to determine a first predicted mute audio feature according to the N quantized audio features and a first neural network; the processing unit is further configured to determine a first non-speech audio corresponding to the original non-speech audio according to the first predicted mute audio feature and a second neural network.

9. The electronic device of claim 8, wherein, The first neural network is an autoregressive-based neural network.

10. The electronic device of claim 8 or 9, wherein, N is a positive integer less than or equal to 3.

11. The electronic device of any of claims 8-10, wherein, The processing unit is further configured to determine a second predicted mute audio feature according to N reference audio features and the first neural network, wherein, in the case of N equal to 1, the N reference audio features are the first predicted mute audio feature, and in the case of N being a positive integer greater than 1, the N reference audio features comprise the first predicted mute audio feature and the last N-1 quantized audio features among the N quantized audio features. The processing unit is further configured to determine, according to the second predicted mute audio feature and the second neural network, a second non-speech audio corresponding to the original non-speech audio.

12. The electronic device of any of claims 8-11, wherein, The obtaining unit is specifically configured to receive a code stream through a low-code-rate audio communication channel, decode and dequantize the code stream to obtain the N quantized audio features. The low-code-rate audio communication channel at least includes a satellite communication channel. The code stream contains N quantized and encoded audio features. The N audio features are extracted from the original non-speech audio. The N quantized audio features correspond to the N audio features one by one.

13. The electronic device of claim 12, wherein The N audio features are mel-frequency spectrum graphs of the original non-speech audio or mel-frequency cepstral coefficients of the original non-speech audio, or the N audio features are features extracted by a third neural network from the original non-speech audio.

14. The electronic device of any of claims 8-13, wherein, The processing unit is further configured to determine, according to the N quantized audio features and the second neural network, N non-speech audios, wherein a reconstructed audio of the original non-speech audio includes the N non-speech audios and the first non-speech audio.

15. An electronic device, comprising: including: A processor configured to be coupled with a memory, read and execute instructions and / or program codes in the memory, to perform the method of any one of claims 1 to 7.

16. A chip system, characterized by including: A logic circuit configured to be coupled with an input / output interface, transmit data through the input / output interface, to perform the method of any one of claims 1 to 7.

17. A computer readable medium characterized by The computer readable medium stores program codes, which, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 7.

Images