Audio frame processing method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for processing audio frames, wherein a target audio model for audio frame processing is split into multiple connected local models, and the method comprises: obtaining a first state parameter corresponding to a previous audio frame before a current audio frame, and using the first state parameter as an input state parameter of the current audio frame; based on the input state parameter, processing the current audio frame using the multiple local models; in the process of processing the current audio frame, generating a second state parameter of the current audio frame, and using the second state parameter as an input state parameter of a next audio frame after the current audio frame. The technical solution provided by the present invention can effectively process audio frames in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to a method, system, device and storage medium for processing an audio frame. Background Art

[0002] In order to improve the playback effect of audio data, the audio data is usually processed by echo cancellation, noise reduction, automatic gain control, etc. With the help of machine learning technology, at this stage, a large number of audio data samples can be learned to train an audio processing model with better effect.

[0003] However, in scenarios where audio frames need to be processed in real time, the trained audio processing model often does not perform well. The reason is that the training samples of the audio processing model are usually audio data of a certain length. During the training of these audio data, multiple inference iterations are performed on the audio data of the previous and next frames to generate processed audio data. However, in the real-time processing scenario of audio frames, each input to the audio processing model is a separate audio frame, so the audio processing model cannot measure the connection between adjacent audio frames, resulting in distortion of the generated audio frames when played continuously.

[0004] It can be seen that a method that can effectively process audio frames in real time is currently needed. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a method, system, device and storage medium for processing an audio frame, which can effectively process the audio frame in real time.

[0006] On the one hand, the present invention provides a method for processing an audio frame, in which a target audio model used for audio frame processing is split into multiple connected local models; the method comprises: obtaining a first state parameter corresponding to an audio frame before a current audio frame, and using the first state parameter as an input state parameter of the current audio frame; based on the input state parameter, processing the current audio frame using the multiple local models; in the process of processing the current audio frame, generating a second state parameter of the current audio frame, and using the second state parameter as the input state parameter of the next audio frame after the current audio frame.

[0007] In one embodiment, the multiple local models include a long short-term memory model and at least one set of codec model pairs, the codec model pairs including a bound encoding model and a decoding model; wherein, for any current decoding model, the input data of the current decoding model includes the output result of the previous decoding model or the output result of the long short-term memory model, and the output result of the encoding model bound to the current decoding model.

[0008] In one embodiment, the target audio model used for audio frame processing is split into multiple connected local models including: dividing the hierarchical structure in the target audio model according to the hierarchical structure contained in the local model to form multiple local models; obtaining a coefficient matrix corresponding to the hierarchical structure in the target audio model, and updating the obtained coefficient matrix to the corresponding hierarchical structure of the local model.

[0009] In one embodiment, the first state parameter is used to characterize the temporary cache data generated by each of the local models when processing the previous audio frame; the second state parameter is used to characterize the temporary cache data generated by each of the local models when processing the current audio frame.

[0010] In one embodiment, the first state parameters include at least the hidden element parameters and memory element parameters corresponding to the long short-term memory model when processing the previous audio frame, the intermediate state parameters corresponding to each of the encoding models when processing the previous audio frame, and the intermediate state parameters corresponding to each of the decoding models when processing the previous audio frame; the second state parameters include at least the hidden element parameters and memory element parameters corresponding to the long short-term memory model when processing the current audio frame, the intermediate state parameters corresponding to each of the encoding models when processing the current audio frame, and the intermediate state parameters corresponding to each of the decoding models when processing the current audio frame.

[0011] In one embodiment, the first state parameter is written into a global cache as a current state parameter, and after the second state parameter is generated, the second state parameter replaces the first state parameter as the current state parameter in the global cache.

[0012] In one embodiment, the method further includes: for the processed current audio frame, determining a sub-segment to be smoothed at the tail of the processed current audio frame, and after adjusting the amplitude of the sub-segment, superimposing it with the adjusted sub-segment at the head of an audio frame after the current audio frame to generate a smoothly processed audio frame.

[0013] On the other hand, the present invention provides an audio frame processing system, in which a target audio model for audio frame processing is split into multiple connected local models; the system includes: a state parameter acquisition unit, used to acquire a first state parameter corresponding to a previous audio frame before a current audio frame, and use the first state parameter as an input state parameter of the current audio frame; an audio frame processing unit, used to process the current audio frame using the multiple local models based on the input state parameter; a state parameter generation unit, used to generate a second state parameter of the current audio frame during the processing of the current audio frame, and the second state parameter is used as the input state parameter of the next audio frame after the current audio frame.

[0014] Another aspect of the present invention provides an audio frame processing device, the device comprising a processor and a memory, the memory being used to store a computer program, and when the computer program is executed by the processor, the above-mentioned audio frame processing method is implemented.

[0015] Another aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned audio frame processing method is implemented.

[0016] The technical solution provided by the present application can split the trained target audio model into multiple connected local models. The significance of model splitting is that when the local model processes the audio frame, it can generate intermediate state parameters, and the state parameters can characterize the characteristic information of the audio frame during the processing. In the present application, the state parameters can be temporarily stored. When the next audio frame needs to be processed, the stored state parameters can be used as the input state parameters of the next audio frame, and processed together with the data of the next audio frame by each local model. In this way, the state parameters of the adjacent audio frames are introduced in the processing of the next audio frame, so that the processing result of the next audio frame can be associated with the processing result of the adjacent audio frame.

[0017] When audio frames are processed in real time in this way, each processing process is combined with the state parameters of adjacent audio frames. As a result, when the processed audio frames are played continuously, the sound effects can be relatively coherent without obvious distortion or stuttering, thereby improving the audio playback effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0019] Figure 1A schematic diagram showing the connection of the split local models in one embodiment of the present invention is shown;

[0020] Figure 2 A schematic diagram showing the steps of a method for processing an audio frame in one embodiment of the present invention is shown;

[0021] Figure 3 A processing schematic diagram of an LSTM model in one embodiment of the present invention is shown;

[0022] Figure 4 A schematic diagram showing inter-frame smoothing processing in one embodiment of the present invention is shown;

[0023] Figure 5 A schematic diagram of functional modules of an audio frame processing system in one embodiment of the present invention is shown;

[0024] Figure 6 A schematic structural diagram of an audio frame processing device in one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] The target audio model trained based on the existing machine learning algorithm often has a better effect when processing audio data in non-real time. The reason is that the training data of the target audio model is usually audio data of a certain length. When processing the audio data as a whole, it can ensure that the processed audio data also has good coherence.

[0027] However, when the existing target audio model processes audio data in real time, it inputs a single audio frame. The target audio model cannot determine the connection between the audio frame and the adjacent previous and next audio frames, which may cause the sound to be incoherent or distorted when the processed audio frames are played continuously.

[0028] In view of this, in one embodiment, the trained target audio model can be split into multiple local models. When the split local models process the input audio frames, some intermediate state parameters will be generated. These state parameters can represent the characteristic information of the audio frames. These state parameters can be temporarily stored in the cache. When processing the current audio frame, the state parameters of the adjacent audio frames are introduced to reuse and update the state parameters, so that the processed audio frames can maintain association with the adjacent audio frames.

[0029] Specifically, the target audio model usually includes multiple neural network layers, which can constitute the hierarchical structure of the target audio model. For example, the target audio model may include convolutional layers, activation layers, normalization layers, deconvolutional layers, etc. According to the functions implemented by each neural network layer in the target audio model, the hierarchical structure of the target audio model can be divided to obtain groupings of the neural network layers. For example, multiple neural network layers in the target audio model jointly implement the encoding function, so this part of the neural network layer can be divided into a group. For another example, multiple neural network layers in the target audio model jointly implement the decoding function, so this part of the neural network layer can also be divided into a group. In the grouped result, the connection relationship between each neural network layer can be retained, and each group can correspond to a local model. In this way, by dividing the hierarchical structure in the target audio model, the hierarchical structures of multiple local models can be formed.

[0030] In this embodiment, the purpose of splitting the target audio model into multiple local models is that the state parameters generated by the original target audio model when processing audio data cannot be obtained externally, but after being split into local models, the state parameters generated by the local models can be obtained and stored externally.

[0031] See also Figure 1 In a specific application example, the local model obtained after the splitting may include a long short-term memory (LSTM) model and at least one set of codec model pairs. The codec model pair may include a bound coding model and a decoding model. This binding relationship between the coding model and the decoding model may be reflected in the processing of the audio frame. Specifically, Figure 1 As shown, for any current decoding model (eg, decoding model N), a portion of input data of the current decoding model may include an output result of a coding model (eg, coding model N) bound to the current decoding model.

[0032] In this application example, the split local models can be arranged in the order of encoding model, long short-term memory model, and decoding model. The audio frame to be processed can be input into the first encoding model, and then can be encoded step by step by the subsequent encoding models. The encoding result obtained by the encoding model can be sent to the LSTM model for processing. The output result of the LSTM model can be decoded layer by layer by the decoding model, and finally the processed audio frame can be output. Figure 1 As shown, when the decoding model performs a decoding operation, the input data includes not only the output result of the bound encoding model, but also the output result of the previous model. The previous model can be an LSTM model or a previous decoding model. For example, for decoding model N, since its previous model is an LSTM model, the input data of decoding model N includes the output result of encoding model N and the output result of LSTM model. For decoding model 1, since its previous model is decoding model 2, the input data of decoding model 1 includes the output result of encoding model 1 and the output result of decoding model 2.

[0033] It should be noted that after dividing the hierarchical structure in the target audio model to form hierarchical structures of multiple local models, since some hierarchical structures in the target audio model originally have coefficient matrices, after the hierarchical structure of the local model is divided, the corresponding coefficient matrices need to be updated to the local model. For example, for neural network layers such as convolutional layers and deconvolutional layers, coefficient matrices are usually contained within the layers. In this case, these coefficient matrices should be updated to the corresponding neural network layers in the local model. In practical applications, the coefficient matrix corresponding to the hierarchical structure can be obtained in the target audio model, and the obtained coefficient matrix can be updated to the corresponding hierarchical structure of the local model, thereby completing the process of splitting the local model.

[0034] See also Figure 2 , a method for processing an audio frame provided in one embodiment of the present application may include the following steps.

[0035] S1: Obtain a first state parameter corresponding to a previous audio frame before a current audio frame, and use the first state parameter as an input state parameter of the current audio frame.

[0036] S2: Based on the input state parameters, the current audio frame is processed using the multiple local models; during the processing of the current audio frame, a second state parameter of the current audio frame is generated, and the second state parameter is used as an input state parameter of a next audio frame after the current audio frame.

[0037] In this embodiment, for each audio frame, after being processed by each local model, while generating the processed audio frame, a corresponding state parameter is also generated, and the state parameter can characterize the characteristic information of the audio frame. In order to improve the processing effect of the audio frame, the state parameter of the previous audio frame can be used as a reference when processing the current audio frame. In this way, the processed current audio frame can have a certain association with the previous audio frame, so that each audio frame is more coherent when played continuously. Specifically, each local model after the splitting can generate corresponding state parameters when processing the audio frame, and this part of the state parameters can be stored as temporary cache data. When processing the next audio frame, the temporary cache data can be used as a reference, such as reusing and updating the temporary cache data, so as to maintain the connection between the previous and next audio frames.

[0038] It can be seen that in order to achieve inter-frame coherence of audio frames, it is necessary to temporarily store the intermediate state data of each network layer in the target audio model, and these intermediate state data can be used to process the next audio frame. In practical applications, if the intermediate state data of each network layer in the target audio model is obtained, it will result in a large amount of data processing, thereby affecting the processing efficiency of the audio data. In view of this, in the present application, the target audio model can be disassembled into local models according to the functions implemented by the network layer. In each local model, a combination of network layers can be included, and these combinations of network layers can jointly implement a specific function. For example, the above-mentioned encoding model can implement the encoding function, the decoding model can implement the decoding function, and so on. Obtaining the intermediate state data in the form of a local model can reduce the amount of data processing, thereby improving the data processing efficiency while ensuring the coherence of the audio frame.

[0039] Taking the state parameters generated by the LSTM model as an example, the state parameters generated by the LSTM model can be hidden meta-parameters and memory meta-parameters. Since the target audio model is split into local models, the state parameters generated by the LSTM model can be obtained and temporarily stored. In a specific application example, the state parameters corresponding to the audio frame can be written into the global cache, so that the state parameters corresponding to the audio frame can be read from the global cache.

[0040] It should be noted that the state parameters written in the global cache can correspond to the identifier of the audio frame. For example, if the identifier of the current audio frame is A01, then the state parameters of the audio frame in the global cache can also be represented by A01. In this way, the state parameters of multiple audio frames can be written in the global cache, and when the state parameters of a certain audio frame are needed, they can be read from the global cache based on the identifier of the audio frame. In addition, in one embodiment, in order to save space in the global cache, the state parameters of only one audio frame can be written in the same time period. For example, when the current audio frame is processed, the state parameters corresponding to the current audio frame can be written to the global cache. If the state parameters of another audio frame already exist in the global cache before the state parameters corresponding to the current audio frame are written, the state parameters of the current audio frame can be used to replace the existing state parameters, thereby ensuring that there are only state parameters of one audio frame in the global cache. Then when processing the next audio frame after the current audio frame, the state parameters of the current audio frame can be read from the global cache.

[0041] Similarly, after the state parameters corresponding to the next audio frame are generated, they can also be written into the global cache. At this time, the state parameters of the next audio frame can replace the state parameters of the current audio frame, thereby ensuring that there are still only state parameters of one audio frame in the global cache.

[0042] See also Figure 3 In this embodiment, when the current audio frame is processed by each local model, the data of the current audio frame can be imported into the first encoding model, and then after being encoded step by step by each encoding model, the encoding result can be sent to the LSTM model. When the LSTM model processes the encoding result of the current audio frame, the first state parameter corresponding to the previous audio frame before the current audio frame can be obtained, and then the first state parameter can be used as the input state parameter of the current audio frame, and processed by the LSTM model together with the encoding result of the first audio frame. Figure 3 In the example, the input data (input) of the LSTM model can be the encoding result of each encoding model for the current audio frame, and the input state parameter can be the first state parameter of the previous audio frame, including the hidden meta-parameter (in_hn) and the memory meta-parameter (in_cn). After being processed by the LSTM model, the output data (output) of the current audio frame can be obtained, and the output data can be used as part of the input data of the decoding model. At the same time, the LSTM model can also generate the second state parameter of the current audio frame. The second state parameter can also include the hidden meta-parameter (out_hn) and the memory meta-parameter (out_cn).

[0043] It should be noted that, in practical applications, the first state parameter and the second state parameter may include not only the state parameter generated by the LSTM model, but also the state parameter generated by the encoding module and the decoding module. The above is just an explanation of the scheme using the LSTM model as an example, and does not mean that the scheme of this application can only be applied to the scenarios of hidden meta-parameters and memory meta-parameters.

[0044] also, Figure 3 The LSTM model shown is only a simplified model. In actual applications, there may be more processing units related to the LSTM model, such as a slice unit, a transpose unit, a squeeze unit, a concat unit, and so on.

[0045] In a practical application, when the first state parameter of the previous audio frame is generated, it can be written into the global cache as the current state parameter. When processing the current audio frame, the first state parameter can be read from the global cache and used as the input state parameter of the current audio frame. After the second state parameter corresponding to the current audio frame is generated, the second state parameter can replace the first state parameter as the current state parameter in the global cache. In this way, in the global cache, there will only be state parameters of one audio frame in the same time period, thereby saving space in the global cache.

[0046] Since there are only one audio frame state parameter in the global cache, whenever an audio frame is processed, the state parameter stored at the current moment can be directly read from the global cache without querying the state parameter according to the audio frame identifier, thereby improving the processing efficiency of the audio frame.

[0047] In this embodiment, the output data of the LSTM model can be input into the decoding model, and the output result of the encoding model bound to the decoding model can also be input into the decoding model, so as to perform the registration decoding process. The final result output by the decoding model can be used as the processed current audio frame.

[0048] It can be seen that since each processing process combines the state parameters of adjacent audio frames, the processed audio frames can have a relatively coherent sound effect when played continuously, without obvious distortion or freeze, thereby improving the audio playback effect.

[0049] In one embodiment, after each audio frame is processed in the above manner, the output audio frame may still have some noise. In order to improve the processing effect of the audio frame, inter-frame smoothing processing can be performed on the output audio frame. Specifically, for the current audio frame, a sub-segment to be smoothed can be determined in the current audio frame. Figure 4As shown in , the sub-segment is usually the head and tail of the current audio frame and the adjacent previous and next audio frames. After determining the sub-segment to be smoothed, the amplitude window can be used to adjust the amplitude of the sub-segment. Figure 4 As shown, the amplitude window can be a broken line window, wherein the broken line window can be composed of two amplitude adjustment lines, wherein the first amplitude adjustment line can adjust the amplitude of the tail sub-segment of the current audio frame, and the second amplitude adjustment line can adjust the amplitude of the head field of the next audio frame. The two adjusted fields can be superimposed, so that the amplitude after superposition can be the same as the amplitude before adjustment. It can be seen that the adjusted sub-segment of the current audio frame can be superimposed with the adjusted sub-segment in the adjacent audio frame, so as to generate the current audio frame after smoothing. The audio frame that has been smoothed between frames can maintain a good correlation with the two adjacent audio frames before and after, so that better effects can be achieved during continuous playback.

[0050] See also Figure 5 The present application also provides an audio frame processing system, wherein a target audio model for audio frame processing is split into a plurality of connected local models; the system comprises:

[0051] a state parameter acquisition unit, configured to acquire a first state parameter corresponding to a previous audio frame before a current audio frame, and use the first state parameter as an input state parameter of the current audio frame;

[0052] an audio frame processing unit, configured to process the current audio frame using the multiple local models based on the input state parameters;

[0053] The state parameter generating unit is used to generate a second state parameter of the current audio frame during the processing of the current audio frame, and the second state parameter is used as an input state parameter of a next audio frame after the current audio frame.

[0054] See also Figure 6 The present application also provides an audio frame processing device, the device includes a processor and a memory, the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned audio frame processing method is implemented.

[0055] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned audio frame processing method is implemented.

[0056] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0057] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0058] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0059] The technical solution provided by the present application can split the trained target audio model into multiple connected local models. The significance of model splitting is that when the local model processes the audio frame, it can generate intermediate state parameters, and the state parameters can characterize the characteristic information of the audio frame during the processing. In the present application, the state parameters can be temporarily stored. When the next audio frame needs to be processed, the stored state parameters can be used as the input state parameters of the next audio frame, and processed together with the data of the next audio frame by each local model. In this way, the state parameters of the adjacent audio frames are introduced in the processing of the next audio frame, so that the processing result of the next audio frame can be associated with the processing result of the adjacent audio frame.

[0060] When audio frames are processed in real time in this way, each processing process is combined with the state parameters of adjacent audio frames. As a result, when the processed audio frames are played continuously, the sound effects can be relatively coherent without obvious distortion or stuttering, thereby improving the audio playback effect.

[0061] Those skilled in the art can understand that all or part of the processes in the above-mentioned implementation methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the implementation methods of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0062] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for processing an audio frame, characterized in that: The target audio model for audio frame processing is split into multiple connected local models; the multiple local models include a long short-term memory model and at least one set of codec model pairs, and the codec model pairs include a bound encoding model and a decoding model; wherein, for any current decoding model, the input data of the current decoding model includes an output result of a previous decoding model or an output result of the long short-term memory model, and an output result of the encoding model bound to the current decoding model; the method includes: Obtaining a first state parameter corresponding to a previous audio frame before a current audio frame, and using the first state parameter as an input state parameter of the current audio frame; the first state parameter is used to characterize temporary cache data generated by each of the local models when processing the previous audio frame; the first state parameter at least includes a hidden element parameter and a memory element parameter corresponding to the long short-term memory model when processing the previous audio frame, an intermediate state parameter corresponding to each of the encoding models when processing the previous audio frame, and an intermediate state parameter corresponding to each of the decoding models when processing the previous audio frame; Based on the input state parameters, the current audio frame is processed using the multiple local models; in the process of processing the current audio frame, a second state parameter of the current audio frame is generated, and the second state parameter is used as the input state parameter of the next audio frame after the current audio frame; the second state parameter is used to characterize the temporary cache data generated by each of the local models when processing the current audio frame; the second state parameter at least includes the hidden element parameters and memory element parameters corresponding to the long short-term memory model when processing the current audio frame, the intermediate state parameters corresponding to each of the encoding models when processing the current audio frame, and the intermediate state parameters corresponding to each of the decoding models when processing the current audio frame; The intermediate state parameters are used to characterize feature information of the audio frame.

2. The method according to claim 1, characterized in that The target audio model for audio frame processing is split into multiple connected local models including: Dividing the hierarchical structure in the target audio model to form hierarchical structures of multiple local models; A coefficient matrix corresponding to the hierarchical structure is obtained in the target audio model, and the obtained coefficient matrix is ​​updated to the corresponding hierarchical structure of the local model.

3. The method according to claim 1, characterized in that: The first state parameter is written into the global cache as the current state parameter, and after the second state parameter is generated, the second state parameter replaces the first state parameter as the current state parameter in the global cache.

4. The method according to claim 1, characterized in that: The method further comprises: For the processed current audio frame, a sub-segment to be smoothed is determined at the tail of the processed current audio frame, and after the amplitude of the sub-segment is adjusted, it is superimposed with the adjusted sub-segment at the head of an audio frame after the current audio frame to generate a smoothly processed audio frame.

5. A system for processing an audio frame, characterized in that: The target audio model for audio frame processing is split into multiple connected local models; the multiple local models include a long short-term memory model and at least one set of codec model pairs, and the codec model pairs include a bound encoding model and a decoding model; wherein, for any current decoding model, the input data of the current decoding model includes an output result of a previous decoding model or an output result of the long short-term memory model, and an output result of the encoding model bound to the current decoding model; the system includes: A state parameter acquisition unit, used to acquire a first state parameter corresponding to a previous audio frame before a current audio frame, and use the first state parameter as an input state parameter of the current audio frame; the first state parameter is used to characterize temporary cache data generated by each of the local models when processing the previous audio frame; the first state parameter at least includes hidden element parameters and memory element parameters corresponding to the long short-term memory model when processing the previous audio frame, intermediate state parameters corresponding to each of the encoding models when processing the previous audio frame, and intermediate state parameters corresponding to each of the decoding models when processing the previous audio frame; an audio frame processing unit, configured to process the current audio frame using the multiple local models based on the input state parameters; A state parameter generating unit, used for generating a second state parameter of the current audio frame during processing of the current audio frame, the second state parameter being used as an input state parameter of a next audio frame after the current audio frame; the second state parameter being used for characterizing temporary cache data generated by each of the local models when processing the current audio frame; the second state parameter at least includes a hidden element parameter and a memory element parameter corresponding to the long short-term memory model when processing the current audio frame, an intermediate state parameter corresponding to each of the encoding models when processing the current audio frame, and an intermediate state parameter corresponding to each of the decoding models when processing the current audio frame; The intermediate state parameters are used to characterize feature information of the audio frame.

6. An audio frame processing device, characterized in that: The device includes a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method for processing an audio frame as claimed in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method for processing an audio frame as described in any one of claims 1 to 4 is implemented.

Images